From 6964659bff8a1e52259fd7a1937e379897bc0a28 Mon Sep 17 00:00:00 2001 From: Joshua Snyder Date: Wed, 8 Jan 2025 09:46:06 +0000 Subject: [PATCH 1/2] chore: update docs to include flox --- .../engineering/developing-locally.md | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/contents/handbook/engineering/developing-locally.md b/contents/handbook/engineering/developing-locally.md index 5e1feb1e9136..ff5c4153fe73 100644 --- a/contents/handbook/engineering/developing-locally.md +++ b/contents/handbook/engineering/developing-locally.md @@ -109,7 +109,23 @@ This is a faster option to get up and running. If you don't want to or can't use git clone https://github.com/PostHog/posthog && cd posthog/ ``` -### Get things up and running +### Environment setup + +You can set up your development environment quickly using [Flox](https://github.com/flox/flox). To do that, you should only need to: +1. Install required system-level dependencies for flox: + +```bash +brew install flox ruff rustup && rustup-init && rustup default stable +``` +2. From the root of the repository, activate the environment: +```bash +flox activate +``` +This gets you a fully-fledged environment under `.flox/` in the repository directory's, with everything needed to run and develop PostHog. Once done, you'll see instructions for migrations and running the app. On the first activation you'll also be prompted if you'd like the environment to be activated automatically. + +### Manual setup + +Alternatively, if you would prefer not to use [Flox](https://github.com/flox/flox), you can manually setup your environment by following the next steps. #### 1. Spin up external services @@ -339,10 +355,6 @@ Open [http://localhost:8010](http://localhost:8010) to see the app. To get some practical test data into your brand-new instance of PostHog, run `DEBUG=1 ./manage.py generate_demo_data`. For a list of useful arguments of the command, run `DEBUG=1 ./manage.py generate_demo_data --help`. -You can also use [mprocs](https://github.com/pvolok/mprocs) to run all development processes in a single terminal window. mprocs provides a clean interface for starting, stopping, and monitoring each service individually, with separate log views for easier debugging. This includes docker compose, so make sure to stop it if it's already running. Once you have mprocs installed, run -```bash -./bin/start-mprocs -``` #### 7. Develop From 1dcf19b791e66d51a42424174c28da3809106a12 Mon Sep 17 00:00:00 2001 From: Michael Matloka Date: Thu, 9 Jan 2025 16:37:01 +0100 Subject: [PATCH 2/2] Tweak the order of things and wording Biggest change to the flow: the /etc/hosts/ change doesn't need to be done in the Flox flow, since the Flox activation script already takes care of this. --- .../engineering/developing-locally.md | 68 ++++++++++--------- 1 file changed, 37 insertions(+), 31 deletions(-) diff --git a/contents/handbook/engineering/developing-locally.md b/contents/handbook/engineering/developing-locally.md index c0f8c7fc0b15..a7b17050ca4e 100644 --- a/contents/handbook/engineering/developing-locally.md +++ b/contents/handbook/engineering/developing-locally.md @@ -71,14 +71,14 @@ This is a faster option to get up and running. If you don't want to or can't use 2. Install the package manager Homebrew by following the [instructions here](https://brew.sh/). -
- After installation, make sure to follow the instructions printed in your terminal to add Homebrew to your{' '} - $PATH. Otherwise the command line will not know about packages installed with brew. -
+
+ After installation, make sure to follow the instructions printed in your terminal to add Homebrew to your{' '} + $PATH. Otherwise the command line will not know about packages installed with brew. +
3. Install [OrbStack](https://orbstack.dev/) – a more performant Docker Desktop alternative – with `brew install orbstack`. Go to OrbStack settings and set the memory usage limit to **at least 4 GB** (or 8 GB if you can afford it) + the CPU usage limit to at least 4 cores (i.e. 400%). You'll want to use Brex for the license if you work at PostHog. -4. Continue with the [common prerequisites for both macOS and Linux](#common-prerequisites-for-both-macos--linux). +4. Continue with [cloning the repository](#cloning-the-repository). #### Ubuntu @@ -89,48 +89,56 @@ This is a faster option to get up and running. If you don't want to or can't use ```bash sudo apt install -y build-essential ``` -3. Continue with the [common prerequisites for both macOS and Linux](#common-prerequisites-for-both-macos--linux). +3. Continue with [cloning the repository](#cloning-the-repository). -#### Common prerequisites for both macOS & Linux +#### Cloning the repository -1. Append line `127.0.0.1 kafka clickhouse` to `/etc/hosts`. You can do it in one line with: - ```bash - echo '127.0.0.1 kafka clickhouse' | sudo tee -a /etc/hosts - ``` +Clone the [PostHog repo](https://github.com/posthog/posthog). All future commands assume you're inside the `posthog/` folder. - ClickHouse and Kafka won't be able to talk to each other without these mapped hosts. +```bash +git clone https://github.com/PostHog/posthog && cd posthog/ +``` - > If you are using a newer (>=4.1) version of Podman instead of Docker, the host machine's `/etc/hosts` is used as the base hosts file for containers by default, instead of container's `/etc/hosts` like in Docker. This can make hostname resolution fail in the ClickHouse container, and can be mended by setting `base_hosts_file="none"` in [`containers.conf`](https://github.com/containers/common/blob/main/docs/containers.conf.5.md#containers-table). +### Instant setup -2. Clone the [PostHog repository](https://github.com/posthog/posthog). All future commands assume you're inside the `posthog/` folder. +You can set your development environment up instantly using [Flox](https://flox.dev/). Flox is a dev environment manager, which takes care of the whole dependency graph needed to develop PostHog – all declared in the repo's `.flox/manifest.toml`. To get this environment going: + +1. Install Flox (plus `ruff` and `rustup` for pre-commit checks outside of the Flox env). ```bash - git clone https://github.com/PostHog/posthog && cd posthog/ + brew install flox ruff rustup && rustup-init && rustup default stable ``` -### Environment setup +2. From the root of the repository, activate the environment. (On first activation, you'll be prompted if you'd like the environment to be activated automatically using `direnv`.) -You can set up your development environment quickly using [Flox](https://github.com/flox/flox). To do that, you should only need to: -1. Install required system-level dependencies for flox: - -```bash -brew install flox ruff rustup && rustup-init && rustup default stable -``` -2. From the root of the repository, activate the environment: -```bash -flox activate -``` -This gets you a fully-fledged environment under `.flox/` in the repository directory's, with everything needed to run and develop PostHog. Once done, you'll see instructions for migrations and running the app. On the first activation you'll also be prompted if you'd like the environment to be activated automatically. + ```bash + flox activate + ``` + +This gets you a fully fledged environment, with its executables and linked libraries stored under `.flox/`. You should now be seeing instructions for migrations and running the app. + +That's it! You can now change PostHog in any way you want. See [Project structure](/handbook/engineering/project-structure) for an intro to the repository's contents. To commit changes, create a new branch based on `master` for your intended change, and develop away. ### Manual setup -Alternatively, if you would prefer not to use [Flox](https://github.com/flox/flox), you can manually setup your environment by following the next steps. +Alternatively, if you'd prefer not to use [Flox-based instant setup](#instant-setup), you can set the environment up manually: #### 1. Spin up external services In this step we will start all the external services needed by PostHog to work. +First, append line `127.0.0.1 kafka clickhouse` to `/etc/hosts`. Our ClickHouse and Kafka data services won't be able to talk to each other without these mapped hosts. +You can do this in one line with: + +```bash +echo '127.0.0.1 kafka clickhouse' | sudo tee -a /etc/hosts +``` + +> If you are using a newer (>=4.1) version of Podman instead of Docker, the host machine's `/etc/hosts` is used as the base hosts file for containers by default, instead of container's `/etc/hosts` like in Docker. This can make hostname resolution fail in the ClickHouse container, and can be mended by setting `base_hosts_file="none"` in [`containers.conf`](https://github.com/containers/common/blob/main/docs/containers.conf.5.md#containers-table). + +Now, start the Docker Compose stack: + ```bash docker compose -f docker-compose.dev.yml up ``` @@ -362,9 +370,7 @@ To get some practical test data into your brand-new instance of PostHog, run `DE #### 7. Develop -This is it! You can now change PostHog in any way you want. See [Project Structure](/handbook/engineering/project-structure) for an intro to the repository's contents. - -To commit changes, create a new branch based on `master` for your intended change, and develop away. Just make sure not use to use `release-*` patterns in your branches unless putting out a new version of PostHog, as such branches have special handling related to releases. +That's it! You can now change PostHog in any way you want. See [Project structure](/handbook/engineering/project-structure) for an intro to the repository's contents. To commit changes, create a new branch based on `master` for your intended change, and develop away. ## Testing