From 54b08b8efdd3a8035bc7be07f7843a4d76bfa4bb Mon Sep 17 00:00:00 2001 From: Michelle Fu <83109586+mifu67@users.noreply.github.com> Date: Tue, 9 Jul 2024 16:22:47 -0700 Subject: [PATCH] fix(environment): restructure subheadings and clarify ingest setup (#1330) * restructure docs and clarify ingest setup * fix copy paste errors * fix link location * fix link location again * fix missing backticks * move getsentry instructions above siloed instance instructions * bug * properly bold text --- .../development/environment/index.mdx | 168 +++++++++--------- 1 file changed, 86 insertions(+), 82 deletions(-) diff --git a/develop-docs/development/environment/index.mdx b/develop-docs/development/environment/index.mdx index 1d9237da662e2..602c23ef9a312 100644 --- a/develop-docs/development/environment/index.mdx +++ b/develop-docs/development/environment/index.mdx @@ -51,76 +51,54 @@ You can create other users with `sentry createuser`. displayed correctly. -## Running siloed instances +### Frontend Only & Backend Only -By default `sentry devserver` will run a monolith mode application server. You can also run ``devserver`` with siloed application instances. Before you do, you need to create split silo databases. +Please refer to [Frontend Development Server](/frontend/development-server/) and [Backend Development Server](/backend/development-server/) for alternative ways to bring up the Sentry UI. -The devserver command supports `--silo` option that lets you create siloed instances. Any workers, consumers, or celery-beat processes will be started with the same silo mode. +### Enabling HTTPS -```shell -# Start control silo servers -sentry devserver --silo=control --celery-beat --workers +You may wish to run the development server in HTTPS mode. This can be done by generating and installing local certificates. -# Start region silo servers -sentry devserver --silo=region --celery-beat --workers --ingest +We will be using [mkcert](https://github.com/FiloSottile/mkcert) to create and install a locally-trusted, development certificate. The following will install `mkcert` and then create and install the local certificates. + +```shell +brew install mkcert +brew install nss # if you use Firefox +yarn mkcert-localhost ``` -Siloed servers have the following port assignments: -- 8000 - Webpack -- 8010 - Region silo webserver -- 8001 - Control silo webserver +Running `sentry devserver` will automatically use HTTPS when the certificates have been installed. -In the above setup your local environment will use org slug domains, and send requests to `us.dev.getsentry.net` (region silo) and `dev.getsentry.net` (control silo). All requests will be proxied by webpack and forwarded to the appropriate server based on hostname. +### Ingestion Pipeline (Relay) -### Ngrok and siloed servers +Some services are not run in all situations. Among those are Relay and the ingest workers. -To combine ngrok and local development servers you’ll need to reserve multiple domains in ngrok, and create a configuration file for ngrok: +**If you need to ingest errors, but don't require metrics ingestion:** Set `SENTRY_USE_RELAY=True` in `~/.sentry/sentry.conf.py`. If `sentry devservices` +is currently up, make sure to restart it after you make the change. This will launch Relay as part of the `devserver` workflow. -```yaml -version: '2' -authtoken: -tunnels: - acme-org: - proto: http - hostname: acme..ngrok.dev - addr: 8000 - control-silo: - proto: http - hostname: .ngrok.dev - host_header: 'rewrite' - addr: 8000 - region-silo: - proto: http - hostname: us..ngrok.dev - addr: 8010 - host_header: 'rewrite' -``` - -Then run all the required servers +Additionally, you can explicitly control this during `devserver` usage with the `--ingest` and `--no-ingest` flags. The `sentry devservices` +command will not update Relay automatically in that case, to do this manually run: ```shell -# Run a control silo with ngrok -sentry devserver --silo=control --ngrok .ngrok.dev - -# Run a region silo without ngrok -sentry devserver --silo=region --ngrok .ngrok.dev - -# Run ngrok -ngrok start --all --config regions.yml +sentry devservices up --skip-only-if relay +sentry devserver --workers --ingest ``` -This setup will result in both the region and control servers responding to different domains. The multi-region setup with ngrok also enables customer-domains and you'll need ngrok domains for each organization you plan on using. In this configuration, CORS will work similar to production. For ngrok setup with non-siloed development server see developement server. +**If you want to enable the entire metrics ingestion pipeline:** You need to add the following to your config at `~/.sentry/sentry.conf.py`: -### Siloed Django Shell +```python +SENTRY_USE_RELAY = True +SENTRY_USE_METRICS_DEV = True +SENTRY_EVENTSTREAM = "sentry.eventstream.kafka.KafkaEventStream" +SENTRY_FEATURES['organizations:metrics-extraction'] = True # Enables session metrics +SENTRY_FEATURES['organizations:transaction-metrics-extraction'] = True # Enables transaction metrics +``` -If you want to use a repl with silo modes active: +After enabling `KafkaEventStream` the `snuba` service has to be reset to pick up the new configuration: ```shell -# Start a control silo shell -SENTRY_SILO_DEVSERVER=1 SENTRY_SILO_MODE=CONTROL getsentry django shell - -# Start a region silo shell -SENTRY_SILO_DEVSERVER=1 SENTRY_SILO_MODE=REGION SENTRY_REGION=us getsentry django shell +sentry devservices rm snuba +sentry devservices up snuba ``` ## Running the Getsentry Development Server @@ -154,57 +132,83 @@ environment, you can create `getsentry/conf/settings/devlocal.py` and put the configuration option overrides there. This module will be automatically imported by `dev.py` if it exists. +To enable the ingest workers, follow the steps described here and run +```shell +getsentry devserver --workers --ingest +``` -### Frontend Only & Backend Only - -Please refer to [Frontend Development Server](/frontend/development-server/) and [Backend Development Server](/backend/development-server/) for alternative ways to bring up the Sentry UI. -### Enabling HTTPS +## Running siloed instances -You may wish to run the development server in HTTPS mode. This can be done by generating and installing local certificates. +By default `sentry devserver` will run a monolith mode application server. You can also run ``devserver`` with siloed application instances. Before you do, you need to create split silo databases. -We will be using [mkcert](https://github.com/FiloSottile/mkcert) to create and install a locally-trusted, development certificate. The following will install `mkcert` and then create and install the local certificates. +The devserver command supports `--silo` option that lets you create siloed instances. Any workers, consumers, or celery-beat processes will be started with the same silo mode. ```shell -brew install mkcert -brew install nss # if you use Firefox -yarn mkcert-localhost +# Start control silo servers +sentry devserver --silo=control --celery-beat --workers + +# Start region silo servers +sentry devserver --silo=region --celery-beat --workers --ingest ``` +Siloed servers have the following port assignments: -Running `sentry devserver` will automatically use HTTPS when the certificates have been installed. +- 8000 - Webpack +- 8010 - Region silo webserver +- 8001 - Control silo webserver +In the above setup your local environment will use org slug domains, and send requests to `us.dev.getsentry.net` (region silo) and `dev.getsentry.net` (control silo). All requests will be proxied by webpack and forwarded to the appropriate server based on hostname. -### Ingestion Pipeline (Relay) +### Ngrok and siloed servers -Some services are not run in all situations. Among those are Relay and the ingest workers. If you need -a more production-like environment in development, you can set `SENTRY_USE_RELAY=True` in `~/.sentry/sentry.conf.py`. If `sentry devservices` is currently up ,make sure to restart it after you make the change. This will launch Relay -as part of the `devserver` workflow. +To combine ngrok and local development servers you’ll need to reserve multiple domains in ngrok, and create a configuration file for ngrok: -Additionally, you can explicitly control this during `devserver` usage with the `--ingest` and `--no-ingest` flags. The `sentry devservices` -command will not update Relay automatically in that case, to do this manually run: +```yaml +version: '2' +authtoken: +tunnels: + acme-org: + proto: http + hostname: acme..ngrok.dev + addr: 8000 + control-silo: + proto: http + hostname: .ngrok.dev + host_header: 'rewrite' + addr: 8000 + region-silo: + proto: http + hostname: us..ngrok.dev + addr: 8010 + host_header: 'rewrite' +``` + +Then run all the required servers ```shell -sentry devservices up --skip-only-if relay -sentry devserver --workers --ingest -``` +# Run a control silo with ngrok +sentry devserver --silo=control --ngrok .ngrok.dev -If you want to enable the entire metrics ingestion pipeline, you need to add the following to your config at `~/.sentry/sentry.conf.py`: +# Run a region silo without ngrok +sentry devserver --silo=region --ngrok .ngrok.dev -```python -SENTRY_USE_RELAY = True -SENTRY_USE_METRICS_DEV = True -SENTRY_EVENTSTREAM = "sentry.eventstream.kafka.KafkaEventStream" -SENTRY_FEATURES['organizations:metrics-extraction'] = True # Enables session metrics -SENTRY_FEATURES['organizations:transaction-metrics-extraction'] = True # Enables transaction metrics +# Run ngrok +ngrok start --all --config regions.yml ``` -After enabling `KafkaEventStream` the `snuba` service has to be reset to pick up the new configuration: +This setup will result in both the region and control servers responding to different domains. The multi-region setup with ngrok also enables customer-domains and you'll need ngrok domains for each organization you plan on using. In this configuration, CORS will work similar to production. For ngrok setup with non-siloed development server see developement server. + +### Siloed Django Shell + +If you want to use a repl with silo modes active: ```shell -sentry devservices rm snuba -sentry devservices up snuba -``` +# Start a control silo shell +SENTRY_SILO_DEVSERVER=1 SENTRY_SILO_MODE=CONTROL getsentry django shell +# Start a region silo shell +SENTRY_SILO_DEVSERVER=1 SENTRY_SILO_MODE=REGION SENTRY_REGION=us getsentry django shell +``` ## Troubleshooting