Sciety

Table of contents

1. Development
   1. Running the app
   2. Running the tests
   3. Linting
   4. Architecture Decision Records
   5. Architecture sketch
2. Operations
   1. Releasing to production
   2. Looking at logs
   3. Updating event data
   4. Dump all data to a CSV
3. License

Development

Requirements

• Docker
• Docker Compose
• GNU Make
• Node.js
• Git LFS

The project contains a Makefile which uses Docker and Docker Compose for development.

Running the app

To build and run the app for development, execute:

make dev

You can now access the app at http://localhost:8080.

Rebuilding the container

Static content is attached to the containers as volumes so most updates are visible without a need to rebuild the container. However, changes to NPM dependencies, for example, require a rebuild. So you may need to execute

make dev

again before running further commands.

Configuring environment variables

You can use a .env file to pass environment variables to the container.

After running make dev the file will contain a set of instructions to follow.

Re-run make dev after modifying this file.

Running the tests

We use Jest to test the app. You can run it by executing:

make test

To run all the regression tests, execute:

make taiko

To run just one regression test file, execute:

make taiko TEST=[file-name].ts

Linting

We lint the app with ESLint. You can run it by:

make lint

You can fix problems, where possible, by executing:

make lint:fix

Operations

The application is deployed on a Kubernetes cluster via an Helm chart.

A staging environment is updated with every new commit on main that passes tests.

A production environment is updated manually by pushing a tag.

Releasing to production

Ensure your current reference is green in CI.

Run make release.

Looking at logs

Logs of all Pods are streamed to AWS CloudWatch for persistence and searchability.

A CloudWatch dashboard graphs log lines representing errors and shows the state of the alarm.

An monitoring SNS topic triggers a lambda function that notifies the Slack #sciety-general channel.

A CloudWatch user journey by IP query is available to track a single client across multiple requests (adjust timeframe and IP).

Updating event data

Run make -j 4 update-event-data.

Substitute 4 with the desired concurrency level.

Dump all data

Run make prod-sql.

At the prompt, execute this command:

\copy (SELECT date, type, payload FROM events ORDER BY date) TO STDOUT WITH CSV;

License

We released this software under the MIT license. Copyright © 2020 eLife Sciences Publications, Ltd.