Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Update contributing guidelines #763

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
a8ec7be
:memo: Update docs contribution guide
pavithraes Feb 6, 2024
498e3b9
Restrucutre contributing section
pavithraes Mar 14, 2024
a8923c8
:memo: Add contribution workflow
pavithraes Mar 19, 2024
6e59ae5
Misc updates
pavithraes Sep 3, 2024
0773607
Update local-setup-ui.md
gabalafou Sep 3, 2024
91a7bf8
More updates based on feedback and based on work on
gabalafou Sep 4, 2024
cabf62b
reword pre-requisites
gabalafou Sep 4, 2024
b2beb2f
forgot corepack
gabalafou Sep 4, 2024
e701804
typo
gabalafou Sep 4, 2024
92ca0d1
Update docusaurus-docs/community/contribute/local-setup-ui.md
gabalafou Sep 4, 2024
fda6d9a
Update docusaurus-docs/community/contribute/local-setup-ui.md
gabalafou Sep 4, 2024
8c9587f
move testing into advanced option
gabalafou Sep 4, 2024
0f5258b
Update docusaurus-docs/community/contribute/local-setup-ui.md
gabalafou Sep 6, 2024
94fd52b
Update docusaurus-docs/community/contribute/local-setup-ui.md
gabalafou Sep 6, 2024
6b3ebb6
Update docusaurus-docs/community/contribute/local-setup-ui.md
gabalafou Sep 6, 2024
a942ee3
Update docusaurus-docs/community/contribute/local-setup-ui.md
gabalafou Sep 6, 2024
d5e1d90
Update docusaurus-docs/community/contribute/local-setup-ui.md
gabalafou Sep 6, 2024
e77173f
Update docusaurus-docs/community/contribute/local-setup-ui.md
gabalafou Sep 6, 2024
05be358
Misc updates
pavithraes Sep 17, 2024
bfa1f6e
Add editable install in without docker setup
pavithraes Sep 17, 2024
8534f0a
merge main
pavithraes Sep 17, 2024
6cbcf70
Merge pull request #6 from conda-incubator/local-setup-ui
pavithraes Oct 1, 2024
6addb97
Update contrib guidelines for jlab extension
pavithraes Oct 1, 2024
7cd8516
Merge main
pavithraes Oct 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
284 changes: 83 additions & 201 deletions docusaurus-docs/community/contribute/contribute-code.md

Large diffs are not rendered by default.

57 changes: 46 additions & 11 deletions docusaurus-docs/community/contribute/contribute-docs.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,43 @@
---
sidebar_position: 2
description: Contribute to conda-store's documentation
---

> **Note**
> This page in active development.
# Contribute documentation

The new conda-store documentation website is built using [Docusaurus 2](https://docusaurus.io/) and organized using the [Diátaxis framework](https://diataxis.fr).
The documentation website is built using [Docusaurus 3][docusaurus] and organized using the [Diátaxis framework][diataxis] for each conda-store library.

## Local Development
## Style guidelines

conda-store documentation follows the [Nebari documentation style guide][nebari-style-guide] including the [capitalization preferences][nebari-style-guide-capitalization], which in-turn derives from the [Google developer documentation style guide][google-style-guide].

## Documentation structure

All three conda-store libraries are documented in the [`conda-store` GitHub repository][cs-github], in the `docusaurus-docs` folder with each library in a separate sub-folder.

## Contribution process

Similar to code contributions, you can open issues to track as discuss documentation updates and file pull requests to contribute changes.

## Local development setup

### Pre-requisites

1. Fork and clone the conda-store repository: `git clone https://github.com/<your-username>/conda-store.git`
2. Install [Node.js](https://nodejs.org/en), and verify the installation with `node -v`
2. Install [Node.js][nodejs], and verify the installed version with `node -v`.

### Local development
:::tip
To create and activate an isolated conda environment with nodejs, run:

:::note
You can also create an isolated environment for development.
```bash
conda create -n conda-store-docs nodejs
conda activate conda-store-docs
```
:::

From the root of the repository, navigate to the `/docusaurus-docs` directory, and run:
### Local development

Navigate to the `/docusaurus-docs` directory, and run:

```bash
npm install
Expand All @@ -32,15 +49,33 @@ You can then start a development server with the following:
npm run start
```

This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
This command starts a local development server at [localhost:3000](http://localhost:3000) which opens automatically in a new browser window. Most changes are reflected live without having to restart the server.
The only time you'd need to restart the server is if you are making changes to the documentation site through the `docusaurus.config.js` file.

### Build website

Run:
:::note
This is rarely required during development, but can be useful for verifying certain changes. The static files should **not** be committed to the repository.
:::

To build the website locally, run:

```bash
npm run build
```

This command generates static content into the `build` directory and can be served using any static contents hosting service.

Currently, conda-store docs are deployed using [Netlify][netlify].


<!-- External links -->

[docusaurus]: https://www.nebari.dev/docs/community/style-guide
[diataxis]: https://diataxis.fr
[nebari-style-guide]: https://www.nebari.dev/docs/community/style-guide
[nebari-style-guide-capitalization]: https://www.nebari.dev/docs/community/style-guide#capitalization
[google-style-guide]: https://developers.google.cn/style
[nodejs]: https://nodejs.org/en
[netlify]: https://www.netlify.com
[cs-github]: https://github.com/conda-incubator/conda-store
1 change: 1 addition & 0 deletions docusaurus-docs/community/contribute/issues.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
---
sidebar_position: 3
description: Best practices for opening issues
---

Expand Down
213 changes: 213 additions & 0 deletions docusaurus-docs/community/contribute/local-setup-core.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
---
sidebar_position: 4
description: Local development setup for conda-store
---

# Local setup for conda-store (core)

There are two main ways to set up your local environment and conda-store services (web UI, API server, database, etc.) for development:

- Using [Docker and Docker compose](#docker-setup-recommended): This is the recommended approach for working on `conda-store-server` library.
- Using [standalone mode](#standalone-setup): Required for running tests and for working on the `conda-store` (client) library.

:::important
You need a [local copy of the `conda-store` repository](community/contribute/contribute-code#setup-for-local-development) for the development setup.
:::

## Docker setup (recommended) 🐳

### Pre-requisites

Install the following dependencies before developing on conda-store:

- [Docker](https://docs.docker.com/engine/install/)
- [Docker compose](https://docs.docker.com/compose/install/)

### Local deployment

To **deploy `conda-store` locally**, run the following command:

```bash
docker compose up --build -d
```

On a fast machine, this should take about 10 seconds, assuming the docker images have been partially built before.

:::note
Most of the conda-store docker images are built/tested for amd64(x86-64). Notice the `architecture: amd64` within the `docker-compose.yaml` files.

There will be a performance impact when building and running on
arm architectures. Otherwise, this workflow has been shown to run and build on OSX.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
arm architectures. Otherwise, this workflow has been shown to run and build on OSX.
ARM architectures. Otherwise, this workflow has been shown to run and build on OSX.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, this workflow has been shown to run and build on OSX.

Assume here we refer to series M?

:::

The following resources will be available on the deployment:

| Resource | Localhost port | username | password |
|----------|----------------|----------|----------|
| **conda-store web server (UI)** ✨| [localhost:8080](http://localhost:8080)| `admin` | `password`|
| [MinIO](https://min.io/) S3 | [localhost:9000](http://localhost:9000) | `admin` | `password` |
| [PostgreSQL](https://www.postgresql.org/) (database: `conda-store`)| [localhost:5432](http://localhost:5432) | `admin` | `password` |
| [Redis](https://www.redis.com/) | [localhost:6379](http://localhost:6379) | - | password |

If you **make any changes** to `conda-store-server`,
run the following to have those changes in the deployment:

```bash
docker compose down -v
docker compose up --build
```

To **stop the deployment**, run:

```bash
docker compose stop
```

Optionally, to remove the containers, run:

```bash
docker compose rm -f
```

## Standalone setup 💻

### Pre-requisites

You need **conda** for this setup, you can install it with the instructions in the [documentation][conda-install].

### Development environment

Create a conda environment with the development dependencies, and activate the environment:

```bash
conda env create -f conda-store-server/environment-dev.yaml
conda activate conda-store-server-dev
```

To install the `conda-store-server` package in editable (development) mode, run the following from the root of the repository:

```bash
python -m pip install -e ./conda-store-server
```

To install the `conda-store` package in editable (development) mode, run the following from the root of the repository:

```bash
python -m pip install -e ./conda-store
```

### Start conda-store in standalone mode

Running `conda-store` in `--standalone` mode launches celery as a
subprocess of the web server.

```bash
python -m conda_store_server.server --standalone
```

Visit [localhost:8080](http://localhost:8080/) from your web browser to access the conda-store web UI. ✨

## Run the test suite ✅

You can run the codebase tests locally to verify your changes before submitting a pull request.
You need [Docker Compose](#pre-requisites) as well as the [conda development environment](#development-environment) to run the complete set of tests.

### conda-store (client)

#### Lint and format

Run the linting and formatting checks with hatch:

```bash
cd conda-store
hatch env run -e dev lint
```

#### Integration tests

These tests are stateful, so clear the state if you previously ran the conda-store-server service on Docker:

```bash
cd conda-store
docker compose down -v # ensure you've cleared state
docker compose up --build
```

Wait until the conda-store-server is running check by visiting [localhost:8080](http://localhost:8080).

Install `conda-store` (client) library in editable mode:

```bash
pip install -e .
```

Execute the scripts in the `tests` directory to run the tests:

```bash
./tests/unauthenticated-tests.sh
./tests/authenticated-tests.sh
./tests/authenticated-token-tests.sh
```

You need to explicitly set some environment variables to run the shebang tests:

```bash
export CONDA_STORE_URL=http://localhost:8080/conda-store
export CONDA_STORE_AUTH=basic
export CONDA_STORE_USERNAME=username
export CONDA_STORE_PASSWORD=password

./tests/shebang.sh
```

### conda-store-server

#### Lint and format

Run the linting and formatting checks with hatch:

```bash
cd conda-store-server
hatch env run -e dev lint
```

#### Package build

Check that the package builds:

```bash
cd conda-store-server
hatch build
```

#### Unit tests

Run the unit tests with pytest:

```bash
cd conda-store-server
pytest
```

#### Integration tests

These tests are stateful, so clear the state if you previously ran the conda-store-server service on Docker:

```bash
cd conda-store-server
docker-compose down -v # ensure you've cleared state
docker-compose up --build
```

Wait until the conda-store-server is running check by visiting [localhost:8080](http://localhost:8080).

Run the tests with hatch:

```bash
hatch env run -e dev playwright-test
hatch env run -e dev integration-test
```
Comment on lines +206 to +209
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have many ways to run the tests, hatch, pytest and calling bash scripts. At least for the Python ones we should stick to only one (either through hatch or pytest directly).

I would suggest hatch as we can also configure it so that we call the bash scripts with hatch https://hatch.pypa.io/latest/config/environment/overview/#scripts and set the env variables there too https://hatch.pypa.io/latest/config/environment/overview/#environment-variables

cc @peytondmurray

Copy link
Contributor

@peytondmurray peytondmurray Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☝️ Note that we will be deprecating these soon when the old admin UI is removed. It doesn't matter to me much what approach we take, this is something we can easily change later on, though I will pretty much always vote for a simple pytest invocation because it's what most folks will be familiar with.


<!-- External links -->

[conda-install]: https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html
Loading
Loading