Skip to content

Commit

Permalink
[DOCS] Add developer documentation section in the docs (#999)
Browse files Browse the repository at this point in the history
* Add new section with developer docs

* Fix name of link

* Add help for PR body
  • Loading branch information
plaguss authored Sep 25, 2024
1 parent d7e61b5 commit a178109
Show file tree
Hide file tree
Showing 3 changed files with 112 additions and 5 deletions.
12 changes: 7 additions & 5 deletions docs/sections/community/contributor.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ hide:
Thank you for investing your time in contributing to the project! Any contribution you make will be reflected in the most recent version of distilabel 🤩.

??? Question "New to contributing in general?"
If you're a new contributor, read the [README](https://github.com/argilla-io/distilabel/blob/develop/README.md) to get an overview of the project. In addition, here are some resources to help you get started with open-source contributions:
If you're a new contributor, read the [README](https://github.com/argilla-io/distilabel/blob/develop/README.md) to get an overview of the project. In addition, here are some resources to help you get started with open-source contributions:

* **Discord**: You are welcome to join the [distilabel Discord community](http://hf.co/join/discord), where you can keep in touch with other users, contributors and the distilabel team. In the following [section](#first-contact-in-discord), you can find more information on how to get started in Discord.
* **Git**: This is a very useful tool to keep track of the changes in your files. Using the command-line interface (CLI), you can make your contributions easily. For that, you need to have it [installed and updated](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) on your computer.
* **GitHub**: It is a platform and cloud-based service that uses git and allows developers to collaborate on projects. To contribute to distilabel, you'll need to create an account. Check the [Contributor Workflow with Git and Github](#contributor-workflow-with-git-and-github) for more info.
* **Developer Documentation**: To collaborate, you'll need to set up an efficient environment. Check the [Installation](../getting_started/installation.md) guide to know how to do it.
* **Discord**: You are welcome to join the [distilabel Discord community](http://hf.co/join/discord), where you can keep in touch with other users, contributors and the distilabel team. In the following [section](#first-contact-in-discord), you can find more information on how to get started in Discord.
* **Git**: This is a very useful tool to keep track of the changes in your files. Using the command-line interface (CLI), you can make your contributions easily. For that, you need to have it [installed and updated](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) on your computer.
* **GitHub**: It is a platform and cloud-based service that uses git and allows developers to collaborate on projects. To contribute to distilabel, you'll need to create an account. Check the [Contributor Workflow with Git and Github](#contributor-workflow-with-git-and-github) for more info.
* **Developer Documentation**: To collaborate, you'll need to set up an efficient environment. Check the [Installation](../getting_started/installation.md) guide to know how to do it.

## First Contact in Discord

Expand Down Expand Up @@ -129,6 +129,8 @@ In addition, on the right side, you can select a reviewer (for instance, if you

Finally, fill in the template carefully and follow the guidelines. Remember to link the original issue and enable the checkbox to allow maintainer edits so the branch can be updated for a merge. Then, click on `Create pull request`.

For the PR body, ensure you give a description of what the PR contains, and add examples if possible (and if they apply to the contribution) to help with the review process. You can take a look at [#PR 974](https://github.com/argilla-io/distilabel/pull/974) or [#PR 983](https://github.com/argilla-io/distilabel/pull/983) for examples of typical PRs.


### Review your pull request

Expand Down
104 changes: 104 additions & 0 deletions docs/sections/community/developer_documentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
---
description: This is a step-by-step guide to help you develop distilabel.
hide:
- footer
---

Thank you for investing your time in contributing to the project!

If you don't have the repository locally, and need any help, go to the [contributor guide](../community/contributor.md) and read the contributor workflow with Git and GitHub first.

## Set up the Python environment

To work on the `distilabel`, you must install the package on your system.

!!! Tip
This guide will use `uv`, but `pip` and `venv` can be used as well, this guide can work quite similar with both options.

From the root of the cloned Distilabel repository, you should move to the distilabel folder in your terminal.

```bash
cd distilabel
```

### Create a virtual environment

The first step will be creating a virtual environment to keep our dependencies isolated. Here we are choosing `python 3.11` ([uv venv](https://docs.astral.sh/uv/pip/environments/) documentation), and then activate it:

```bash
uv venv .venv --python 3.11
source .venv/bin/activate
```

### Install the project

Installing from local (we are using [`uv pip`](https://docs.astral.sh/uv/pip/packages/)):

```bash
uv pip install -e .
```

We have extra dependencies with their name, depending on the part you are working on, you may want to install some dependency (take a look at `pyproject.toml` in the repo to see all the extra dependencies):

```bash
uv pip install -e ".[vllm,outlines]"
```

### Linting and formatting

To maintain a consistent code format, install the pre-commit hooks to run before each commit automatically (we rely heavily on [`ruff`](https://docs.astral.sh/ruff/)):

```bash
uv pip install -e ".[dev]"
pre-commit install
```

### Running tests

All the changes you add to the codebase should come with tests, either `unit` or `integration` tests, depending on the type of change, which are placed under `tests/unit` and `tests/integration` respectively.

Start by installing the tests dependencies:

```bash
uv pip install ".[tests]"
```

Running the whole tests suite may take some time, and you will need all the dependencies installed, so just run your tests, and the whole tests suite will be run for you in the CI:

```bash
# Run specific tests
pytest tests/unit/steps/generators/test_data.py
```

## Set up the documentation

To contribute to the documentation and generate it locally, ensure you have installed the development dependencies:

```bash
uv pip install -e ".[docs]"
```

And run the following command to create the development server with `mkdocs`:

```bash
mkdocs serve
```

### Documentation guidelines

As mentioned, we use mkdocs to build the documentation. You can write the documentation in `markdown` format, and it will automatically be converted to HTML. In addition, you can include elements such as tables, tabs, images, and others, as shown in this guide. We recommend following these guidelines:

- Use clear and concise language: Ensure the documentation is easy to understand for all users by using straightforward language and including meaningful examples. Images are not easy to maintain, so use them only when necessary and place them in the appropriate folder within the docs/assets/images directory.

- Verify code snippets: Double-check that all code snippets are correct and runnable.

- Review spelling and grammar: Check the spelling and grammar of the documentation.

- Update the table of contents: If you add a new page, include it in the relevant index.md or the mkdocs.yml file.

### Components gallery

The components gallery section of the documentation is automatically generated thanks to a custom plugin, it will be run when `mkdocs serve` is called. This guide to the steps helps us visualize each step, as well as examples of use.

!!! Note
Changes done to the docstrings of `Steps/Tasks/LLMs` won't appear in the components gallery automatically, you will have to stop the `mkdocs` server and run it again to see the changes, everything else is reloaded automatically.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -252,4 +252,5 @@ nav:
- Community:
- sections/community/index.md
- How to contribute?: sections/community/contributor.md
- Developer Documentation: sections/community/developer_documentation.md
- Issue dashboard: sections/community/popular_issues.md

0 comments on commit a178109

Please sign in to comment.