From a178109b62db95c154b0dbae0a4b5efb7dd095f9 Mon Sep 17 00:00:00 2001 From: Agus Date: Wed, 25 Sep 2024 16:30:30 +0200 Subject: [PATCH] [DOCS] Add developer documentation section in the docs (#999) * Add new section with developer docs * Fix name of link * Add help for PR body --- docs/sections/community/contributor.md | 12 +- .../community/developer_documentation.md | 104 ++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 112 insertions(+), 5 deletions(-) create mode 100644 docs/sections/community/developer_documentation.md diff --git a/docs/sections/community/contributor.md b/docs/sections/community/contributor.md index bfc5f287c4..180c46929e 100644 --- a/docs/sections/community/contributor.md +++ b/docs/sections/community/contributor.md @@ -7,12 +7,12 @@ hide: Thank you for investing your time in contributing to the project! Any contribution you make will be reflected in the most recent version of distilabel 🤩. ??? Question "New to contributing in general?" - If you're a new contributor, read the [README](https://github.com/argilla-io/distilabel/blob/develop/README.md) to get an overview of the project. In addition, here are some resources to help you get started with open-source contributions: + If you're a new contributor, read the [README](https://github.com/argilla-io/distilabel/blob/develop/README.md) to get an overview of the project. In addition, here are some resources to help you get started with open-source contributions: - * **Discord**: You are welcome to join the [distilabel Discord community](http://hf.co/join/discord), where you can keep in touch with other users, contributors and the distilabel team. In the following [section](#first-contact-in-discord), you can find more information on how to get started in Discord. - * **Git**: This is a very useful tool to keep track of the changes in your files. Using the command-line interface (CLI), you can make your contributions easily. For that, you need to have it [installed and updated](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) on your computer. - * **GitHub**: It is a platform and cloud-based service that uses git and allows developers to collaborate on projects. To contribute to distilabel, you'll need to create an account. Check the [Contributor Workflow with Git and Github](#contributor-workflow-with-git-and-github) for more info. - * **Developer Documentation**: To collaborate, you'll need to set up an efficient environment. Check the [Installation](../getting_started/installation.md) guide to know how to do it. + * **Discord**: You are welcome to join the [distilabel Discord community](http://hf.co/join/discord), where you can keep in touch with other users, contributors and the distilabel team. In the following [section](#first-contact-in-discord), you can find more information on how to get started in Discord. + * **Git**: This is a very useful tool to keep track of the changes in your files. Using the command-line interface (CLI), you can make your contributions easily. For that, you need to have it [installed and updated](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) on your computer. + * **GitHub**: It is a platform and cloud-based service that uses git and allows developers to collaborate on projects. To contribute to distilabel, you'll need to create an account. Check the [Contributor Workflow with Git and Github](#contributor-workflow-with-git-and-github) for more info. + * **Developer Documentation**: To collaborate, you'll need to set up an efficient environment. Check the [Installation](../getting_started/installation.md) guide to know how to do it. ## First Contact in Discord @@ -129,6 +129,8 @@ In addition, on the right side, you can select a reviewer (for instance, if you Finally, fill in the template carefully and follow the guidelines. Remember to link the original issue and enable the checkbox to allow maintainer edits so the branch can be updated for a merge. Then, click on `Create pull request`. +For the PR body, ensure you give a description of what the PR contains, and add examples if possible (and if they apply to the contribution) to help with the review process. You can take a look at [#PR 974](https://github.com/argilla-io/distilabel/pull/974) or [#PR 983](https://github.com/argilla-io/distilabel/pull/983) for examples of typical PRs. + ### Review your pull request diff --git a/docs/sections/community/developer_documentation.md b/docs/sections/community/developer_documentation.md new file mode 100644 index 0000000000..ccc0795f87 --- /dev/null +++ b/docs/sections/community/developer_documentation.md @@ -0,0 +1,104 @@ +--- +description: This is a step-by-step guide to help you develop distilabel. +hide: + - footer +--- + +Thank you for investing your time in contributing to the project! + +If you don't have the repository locally, and need any help, go to the [contributor guide](../community/contributor.md) and read the contributor workflow with Git and GitHub first. + +## Set up the Python environment + +To work on the `distilabel`, you must install the package on your system. + +!!! Tip + This guide will use `uv`, but `pip` and `venv` can be used as well, this guide can work quite similar with both options. + +From the root of the cloned Distilabel repository, you should move to the distilabel folder in your terminal. + +```bash +cd distilabel +``` + +### Create a virtual environment + +The first step will be creating a virtual environment to keep our dependencies isolated. Here we are choosing `python 3.11` ([uv venv](https://docs.astral.sh/uv/pip/environments/) documentation), and then activate it: + +```bash +uv venv .venv --python 3.11 +source .venv/bin/activate +``` + +### Install the project + +Installing from local (we are using [`uv pip`](https://docs.astral.sh/uv/pip/packages/)): + +```bash +uv pip install -e . +``` + +We have extra dependencies with their name, depending on the part you are working on, you may want to install some dependency (take a look at `pyproject.toml` in the repo to see all the extra dependencies): + +```bash +uv pip install -e ".[vllm,outlines]" +``` + +### Linting and formatting + +To maintain a consistent code format, install the pre-commit hooks to run before each commit automatically (we rely heavily on [`ruff`](https://docs.astral.sh/ruff/)): + +```bash +uv pip install -e ".[dev]" +pre-commit install +``` + +### Running tests + +All the changes you add to the codebase should come with tests, either `unit` or `integration` tests, depending on the type of change, which are placed under `tests/unit` and `tests/integration` respectively. + +Start by installing the tests dependencies: + +```bash +uv pip install ".[tests]" +``` + +Running the whole tests suite may take some time, and you will need all the dependencies installed, so just run your tests, and the whole tests suite will be run for you in the CI: + +```bash +# Run specific tests +pytest tests/unit/steps/generators/test_data.py +``` + +## Set up the documentation + +To contribute to the documentation and generate it locally, ensure you have installed the development dependencies: + +```bash +uv pip install -e ".[docs]" +``` + +And run the following command to create the development server with `mkdocs`: + +```bash +mkdocs serve +``` + +### Documentation guidelines + +As mentioned, we use mkdocs to build the documentation. You can write the documentation in `markdown` format, and it will automatically be converted to HTML. In addition, you can include elements such as tables, tabs, images, and others, as shown in this guide. We recommend following these guidelines: + +- Use clear and concise language: Ensure the documentation is easy to understand for all users by using straightforward language and including meaningful examples. Images are not easy to maintain, so use them only when necessary and place them in the appropriate folder within the docs/assets/images directory. + +- Verify code snippets: Double-check that all code snippets are correct and runnable. + +- Review spelling and grammar: Check the spelling and grammar of the documentation. + +- Update the table of contents: If you add a new page, include it in the relevant index.md or the mkdocs.yml file. + +### Components gallery + +The components gallery section of the documentation is automatically generated thanks to a custom plugin, it will be run when `mkdocs serve` is called. This guide to the steps helps us visualize each step, as well as examples of use. + +!!! Note + Changes done to the docstrings of `Steps/Tasks/LLMs` won't appear in the components gallery automatically, you will have to stop the `mkdocs` server and run it again to see the changes, everything else is reloaded automatically. diff --git a/mkdocs.yml b/mkdocs.yml index b91cd814fa..2ef6234673 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -252,4 +252,5 @@ nav: - Community: - sections/community/index.md - How to contribute?: sections/community/contributor.md + - Developer Documentation: sections/community/developer_documentation.md - Issue dashboard: sections/community/popular_issues.md