Skip to content

Commit

Permalink
update contributing
Browse files Browse the repository at this point in the history
  • Loading branch information
sh-rp committed Apr 3, 2024
1 parent 204bc80 commit 852987c
Showing 1 changed file with 14 additions and 37 deletions.
51 changes: 14 additions & 37 deletions docs/examples/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,50 +4,27 @@ Note: All paths in this guide are relative to the `dlt` repository directory.

## Add snippet

- Go to `docs/website/docs/examples/`.
- Copy one of the examples, rename scripts.
- Modify the script in `<example-name>/code/<snippet-name>-snippets.py`:
- The whole example code should be inside of `def <snippet-name>_snippet()` function.
- Use tags `# @@@DLT_SNIPPET_START example` and `# @@@DLT_SNIPPET_END example` to indicate which part of the code will be auto-generated in the final script `docs/examples/<examlple-name>/<snippet-name>.py`.
- Use additional tags as `# @@@DLT_SNIPPET_START smal_part_of_code` to indicate which part of the code will be auto-inserted into a text document `docs/website/docs/examples/<example-name>/index.md` in the form of a code snippet.
- Modify .`dlt/secrets.toml` and `configs.toml` if needed.
- Modify `<example-name>/index.md`:
- In the section `<Header info=` add the tl;dr for your example, it should be short but informative.
- Set `slug="<example-name>" run_file="<snippet-name>" />`.
- List what users will learn from this example. Use bullet points and link corresponding documentation pages.
- Use tags `<!--@@@DLT_SNIPPET ./code/<snippet-name>-snippets.py::smal_part_of_code-->` to insert example code snippets. Do not write them manually!

## Add tests

- Do not forget to add tests to `<example-name>/code/<snippet-name>-snippets.py`.
- They could be short asserts, code should work.
- Use `# @@@DLT_REMOVE` to remove test code from final code example.
- Test your snippets locally first with command:
- `cd docs/website/docs/examples/<example-name>/code && pytest --ignore=node_modules -s -v`.
- Add `@skipifgithubfork` decorator to your main snippet function, look [example](https://github.com/dlt-hub/dlt/blob/master/docs/website/docs/examples/chess_production/code/chess-snippets.py#L1-L4).

## Run npm start
- Go to `docs/examples/`.
- Copy the template in `./_template/..`.
- Make sure the folder and your examples script have the same name
- Update the doc string which will compromise the generated markdown file, check the other examples how it is done
- If your example requires any secrets, add the vars to the example.secrects.toml but do not enter the values.
- Add your example code, make sure you have a `if __name__ = "__main__"` clause in which you run the example script, this will be used for testing
- You should add one or two assertions after running your example and maybe also `load_info.raise_on_failed_jobs()`, this will help greatly with testing

## Testing
- You can test your example simply by running your example script from your example folder. On CI a test will be automatically generated.

## Checking your generated markdown

The command `npm start` starts a local development server and opens up a browser window.

- To install npm read [README](../website/README.md).
- This command will generate a clean example script in `docs/examples/<examlple-name>` folder based on `docs/website/docs/examples/<example-name>/code/<snippet-name>-snippets.py`.
- Also, this command automatically inserts code snippets to `docs/website/docs/examples/<example-name>/index.md`.
- You should your example be automatically added to the examples section in the local version of the docs. Check the rendered output and see wether it looks the way you intended.

## Add ENV variables

If you use any secrets for the code snippets, e.g. Zendesk requires credentials. You need to add them to GitHub Actions in ENV style:

- First, add the variables to `.github/workflows/test_doc_snippets.yml`:

Example:

```yaml
# zendesk vars for example
SOURCES__ZENDESK__CREDENTIALS: ${{ secrets.ZENDESK__CREDENTIALS }}
```
- Ask dlt team to add them to the GitHub Secrets.
If you use any secrets for the code snippets, e.g. Zendesk requires credentials. Please talk to us. We will add them to our google secrets vault.

## Add dependencies

Expand Down

0 comments on commit 852987c

Please sign in to comment.