This document contains instructions for collaborating on the different libraries of Nixtla.
Sometimes, diving into a new technology can be challenging and overwhelming. We've been there too, and we're more than ready to assist you with any issues you may encounter while following these steps. Don't hesitate to reach out to us on Slack. Just give Azul a ping, and she'll be glad to assist you.
- Prerequisites
- Git
fork-and-pull
worklow - Set Up a Virtual Environment
- Install required libraries for development
- Start editable mode
- Set Up your Notebook based development environment
- Start Coding
- Example with Screen-shots
- GitHub: You should already have a GitHub account and a basic understanding of its functionalities. Alternatively check this guide.
- uv: You need to have
uv
installed. You can refer to the docs in order to install it.
1. Fork the Project: Start by forking the Nixtla repository to your own GitHub account. This creates a personal copy of the project where you can make changes without affecting the main repository.
2. Clone the Forked Repository
Clone the forked repository to your local machine using git clone --recursive https://github.com/<your-username>/statsforecast.git
. This allows you to work with the code directly on your system.
3. Create a Branch:
Branching in GitHub is a key strategy for effectively managing and isolating changes to your project. It allows you to segregate work on different features, fixes, and issues without interfering with the main, production-ready codebase.
-
Main Branch: The default branch with production-ready code.
-
Feature Branches: For new features, create branches prefixed with 'feature/', like
git checkout -b feature/new-model
. -
Fix Branches: For bug fixes, use 'fix/' prefix, like
git checkout -b fix/forecasting-bug
. -
Issue Branches: For specific issues, use
git checkout -b issue/issue-number
orgit checkout -b issue/issue-description
.
After testing, branches are merged back into the main branch via a pull request, and then typically deleted to maintain a clean repository. You can read more about github and branching here.
If you want to use Docker or Codespaces, let us know opening an issue and we will set you up.
Next, you'll need to create a virtual environment. uv is an open-source package management and environment management system that runs on Windows, macOS, and Linux. It allows you to create isolated environments with the dependencies required for a specific project.
First, ensure you have uv installed on your system. Alternatively, refer to the installation docs.
Then, you can create a new environment using uv venv
. This will use your default python interpreter, you can also define a specific python version (which will be downloaded if you don't have it already) by providing the --python
flag. For example, uv venv --python 3.12
.
Activate your new environment with source .venv/bin/activate
for MacOS and Linux or .\.venv\Scripts\activate
for Windows.
The setup.py
file contains all the dependencies required for the project. To install these dependencies you can use uv pip install -r setup.py --extra dev
We use pre-commit to ease the development process, which run some checks before you make a commit to have a faster feedback loop.
To setup the pre-commit hooks run: pre-commit install
Install the library in editable mode using uv pip install --no-build-isolation -e .
(this requires a C++ compiler).
By using the -e
flag the package is linked directly to the source code, allowing any changes made to the source code to be immediately reflected in your Python environment without the need to reinstall the package. This is useful for testing changes during package development.
If you're working on the C++ code, you'll need to re-compile the shared library, which can be done with: python setup.py build_ext --inplace
(this will compile it into the build
directory and copy it to the python package location).
Notebook-based development refers to using interactive notebooks, such as Jupyter Notebooks, for coding, data analysis, and visualization. Here's a brief description of its characteristics:
-
Interactivity: Code in notebooks is written in cells which can be run independently. This allows for iterative development and testing of small code snippets.
-
Visualization: Notebooks can render charts, tables, images, and other graphical outputs within the same interface, making it great for data exploration and analysis.
-
Documentation: Notebooks support Markdown and HTML, allowing for detailed inline documentation. Code, outputs, and documentation are in one place, which is ideal for tutorials, reports, or sharing work.
For notebook based development you'll need nbdev
and a notebook editor (such as VS Code, Jupyter Notebook or Jupyter Lab). nbdev
and jupyter have been installed in the previous step. If you use VS Code follow this tutorial.
nbdev makes debugging and refactoring your code much easier than in traditional programming environments since you always have live objects at your fingertips. nbdev
also promotes software engineering best practices because tests and documentation are first class.
All your changes must be written in the notebooks contained in the library (under the nbs
directory). Once a specific notebook is open (more details to come), you can write your Python code in cells within the notebook, as you would do in a traditional Python development workflow. You can break down complex problems into smaller parts, visualizing data, and documenting your thought process. Along with your code, you can include markdown cells to add documentation directly in the notebook. This includes explanations of your logic, usage examples, and more. Also, nbdev
allows you to write tests inline with your code in your notebook. After writing a function, you can immediately write tests for it in the following cells.
Once your code is ready, nbdev
can automatically convert your notebook into Python scripts. Code cells are converted into Python code, and markdown cells into comments and docstrings.
Open a jupyter notebook using jupyter lab
(or VS Code).
-
Make Your Changes: Make changes to the codebase, ensuring your changes are self-contained and cohesive.
-
Commit Your Changes: Add the changed files using
git add [your_modified_file_0.ipynb] [your_modified_file_1.ipynb]
, then commit these changes usinggit commit -m "<type>: <Your descriptive commit message>"
. Please use Conventional Commits -
Push Your Changes: Push your changes to the remote repository on GitHub with
git push origin feature/your-feature-name
. -
Open a Pull Request: Open a pull request from your new branch on the Nixtla repository on GitHub. Provide a thorough description of your changes when creating the pull request.
-
Wait for Review: The maintainers of the Nixtla project will review your changes. Be ready to iterate on your contributions based on their feedback.
Remember, contributing to open-source projects is a collaborative effort. Respect the work of others, welcome feedback, and always strive to improve. Happy coding!
Nixtla offers the possibility of assisting with stipends for computing infrastructure for our contributors. If you are interested, please join our slack and write to Azul or Max.
You can find a detailed step by step buide with screen-shots below.
The first thing you need to do is create a fork of the GitHub repository to your own account:
Your fork on your account will look like this:
In that repository, you can make your changes and then request to have them added to the main repo.
In this tutorial, we are using Mac (also compatible with other Linux distributions). If you are a collaborator of Nixtla, you can request an AWS instance to collaborate from there. If this is the case, please reach out to Max or Azul on Slack to receive the appropriate access. We also use Visual Studio Code, which you can download from here.
Once the repository is created, you need to clone it to your own computer. Simply copy the repository URL from GitHub as shown below:
Then open Visual Studio Code, click on "Clone Git Repository," and paste the line you just copied into the top part of the window, as shown below:
Select the folder where you want to copy the repository:
And choose to open the cloned repository:
You will end up with something like this:
In this section, we assume that we want to increase the default number of windows used to create prediction intervals from 2 to 3. The first thing we need to do is create a specific branch for that change using git checkout -b [new_branch]
like this:
Once created, open the notebook you want to modify. In this case, it's nbs/utils.ipynb
, which contains the metadata for the prediction intervals. After opening it, click on the environment you want to use (top right) and select the .venv
environment.
Next, execute the notebook and make the necessary changes. In this case, we want to modify the PredictionIntervals
class:
We will change the default value of n_window
from 2 to 3:
Once you have made the change and performed any necessary validations, it's time to convert the notebook to Python modules. To do this, simply use nbdev_export
in the terminal.
You will see that the mlforecast/utils.py
file has been modified (the changes from nbs/utils.ipynb
are reflected in that module). Before committing the changes, we need to clean the notebooks using the command ./action_files/clean_nbs
.
Once you have done the above, simply add the changes using git add nbs/utils.ipynb mlforecast/utils.py
:
Create a descriptive commit message for the changes using git commit -m "[description of changes]"
:
Finally, push your changes using git push
:
In GitHub, open your repository that contains your fork of the original repo. Once inside, you will see the changes you just pushed. Click on "Compare and pull request":
Include an appropriate title for your pull request and fill in the necessary information. Once you're done, click on "Create pull request".
Finally, you will see something like this:
- This file was generated using this file. Please change that file if you want to enhance the document.