Skip to content

Commit

Permalink
Fill in documentation (#12)
Browse files Browse the repository at this point in the history
Updating documentation by removing aieng_template text and adding text from the README and project proposal.
  • Loading branch information
lotif authored Mar 18, 2024
1 parent d8f7055 commit f93ee3e
Show file tree
Hide file tree
Showing 3 changed files with 98 additions and 92 deletions.
71 changes: 17 additions & 54 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,57 +12,20 @@ api
```

# TODO: modify this when working on updating documentation

This template repository can be used to bootstrap AI Engineering project repositories
on Github! The template is meant for python codebases since Python is the most commonly
used language by our team.

The template includes:

- [pyproject.toml](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/)
file to specify repository information and manage dependencies using
[Poetry](https://python-poetry.org/).

- [README.md](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-readmes) which should have basic information on why the project is
useful, installation instructions and other information on how users can get started.

- [.pre-commit-config.yaml](https://pre-commit.com/) for running pre-commit hooks that
check for code-style, apply formatting, check for type hints and run tests.

- [.github/pull_request_template.md](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/creating-a-pull-request-template-for-your-repository) for PRs.

- [.github/ISSUE_TEMPLATE](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository) for bug reports and issues that can be raised on the repository.

- [.github/workflows](https://docs.github.com/en/actions/using-workflows) for running CI
workflows using Github actions. The template includes CI workflows for code checks,
documentation building and releasing python packages to PyPI.

- [LICENSE.md](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository) for adding a license to the project repository.
By default, this is the [Apache-2.0 license](http://www.apache.org/licenses/). Please
change according to your project!

- [docs](https://pradyunsg.me/furo/) for adding project documentation. Typically
projects should have API reference documentation, user guides and tutorials.

- [CONTRIBUTING.md](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/setting-guidelines-for-repository-contributors) with basic guidelines on how others can
contribute to the repository.

- [CODE_OF_CONDUCT.md](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-code-of-conduct-to-your-project) with standards on how the community engages in
a healthy and constructive manner.

- [.gitignore](https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files)
with some standard file extensions to be ignored by git. Please add/modify as necessary.

- [codecov.yml](https://docs.codecov.com/docs/codecov-yaml) for using codecov.io to
generate code coverage information for your repository. You would need to add codecov.io
app as an [integration to your repository](https://docs.codecov.com/docs/how-to-create-a-github-app-for-codecov-enterprise).


If you are starting a new project, you can navigate to the [Use this template](https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template) button
on the top right corner of the [template repository home page](https://github.com/VectorInstitute/FLorist)
which will allow you to bootstrap your project repo using this template.

Please check out the user guide page for more detailed information on using the
template features. For exisiting projects, the [user guide](user_guide.md)
can be followed to migrate to following the template more closely.
FLorist is a platform to launch and monitor Federated Learning (FL) training jobs. Its goal is
to bridge the gap between state-of-the-art FL algorithm implementations and their applications
by providing a system to easily kick off, orchestrate, manage, collect, and summarize the results
of [FL4Health](https://github.com/VectorInstitute/FL4Health) training jobs.

As Federated Learning has a client and a server side, FLorist also has client and server-side
“modes” to orchestrate the training process on both sides. FLorist will have long-running
processes on the clients, which will be waiting for instructions from the FLorist server to
start training. Once the FL server is started, FLorist will monitor both server and client
processes as well as its own FL server process while collecting their progress to be displayed
in a web UI for monitoring.

At the end of training, it will save the results in a database and also provide access to the
training artifacts (e.g. model files). For a visual representation of the system, please check
the diagram below.

![system_diagram.png](system_diagram.png)
Binary file added docs/source/system_diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
119 changes: 81 additions & 38 deletions docs/source/user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,61 +2,104 @@

# User Guide

# TODO: modify this when working on updating documentation
## Setting up

## pyproject.toml file and dependency management
### Install dependencies

If your project doesn't have a pyproject.toml file, simply copy the one from the
template and update file according to your project.
To install the project dependencies, first you need to create a virtual environment.
The easiest way is by using the [virtualenv](https://pypi.org/project/virtualenv/) package:

For managing dependencies, [Poetry](https://python-poetry.org/) is the recommended tool
for our team. Hence, install Poetry to setup the development virtual environment. Poetry
supports [optional dependency groups](https://python-poetry.org/docs/managing-dependencies/#optional-groups)
which help manage dependencies for different parts of development such as `documentation`,
`testing`, etc. The core dependencies are installed using the command:
```shell
virtualenv venv
source venv/bin/activate
```

We use [Poetry](https://python-poetry.org/) to manage back-end dependencies:

```shell
pip install --upgrade pip poetry
poetry install
```

### Install Yarn

We use [Yarn](https://yarnpkg.com/) to manage front-end dependencies. Install it on MacOS
using [Homebrew](https://brew.sh/):

```bash
python3 -m poetry install
```shell
brew install yarn
```

Additional dependency groups can be installed using the `--with` flag. For example:
Then install the project dependencies:
```shell
yarn
```

### Pulling Redis' Docker

Redis is used to fetch the metrics reported by servers and clients during their runs.

```bash
python3 -m poetry install --with docs,test

If you don't have Docker installed, follow [these instructions](https://docs.docker.com/desktop/)
to install it. Then, pull [Redis' official docker image](https://hub.docker.com/_/redis)
(we currently use version 7.2.4):
```shell
docker pull redis:7.2.4
```

## documentation
## Running the server

If your project doesn't have documentation, copy the directory named `docs` to the root
directory of your repository. The provided source files use [Furo](https://pradyunsg.me/furo/),
a clean and customisable Sphinx documentation theme.
### Start server's Redis instance

In order to build the documentation, install the documentation dependencies as mentioned
in the previous section, navigate to the `docs` folder and run the command:
If it's your first time running it, create a container and run it with the command below:
```shell
docker run --name redis-florist-server -d -p 6379:6379 redis:7.2.4 redis-server --save 60 1 --loglevel warning
```

```bash
make html
From the second time on, you can just start it:
```shell
docker start redis-florist-server
```

You can configure the documentation by updating the `docs/source/conf.py`. The markdown
files in `docs/source` can be updated to reflect the project's documentation.
### Start back-end and front-end servers

Use Yarn to run both the back-end and front-end on server mode:

## github actions
```shell
yarn dev
```

The front-end will be available at `http://localhost:3000`. If you want to access
back-end APIs individually, they will be available at `https://localhost:8000`.

## Running the client

### Start client's Redis instance

If it's your first time running it, create a container and run it with the command below:
```shell
docker run --name redis-florist-client -d -p 6380:6379 redis:7.2.4 redis-server --save 60 1 --loglevel warning
```

From the second time on, you can just start it:
```shell
docker start redis-florist-client
```

### Start back-end and front-end servers

To start the client back-end service:

```shell
uvicorn florist.api.client:app --reload --port 8001
```

The template consists of some github action continuous integration workflows that you
can add to your repository.
The service will be available at `http://localhost:8001`.

The available workflows are:
# Contributing to the project

- [code checks](https://github.com/VectorInstitute/FLorist/blob/main/.github/workflows/code_checks.yml): Static code analysis, code formatting and unit tests
- [documentation](https://github.com/VectorInstitute/FLorist/blob/main/.github/workflows/docs_deploy.yml): Project documentation including example API reference
- [integration tests](https://github.com/VectorInstitute/FLorist/blob/main/.github/workflows/integration_tests.yml): Integration tests
- [publish](https://github.com/VectorInstitute/FLorist/blob/main/.github/workflows/publish.yml):
Publishing python package to PyPI. Create a `PYPI_API_TOKEN` and add it to the
repository's actions [secret variables](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions)
in order to publish PyPI packages when new software releases are created on Github.
If you are a developer of the team or someone who wants to contribute to the project,
please refer to the [CONTRIBUTING.md](https://github.com/VectorInstitute/FLorist/blob/main/CONTRIBUTING.md) file.

The test workflows also compute coverage and upload code coverage metrics to
[codecov.io](https://app.codecov.io/gh/VectorInstitute/FLorist). Create a
`CODECOV_TOKEN` and add it to the repository's actions [secret variables](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions).
Please read and adhere to our [Code of Conduct](https://github.com/VectorInstitute/FLorist/blob/main/CODE_OF_CONDUCT.md)
when making contributions to the project.

0 comments on commit f93ee3e

Please sign in to comment.