Skip to content

UofT-EcoSystem/algorithmic-efficiency

 
 

Repository files navigation

MLCommons™ Algorithmic Efficiency


MLCommons Logo

InstallationRulesContributingLicense

CI Lint License: Apache 2.0 Code style: yapf


MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models. This repository holds the competition rules and the benchmark code to run it.

Installation

  1. Create new environment, e.g. via conda or virtualenv:

    Python minimum requirement >= 3.7

     sudo apt-get install python3-venv
     python3 -m venv env
     source env/bin/activate
  2. Clone this repository:

    git clone https://github.com/mlcommons/algorithmic-efficiency.git
    cd algorithmic-efficiency
  3. Install the algorithmic_efficiency package:

    pip3 install -e .

    Depending on the framework you want to use (e.g. JAX or PyTorch) you need to install them as well. You could either do this manually or by adding the corresponding options:

    JAX (GPU)

    pip3 install -e .[jax-gpu] -f 'https://storage.googleapis.com/jax-releases/jax_releases.html'

    JAX (CPU)

    pip3 install -e .[jax-cpu]

    PyTorch

    pip3 install -e .[pytorch] -f 'https://download.pytorch.org/whl/torch_stable.html'

    Development

    To use the development tools such as pytest or pylint use the dev option:

    pip3 install -e .[dev]

Docker

Docker is the easiest way to enable PyTorch/JAX GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).

Docker requirements

  • Install Docker on your local host machine.

  • For GPU support on Linux, install NVIDIA Docker support.

    • Take note of your Docker version with docker -v. Versions earlier than 19.03 require nvidia-docker2 and the --runtime=nvidia flag. On versions including and after 19.03, you will use the nvidia-container-toolkit package and the --gpus all flag. Both options are documented on the page linked above.

Setup

  1. Clone this repository:

    git clone https://github.com/mlcommons/algorithmic-efficiency.git
  2. Build Docker

    cd algorithmic-efficiency/ && sudo docker build -t algorithmic-efficiency .
  3. Run Docker

    sudo docker run --gpus all -it --rm -v $PWD:/home/ubuntu/algorithmic-efficiency --ipc=host algorithmic-efficiency

    Currently docker method installs both PyTorch and JAX

Running a workload

JAX

python3 algorithmic_efficiency/submission_runner.py --framework=jax --workload=mnist_jax --submission_path=baselines/mnist/mnist_jax/submission.py --tuning_search_space=baselines/mnist/tuning_search_space.json

PyTorch

python3 algorithmic_efficiency/submission_runner.py --framework=pytorch --workload=mnist_pytorch --submission_path=baselines/mnist/mnist_pytorch/submission.py --tuning_search_space=baselines/mnist/tuning_search_space.json

Rules

The rules for the MLCommons Algorithmic Efficency benchmark can be found in the seperate rules document. Suggestions, clarifications and questions can be raised via pull requests.

Contributing

If you are interested in contributing to the work of the working group, feel free to join the weekly meetings, open issues, and see the MLCommons contributing guidelines. TEST

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 99.3%
  • Other 0.7%