Use pip-tools to build requirements files

[keimlink]

One of the main goals of using requirement files is to guarantee repeatable installations. Therefore pip should be used with the --no-deps option. This requires all dependencies to be listed in the requirements files. Maintaining these dependencies can be a huge burden.

A solution to make the maintenance of requirements files easier is pip-tools. It provides a pip-compile comamnd that compiles a requirements.in file to a requirements.txt file.

First example:

$ cat requirements/django.in
Django==1.7.4
-e [email protected]:bsvetchine/django-configurations.git@f514eac8afd971b9e94782c20693c59c3b46275a#egg=django-configurations
psycopg2
$ pip-compile --dry-run requirements/django.in
#
# This file is autogenerated by pip-compile
# Make changes in django.in, then run this to update:
#
#    pip-compile django.in
#
-e [email protected]:bsvetchine/django-configurations.git@f514eac8afd971b9e94782c20693c59c3b46275a#egg=django-configurations
Django==1.7.4
psycopg2==2.6.1
Dry-run, so nothing updated.

The example above shows:

• Already pinned requirements are not modified
• Unpinned requirements updated to the latest release
• Editable VCS URLs are supported which allows forks to be used
• All requirements are sorted

Second example:

$ cat requirements/test.in
pytest
freezegun
pytest-factoryboy
$ pip-compile --dry-run requirements/test.in
#
# This file is autogenerated by pip-compile
# Make changes in test.in, then run this to update:
#
#    pip-compile test.in
#
factory-boy==2.5.2        # via pytest-factoryboy
freezegun==0.3.4
inflection==0.3.1         # via pytest-factoryboy
py==1.4.30                # via pytest
pytest-factoryboy==1.1.2
pytest==2.7.2
python-dateutil==2.4.2    # via freezegun
six==1.9.0                # via freezegun, python-dateutil
Dry-run, so nothing updated.

The example above shows:

• Dependencies are added to the compiled requirements file
• Dependencies are annotated with the package they are required by

[rm--]

At what point do you want to include this? Another make target? Run manual or automatically?

[keimlink]

IMO there should be a new target compile-requirements. It should compile all requirements in a single task.

[rm--]

I created a issue dealing with our problem.