add requirements management policy #122
Conversation
dhellmann
force-pushed
the
requirements-management
branch
from
f3993dc to
3961e0d
Compare
|
Several months ago, I experiemented with https://pypi.org/project/pip-compile-multi/ to generate a set of constraint files for our entire dependency chain. Yesterday I revived my old PR and tried again. pip-compile-multi breaks because packages like flash-attn have broken packaging metadata. Doug filed a ticket a while ago, but upstream hasn't agreed to accept the fix. Related InstructLab ticket and PR to constraint versions: |
|
The topic of version bounds came up in internal channels. The blog post https://iscinumpy.dev/post/bound-version-constraints/ explains very well how to deal with version constraints in Python. It links to several other blog posts about semver and version handling in Python ecosystem, too. |
There was a problem hiding this comment.
Great document, thanks! Do you think it would be beneficial to include a practice of documenting any changes to the dependency requirements? This could include the rationale for setting or changing version ranges, providing context for future maintainers and downstream builds?
EDIT: "squash and merge" is not enabled in this repository so please do it locally and push again, thanks!
That sounds like a good follow-up for someone else to do, yes. |
dhellmann
force-pushed
the
requirements-management
branch
from
8f2dae9 to
cec6bd1
Compare
Done |
docs/dependency-management.md
Outdated
| That information is useful for users and re-packagers to understand which versions of dependencies are compatible with more specificity than the ranges provide. | ||
| Tools like Dependabot will submit PRs to automatically update those pins to help us keep up with new releases of all of our dependencies. | ||
|
|
||
| Pinning to specific versions in the package dependencies in `pyproject.toml` or `requirements.txt` is not a good practice. |
There was a problem hiding this comment.
in line 16, it suggests to use pinned versions in requirements.txt for CI, but here it suggests using pinned versions in requirement.txt is not a good practice. Just found this confusing.
There was a problem hiding this comment.
You don't want to pin your package dependencies. You do want to pin your CI packages. That's done in separate files. Both tend to be named requirements.txt. The earlier text had said requirements.in but that was changed to requirements.txt.
I don't know if you're using separate requirements/constraints lists for CI, yet.
There was a problem hiding this comment.
I made that suggestion assuming it was a typo - if not feel free to change it back
There was a problem hiding this comment.
The point of the suggestions is getting lost in the concern about specific filenames, so I've dropped the filenames and focused on how the requirements are used in different cases.
If this part is still confusing, I can just remove it all. The most important part for me right now is that the team understands their responsibility for keeping minimum versions up to date.
Signed-off-by: Doug Hellmann <[email protected]>
dhellmann
force-pushed
the
requirements-management
branch
from
cec6bd1 to
c4378ce
Compare
This document describes the policies for adding and updating build and runtime dependencies of all InstructLab components.