tracker: forcing rebuild of wheels

[shubhbapna]

When running build-sequence with the -–skip-existing flag we need a way to force builds of wheels whose version numbers haven't changed. We might need to do this for various reasons: changes in sdist, env variables, plugins, container image changes etc. The idea is to introduce a changelog per package per version that will bump the build tag when publishing to a pypi index.

TODO:

• Add package-version-specific changelog to settings #317
• Change filename of wheel with build number based on length of changelog #318
• Add extra build-log metadata to wheel content #319
• Add global changelog to settings to trigger a rebuild of every package with a new build tag #347
• Update docs to explain the use of changelogs #321

Good to haves:

• A linter that scans plugin and env file to notify the user of any changes so that they can update the changelog

[shubhbapna]

@dhellmann there might also be another potential issue:

Say bootstrap has resolved A, B and C and say there was only a version bump for B. Also B is a direct dependency of C. If we run build-sequence with --skip-existing then only B would be built. Shouldn't in this case C also be built? We aren't currently doing this right? Do we have to worry about this?

[dhellmann]

@dhellmann there might also be another potential issue:

Say bootstrap has resolved A, B and C and say there was only a version bump for B. Also B is a direct dependency of C. If we run build-sequence with --skip-existing then only B would be built. Shouldn't in this case C also be built? We aren't currently doing this right? Do we have to worry about this?

Yes, that's a good point. We don't want an update to some low-level package to trigger a complete rebuild, but we also want to ensure we have a way to force that rebuild when it's needed.

If we know that changing B means we have to build C, we could add a changelog entry to C with something like "updated B to version X" to cause that, right?

[shubhbapna]

Yes, that's a good point. We don't want an update to some low-level package to trigger a complete rebuild, but we also want to ensure we have a way to force that rebuild when it's needed.
If we know that changing B means we have to build C, we could add a changelog entry to C with something like "updated B to version X" to cause that, right?

Summarizing our discussion:

It is difficult to identify whether a version bump is significant enough to cause a rebuild of any package that depends on it. So we will leave it to the user to update the changelog. There might be times where we need a to trigger a build of every package (eg: maybe when setuptools get upgraded to some version with a lot of changes). In this case we need a way for the user to tell fromager to rebuild everything with a new build tag. Adding a global changelog will be helpful for this

[shubhbapna]

I think we got everything that we needed, marking it closed