Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retiree-Packages #325

Open
stschiff opened this issue Dec 6, 2024 · 1 comment
Open

Retiree-Packages #325

stschiff opened this issue Dec 6, 2024 · 1 comment

Comments

@stschiff
Copy link
Member

stschiff commented Dec 6, 2024

As discussed on December 5th, we may need an option to "retire" packages on the server. This is less trivial than it sounds, since our trident timetravel mechanism will automatically (and nightly) fetch all packages, even old ones from our git repository. Even if we remove a package from the repository's master branch, it will remain in the record and therefore it will be fetched by timetravel. This is intentional: For backwards-compatibility, we want to continue serving old packages, even if they have been superseded by other packages or terminated. An example is package 2019_BraceNatureEcologyEvolution, which we removed as it turned out to duplicate another package named 2019_Brace_Britain. It will however forever live in our record and hence it will be served by our server APIs. It will even get marked as latest by clients, because some version of it will be the latest, by definition.

There are a number of ideas that we discussed:

1.) We could simply remove that package from the community-archive chronicle file. This will practically erase all traces of this package, and our nightly trident timetravel on the server will stop serving this package at all. This option is easy to implement, but it makes a somewhat opaque manual change that breaks our principle of chronicling all old packages. There is no obvious way to document that change so that we can later understand that one entry is missing. Also: It will break any previous workflows from users who have used that package and will be surprised to not be able to fetch it anymore. I think this option is therefore out! Breaking changes are not acceptable, and for whatever reason a user might have to still use a retired package, we shouldn't prevent them from it.

2.) (this was the option we discussed as final solution) We could define a new "retired_packages" file, which we track on the GitHub-repository of the community-archive, which lists packages that we want to retire. We defined in our discussion what "retired" really means: A retired package is a package that will never be marked as latest, not even its formally latest version. One thing to think through here is the fact that currently none of our APIs supports requesting only "latest" packages. Rather: the /packages API simply lists all packages, in all versions, and the /zip_file API serves any single package, either specified with version, or without, in which case it serves the latest. The whole "latest" classification happens entirely inside of trident list and related commands on the client side. So in order to get this option implemented, we need to implement a new API option onlyLatest for /packages, which would then skip a package whose package name is in the "retired_packages" file. This would then also require changing trident list --remote --packages --onlyLatest internally, to use this API option instead of determining "latest" packages on the client side. Similar changes might be necessary in trident fetch --downloadAll. We might also want to change the behavior of the /zip_file API in case a package name is requested without version. In that case we could return an error if a retired package is requested, and only serve it when requested with a specific version.

So this option is doable, but I'm slightly worried about the changes and any possible unforeseen consequences.

3.) We could augment a package's POSEIDON.yml file of a retired package, with an undocumented retired: true field, only on the Community Archive. This would be ignored in all CLI subcommands except for trident serve, which would skip such a package in /packages&onlyLatest and /zip_file for a package name without version, similarly to Option 2. This would be very similar to Option 2 altogether, and require the same changes in trident serve, trident list and trident fetch. We did not prefer this option over Option 2 in the end, because the documentation would be more hidden and less obvious than a "retired_packages" file.

@stschiff
Copy link
Member Author

stschiff commented Dec 6, 2024

As a slight edit of Option 2, let's call it Option 2a, I propose:

  • We do not tinker with the "latest" definition, and leave that entirely on the client side. Instead, we really just add an option to the /packages, /groups and /individuals APIs which we may call &includeRetired=True. If requested like this, it serves entities from all packages, including retired ones, but by default these APIs would not serve retired ones, not even in their latest version. The behavior of /zip_file would be changed as suggested above in Option 2.

This would - in my view - keep the server APIs cleaner, and prevent us from having to move the entire latest computation onto the server but keep that on the client-side.

We could then consider adding an option to trident list --remote --includeRetired, which would then use the new API flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant