Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved versioning #738

Merged
merged 10 commits into from
Nov 19, 2023
Merged

Improved versioning #738

merged 10 commits into from
Nov 19, 2023

Conversation

matthijskooijman
Copy link
Member

This implements the improvements suggested at #608 (comment). Practically, this PR makes the following improvements:

  • The version number for releases is now in a single file called src/hamster/VERSION instead of being hardcoded in multiple places.
  • The git version determination is also no longer duplicated in two places.
  • The fallback version for unreleased versions is now unreleased instead of 3.0.2 (i.e. when running from a source checkout or tarball that is not from a released version, and git cannot be used to determine the version, this no longer incorrectly shows 3.0.2).
  • The version number detected from git now has the v prefix stripped, making version numbers consistent between git checkouts and release tarballs.
  • The flatpak build now knows about the version number it contains.
  • The flatpak build now can actually use the metainfo.xml file.

I'm marking this as a draft since I still need to do a bit more testing, and because I found out just before I ran out of time for today, that flatpak seems to insist on having a release date with the version number in metainfo.xml and otherwise just ignores the version number entirely (F: Ignoring release element without timestamp or date). To solve this, we should either:

  1. Autodetect the release date (meaning take it from the git tag or commit date if available, and, I guess, add a RELEASE_DATE next to VERSION for release tarballs).
  2. Just add a fake date (something like a zero timestamp or date far into the past that is clearly not meaningful)

I've implemented 2. for now, but comments are welcome.

Copy link

Automatically generated build artifacts for commit 3d161a8 (note: these links will expire after some time):


To test this PR, download and unzip the flatpak application above and then install with:

flatpak install Hamster.flatpak

@GeraldJansen
Copy link
Contributor

Looking good. My only suggestion for a fallback version is that "3.0.2-derived" or some such notation would be more informative than "unreleased". After the 3.0.3 release the VERSION file would be updated to ""3.0.3-derived". However, this is of marginal importance so feel free to drive on!

@matthijskooijman
Copy link
Member Author

Looking good. My only suggestion for a fallback version is that "3.0.2-derived" or some such notation would be more informative than "unreleased". After the 3.0.3 release the VERSION file would be updated to ""3.0.3-derived". However, this is of marginal importance so feel free to drive on!

Right, that would make some sense, but be slightly harder to maintain (but might be good to do if we automate that later). In practice, it should not matter much, since the unreleased should hardly ever be used, since during normal development git version will be used anyway.

Here's a bit of documentation (that I wanted to write anyway) about how the versioning now works (to be put somewhere):

For determining the hamster version, there are currently two interfaces:

  • The hamster.__version__ variable is to be used from within hamster itself, to figure out the version of the code that is currently running. This works both when installed or running out of a source tree (git checkout or unpacked tarball). When not installed, this version includes an "(uninstalled)" suffix.
  • The src/hamster/version.py script can be run by itself to get the version of the source tree, to be used by e.g. waf or later github actions to get the source tree version (does not work when installed). This does not return the "(uninstalled)" suffix.

To find the actual version number, three options are tried in order:

  1. The hamster/defs.py file. This file is not present in the source tree, but generated by waf during install. This is only considered for hamster.__version__, not for the src/hamster/version.py script.
  2. The git version. This runs git describe with some options to figure out the most recent tag, any commits made on top of this, and if the source tree is modified. For example, 3.0.2-75-ga0fe41d0+ indicates that the most recent tag is 3.0.2, 75 commits have been made since then, the most recent commit is a0fe41d and the source tree was modified. When the current version is an unmodified tag, this returns just the version number.
  3. The contents of src/hamster/VERSION. This is used for source trees without git information (typically when installing from a tarball git export for releases). This file only has meaningful content in git for release commits, and is reverted to a plain unreleased value directly after the release (but during development the git version should usually be used).

@GeraldJansen
Copy link
Contributor

documentation ... (to be put somewhere)

Created wiki page Versioning, for now.

@aquaherd
Copy link
Member

There is a .bumpversion and setup.py in the root. You may want to review or drop them in this endeavour.

Copy link

Automatically generated build artifacts for commit 7f9bdb3 (note: these links will expire after some time):


To test this PR, download and unzip the flatpak application above and then install with:

flatpak install Hamster.flatpak

@matthijskooijman matthijskooijman force-pushed the improved-versioning branch 3 times, most recently from 13d177e to c01b74c Compare November 19, 2023 11:52
Copy link

Automatically generated build artifacts for commit c01b74c (note: these links will expire after some time):


To test this PR, download and unzip the flatpak application above and then install with:

flatpak install Hamster.flatpak

Copy link

Automatically generated build artifacts for commit 4bf6e78 (note: these links will expire after some time):


To test this PR, download and unzip the flatpak application above and then install with:

flatpak install Hamster.flatpak

@matthijskooijman
Copy link
Member Author

There is a .bumpversion and setup.py in the root. You may want to review or drop them in this endeavour.

Good point, I've removed them. I've also pushed a few other commits and did some more testing, I think this is now ready for final review. @aquaherd, @DirkHoffmann, maybe you could have another review pass over the new commits?

If ok, let me handle the merge, there is still a fixup commit to be squashed before merging.

@matthijskooijman matthijskooijman marked this pull request as ready for review November 19, 2023 12:13
@@ -44,6 +44,10 @@
</screenshots>
​<translation type="gettext">hamster</translation>
<url type="homepage">https://github.com/projecthamster/hamster/wiki</url>
<releases>
<!-- Provide a (clearly) fake date, otherwise flatpak will ignore the entry -->
<release version="@VERSION@" date="1980-01-01" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

date should be an easter egg: "2007-08-01" Toms first commit to the repo

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid that such a date might actually look like its meaningful, 1980-01-01 is even more clearly invalid/fake. I'm not entirely sure where this date will actually show up, though, but I'm inclined to leave it as is?

@aquaherd
Copy link
Member

aquaherd commented Nov 19, 2023

I have built a tar release with ./waf dist but the magical substitution from defs.py.in to defs.py did not happen. Does the release pipeline handle this?

AFAIK, downstream packagers prefer to download the tar.gz file and do not do a git checkout from github just to get a version string.

@matthijskooijman
Copy link
Member Author

I have built a tar release with ./waf dist but the magical substitution from defs.py.in to defs.py did not happen. Does the release pipeline handle this?

Right, I did not look at that case yet. I think it should (be made to) include the src/hamster/VERSION file, and then everything would be ok. Let me check that.

AFAIK, downstream packagers prefer to download the tar.gz file and do not do a git checkout from github just to get a version string.

For at least Debian, I think that is the case, though the previous release did not have a tarball from us, so I'm assuming @rhertzog has used a git export or something like that.

Also, the approach with adding a VERSION file manually in git (as opposed to auto-generating a tarball with a github workflow or so), has the advantage that you can just export a tarball from git and have that work (and is, at least for now, simpler so a better way to move forward).

The question is, though, should we (as part of the release process) generate such a tarball (either directly from git, or using ./waf dist) and publish it as part of the github release maybe?

@matthijskooijman
Copy link
Member Author

I just tried ./waf dist, which seems to basically just tar up the current directory (including any temporary and .gitignored files lying around), so it seems to be easy to create bundles with too much files in them. But src/hamster/VERSION is also included, so it seems things should work fine for release builds (i.e. when src/hamster/VERSION contains a meaningful version, for a dist created from a random git commit, you get "unreleased" as the version number, but I'm not sure if that's a case that we need to better support, probably not?).

@matthijskooijman matthijskooijman mentioned this pull request Nov 19, 2023
@aquaherd
Copy link
Member

The problem is the packagers: They only monitor the most recent tag, add that to a github URL and then unpack the github generated tar file, dropping the first folder level that is the only thing that contains a string like v3.0.3.

Previously, the 3.0.2 was inside the sources. I have not yet seen a release build so I ight be wrong. I don't see any modification on how the release tar is built here, so I assume it is just a tar from a git tag, that does not include the tag name in the tar file.

@matthijskooijman
Copy link
Member Author

Previously, the 3.0.2 was inside the sources. I have not yet seen a release build so I ight be wrong. I don't see any modification on how the release tar is built here, so I assume it is just a tar from a git tag, that does not include the tag name in the tar file.

The plan is to write 3.0.3 to src/hamster/VERSION in a commit, tag that, and then revert to unreleased in a subsequent commit, exactly so the workflow you describe should work. See also #686 (comment)

@DirkHoffmann
Copy link
Member

Not sure, if it is relevant for this issue, and if it is to be fixed on your side or on my test system, but I get these two messages from flatpak when installing Hamster-3.0.2-81-g14262.flatpak:

Info: runtime org.gnome.Platform branch 41 is end-of-life, with reason:
   The GNOME 41 runtime is no longer supported as of September 17, 2022. Please ask your application developer to migrate to a supported platform.
Info: applications using this runtime:
   org.gnome.Hamster

Info: runtime org.freedesktop.Platform.GL.default branch 21.08 is end-of-life, with reason:
   org.freedesktop.Platform 21.08 is no longer receiving fixes and security updates. Please update to a supported runtime version.
Info: applications using this runtime:
   org.gnome.Hamster

@DirkHoffmann
Copy link
Member

Looking good. My only suggestion for a fallback version is that "3.0.2-derived" or some such notation would be more informative than "unreleased".

Shouldn't it be "rcN" (with N=1, 2, …)?

@matthijskooijman
Copy link
Member Author

The GNOME 41 runtime is no longer supported as of September 17, 2022. Please ask your application developer to migrate to a supported platform.

This I fixed today in #739, but this PR is not rebased on top of that yet.

Shouldn't it be "rcN" (with N=1, 2, …)?

That would make sense for explicit release candidate images, but @GeraldJansen is talking about the fallback VERSION file contents, that gets used when an unreleased source tree is used in a context where no git information is available.

@matthijskooijman
Copy link
Member Author

I think all concerns have been answered. In the interest of progress, I'm going to go ahead and merge this already, so we can prepare for the last steps of the release. If there's anything that still needs to be changed or discussed, before or after the release, feel free to leave more comments.

Previously, if not installed and no git version could be deduced,
a hardcoded 3.0.2 version was used. Now, this fallback version is moved
into an external VERSION file and is changed to "unreleased" instead of
"3.0.2", to ensure that a proper version is only returned when it is
really correct.

This also makes it easier to update the version number for releases, and
prepares for having a single place in the code to store the release version.
This prepares for using this code from waf, since it can now be imported
without additional (gtk) dependencies.
This allows running the src/hamster/version.py file as a standalone
script that just outputs the version number and lets wscript call that.

This ensures that the exact same version number is used between
installed and non-installed versions, and removes the second place where
the release version was previously hardcoded.

To do this, the version.py code is split into a part that decides the
version when installed, and a bit that decides the version when running
from the source tree. This is needed since when running the script from
waf, sys.path is not set up to allow importing hamster.defs, and trying
this might result in importing hamster.defs from a system-wide installed
hamster instead of the current source dir. So just skip the
installed-version detection entirely when running from waf, it is not
relevant anyway.
Released versions omitted the v prefix (to make proper semantic
versions), but version numbers autodetected from git would not, leading
to discrepancies. This fixes that.
This file is mostly used by packaging tools, such as flatpak, to get
info about the application. Since flatpak uses org.gnome.Hamster as the
main package id, the metainfo.xml must be named the same for it to be
used.

Possibly more things need to be renamed for all application ids to be
consistent, but it is non-obvious what the best approach is there (see
also #725 for details). For now, this pragmatically renames metainfo.xml
to ensure it is found by flatpak.
This is now done by waf at install time and ensures that flatpak knows
the hamster version of the build.
Without a release date, flatpak ignores the version number, but
autodetecting the release date (and carrying it into tarballs) might be
too much work, so just hardcode an obviously fake date for now.
This was not actually used in practice and no longer matches the recent
changes to the version handling, so remove it.
This was added a long time ago only for the database backend, but seems
to be outdated. In general, hamster cannot be installed using setup.py
anyway, since it requires dbus service files, gsettings schemas, etc. so
waf is used for all that. Remove setup.py which is no longer used
anyway.

This fixes #581.
Note: For pull requests, the version that is tested is an autogenerated
temporary merge commit (which is good, since it tests whether things
work *after* merging), but that commit id is also included in the
filename which might be bit suprising. To be considered later.
@matthijskooijman matthijskooijman merged commit cc01e16 into master Nov 19, 2023
8 checks passed
wscript Show resolved Hide resolved
This was referenced Nov 20, 2023
@aquaherd aquaherd deleted the improved-versioning branch November 22, 2023 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants