List updated packages

[rinigus]

As I couldn't find any issue filed regarding it, I am filing it here.

One frequent request that we are getting is:

Please consider making the Chum GUI app show all new submissions somehow. Currently it only shows updates to installed packages, whereas in order to check if there’s some new application it takes browsing the whole list of all packages, which (as the list gets longer and longer) gets really inconvenient and time consuming.

I would very much prefer to have it managed the Storeman way, i.e. to have a list of all newly submitted packages, be it a wholly new release or an update. One look at the list and I always know what’s new. Whereas in Chum GUI I can only see if any of the apps I already have installed has been updated, and to find out what else is new it takes scrolling the whole list and trying to spot some new items on it…

From https://forum.sailfishos.org/t/announcing-sailfishos-chum/7381/122

[rinigus]

To my knowledge, it is not possible to get package build timestamp from pkcon. So, the best solution I have came up so far would be to maintain a list by Chum GUI that would be updated every time repo is refreshed. Older list would be compared with a new one and the diff would indicate which packages are newer. Based on such crude approach, we can get a rough estimate on which packages are new(er).

Any better ways of doing it?

[piggz]

In my mind, we need to get an update time at the individual package level. Its not good enough to scrape the package repo date/times as that would result in updated times whenever a rebuild is triggered, eg, when a new sfos release is built.

The best source of update time is at the package source view, however, there is then the task of relating each source packge to a binary package. I think if we think through the issues we can come up with something. We probably need to document what is possible with the API, and what might need to be done with scraping?

[poetaster]

Maybe I'm being naive, but can't version/tag be sufficient if it's just order? All packages with changed release values are newer? At most you would need to store version. That's not date or time based, just newer release based, but if a package has changed release since last chum gui run, it's a newer package ;)

[poetaster]

And of course, this only works when there is activity and only ever for a subset of rpms.

[piggz]

I think users would expect a "recent changes" to list all packages changed in the last X days *where X ~ 10" ?

[poetaster]

I think users would expect a "recent changes" to list all packages changed in the last X days *where X ~ 10" ?

As a user, I'm generally interested in recent changes but this has no specific time dimension. But I'm not a normal user. I noticed that, when updating packages on openrepos, I often navigate to the start page when I'm done to see recent activity. That's generally just the last 20 packages that have been updated.

It's just an idea for something that could easily be achieved with info that's available. It's also always relative but for those who check every day?

[rinigus]

The info that available immediately is pkcon package string. That doesn't contain when the package is rebuilt, just version. Version contains 2 parts - one set by package SPEC and the other one incremented by OBS. I presume we can assume that SPEC-set version would be the same for the OBS induced rebuilds. This can be used to flag "new" packages.

Other option would involve parsing repo XML files (as in https://repo.sailfishos.org/obs/sailfishos:/chum/4.4.0.64_aarch64/repodata/) by GUI, as those have build times. But that would get false positives induced by OBS rebuilds.

Finally, we can setup our scraper on PCs / servers, periodically check all OBS repos and create some intermediate file that is then pushed to some cloud storage.

Out of those, the first solution is simplest. But it will fail "~10 days" condition as it would depend on how frequently user is starting Chum GUI.

As for connecting RPMs to source RPMs, that one is tricky.

[poetaster]

one set by package SPEC

I believe, for the fast and dirty version of 'recent' that's all you need to store. Just the spec version. All packages with a newer version string are 'recent'. done. There is no 'window' that's ideal. There is the 'ideal state' of knowing when something was built, BUT, even then that could be the same version with a new build. So, version is king, or?

[Olf0]

I think users would expect a "recent changes" to list all packages changed in the last X days *where X ~ 10" ?

To list all new and updated apps since one opened the SailfishOS:Chum GUI app the last time is better than a time based approach, IMO. For this to save a simple list of all available RPMs after the repository has been refreshed is sufficient to perform such a comparison when starting next time (and then save the newly retrieved list, replacing the old one). Only the package lists for the regular and the testing SailfishOS:Chum repositories have to be saved separately. As metadata a timestamp for each of these two lists when it was saved would allow for displaying a headline as "New and updated packages in the SailfishOS:Chum[:Testing] repository since <save date&time>:".