apt_get package module upgrades

[vali-um]

Hi,

tl;dr: package module apt_get is using upgrade, not dist-upgrade for determining updateable packages and therefor will not update from new sources

I recently tried to add a repository containing a patched package version and installing that with cfengine. (my servers are running debian)
I tried to achieve this with a package promise with version => latest and promising a /etc/sources.list.d file.
I then realized that the package would notbe updated and started messing around with removing the package list databases in the state dir to force an update.
Still not successful i asked on IRC and Craig Comstock tried to help me debugging the issue which lead to realization:
The package module apt_get is using upgrade, not dist-upgrade for determining updateable packages and therefor will not update packages from new sources because apt policies prevent that. In the same callout the flag --ignore-hold is used, which overrides some apt policies...
If i read the old sources correctly the command apt list --upgradeable was used previously to do this which would list a upgradeable package from a new source.
Does anyone know if there is a reason to do this? or might this just be a bug in the apt_get package module?

Looking forward to read your opinion on this.

Cheers, Valentin

[vali-um]

I got part of that explaination worng.
The patched package i use also requires an additional dependency.
apt-get upgrade will install from a new source, but it won't install additional dependencies.
I suggest using the "--with-new-pkgs" flag to determine upgradeable packages.

Cheers, Valentin

[nickanderson]

From the apt-get man pages (apt - 2.3.9):

• upgrade: upgrade is used to install the newest versions of all packages
  currently installed on the system from the sources enumerated in
  /etc/apt/sources.list. Packages currently installed with new versions
  available are retrieved and upgraded; under no circumstances are currently
  installed packages removed, or packages not already installed retrieved and
  installed. New versions of currently installed packages that cannot be
  upgraded without changing the install status of another package will be left
  at their current version. An update must be performed first so that apt-get
  knows that new versions of packages are available.

• dist-upgrade: dist-upgrade in addition to performing the function of upgrade, also
  intelligently handles changing dependencies with new versions of packages;
  apt-get has a "smart" conflict resolution system, and it will attempt to
  upgrade the most important packages at the expense of less important ones if
  necessary. The dist-upgrade command may therefore remove some packages. The
  /etc/apt/sources.list file contains a list of locations from which to retrieve
  desired package files. See also =apt_preferences=(5) for a mechanism for
  overriding the general settings for individual packages.

Given the above, I suspect that upgrade was deemed the safest option. Generally CFEngine has been very cautious about any kind of removal but things change based on user feedback. e.g. now the default options include --allow-downgrades, --allow-remove-essential, and --allow-change-held-packages.

Looking at the code, I don't see where apt was previously used, just the comment noting the addition of –ignore-hold made the list of packages similar to that returned by apt list --upgradeable.

I don't think I have an objection to adding --with-new-pkgs.

CFE-2855 is the ticket that tracks the change where --ignore-hold was introduced. It was filed by @basvandervlies. @basvandervlies what do you think about this proposal to add --with-new-pkgs to the list of options used when listing package updates?

[vali-um]

Thanks for your work

Given the above, I suspect that upgrade was deemed the safest option.

I agree that this is a good and safe option.

I think that --with-new-pkgs would be a good option, as you would also be able to install the latest versions of some packages like linux-headers-amd64 which are purely there to depend on the header package of the latest version, e.g. linux-headers-5.10.0-10-amd64, and therefor needs to install new packages to stay up to date.