Use permanent links for ELPA/MELPA packages

[spacekitteh]

Whenever a new version of a package is released on ELPA/MELPA, and this overlay isn't updated, we can no longer build the derivation. This is because the overlay references non-permanent source URLs (such as https://elpa.gnu.org/packages/auctex-13.0.4.tar). They should reference the git directly, such as http://git.savannah.gnu.org/cgit/emacs/elpa.git/snapshot/elpa-04d69c53183ef8a5e7593e35355e55896bf079d3.tar.gz or the actual git repo commit itself.

[talyz]

This sounds like an issue that needs to be fixed in emacs2nix, see nix-community/emacs2nix#55.

[collares]

One other idea is to construct a fetchurl-like function that tries both the normal URL and the LZMA-compressed URL (all older versions exist on ELPA, but they are compressed), producing a fixed-output derivation; emacs2nix provides us with the post-decompression hash, so that's doable. Then pass that to the automatically-generated file instead of the usual fetchurl function, requiring no changes to emacs2nix.

[talyz]

Yeah, but unless we want to modify the files after running emacs2nix or load the package set in an interesting way, it would also require modification to elpa2nix. Since this affects nixpkgs as well, fixing nix-community/emacs2nix#55 is probably the way to go.

[collares]

@adisbladis is a hero and fixed this at NixOS/nixpkgs#132937 :) Thanks!!!

[adisbladis]

Closing because of NixOS/nixpkgs#132937

[jorsn]

@adisbladis is a hero and fixed this at NixOS/nixpkgs#132937 :) Thanks!!! -- #125 (comment)

I totally agree with the “hero”! But is this really fixed?

For my application the approach used in the fix worked so far.
Nevertheless, were you aware @adisbladis that it is not really reprodicible, since elpa has no clear policy on which releases are kept in compressed form, and how long they are kept?

The corresponding cite from https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46441:

there's no way to get
a permanent link at all: while we do keep some old versions, we don't
keep them all. Currently, the limit is set at 20, meaning that we keep
up to 20 old versions around per package, and once we hit that limit we
start pruning the old versions according to a heuristic that intends to
guess which versions are more important to keep.

Probably reproducible approaches would be (a) to use urls from archive.org or (b) use the elpa git repo. I demonstrated how to do the latter here. This would probably be done by emacs2nix. I admit it is not totally convenient since the git repo does not necessarily contain exactly the same version as elpa, and one needs to download the 250 MB git repo, instead of only a list of packages as in the current emacs2nix.
Also, none of the maintainers of emacs2nix reacted to my issue.

[ulysses4ever]

and one needs to download the 250 MB git repo, instead of only a list of packages as in the current emacs2nix.

There are probably ways around it. One that comes to mind is to have GitHub mirrors for ELPA etc. (there was one for ELPA even; and there's one for Emacs) and use GitHub API to the sources (via https://github.com/{username}/{projectname}/archive/{sha}.zip).

[jorsn]

For building the packages, you're probably right. But the only way to get the right git revisions for the package versions seems to be to scrape the git log.