Mismatching hashes and dependency versions for gradle2nix itself #26

mschwaig · 2020-11-18T11:24:50Z

I just tried to build gradle2nix with the command suggested in the README.md:

nix build -f "https://github.com/tadfisher/gradle2nix/archive/master.tar.gz"

which always seems to give me a mismatching hash for different random parts of jackson.

For example with the latest nix docker image:

$ nix build -f "https://github.com/tadfisher/gradle2nix/archive/master.tar.gz"
waiting for locks or build slots...
hash mismatch in fixed-output derivation '/nix/store/q4apxx0rdz72axxhi7c4gcghsf0vmj3d-jackson-core-2.11.0.rc1.pom':
  wanted: sha256:1jx8piydld4km6rhlxxzwwx70277nr75g7zbvkbg47zd2sm9ljk6
  got:    sha256:182jz7hf4b5ywphziw0647kl7r2y0fm49h9xnpv8w9vrsq27bqbc
cannot build derivation '/nix/store/q987m8dns0cbsrhz47xj1xdxh89kx9ja-jackson-core-2.11.0.rc1.pom.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/f6i3wy1p7q71gy68yb3qkbii2vpiwk5f-gradle2nix-gradle-plugin-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/8b19gkvn40l7l1valmyjdy89glrbf95c-init.gradle.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/cmnmm9sg2l6d7jwbizj2jvsys2kjpfhh-gradle2nix-1.0.0-rc2.drv': 1 dependencies couldn't be built
error: build of '/nix/store/cmnmm9sg2l6d7jwbizj2jvsys2kjpfhh-gradle2nix-1.0.0-rc2.drv' failed

And with my local machine running unstableNix:

$ nix build -f gradle2nix.nix
warning: unknown setting 'extra-sandbox-paths'
warning: unknown setting 'extra-sandbox-paths'
error: --- Error --- nix-daemon
hash mismatch in fixed-output derivation '/nix/store/rfmm7n5sqi52sbaaklvjqmk3dwjc7z0b-jackson-parent-2.11.pom.drv':
  specified: sha256-wbChPYz+Y7QLD66EWLmUYxae0MYPf8lj7z8FJXFQy+g=
     got:    sha256-YXSwNaEPVu8DtZPoLCjfnlMFVcJRiAFd1x+wu3k65RY=
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/4falvcj3mk9k2fwl1ygikzn4lm0gx9bz-jackson-parent-2.11.pom.drv' failed to build
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/nxlbrgh6k96l24gvmyqp8iq8194xvqhv-gradle2nix-gradle-plugin-env.drv' failed to build
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/043gm5jm164nq82a5fiwljcg0pyxx55w-init.gradle.drv' failed to build
error: --- Error ------------------------------------------------------------------------------------- nix
1 dependencies of derivation '/nix/store/s05zvjxgzs75gp8nl7syqy9flymm8nm5-gradle2nix-1.0.0-rc2.drv' failed to build

Looking into that a bit I saw that the jackson version in fixtures/basic/basic-kotlin-project/kotlin/build.gradle.kts also does not match the jackson version in gradle-env.json which looks wrong to me and leads me to think the mismatch might simply be because some version change got lost in either one of those files.

I have not looked into this further or actually gotten gradle2nix to build yet.

The text was updated successfully, but these errors were encountered:

tadfisher · 2020-11-19T18:59:31Z

This is strange, I'm not sure why jackson is being pulled in at all. Let me investigate.

mschwaig · 2020-11-19T19:55:18Z

With the changes I made in #27 I was able to bypass the issue for now and and to then successfully apply gradle2nix to one of Gradle's builtin kotlin hello world projects and package it as a flake. 👍

Thanks for making that possible.

tadfisher · 2020-11-23T03:35:53Z

Does this work with the latest commit on master? That version of jackson is actually required by a plugin dependency.

stites · 2020-11-23T18:29:42Z

This is still broken in master, as far as I know, but not with jackson. For me I see this with javax.servlet-api:

✦ ❯ nix build -f "https://github.com/tadfisher/gradle2nix/archive/master.tar.gz"
warning: unknown setting 'extra-sandbox-paths'                                                                                                           
warning: unknown setting 'extra-sandbox-paths'
error: --- Error --- nix-daemon
hash mismatch in fixed-output derivation '/nix/store/m4mwk5idw1xahkdp5a6k82i4bhrs8nqx-javax.servlet-api-3.1.0.pom.drv':
  specified: sha256-sxEJ4i6j8t8a15VUMucYo13vUK5sGWmANK+ooM+ekGk=
     got:    sha256-b60q4KOgDBRbpXdNZxlmeCkcwgC9Cb0GJu8R/yC03Bo=
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/fbchzvf4413820ymwh4hvjambvr31k48-javax.servlet-api-3.1.0.pom.drv' failed to build
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/7chxzznq0ini5clc4imgxwkkqpgl2jkh-gradle2nix-gradle-project-env.drv' failed to build
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/kgcqdafk9dp0zn9r44l86sfq8vgykh0r-init.gradle.drv' failed to build
error: --- Error --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
1 dependencies of derivation '/nix/store/a59vn6cradqc9p7zc6cpwzkrxq5iac2n-gradle2nix-1.0.0-rc2.drv' failed to build

mschwaig · 2020-11-23T19:11:39Z

Looks like I'm seeing the same mismatching hashes as @stites right now. Thanks for confirming that this does not only happen for me.

$ nix build -f "https://github.com/tadfisher/gradle2nix/archive/master.tar.gz"
warning: unknown setting 'extra-sandbox-paths'
warning: unknown setting 'extra-sandbox-paths'
error: --- Error --- nix-daemon
hash mismatch in fixed-output derivation '/nix/store/ali7yir0h9z7dbsy8xz95ish9vsbbms4-javax.servlet-api-3.1.0.pom.drv':
  specified: sha256-sxEJ4i6j8t8a15VUMucYo13vUK5sGWmANK+ooM+ekGk=
     got:    sha256-b60q4KOgDBRbpXdNZxlmeCkcwgC9Cb0GJu8R/yC03Bo=
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/v6flb992d2qlnc3rxfj0g905xbiys9qd-javax.servlet-api-3.1.0.pom.drv' failed to build
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/pxn87jkhn4y4ry4gyvpg11lz768nc4dj-gradle2nix-gradle-project-env.drv' failed to build
error: --- Error --- nix-daemon
1 dependencies of derivation '/nix/store/lbijb7hr77dm086wckrbwlj7qvasy7fw-init.gradle.drv' failed to build
error: --- Error ------------------------------------------------------------------------------------- nix
1 dependencies of derivation '/nix/store/z1apk82sx0hpp00j6rd2mkq6q8mklqdg-gradle2nix-1.0.0-rc2.drv' failed to build

tadfisher · 2020-11-23T19:56:28Z

Ah, once I run nix-collect-garbage to clear out my existing dependency derivations, I see this too. Investigating.

stites · 2020-11-23T20:00:23Z

@tadfisher see #28

mschwaig · 2020-11-23T20:47:27Z

#28 is basically the same thing that I did in #27, but for a different dependency.

Since gradle2nix depends on gradle-env.json and gradle-env.nix it always indirectly depends on a previous version of itself.
The hashes inside gradle-env.json naturally are the part of that that can very easily break.

Maybe it would be helpful to have a flag inside gradle-env.nix or somewhere that can disable checking the hashes, so that people can get to a working version of gradle2nix in a first step and then generate working hashes again in a second step in order to bootstrap independently of the provided hashes.

That does not help really fix any specific problem, but I think it would be useful for users to recover from things like this and I think it might have other uses as well.

tadfisher · 2020-11-23T21:03:32Z

@mschwaig That's not really a viable solution; if gradle2nix generates bad hashes for itself then it will do so for any project you run it on, making it a non-usable tool.

Anyway, what's going on is that we are hashing the artifact in the Gradle cache, which is apparently not the same as the artifact from the source repo. I'm not sure what Gradle is doing to alter these files (perhaps it's regenerating metadata) but the fix is to hash the artifact as we resolve it ourselves. I'll have a PR up shortly.

tadfisher · 2020-11-23T21:32:46Z

Ah, that file is being served with CRLF (Windows) line separators, and Gradle is saving it to the cache with LF (Unix) line separators.

I've moved hash generation to our own resolvers, but this is extremely slow as we have to redownload every dependency. It might be worth doing this only for POM artifacts just to mitigate this issue, or somehow normalizing POMs at fetch time.

tadfisher · 2020-11-23T21:41:22Z

This is the root cause, FWIW: gradle/gradle#9072

tadfisher · 2020-11-23T22:31:29Z

@stites @mschwaig Try it now?

mschwaig · 2020-11-23T23:00:41Z

gradle2build builds successfully now, thanks.

stites · 2020-11-24T14:33:25Z

Works for me as well! Thanks!

ghost · 2020-12-06T05:10:29Z

Still (or again?) having this issue with the following command:

nix build -f "https://github.com/tadfisher/gradle2nix/archive/master.tar.gz"

The output is the following:

downloading 'https://github.com/tadfisher/gradle2nix/archive/master.tar.gz'
waiting for locks or build slots...
hash mismatch in fixed-output derivation '/nix/store/9w04ibi53kngj4k2fsv0ll0i19j6lnwr-jackson-bom-2.12.0.pom':
  wanted: sha256:082nadlh7mz44qikkm231188isdjm6v7hbxd57b3a5yxdjvdm1iq
  got:    sha256:0f8vg5jnyjcdxmcqm7m3iqn5mk2x08qvnp6iz17ml631pzyv4rzk
cannot build derivation '/nix/store/3pmb6d32z43qm8jbypwv3acnaw059ymm-jackson-bom-2.12.0.pom.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/16676hx9j24xqarfqs2wyk7j2y3bkvq0-gradle2nix-gradle-plugin-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/z4cgi26nwqfvd9psdvv6jzjcg61gpvwc-init.gradle.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/k7mc4nm19azwg72zvc7yjppdm57d7d65-gradle2nix-1.0.0-rc2.drv': 1 dependencies couldn't be built
[137 built (1 failed), 0.7 MiB DL]
error: build of '/nix/store/k7mc4nm19azwg72zvc7yjppdm57d7d65-gradle2nix-1.0.0-rc2.drv' failed

jeeger · 2020-12-09T15:07:32Z

Yep, same problem here:

building '/nix/store/ypl5afl0g6iadvbhhilnjnb7yx5l7zi2-jackson-annotations-2.12.0.pom.drv'...

trying https://plugins.gradle.org/m2/com/fasterxml/jackson/core/jackson-annotations/2.12.0/jackson-annotations-2.12.0.pom
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
direnv: ([/usr/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  6038  100  6038    0     0  12423      0 --:--:-- --:--:-- --:--:-- 57504
hash mismatch in fixed-output derivation '/nix/store/xwmnfx609qakriq27zvshrgkld9xi6qf-jackson-annotations-2.12.0.pom':
  wanted: sha256:1x8yrrnpfniphk9faylr9c1j69wkz6ff02mzimq6q1myk56hfbm8
  got:    sha256:15h29c2zy764j34x653gn7r56v7sbh4ynqmphay42h6a0cva0l2w
cannot build derivation '/nix/store/cgrzkb1jfcsp48hmcbdkn44ndbrsy0bm-jackson-annotations-2.12.0.pom.drv': 1 dependencies couldn't be built
building '/nix/store/3lx6sklaqvnbifbvx4qcmmmnaggd312a-jackson-base-2.12.0.pom.drv'...
cannot build derivation '/nix/store/2mgylfnqmm7254qji1wfs9cf7iqknbzx-gradle2nix-gradle-plugin-env.drv': 1 dependencies couldn't be built
building '/nix/store/mhmpnhwijkayj6sz0khf9j3i9n46fj1c-javalin-3.12.0.jar.drv'...
cannot build derivation '/nix/store/fvwmp444i6chz0fxi0pd62xp82qcni7m-init.gradle.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/x2x0w5nyi0n0298hgbjj13g4vq4sy529-gradle2nix-1.0.0-rc2.drv': 1 dependencies couldn't be built
error: build of '/nix/store/x2x0w5nyi0n0298hgbjj13g4vq4sy529-gradle2nix-1.0.0-rc2.drv' failed

tadfisher · 2020-12-09T20:50:02Z

This issue is caused by me; when I hack on gradle2nix, it's pulling hashes from my local gradle cache, which normalizes certain artifacts such as POM files. It's time to add some CI to avoid this.

mschwaig · 2020-12-11T09:33:59Z

This is the .gitlab-ci.yml file that I used for reproducing this issue inside GitLab CI, in case it is helpful.

image: nixos/nix:latest

default:
  tags:
    - docker-images

stages:
  - test

before_script:
  - nix-channel --add https://nixos.org/channels/nixos-unstable nixpkgs
  - nix-channel --update

dockertest:
  stage: test
  script:
    - nix build -f "https://github.com/tadfisher/gradle2nix/archive/master.tar.gz"

To test for that kind of issue it's actually nice that starts from a docker image with nothing relevant in /nix/store each time.

spencerpogo · 2022-01-18T01:47:20Z

I'm unable to run this tool due to this error.

❯ nix run github:tadfisher/gradle2nix -- --help
error: hash mismatch in fixed-output derivation '/nix/store/6hhf9myxx3wxl38k89l0psm1p6b7006y-jackson-parent-2.12.pom.drv':
         specified: sha256-YqocFnmt4J8XPb8bbDLTXFXnWAAjj9XkjxOqQzfAh1s=
            got:    sha256-Z749r6r4zhcX56fefbkrpDPfkjtpekDJdDpUzJ/FNjc=

Pythoner6 · 2024-02-09T12:36:38Z

Hmm, I'm getting the opposite error right now:

error: hash mismatch in fixed-output derivation '/nix/store/a56mz008i9cnfmxrqszlfcvb3ysjq79l-jackson-parent-2.12.pom.drv':
         specified: sha256-Z749r6r4zhcX56fefbkrpDPfkjtpekDJdDpUzJ/FNjc=
            got:    sha256-YqocFnmt4J8XPb8bbDLTXFXnWAAjj9XkjxOqQzfAh1s=

mschwaig mentioned this issue Nov 18, 2020

manually downgrade pinned jackson dependency #27

Closed

stites mentioned this issue Nov 23, 2020

make gradle2nix build using nix-prefetch-url sha256 #28

Closed

tadfisher closed this as completed Nov 24, 2020

tadfisher reopened this Dec 9, 2020

chriswarbo mentioned this issue Aug 10, 2022

Fix bad hash Zipabout/gradle2nix#1

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mismatching hashes and dependency versions for gradle2nix itself #26

Mismatching hashes and dependency versions for gradle2nix itself #26

mschwaig commented Nov 18, 2020

tadfisher commented Nov 19, 2020

mschwaig commented Nov 19, 2020

tadfisher commented Nov 23, 2020

stites commented Nov 23, 2020 •

edited

Loading

mschwaig commented Nov 23, 2020

tadfisher commented Nov 23, 2020

stites commented Nov 23, 2020

mschwaig commented Nov 23, 2020

tadfisher commented Nov 23, 2020

tadfisher commented Nov 23, 2020

tadfisher commented Nov 23, 2020

tadfisher commented Nov 23, 2020

mschwaig commented Nov 23, 2020

stites commented Nov 24, 2020

ghost commented Dec 6, 2020

jeeger commented Dec 9, 2020

tadfisher commented Dec 9, 2020

mschwaig commented Dec 11, 2020 •

edited

Loading

spencerpogo commented Jan 18, 2022

Pythoner6 commented Feb 9, 2024

Mismatching hashes and dependency versions for gradle2nix itself #26

Mismatching hashes and dependency versions for gradle2nix itself #26

Comments

mschwaig commented Nov 18, 2020

tadfisher commented Nov 19, 2020

mschwaig commented Nov 19, 2020

tadfisher commented Nov 23, 2020

stites commented Nov 23, 2020 • edited Loading

mschwaig commented Nov 23, 2020

tadfisher commented Nov 23, 2020

stites commented Nov 23, 2020

mschwaig commented Nov 23, 2020

tadfisher commented Nov 23, 2020

tadfisher commented Nov 23, 2020

tadfisher commented Nov 23, 2020

tadfisher commented Nov 23, 2020

mschwaig commented Nov 23, 2020

stites commented Nov 24, 2020

ghost commented Dec 6, 2020

jeeger commented Dec 9, 2020

tadfisher commented Dec 9, 2020

mschwaig commented Dec 11, 2020 • edited Loading

spencerpogo commented Jan 18, 2022

Pythoner6 commented Feb 9, 2024

stites commented Nov 23, 2020 •

edited

Loading

mschwaig commented Dec 11, 2020 •

edited

Loading