Use `nix-eval-jobs` and delete `hydra-eval-jobs` #1421

Ericson2314 · 2024-11-13T21:50:02Z

nix-eval-jobs was forked from hydra-eval-jobs originally, for others to use. That's great! But maintaining two copies is no good. But we use that.

Original author is @delroth, for Lix's Hydra, thank you!

Draft because some tests fail. I am not sure why. Maybe there are fixes on the Lix branch that can be cherry-picked to fix? Not sure!

zowoq · 2024-11-13T23:06:59Z

src/script/hydra-eval-jobset

-                "--flake", $flakeRef,
-                "--gc-roots-dir", getGCRootsDir,
-                "--max-jobs", 1);
+        @cmd = ("nix-eval-jobs", "--flake", $flakeRef . '#hydraJobs');


Uses hydraJobs here and the original hydra-eval-jobs has hydraJobs and checks:

hydra/src/hydra-eval-jobs/hydra-eval-jobs.cc

Lines 148 to 152 in f974891

auto aHydraJobs = vOutputs->attrs()->get(state.symbols.create("hydraJobs"));

if (!aHydraJobs)

aHydraJobs = vOutputs->attrs()->get(state.symbols.create("checks"));

if (!aHydraJobs)

throw Error("flake '%s' does not provide any Hydra jobs or checks", flakeRef);

Ah found more commits that are relevant.

The Lix commits don't address this issue, they only have support for hydraJobs.

I think it would be important to keep the checks supports

Ericson2314 · 2024-11-13T23:47:35Z

I left a comment on nix-community/nix-eval-jobs#330, I think that is causing the failure. CC @Mic92.

Ericson2314 · 2024-11-14T02:51:59Z

nix-community/nix-eval-jobs#336 this I thought would help but is not sufficient hmm

Ericson2314 · 2024-11-14T03:05:52Z

( STDERR )  job  1    {"attr":"caDependingOnCA","attrPath":["caDependingOnCA"],"error":"error: derivation 'caDependingOnCA' does not have valid outputs: error:\n              â¦ while evaluating the output path of a derivation\n                at <nix/derivation-internal.nix>:19:9:\n                  18|       value = commonAttrs // {\n                  19|         outPath = builtins.getAttr outputName strict;\n                    |         ^\n                  20|         drvPath = strict.drvPath;\n\n              error: path '/0rmq7bvk2raajd310spvd416f2jajrabcg6ar706gjbd6b8nmvks' is not in the Nix store"}
( STDERR )  job  1    {UNKNOWN}: asdfasdf at /home/jcericson/src/hydra/master/src/script/hydra-eval-jobset line 412. at /nix/store/m07199rzw9ja8v6ba75r4ak5iy92ps9i-hydra-perl-deps/lib/perl5/site_perl/5.38.2/Catalyst/Model/DBIC/Schema.pm line 526

ah nix-eval-jobs needs more changes.

dasJ · 2024-11-14T10:51:18Z

A relevant thing to consider here is whether we want an official NixOS project to be dependant on a Nix Community project. As of now (afaik), the NixOS GitHub namespace is mostly self-contained. I see these solutions:

Close this PR
Migrate nix-eval-jobs to the NixOS namespace (or even nix repo?)
Just live with the fact

I personally prefer the latter two options over the first one since it would get rid of code duplication and remove a component from Hydra that only a few people are proficient in and that gets a lot less maintenance than nix-eval-jobs.

Maybe that's a question for the new SC?

Mic92 · 2024-11-14T11:10:05Z

Happy to move the project to the NixOS org if I can keep maintaining it there. We would loose aarch64-linux builds because those runners are currently sponsored by namespace in nix-community. I can attach my buildbot-nix installation to this org to compensate.

Mic92 · 2024-11-14T11:17:36Z

Maybe that's a question for the new SC?

That's more a technical question for the Hydra/Nix team, unless we feel to overwhelmed what to do and need to escalate.

Ericson2314 · 2024-11-14T18:24:46Z

OK it builds now (CI should say so in a moment).

Ericson2314 · 2024-11-14T18:29:33Z

Keeping draft just until nix-community/nix-eval-jobs#337 is merged.

Ericson2314 · 2024-11-15T16:17:43Z

OK undrafting because we're back on nix-eval-jobs main.

@dasJ I am fine resolving the nix-community thing later, because Hydra is a funny project --- it's official because history and because hydra.nixos.org, not because it is up to the quality we generally expect for 3rd-party use.

Mindavi

The perl bits look generally ok to me but not my expertise. Maybe put somewhere a bit clearer that the constituents are not supported anymore. That's not used on hydra.nixos.org, right?

src/script/hydra-eval-jobset

dasJ

As stated before, I do find this move to be a good idea. Here are some things I noticed while reading the code

dasJ · 2024-11-16T20:55:16Z

flake.nix

@@ -8,14 +8,20 @@
  inputs.nix.inputs.nixpkgs.follows = "nixpkgs";
  inputs.nix.inputs.libgit2.follows = "libgit2";

+  inputs.nix-eval-jobs.url = "github:nix-community/nix-eval-jobs";


Can we somehow get that rev into the footer in src/root/layout.tt? I'm thinking something like Hydra 1.0 (using nix-eval-jobs abcdef and nix-2.18)

dasJ · 2024-11-16T20:56:20Z

src/script/hydra-eval-jobset

-                "--flake", $flakeRef,
-                "--gc-roots-dir", getGCRootsDir,
-                "--max-jobs", 1);
+        @cmd = ("nix-eval-jobs", "--flake", $flakeRef . '#hydraJobs');


I think it would be important to keep the checks supports

dasJ · 2024-11-16T20:57:18Z

src/script/hydra-eval-jobset

-    push @cmd, "--no-allow-import-from-derivation" if $config->{allow_import_from_derivation} // "true" ne "true";
+    push @cmd, ("--gc-roots-dir", getGCRootsDir);
+    push @cmd, ("--max-jobs", 1);
+    push @cmd, "--meta";


Wills there two parameters change anything relevant?

$ git log -G --max-memory-size commit 4b107e6ff36bd89958fba36e0fe0340903e7cd13 Author: Pierre Bourdon <[email protected]> Date: Mon Jul 22 23:16:29 2024 +0200 hydra-eval-jobset: pass --workers and --max-memory-size to n-e-j Lost in the h-e-j -> n-e-j migration, causing evaluation to always be single threaded and limited to 4GiB RAM. Follow the config settings like h-e-j used to do (via C++ code). commit 6f1d68bda483b2ed3b1c1f0a696e8fe300108853 Author: Eelco Dolstra <[email protected]> Date: Wed Feb 19 20:36:52 2020 +0100 Revert "hydra-eval-jobs -> nix eval-hydra-jobs" This reverts commit 345512a6d0743eaac9d451833ecc1a71d12809c9. commit 345512a6d0743eaac9d451833ecc1a71d12809c9 Author: Eelco Dolstra <[email protected]> Date: Sat Feb 15 00:03:58 2020 +0100 hydra-eval-jobs -> nix eval-hydra-jobs

It's purpose seems to be to change a setting that hydra-eval-jobs was changing programmatically before.

src/script/hydra-eval-jobset

dasJ · 2024-11-16T20:59:47Z

src/script/hydra-eval-jobset

-    print STDERR "$stderr";
+            if (defined $err) {
+                print STDERR "$err";
+                undef $err;


Not sure about the API, but is it a good idea to undef this? Maybe set it to an emtpy string instead? Feels like that would die horribly if the process decides to do stdout, then stderr, then stdout again. Not sure why it would, but better safe than sorry

I think every loop it will define them if they need to exists. Perl is weird! Why can't they just return a tuple!

dasJ · 2024-11-16T21:03:40Z

src/script/hydra-eval-jobset

@@ -709,36 +744,20 @@ sub checkJobsetWrapped {
        return;
    }

-    die "Jobset contains a job with an empty name. Make sure the jobset evaluates to an attrset of jobs.\n"


Is this a relevant check? Kind of surprising to see it gone

src/script/hydra-eval-jobset

dasJ · 2024-11-17T08:17:45Z

flake.nix

@@ -8,14 +8,20 @@
  inputs.nix.inputs.nixpkgs.follows = "nixpkgs";
  inputs.nix.inputs.libgit2.follows = "libgit2";

+  inputs.nix-eval-jobs.url = "github:nix-community/nix-eval-jobs";
+  inputs.nix-eval-jobs.inputs.nixpkgs.follows = "nixpkgs";


Something I forgot in the last review: I think we should also force the nix version of hydra into the package, otherwise we might depend on 2 nix versions

That's actually a good one and could be a problem if hydra or nix-eval-jobs doesn't keep up with nix releases.

I can maintain release branches for different nix versions again if this is needed.

Ericson2314 · 2024-11-17T21:23:57Z

Thanks for this good review, both of you. In light of these things. I am going to try to flip the order: re-draft this, and get Mesonification done first.

(cherry picked from commit 684cc50)

(cherry picked from commit 6d4ccff)

(cherry picked from commit ed7c587)

nix-eval-jobs streams output, unlike hydra-eval-jobs. Now that we've migrated, we can use this to: 1. Use less RAM by avoiding buffering a whole eval's worth of metadata into a Perl string and an array of JSON objects. 2. Make evals latency a bit lower by allowing the queue runner to start ingesting builds faster. (cherry picked from commit b0e9b4b)

This partially reverts commit 370a4bf. (cherry picked from commit cdfc5c81e8037d3e4818a3e459d0804b2c157ea9)

Lost in the h-e-j -> n-e-j migration, causing evaluation to always be single threaded and limited to 4GiB RAM. Follow the config settings like h-e-j used to do (via C++ code). (cherry picked from commit 4b107e6)

Ericson2314 · 2024-11-25T16:14:51Z

@Ma27 any idea why the constituents GC test is failing?

Mindavi · 2024-11-25T17:34:07Z

I've seen (presumably) races in some tests, this could be one too. But may as well be a valid fail.

Ericson2314 · 2024-11-25T17:42:15Z

It happened very reliably for me in the dev shell.

Ericson2314 marked this pull request as draft November 13, 2024 21:50

Ericson2314 mentioned this pull request Nov 13, 2024

autotools -> meson #1422

Merged

zowoq reviewed Nov 13, 2024

View reviewed changes

Ericson2314 mentioned this pull request Nov 13, 2024

improve compatibility if ca derivations are present nix-community/nix-eval-jobs#330

Merged

Ericson2314 mentioned this pull request Nov 14, 2024

Ensure that we still create map kv pairs for floating CA outputs nix-community/nix-eval-jobs#336

Merged

Ericson2314 mentioned this pull request Nov 14, 2024

PackageInfo::queryOutputs should not trip up on placeholders NixOS/nix#11885

Open

Ericson2314 force-pushed the nix-eval-jobs branch from 498b283 to d0cfbed Compare November 14, 2024 18:21

Ericson2314 mentioned this pull request Nov 14, 2024

Make fallback logic for CA derivations more flexible nix-community/nix-eval-jobs#337

Merged

Ericson2314 marked this pull request as ready for review November 14, 2024 18:28

Ericson2314 marked this pull request as draft November 14, 2024 18:28

Ericson2314 marked this pull request as ready for review November 15, 2024 15:57

Ericson2314 force-pushed the nix-eval-jobs branch from d0cfbed to 8a49ecb Compare November 15, 2024 15:57

Mindavi approved these changes Nov 15, 2024

View reviewed changes

Mic92 reviewed Nov 16, 2024

View reviewed changes

src/script/hydra-eval-jobset Show resolved Hide resolved

dasJ requested changes Nov 16, 2024

View reviewed changes

dasJ reviewed Nov 17, 2024

View reviewed changes

Ericson2314 marked this pull request as draft November 17, 2024 21:24

Ericson2314 force-pushed the nix-eval-jobs branch 2 times, most recently from a3e3fe9 to d2227af Compare November 17, 2024 23:26

flake: add nix-eval-jobs as input

314e7f2

(cherry picked from commit 684cc50)

Ma27 and others added 6 commits November 25, 2024 11:08

TEMP BUMP for nix-community/nix-eval-jobs#340

b133e2e

hydra-eval-jobset: use nix-eval-jobs instead of hydra-eval-jobs

61e34ce

(cherry picked from commit 6d4ccff)

hydra-eval-jobs: remove, replaced by nix-eval-jobs

57c6e49

(cherry picked from commit ed7c587)

Bring back constituents

72e2516

This partially reverts commit 370a4bf. (cherry picked from commit cdfc5c81e8037d3e4818a3e459d0804b2c157ea9)

hydra-eval-jobset: pass --workers and --max-memory-size to n-e-j

d97956d

Lost in the h-e-j -> n-e-j migration, causing evaluation to always be single threaded and limited to 4GiB RAM. Follow the config settings like h-e-j used to do (via C++ code). (cherry picked from commit 4b107e6)

Ericson2314 force-pushed the nix-eval-jobs branch from d2227af to d97956d Compare November 25, 2024 16:08

Ericson2314 assigned Ma27 Nov 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `nix-eval-jobs` and delete `hydra-eval-jobs` #1421

Use `nix-eval-jobs` and delete `hydra-eval-jobs` #1421

Ericson2314 commented Nov 13, 2024

zowoq Nov 13, 2024 •

edited

Loading

Ericson2314 Nov 13, 2024

Ericson2314 Nov 13, 2024

zowoq Nov 14, 2024 •

edited

Loading

dasJ Nov 16, 2024

Ericson2314 commented Nov 13, 2024

Ericson2314 commented Nov 14, 2024

Ericson2314 commented Nov 14, 2024

dasJ commented Nov 14, 2024 •

edited

Loading

Mic92 commented Nov 14, 2024 •

edited

Loading

Mic92 commented Nov 14, 2024

Ericson2314 commented Nov 14, 2024 •

edited

Loading

Ericson2314 commented Nov 14, 2024

Ericson2314 commented Nov 15, 2024

Mindavi left a comment

dasJ left a comment •

edited

Loading

dasJ Nov 16, 2024

dasJ Nov 16, 2024

dasJ Nov 16, 2024

Ericson2314 Nov 17, 2024

dasJ Nov 16, 2024

Ericson2314 Nov 17, 2024 •

edited

Loading

dasJ Nov 16, 2024

dasJ Nov 17, 2024

Mindavi Nov 17, 2024

Mic92 Nov 18, 2024

Ericson2314 commented Nov 17, 2024

Ericson2314 commented Nov 25, 2024

Mindavi commented Nov 25, 2024

Ericson2314 commented Nov 25, 2024

	auto aHydraJobs = vOutputs->attrs()->get(state.symbols.create("hydraJobs"));
	if (!aHydraJobs)
	aHydraJobs = vOutputs->attrs()->get(state.symbols.create("checks"));
	if (!aHydraJobs)
	throw Error("flake '%s' does not provide any Hydra jobs or checks", flakeRef);

Use nix-eval-jobs and delete hydra-eval-jobs #1421

Are you sure you want to change the base?

Use nix-eval-jobs and delete hydra-eval-jobs #1421

Conversation

Ericson2314 commented Nov 13, 2024

zowoq Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zowoq Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ericson2314 commented Nov 13, 2024

Ericson2314 commented Nov 14, 2024

Ericson2314 commented Nov 14, 2024

dasJ commented Nov 14, 2024 • edited Loading

Mic92 commented Nov 14, 2024 • edited Loading

Mic92 commented Nov 14, 2024

Ericson2314 commented Nov 14, 2024 • edited Loading

Ericson2314 commented Nov 14, 2024

Ericson2314 commented Nov 15, 2024

Mindavi left a comment

Choose a reason for hiding this comment

dasJ left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ericson2314 Nov 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ericson2314 commented Nov 17, 2024

Ericson2314 commented Nov 25, 2024

Mindavi commented Nov 25, 2024

Ericson2314 commented Nov 25, 2024

Use `nix-eval-jobs` and delete `hydra-eval-jobs` #1421

Use `nix-eval-jobs` and delete `hydra-eval-jobs` #1421

zowoq Nov 13, 2024 •

edited

Loading

zowoq Nov 14, 2024 •

edited

Loading

dasJ commented Nov 14, 2024 •

edited

Loading

Mic92 commented Nov 14, 2024 •

edited

Loading

Ericson2314 commented Nov 14, 2024 •

edited

Loading

dasJ left a comment •

edited

Loading

Ericson2314 Nov 17, 2024 •

edited

Loading