Fix listing of reprod dirs #117

bedroge · 2025-10-28T15:29:20Z

I noticed a few times that the listing of tarballs sometimes contains the reprod root dir as well, e.g.:

reprod directories under 2025.06/software/linux/x86_64/intel/icelake/reprod
Catch2/2.13.10-GCCcore-13.3.0/20251028_125714UTC
Cython/3.0.10-GCCcore-13.3.0/
Cython/3.0.10-GCCcore-13.3.0/20251028_125607UTC
Eigen/3.4.0-GCCcore-13.3.0/
Eigen/3.4.0-GCCcore-13.3.0/20251028_125648UTC
hypothesis/6.103.1-GCCcore-13.3.0/
hypothesis/6.103.1-GCCcore-13.3.0/20251028_125225UTC
meson-python/0.16.0-GCCcore-13.3.0/
meson-python/0.16.0-GCCcore-13.3.0/20251028_125244UTC
pybind11/2.12.0-GCC-13.3.0/
pybind11/2.12.0-GCC-13.3.0/20251028_130007UTC
SciPy-bundle/2024.05-gfbf-2024a/
SciPy-bundle/2024.05-gfbf-2024a/20251028_135038UTC

It should not show SciPy-bundle/2024.05-gfbf-2024a/, only SciPy-bundle/2024.05-gfbf-2024a/20251028_135038UTC. As you can guess from this example, it only happens when there's more than a single application in the tarball. I'll reproduce that here first, then add a fix to test it.

bedroge · 2025-10-28T15:29:40Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

eessi-bot-aws · 2025-10-28T15:29:47Z

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2025.10/pr_117/99940

date	job status	comment
Oct 28 15:29:46 UTC 2025	submitted	job id `99940` awaits release by job manager
Oct 28 15:30:25 UTC 2025	released	job awaits launch by Slurm scheduler
Oct 28 15:32:05 UTC 2025	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-99940.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen2-17616654360.tar.gz` size: 0 MiB (45 bytes) entries: 0 modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all no module files in tarball software under 2025.06/software/linux/x86_64/amd/zen2/software no software packages in tarball reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod no reprod directories in tarball other under 2025.06/software/linux/x86_64/amd/zen2 no other files in tarball
Oct 28 15:32:05 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 0/0 test case(s) from 0 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-99940.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2025-10-28T15:35:04Z

Used the wrong version of the easystack, let's try that again.

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

eessi-bot-aws · 2025-10-28T15:35:12Z

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2025.10/pr_117/99952

date	job status	comment
Oct 28 15:35:10 UTC 2025	submitted	job id `99952` awaits release by job manager
Oct 28 15:36:06 UTC 2025	released	job awaits launch by Slurm scheduler
Oct 28 15:37:49 UTC 2025	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-99952.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen2-17616657770.tar.gz` size: 0 MiB (45 bytes) entries: 0 modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all no module files in tarball software under 2025.06/software/linux/x86_64/amd/zen2/software no software packages in tarball reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod no reprod directories in tarball other under 2025.06/software/linux/x86_64/amd/zen2 no other files in tarball
Oct 28 15:37:49 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 0/0 test case(s) from 0 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-99952.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2025-10-28T15:48:42Z

Ah, and used the wrong version in the build command...

bot: build repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

eessi-bot-aws · 2025-10-28T15:48:49Z

New job on instance eessi-bot-mc-aws for repository eessi.io-2023.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2025.10/pr_117/99953

date	job status	comment
Oct 28 15:48:47 UTC 2025	submitted	job id `99953` awaits release by job manager
Oct 28 15:48:52 UTC 2025	released	job awaits launch by Slurm scheduler
Oct 28 15:50:24 UTC 2025	running	job `99953` is running
Oct 28 15:55:04 UTC 2025	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-99953.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-amd-zen2-17616666670.tar.gz` size: 0 MiB (674346 bytes) entries: 115 modules under 2023.06/software/linux/x86_64/amd/zen2/modules/all `cowsay/3.04.lua` `file/5.43-GCCcore-13.2.0.lua` software under 2023.06/software/linux/x86_64/amd/zen2/software `cowsay/3.04` `file/5.43-GCCcore-13.2.0` reprod directories under 2023.06/software/linux/x86_64/amd/zen2/reprod no reprod directories in tarball other under 2023.06/software/linux/x86_64/amd/zen2 no other files in tarball
Oct 28 15:55:04 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ OK ] ( 1/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/29Aug2024-foss-2023b-kokkos %scale=1_node /aeb2d9df @BotBuildTests:x86_64_amd_zen2+default P: perf: 440.798 timesteps/s (r:0, l:None, u:None) [ OK ] ( 2/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/2Aug2023_update2-foss-2023a-kokkos %scale=1_node /04ff9ece @BotBuildTests:x86_64_amd_zen2+default P: perf: 449.058 timesteps/s (r:0, l:None, u:None) [ OK ] ( 3/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /775175bf @BotBuildTests:x86_64_amd_zen2+default P: latency: 1.81 us (r:0, l:None, u:None) [ OK ] ( 4/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /52707c40 @BotBuildTests:x86_64_amd_zen2+default P: latency: 1.72 us (r:0, l:None, u:None) [ OK ] ( 5/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /b1aacda9 @BotBuildTests:x86_64_amd_zen2+default P: latency: 3.99 us (r:0, l:None, u:None) [ OK ] ( 6/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /c6bad193 @BotBuildTests:x86_64_amd_zen2+default P: latency: 4.04 us (r:0, l:None, u:None) [ OK ] ( 7/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /15cad6c4 @BotBuildTests:x86_64_amd_zen2+default P: latency: 0.59 us (r:0, l:None, u:None) [ OK ] ( 8/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /6672deda @BotBuildTests:x86_64_amd_zen2+default P: latency: 0.56 us (r:0, l:None, u:None) [ OK ] ( 9/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /2a9a47b1 @BotBuildTests:x86_64_amd_zen2+default P: bandwidth: 7476.57 MB/s (r:0, l:None, u:None) [ OK ] (10/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /1b24ab8e @BotBuildTests:x86_64_amd_zen2+default P: bandwidth: 7367.71 MB/s (r:0, l:None, u:None) [ PASSED ] Ran 10/10 test case(s) from 10 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-99953.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2025-10-28T15:56:55Z

Argh, 2023.06 didn't have reprod dirs. Switching back to 2025.06...

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

eessi-bot-aws · 2025-10-28T15:57:02Z

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2025.10/pr_117/99954

date	job status	comment
Oct 28 15:57:00 UTC 2025	submitted	job id `99954` awaits release by job manager
Oct 28 15:57:34 UTC 2025	released	job awaits launch by Slurm scheduler
Oct 28 15:59:05 UTC 2025	running	job `99954` is running
Oct 28 16:00:54 UTC 2025	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-99954.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen2-17616671730.tar.gz` size: 0 MiB (875545 bytes) entries: 141 modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all `cowsay/3.04.lua` `file/5.46-GCCcore-13.3.0.lua` software under 2025.06/software/linux/x86_64/amd/zen2/software `cowsay/3.04` `file/5.46-GCCcore-13.3.0` reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod `cowsay/3.04/20251028_155800UTC` `file/5.46-GCCcore-13.3.0/` `file/5.46-GCCcore-13.3.0/20251028_155925UTC` other under 2025.06/software/linux/x86_64/amd/zen2 no other files in tarball
Oct 28 16:00:54 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 0/0 test case(s) from 0 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-99954.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2025-10-28T16:02:00Z

Finally, the tarball listing in #117 (comment) shows the issue:

reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod
cowsay/3.04/20251028_155800UTC
file/5.46-GCCcore-13.3.0/
file/5.46-GCCcore-13.3.0/20251028_155925UTC

Now fixing the issue by checking the number of fields (instead of rows; this is a bug) in the bot's check-build.sh script when splitting by /. After removing the prefix it should have at least 4 elements: app/version/timestamp/empty, the empty one is because of the trailing slash.

bedroge · 2025-10-28T16:02:15Z

Testing the fix...

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

eessi-bot-aws · 2025-10-28T16:02:21Z

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2025.10/pr_117/99955

date	job status	comment
Oct 28 16:02:20 UTC 2025	submitted	job id `99955` awaits release by job manager
Oct 28 16:03:23 UTC 2025	released	job awaits launch by Slurm scheduler
Oct 28 16:04:54 UTC 2025	running	job `99955` is running
Oct 28 16:06:42 UTC 2025	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-99955.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen2-17616675210.tar.gz` size: 0 MiB (875243 bytes) entries: 141 modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all `cowsay/3.04.lua` `file/5.46-GCCcore-13.3.0.lua` software under 2025.06/software/linux/x86_64/amd/zen2/software `cowsay/3.04` `file/5.46-GCCcore-13.3.0` reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod `cowsay/3.04/20251028_160350UTC` `file/5.46-GCCcore-13.3.0/20251028_160513UTC` other under 2025.06/software/linux/x86_64/amd/zen2 no other files in tarball
Oct 28 16:06:42 UTC 2025	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ PASSED ] Ran 0/0 test case(s) from 0 check(s) (0 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-99955.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2025-10-28T16:07:29Z

That worked!

reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod
cowsay/3.04/20251028_160350UTC
file/5.46-GCCcore-13.3.0/20251028_160513UTC

Removing the easystack used for testing, this should be good to go. Doesn't require a build/deploy.

casparvl

Lgtm!

dummy easystack for testing

dc1414e

bedroge marked this pull request as draft October 28, 2025 15:29

bedroge added 2 commits October 28, 2025 16:34

use 2023.06

80f751d

use version supported in 2023.06

86d41a4

use 2025.06

18f6e4a

only include the relevant reprod dirs in the tarball listing

ec0fd75

remove easystack used for testing

5efd7a2

bedroge marked this pull request as ready for review October 28, 2025 16:07

bedroge added the ready-to-review label Oct 28, 2025

restore original file

6a9ac75

casparvl approved these changes Oct 28, 2025

View reviewed changes

bedroge closed this Oct 28, 2025

bedroge reopened this Oct 28, 2025

casparvl merged commit 9cec653 into EESSI:main Oct 29, 2025
126 of 128 checks passed

bedroge deleted the fix_reprod_listing branch October 29, 2025 08:33

Fix listing of reprod dirs #117

Fix listing of reprod dirs #117

Uh oh!

Conversation

bedroge commented Oct 28, 2025

Uh oh!

bedroge commented Oct 28, 2025

Uh oh!

eessi-bot-aws bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Oct 28, 2025

Uh oh!

eessi-bot-aws bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Oct 28, 2025

Uh oh!

eessi-bot-aws bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Oct 28, 2025

Uh oh!

eessi-bot-aws bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Oct 28, 2025

Uh oh!

bedroge commented Oct 28, 2025

Uh oh!

eessi-bot-aws bot commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Oct 28, 2025

Uh oh!

casparvl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

eessi-bot-aws bot commented Oct 28, 2025 •

edited

Loading

eessi-bot-aws bot commented Oct 28, 2025 •

edited

Loading

eessi-bot-aws bot commented Oct 28, 2025 •

edited

Loading

eessi-bot-aws bot commented Oct 28, 2025 •

edited

Loading

eessi-bot-aws bot commented Oct 28, 2025 •

edited

Loading