Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cargo gpu build fails with "missing field dylib_path" #52

Closed
trival opened this issue Feb 28, 2025 · 6 comments
Closed

cargo gpu build fails with "missing field dylib_path" #52

trival opened this issue Feb 28, 2025 · 6 comments

Comments

@trival
Copy link

trival commented Feb 28, 2025

After updating cargo gpu to commit 95efce2 or any more recent version, my shaders stop compiling.

The last working version for me is 70224f6

The error message is:

🦀 Running `spirv-builder-cli` to compile shader at /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader...
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("missing field `dylib_path`", line: 31, column: 1)', src/main.rs:52:47

I am on Ubuntu 24.04, using a regular rustup setup.

Running RUST_BACKTRACE=1 RUST_LOG=trace cargo gpu build results in:

[2025-02-28T10:33:54Z TRACE cargo_gpu] CLI args: [
        "/home/trival/.cargo/bin/cargo-gpu",
        "build",
    ]
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Querying Cargo metadata for "./Cargo.toml"
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/drafts/ball/Cargo.toml?
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/trivalibs/crates/trivalibs/Cargo.toml?
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/trivalibs/crates/trivalibs_core/Cargo.toml?
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/trivalibs/crates/trivalibs_macros/Cargo.toml?
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/trivalibs/crates/trivalibs_painter/Cargo.toml?
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/Cargo.toml?
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/drafts/ball/shader/Cargo.toml?
[2025-02-28T10:33:54Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml?
[2025-02-28T10:33:54Z DEBUG cargo_gpu] building with final merged arguments: Build {
        install: Install {
            spirv_install: InstallArgs {
                dylib_path: "INTERNALLY_SET",
                shader_crate: "./",
                spirv_builder_source: None,
                spirv_builder_version: None,
                rust_toolchain: None,
                force_spirv_cli_rebuild: false,
                auto_install_rust_toolchain: false,
            },
        },
        build_args: BuildArgs {
            output_dir: "./",
            no_default_features: false,
            features: [],
            target: "spirv-unknown-vulkan1.2",
            shader_target: "spirv-unknown-vulkan1.2",
            deny_warnings: false,
            debug: false,
            capability: [],
            extension: [],
            multimodule: false,
            spirv_metadata: None,
            relax_struct_store: false,
            relax_logical_pointer: false,
            relax_block_layout: false,
            uniform_buffer_standard_layout: false,
            scalar_block_layout: false,
            skip_block_layout: false,
            preserve_bindings: false,
        },
    }
[2025-02-28T10:33:54Z INFO  cargo_gpu::install] cache directory is '/home/trival/.cache/rust-gpu'
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_source] Running `cargo tree` on /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader
[2025-02-28T10:33:54Z TRACE cargo_gpu::spirv_source]   found Some("spirv-std v0.9.0")
[2025-02-28T10:33:54Z TRACE cargo_gpu::spirv_source] parsing spirv-std source and version from def: 'spirv-std v0.9.0'
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_source] Parsed `rust-gpu` source and version: CratesIO("v0.9.0")
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_source] Not cloning `rust-gpu` repo (https://github.com/Rust-GPU/rust-gpu) as it already exists at /home/trival/.cache/rust-gpu/rust-gpu-repo/v0_9_0
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_source] Checking out `rust-gpu` repo at /home/trival/.cache/rust-gpu/rust-gpu-repo/v0_9_0 to v0.9.0
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_source] Getting `rust-gpu` version date from /home/trival/.cache/rust-gpu/rust-gpu-repo/v0_9_0
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_source] Parsed date for version v0.9.0: 2023-07-25
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_source] Parsing `rust-toolchain.toml` at "/home/trival/.cache/rust-gpu/rust-gpu-repo/v0_9_0" for the used toolchain
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_source] Parsed version, date and toolchain channel from shader-defined `rust-gpu`: CratesIO("v0.9.0"), 2023-07-25, nightly-2023-05-27
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_cli] toolchain nightly-2023-05-27 is already installed
[2025-02-28T10:33:54Z DEBUG cargo_gpu::spirv_cli] all required components are installed
[2025-02-28T10:33:54Z INFO  cargo_gpu::install] cargo-gpu artifacts are already installed in '/home/trival/.cache/rust-gpu/spirv-builder-cli/v0_9_0+nightly-2023-05-27'
[2025-02-28T10:33:54Z INFO  cargo_gpu::install] ...and so we are aborting the install step.
[2025-02-28T10:33:54Z DEBUG cargo_gpu::build] ensuring output-dir './' exists
[2025-02-28T10:33:54Z DEBUG cargo_gpu::build] canonicalized output dir: "/home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader"
[2025-02-28T10:33:54Z INFO  cargo_gpu::build] using spirv-builder-cli arg: {
      "build": {
        "capability": [],
        "debug": false,
        "deny_warnings": false,
        "extension": [],
        "features": [],
        "multimodule": false,
        "no_default_features": false,
        "output_dir": "/home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader",
        "preserve_bindings": false,
        "relax_block_layout": false,
        "relax_logical_pointer": false,
        "relax_struct_store": false,
        "scalar_block_layout": false,
        "shader_target": "/home/trival/.cache/rust-gpu/target-specs/spirv-unknown-vulkan1.2.json",
        "skip_block_layout": false,
        "spirv_metadata": "None",
        "target": "spirv-unknown-vulkan1.2",
        "uniform_buffer_standard_layout": false
      },
      "install": {
        "auto_install_rust_toolchain": false,
        "dylib_path": "/home/trival/.cache/rust-gpu/spirv-builder-cli/v0_9_0+nightly-2023-05-27/librustc_codegen_spirv.so",
        "force_spirv_cli_rebuild": false,
        "rust_toolchain": null,
        "shader_crate": "/home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader",
        "spirv_builder_source": null,
        "spirv_builder_version": null
      }
    }
🦀 Running `spirv-builder-cli` to compile shader at /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader...
[2025-02-28T10:33:54Z TRACE spirv_builder_cli] setting RUSTUP_TOOLCHAIN = 'nightly-2023-05-27'
[2025-02-28T10:33:54Z DEBUG spirv_builder_cli] running spirv-builder-cli from '/home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader'
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("missing field `dylib_path`", line: 31, column: 1)', src/main.rs:52:47
stack backtrace:
   0: rust_begin_unwind
             at /rustc/1a5f8bce74ee432f7cc3aa131bc3d6920e06de10/library/std/src/panicking.rs:578:5
   1: core::panicking::panic_fmt
             at /rustc/1a5f8bce74ee432f7cc3aa131bc3d6920e06de10/library/core/src/panicking.rs:67:14
   2: core::result::unwrap_failed
             at /rustc/1a5f8bce74ee432f7cc3aa131bc3d6920e06de10/library/core/src/result.rs:1651:5
   3: spirv_builder_cli::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[2025-02-28T10:33:54Z ERROR cargo_gpu] build failed
    
    Stack backtrace:
       0: anyhow::error::<impl anyhow::Error>::msg
       1: cargo_gpu::build::Build::run
       2: cargo_gpu::main
       3: std::sys::backtrace::__rust_begin_short_backtrace
       4: std::rt::lang_start::{{closure}}
       5: std::rt::lang_start_internal
       6: main
       7: __libc_start_call_main
                 at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
       8: __libc_start_main_impl
                 at ./csu/../csu/libc-start.c:360:3
       9: _start
Error: build failed

This version of cargo-gpu is also the first one where cargo gpu install --auto-install-rust-toolchain --force-spirv-cli-rebuild fails with:

error: package `clap_derive v4.5.28` cannot be built because it requires rustc 1.74 or newer, while the currently active rustc version is 1.71.0-nightly
Either upgrade to rustc 1.74 or newer, or use

This is probably the same as #45

Unfortunately, #41 doesn't fix the issue for me. Both cargo gpu build and cargo gpu install fail with error No such file or directory (os error 2):

[2025-02-28T10:19:32Z TRACE cargo_gpu] CLI args: [
        "/home/trival/.cargo/bin/cargo-gpu",
        "build",
    ]
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Querying Cargo metadata for "./Cargo.toml"
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/drafts/ball/Cargo.toml?
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/trivalibs/crates/trivalibs/Cargo.toml?
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/trivalibs/crates/trivalibs_core/Cargo.toml?
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/trivalibs/crates/trivalibs_macros/Cargo.toml?
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/trivalibs/crates/trivalibs_painter/Cargo.toml?
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/Cargo.toml?
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/drafts/ball/shader/Cargo.toml?
[2025-02-28T10:19:32Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml == /home/trival/code/personal/rust/graphics/rust-graphics/drafts/textures/shader/Cargo.toml?
[2025-02-28T10:19:32Z DEBUG cargo_gpu] building with final merged arguments: Build {
        install: Install {
            spirv_install: InstallArgs {
                dylib_path: "INTERNALLY_SET",
                shader_crate: "./",
                spirv_builder_source: None,
                spirv_builder_version: None,
                rust_toolchain: None,
                force_spirv_cli_rebuild: false,
                auto_install_rust_toolchain: false,
                force_overwrite_lockfiles_v4_to_v3: false,
            },
        },
        build_args: BuildArgs {
            output_dir: "./",
            watch: false,
            no_default_features: false,
            features: [],
            target: "spirv-unknown-vulkan1.2",
            shader_target: "spirv-unknown-vulkan1.2",
            deny_warnings: false,
            debug: false,
            capability: [],
            extension: [],
            multimodule: false,
            spirv_metadata: None,
            relax_struct_store: false,
            relax_logical_pointer: false,
            relax_block_layout: false,
            uniform_buffer_standard_layout: false,
            scalar_block_layout: false,
            skip_block_layout: false,
            preserve_bindings: false,
        },
    }
[2025-02-28T10:19:32Z INFO  cargo_gpu::install] cache directory is '/home/trival/.cache/rust-gpu'
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_cli] Ensuring no v3/v4 `Cargo.lock` conflicts from workspace Rust...
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_cli] user's Rust is v1.87.0, so no v3/v4 conflicts possible.
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_source] Running `cargo tree` on ./
[2025-02-28T10:19:32Z TRACE cargo_gpu::spirv_source]   found Some("spirv-std v0.9.0")
[2025-02-28T10:19:32Z TRACE cargo_gpu::spirv_source] parsing spirv-std source and version from def: 'spirv-std v0.9.0'
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_source] Parsed `rust-gpu` source and version: CratesIO("v0.9.0")
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_source] Not cloning `rust-gpu` repo (https://github.com/Rust-GPU/rust-gpu) as it already exists at /home/trival/.cache/rust-gpu/rust-gpu-repo/v0_9_0
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_source] Checking out `rust-gpu` repo at /home/trival/.cache/rust-gpu/rust-gpu-repo/v0_9_0 to v0.9.0
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_source] Getting `rust-gpu` version date from /home/trival/.cache/rust-gpu/rust-gpu-repo/v0_9_0
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_source] Parsed date for version v0.9.0: 2023-07-25
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_source] Parsing `rust-toolchain.toml` at "/home/trival/.cache/rust-gpu/rust-gpu-repo/v0_9_0" for the used toolchain
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_source] Parsed version, date and toolchain channel from shader-defined `rust-gpu`: CratesIO("v0.9.0"), 2023-07-25, nightly-2023-05-27
[2025-02-28T10:19:32Z DEBUG cargo_gpu::spirv_cli] Ensuring no v3/v4 `Cargo.lock` conflicts from shader's Rust...
[2025-02-28T10:19:33Z DEBUG cargo_gpu::spirv_cli] shader's Rust is v1.71.0, so checking both shader and workspace `Cargo.lock` manifest versions...
[2025-02-28T10:19:33Z ERROR cargo_gpu] No such file or directory (os error 2)
    
    Stack backtrace:
       0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
       1: cargo_gpu::spirv_cli::SpirvCli::handle_conflicting_cargo_lock_v4
       2: cargo_gpu::spirv_cli::SpirvCli::new
       3: cargo_gpu::install::Install::spirv_cli
       4: cargo_gpu::install::Install::run
       5: cargo_gpu::build::Build::run
       6: cargo_gpu::main
       7: std::sys::backtrace::__rust_begin_short_backtrace
       8: std::rt::lang_start::{{closure}}
       9: std::rt::lang_start_internal
      10: main
      11: __libc_start_call_main
                 at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
      12: __libc_start_main_impl
                 at ./csu/../csu/libc-start.c:360:3
      13: _start
Error: No such file or directory (os error 2)

I'm not sure if the issue is a duplicate, since the initial error message is different than in other reported issues.
@tombh
Copy link
Collaborator

tombh commented Mar 1, 2025

This is very likely a bug in the PR, thanks for trying it out. When I wrote the PR I was testing against a shader crate that lives in a workspace, but is actually excluded, which it has its own Cargo.lock (as well as there being a Cargo.lock in the containing workspace). I suspect your shader lives in a workspace, but isn't excluded, so doesn't have its own Cargo.lock.

So I've pushed a fix to #41 where it only checks the shader's Cargo.lock if it has one. It was force pushed, so a simple git pull won't be enough to get the change, perhaps a git pull --force will do it.

@trival
Copy link
Author

trival commented Mar 2, 2025

Thank you. Yes that was the case, the shader crate is not excluded from the workspace. I have tried your recent additions and can confirm that it works now.

Interestingly: Checking out the repo and running cargo run --bin cargo-gpu -- build --shader-crate /path/to/your/shader/crate/root --force-spirv-cli-rebuild did not work because of cargo version 3/4 issue. It did override the workspace cargo.lock and then failed.

But installing cargo-gpu using cargo install --git https://github.com/tombh/cargo-gpu.git cargo-gpu --rev edb706a1063a75466c4a6da3b7d9eb58e3a736a8 and then normally running cargo gpu build --force-spirv-cli-rebuild from the shader crate root dir works fine for me.

So the issue is closed by #41

@trival trival closed this as completed Mar 2, 2025
@tombh
Copy link
Collaborator

tombh commented Mar 2, 2025

Great that it works. Thanks for you help.

That's a curious wrinkle that the cargo run --bin method didn't work. It could be something to do with the --shader-crate path, I'll bare it in mind.

Just a quick question, did the cargo gpu method correctly detect and handle the "cargo version 3/4" issue?

@trival
Copy link
Author

trival commented Mar 2, 2025

both cargo run --bin cargo-gpu from the checked out repo, as well as cargo gpu build using the PR commit fail if the workspace Cargo.lock uses version = 4 with:

🦀 Running `spirv-builder-cli` to compile shader at /home/trival/code/rust-graphics/drafts/textures/shader...
error: failed to parse lock file at: /home/trival/code/rust-graphics/Cargo.lock

Caused by:
  lock file version `4` was found, but this version of Cargo does not understand this lock file, perhaps Cargo needs to be updated?
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: BuildFailed', src/main.rs:145:38
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[2025-03-02T15:51:31Z ERROR cargo_gpu] build failed

This happens even if I add the --force-overwrite-lockfiles-v4-to-v3, which doesn't seem to have any effect in my setup.

If I change the version in the Cargo.lock manually to 3, cargo gpu build in the shader crate respects it and leaves it untouched, but running from the repo overrides it with 4 again before compiling the shader, and then emits above error.

I am not sure how the cargo version issue could be handled otherwise.

@trival
Copy link
Author

trival commented Mar 2, 2025

Actually there is a special case: Running from the repo without the --force-overwrite-lockfiles-v4-to-v3 shows an informative error message, advising to use that option. But using cargo gpu does not show this info, only the failed to parse lock file error.

@tombh
Copy link
Collaborator

tombh commented Mar 9, 2025

Okay, I'm not quite sure what's going on. The main thing is that you're able to recover and compile the shader. So let's merge the PR anyway and then we should have a more consistent base to come up with reproducible steps for the issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tombh @trival