Expand SPIR-V features

[DialecticalMaterialist]

Currently this adds support for every SPIR-V Extension and Capability. You enable them by setting which ones you want in a featureSet then assigning it to the cpu_features_add in your target. There is also now support for setting some Execution Modes via the calling convention. Following is an example setting the SPIR-V version to 1.6, enabling the SPV_EXT_mesh_shader extension and enabling the MeshShadingEXT capability it also adds the required Execution Modes via the calling convention.

build.zig

const std = @import("std");
pub fn build(b: *std.Build) void {
    const spirv_target = b.resolveTargetQuery(.{
        .cpu_arch = .spirv64,
        .cpu_model = .{ .explicit = &std.Target.spirv.cpu.generic },
        .cpu_features_add = std.Target.spirv.featureSet(&[_]std.Target.spirv.Feature{ .mesh_shading_ext, .v1_6 }),
        .os_tag = .opengl,
        .ofmt = .spirv,
        .abi = .none,
    });

    const shader = b.addObject(.{
        .name = "shader",
        .root_module = b.createModule(.{
            .root_source_file = b.path("shader.zig"),
            .target = spirv_target,
            .optimize = .ReleaseFast,
        }),
        .use_llvm = false,
    });

    b.getInstallStep().dependOn(&b.addInstallFile(shader.getEmittedBin(), "../shader.spv").step);
}

shader.zig

export fn main_mesh() callconv(.{ .spirv_mesh = .{
    .stage_output = .output_triangles,
    .max_primitives = 10,
    .max_vertices = 30,
} }) void {}

shader.spv

; SPIR-V
; Version: 1.6
; Generator: Khronos; 41
; Bound: 34
; Schema: 0
               OpCapability Shader
               OpCapability Matrix
               OpCapability Int64
               OpCapability MeshShadingEXT
               OpCapability Int8
               OpCapability Int16
               OpExtension "SPV_EXT_mesh_shader"
               OpMemoryModel Logical GLSL450
               OpEntryPoint MeshEXT %25 "main_mesh"
               OpExecutionMode %25 OutputVertices 30
               OpExecutionMode %25 OutputPrimitivesEXT 10
               OpExecutionMode %25 OutputTrianglesEXT
               OpSourceExtension "zig_errors:"
               OpSource Zig 0
               OpName %void "void"
               OpName %shader_main_mesh "shader.main_mesh"
       %void = OpTypeVoid
         %26 = OpTypeFunction %void
%shader_main_mesh = OpFunction %void None %26
          %4 = OpLabel
               OpReturn
          %5 = OpLabel
               OpUnreachable
               OpFunctionEnd
         %25 = OpFunction %void None %26
         %32 = OpLabel
         %33 = OpFunctionCall %void %shader_main_mesh
               OpReturn
               OpFunctionEnd
               

As we can see the version is correctly set, the extension is there and the capability as well as the required Execution Modes.

[alichraghi]

I strongly believe that we should not support OpCapability/OpExtension instructions in assembler and instead populate std.Target.spirv.Feature.

CC @Snektron

[DialecticalMaterialist]

That would require adding them to src/codegen/spirv/Module.zig correct? the finalize function would probably need some restructuring or else it would be filled with a bunch of if statements.

[alichraghi]

Yes but please wait until #24661 is merged before starting work on this.

[DialecticalMaterialist]

Ah my bad but I not start anything big anyways. Is it okay to start working on update_cpu_features.zig?

[alichraghi]

Sure, feel free to work on whatever you like.
I just didn't want you to re-do work if my PR got merged sooner.

[DialecticalMaterialist]

I have now added all capabilities in std.Target.spirv.

One important change is the renaming of .arbitrary_precision_integers to .arbitrary_precision_integers_intel. It already had this name in the spirv spec zig file but not in std. I renamed to get consistency with all the new capabilities.

Also I'm pretty sure I added all the capabilities at least all from https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#_capability. Even the reserved ones, the ones which didn't have their SPIR-V version target stated there I used the version that the extension who implemented the capability required so might not be fully correct? I also stated in the description if the capability should enable an Extension but I did not state which other capabilities it implicitly used this went against how it was implemented before with .variable_pointers but maybe someone else can add it.

[DialecticalMaterialist]

Ah wait I dumb. I now see that https://raw.githubusercontent.com/KhronosGroup/SPIRV-Headers/refs/heads/main/include/spirv/unified1/spirv.core.grammar.json contains the capabilities with their extensions. Okay then that can be automated with gen_spirv_spec and update_cpu_features

[alichraghi]

Yes. Note that i reduced the instructions sets to only core, opencl and glsl so don't bother supporting others.

[DialecticalMaterialist]

Yes. Note that i reduced the instructions sets to only core, opencl and glsl so don't bother supporting others.

What do you mean by this?

[alichraghi]

cd4b03c#diff-c4a2f6808858b44b22c5ea1b6552c3b350020a84e141746bbb0cedbea4dd4f3d

[DialecticalMaterialist]

That is on your spv4 branch yes? I'm going to add modify gen_spirv_spec soon to generate Extensions and Capabilities which has version and requirements, should I rebase on your fork?

[alichraghi]

Yes. Thank you!

[DialecticalMaterialist]

Would I need to move this pr to your fork then?

[DialecticalMaterialist]

I'll continue working on this then, I can rebase once your is merged

[DialecticalMaterialist]

@alichraghi would you know how we can access spec.zig in update_cpu_features.zig?

[alichraghi]

Hmm that would be tricky because we now have to pass --dep which is not desirable so we probably have no other choice but adding a step like update-cpu-features to build.zig.

CC @alexrp

TLDR; We need to import src/codegen/spirv/spec.zig in update_cpu_features.zig to auto-generate SPIR-V CPU features

[DialecticalMaterialist]

How would one do it with --dep? So that I can continue working on update_cpu_features.zig will waiting for proper way.

[alichraghi]

zig run --dep spirv_spec -Mroot=tools/update_cpu_features.zig -Mspirv_spec=src/codegen/spirv/spec.zig

// In update_cpu_features.zig
const spirv_spec = @import("spirv_spec");

[DialecticalMaterialist]

Thanks, good news is that this is only thing left for it work. I compiled latest commit and it works, it emits correct OpCapability and OpExtension, only stuff that don't work are some capabilities that I did not define in spirv.Feature, from the first commits. But now when update_cpu_features works it should be done

[DialecticalMaterialist]

It should be done, I have yet to test since I'm waiting on compilation. I had to change the design a little bit since there were some limitations with CpuFeatures. But let me know what you think of it now. All Extensions and Capabilities are cpu features and if you add a Capability that requires an Extension or specific version that will be forwarded to codegen. Sadly we can't get dependencies for Extensions since they aren't even in the json file we use for spirv spec, I get them by checking all the dependencies of the capabilities in the spec lol. So using an Extension that requires another one(Don't even know if that exists) it will not add the other one.

Should I revert the changes in std.gpu and open this again and do another pr for adding stuff to std.gpu or should i keep this as draft?

[DialecticalMaterialist]

From my testing everything seems to be working, only problem is if you try to compile with allfeatures in your extra cpu features. Then it will cause index out of bound in Target.zig since it's not supposed to handle that many features, but I can't really see when you would do this.

[alichraghi]

Should I revert the changes in std.gpu and open this again and do another pr for adding stuff to std.gpu or should i keep this as draft?

I don't see the need for another PR as long as you can split all these commits into two auto-generate SPIR-V features and inline std.gpu functions. But even if you can't do that, the changes on latter is small enough to be hidden in a squash merge.

Then it will cause index out of bound in Target.zig

This won't be a problem you once my PR is merged since the unnecessary sets are removed.

[DialecticalMaterialist]

This won't be a problem you once my PR is merged since the unnecessary sets are removed.

Good to know.

There's still the problem of how we should deal with update_cpu_features.zig and its dependency on spec.zig, since that is causing the ci to fail

[DialecticalMaterialist]

@alichraghi do you know why openGL does not define bit sizes of C types in std.Target and instead just panics? Should it not have the same as mesa3d?

[alichraghi]

Likely because nobody added them and I have no clue what the correct values would be.

[alichraghi]

Yes it's because .spirv_device is the equivalent of .c calling convention and there's likely a behavior test that uses it. Replace unrechables with continue until we actually fix it later.

[DialecticalMaterialist]

@alichraghi you know why this still isn't merged?

[alexrp]

Since this touches on std.Target, I would like to take a look at it. I'll try to get to it soon, but I'm currently clearing my backlog for 0.15.0.

[DialecticalMaterialist]

Do you know how I can reply to your reviews without having to do this?

[alexrp]

Do you know how I can reply to your reviews without having to do this?

Just make sure that you don't have an in-progress review, since as you've noticed, GitHub will delay posting your comments until you submit it. If you have in-progress review, you can abandon it to get back to a state where you can just post immediate comments.

[DialecticalMaterialist]

Should I resolve the bit size thing?

[alexrp]

No more comments from me, so I'll hand this off to @alichraghi.

[alichraghi]

LGTM! Nice work! I think Sema/InternPool changes need to be reviewed by @mlugg tho.