NZSL Modules and imports

Each .nzsl file defines a module as its first statement.

Example:

[nzsl_version("1.0")]
module;

This is required as it gives two very important pieces of information to the compiler:

The NZSL version this file was written against (currently only 1.0 is available but this is expected to change in the future).
The module name, if any.

Module name must follow the same rules as regular identifiers, except they can contain dots (Engine.Constants is a valid name and has no additional meaning from Engine_Constants).

Here's an example where we are declaring an anonymous module, which is common for shaders directly given to the application/compiler, but giving it a name makes it importable in other shader sources, example:

Debug.nzsl:

[nzsl_version("1.0")]
module Debug;

[export]
fn GetDebugColor() -> vec3[f32]
{
	return vec3[f32](0.0, 1.0, 0.0);
}

Forward.nzsl:

[nzsl_version("1.0")]
module;

import GetDebugColor from Debug;

struct FragOut
{
	[location(0)] color: vec3[f32]
}

[entry(frag)]
fn main() -> FragOut
{
	let output: FragOut;
	output.color = GetDebugColor();
	return output;
}

The keys to this system are the import statement and the [export] attributes, which work together to make functions, structs, etc. usable in other files.

So, how does it work?

What's the difference between modules and #include?

Some C-like languages (such as C, C++, HLSL, GLSL with extensions, and more) uses the #include directive to import content from other files.

#include simply copy the content of a file (generally a header) inside another, recursively, and then the compiler compiles the big concatenation of all those files.
This means headers are processed (parsed and compiled) multiple times which slows down compilation. You also cannot choose what you want to import (including a file declaring 100 functions to only use one means 99 functions declarations will be parsed for nothing).

NZSL (and many modern languages) uses modules which have the following advantage:

Every module has its own scope, importing a module doesn't leak its scope into yours.
Modules needs to be parsed and compiled only once
You can choose what you want to import from a module
As each module is parsed separately, a single module doesn't have to compile with other NZSL versions than requested (see "Module versions" section)

Non-necessary statements are discarded except when required, for example:

Module:

[nzsl_version("1.0")]
module EngineStructs;

option MaxLightCount: u32;

[layout(std140)]
struct Light
{
	type: i32,
	color: vec4[f32],
	factor: vec2[f32],
	parameter1: vec4[f32],
	parameter2: vec4[f32],
	parameter3: vec4[f32],
	hasShadowMapping: u32
}

[export]
[layout(std140)]
struct LightData
{
	lights: array[Light, MaxLightCount],
	lightCount: u32,
}

struct WorldTransform
{
	transformMatrix: mat4[f32]
}

Shader:

[nzsl_version("1.0")]
module;

import LightData from Module;

// LightData identifier can be used

fn GetLightColor(data: LightData, index: u32) -> vec3[f32]
{
	return data.lights[index].color; // While we can't use the Light identifier, we can access it indirectly
}

Here LightData exists as we asked for it, Light exists as well (as it's required for LightData) but its identifier doesn't (which mean there's no Light identifier here). WorldTransform simply doesn't exist as we did not ask for it (and it's not required for LightData definition).

Import

import statements can appear anywhere in a source file and can import from 1 to N identifiers exported in another module.
Modules are exclusively referenced by the name they defined in their module directive, their file path don't play any role in the name passed to import.

Examples:

import Identifier from Module;

Imports the Identifier from the referenced module.

import Identifier as Id from Module;

Imports the Identifier from the referenced module under the name Id.

import Identifier, Identifier2 from Module;

Imports both Identifier and Identifier2 from the referenced module

import * from Module;

Imports every exported identifier from the referenced module.

Those can be combined:

import Identifier, Identifier2 as Id2, * from Module;

Imports Identifier, Identifier2 under the name Id2 and everything else under its original name from the referenced module.

Multiple import from the same module are supported as well:

import Identifier from Module;

// Identifier can be used here

import Identifier2 from Module;

// Identifier and Identifier2 can be used here

It is an error to use multiple wildcards or import the same identifier twice in the same import directive:

import *, * from Module; // CImportMultipleWildcard error: only one wildcard can be present in an import directive
import * as Y from Module; // CImportWildcardRename error: wildcard cannot be renamed
import X, X from Module; // CImportIdentifierAlreadyPresent error: X identifier was already imported

However, importing the same identifier twice in two separate import statements is supported:

import X from Module;
import X from Module; //< does not trigger an error (but does nothing)

this can be useful to import one identifier to two separate names:

import X as FirstId from Module;
import X as SecondId from Module;

// FirstId and SecondId reference the same identifier

Export

In a module, only statements decorated with [export] can be imported in another module.
The following statements can be decorated with the [export] attribute:

Const
Functions declarations
Structure declarations

In the future, it may be possible to export something under another name:

[export("Foo")]
struct Bar {}

// Bar struct is exported as Foo

How does the compiler finds modules?

Since modules are found by their name and not by their path, we must give the compiler a way to find the corresponding module based on its name.

Standalone compiler (nzslc)

All you have to do is to give the compiler a module file, or directory, which will be parsed (not compiled) in a quick process to know about it.

This is done using the -m or --module parameter:

nzslc -m file.nzsl -m modules/ --compile=spv shader.nzsl

Here, the file.nzsl module will be parsed and registered as an available module. The modules folder will be recursively explored and every .nzsl (textual NZSL) and .nzslb (binary NZSL) file will be registered.

All specified modules are registered before the file compilation begins.

NZSL Library

The library gives you more freedom about how modules are resolved. All you have to do is to fill the shaderModuleResolver field of the nzsl::ShaderWriter::States parameter of a generator with the instance of a class derived from nzsl::ModuleResolver.

namespace nzsl
{
	class ModuleResolver
	{
		public:
...
			virtual Ast::ModulePtr Resolve(const std::string& moduleName) = 0;

			NazaraSignal(OnModuleUpdated, ModuleResolver* /*resolver*/, const std::string& /*moduleName*/);
	};
}

The Resolve methods will be called once for every import statement found in the shader and the modules it uses, and has to return the nzsl::Ast::ModulePtr (an alias of std::shared_ptr<nzsl::Ast::Module>) of the corresponding module (or an empty pointer if the module couldn't be found).
The returned module can be at any compilation step (just parsed or already processed).

The easiest way to use modules with the NZSL library is to rely on the nzsl::FilesystemModuleResolver class, which implements module registering (from memory, files and directories) and can be used multiple times.

#include <NZSL/FilesystemModuleResolver.hpp>
#include <NZSL/Parser.hpp>
#include <NZSL/SpirvWriter.hpp>

int main()
{
	nzsl::Ast::ModulePtr shaderAst = nzsl::ParseFromFile("pbr.nzsl");

	std::shared_ptr<nzsl::FilesystemModuleResolver> resolver = std::make_shared<nzsl::FilesystemModuleResolver>();
	resolver->RegisterModule("Module.nzsl");
	resolver->RegisterModuleDirectory("ModuleDirectory");

	nzsl::ShaderWriter::States states;
	states.shaderModuleResolver = resolver;

	nzsl::SpirvWriter spirvWriter;
	std::vector<std::uint32_t> spirv = spirvWriter.Generate(shaderAst, states);
	// Give SPIR-V to Vulkan
}

Additionally, if the library was compiled with efsw support (which is the case by default on desktop systems), you can set the second parameter of the RegisterModuleDirectory method to true to enable filesystem watching.
This will make FilesystemModuleResolver watch the folder for changes, which means it will reparse modules as soon as their files change on the filesystem (which will trigger the OnModuleUpdated signal). This can be used to support hotreloading of shaders.

Module versions

The reason every module has to be associated with a version is to give the language the capability to evolve and break things without losing the ecosystem, in a way similar to Rust Editions.

For example, if in the future we were to decide that NZSL 2.0 uses the a C++-like syntax for functions declarations and inner types, this won't split the ecosystem.

Example:

Module:

// This is just an example, NZSL 2.0 won't be as ugly as this

[[nzsl_version("2.0")]]
module("NextGenModule");

[[public]] // C++ attribute syntax, public replacing export
Vector<3, float> GetColor() // C-like function definition and subtypes
{
	return Vector<3, float>{ 1.0, 0.0, 1.0 }; //< bracket initialization
}

Shader:

// Good old NZSL 1.0

[nzsl_version("1.0")
module;

import GetColor from NextGenModule;

struct FragOut
{
	[location(0)] color: vec3[f32]
}

[entry(frag)]
fn main() -> FragOut
{
	let output: FragOut;
	output.color = GetColor(); // no problem, vec3[f32] in this module and Vector<3, float> in NextGenModule are recognized as the same type
	return output;
}

This is made possible by the usage of a NZSL 2.0 compiler, as it will be able to recognize both NZSL 1.0 and NZSL 2.0 and do what's needed to make them work together.
This way, a NZSL 2.0 shader can still use NZSL 1.0 modules, and vice versa.

Of course this only works as long as common features are used, if NZSL 2.0 introduces a new type or statement that cannot be represented with NZSL 1.0, using it directly wouldn't be possible.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly