pre-release v23.9.186

AE/CMakeLists.txt

@@ -3,7 +3,7 @@
 cmake_minimum_required( VERSION 3.10 FATAL_ERROR )
 
 project( "AE"
-         VERSION 23.9.185   # year, month, version
+         VERSION 23.9.186   # year, month, version
          LANGUAGES C CXX )
 if (APPLE)
     enable_language(OBJC)

AE/LICENSE.md

@@ -18,7 +18,7 @@ Compilation will fail if the required license is not compatible with third-party
 **Warning**: In cmake use only PUBLIC visibility for `target_link_libraries()`, otherwise it breaks license detection.
 
 **Warning**: You must manually check the shader source code for the presence of third-party dependencies with different licenses.
-Detecting license for shader code is not implemented yet. Most third-party shaders are located in `engine/shared_data/3party_shaders` folder.<br/>
+Detecting license for shader code is not implemented yet. Most third-party shaders are located in `AE/engine/shared_data/3party_shaders` folder.<br/>
 Some shader code distributed under 'free for non-commercial use' license or doesn't have any license information.
 
 

AE/engine/Changelog.md

@@ -1,5 +1,7 @@
 
-## 23.09.xxx
+## 23.09.186
+
+ - WinAPI: fixed incorrect render area when switching to/from HDR mode.
 
 
 ## 23.08.183

AE/engine/docs/DeviceProperties.md

@@ -1,9 +1,14 @@
 source: [DeviceProperties.h](../src/graphics/Public/DeviceProperties.h)
 
+Use `DeviceProperties const&  RenderTaskScheduler().GetDeviceProperties()` to get runtime limits for the current GPU.
+
+Use `static constexpr DeviceProperties  DeviceLimits;` to get compile time limits which is compatible with most GPUs.
+
+
 ## ResourceAlignment
 
-This constants is used to calculate data alignment for uniform, storage and vertex buffers.
-Constants used in C++ code to store data for GPU usage.
+These constants are used to calculate data alignment for uniform, storage and vertex buffers.
+Constants are used in C++ code to store data for accessing them in the GPU.
 
 * `minUniformBufferOffsetAlign`</br>
 `minStorageBufferOffsetAlign`</br>

AE/engine/docs/FeatureSet.md

@@ -2,6 +2,10 @@ source: [FeatureSet.h](../src/graphics/Public/FeatureSet.h)
 
 Some kind of [Vulkan Profiles](https://github.com/KhronosGroup/Vulkan-Profiles) and [Metal Feature Set Tables](https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf).
 
+Use `FeatureSet` to get GPU limits and features in resource compilation stage.
+
+Use `FeatureSet const&  RenderTaskScheduler().GetFeatureSet()` to get cross-platform limits and features of the current GPU in runtime.
+
 ## Render States
 
 * __alphaToOne__<br/>

AE/engine/docs/GraphicsResources.md

@@ -0,0 +1,33 @@
+
+## Graphics IDs
+
+ID is a [trivial type](https://learn.microsoft.com/en-us/cpp/cpp/trivial-standard-layout-and-pod-types?view=msvc-170) which contains index in pool and generation counter.<br/>
+Source code: [HandleTmpl](../src/base/Utils/HandleTmpl.h), [Graphics IDs](../src/graphics/Public/IDs.h).
+
+Supports strong and weak references:<br/>
+`BufferID` - weak reference.<br/>
+`Strong<BufferID>` - strong reference.<br/>
+Conversion from strong ref to weak ref has zero cost.<br/>
+Weak reference can be converted to the strong ref by using `IResourceManager::AcquireResource()`.
+
+### IDs in command context
+
+Inside command context resource internal data will be retrieved by `ResourceManager::GetResourcesOrThrow()` which throws exception if ID is not valid - resource is deleted.
+
+### IDs in ResourceManager
+
+Exceptions are not used here. Error will be generated if ID is not valid and method will return `false`.
+
+
+## ResourceManager
+
+### Resource creation
+
+`Create*()` methods doesn't check if resource description is supported by the GPU. Vulkan implementation may create resource even if it is not supported by the current GPU.
+
+If the engine is compiled with `AE_GRAPHICS_STRONG_VALIDATION=ON` then `Create*()` methods will use the `IsSupported()` method to check whether the resource description is supported by the GPU. If the description is not supported, the resource will not be created and an error will be generated.
+
+### Resource description validation
+
+Use `IsSupported()` method before creating new resource to check if the resource description is supported by the GPU.
+

AE/engine/docs/HDR_Display.md

@@ -2,18 +2,18 @@
 
 __Non-HDR (RGBA8 / RGB10A2 formats)__
 
-* Color values above 1.0 is clamped to 1.0.
+* Color values above 1.0 are clamped to 1.0.
 
 __HDR (RGBA16F format)__
 
-* Display uses internal tonemapping to convert color values above 1.0 to another color, more often, the color tends to white.
-* Transition to white color depends on luminance, green is the brightest and turns white earlier.
-* There is no way to programmatically find out the characteristics of the built-in tone mapping, so it's better to use your own, and adjust the brightness of the screen with values greater than 1.0.
+* Display uses internal tonemapping to convert color values above 1.0 to another color, more often the color tends to white.
+* The transition to white depends on luminance, green is the brightest and turns white earlier.
+* There is no way to programmatically find out the characteristics of the built-in tone mapping, so it's better to use your own and adjust the brightness of the screen with values exceeding 1.0.
 
 
 ## Monitor
 
-Samsung monitor with VA matrix with 1000 nit brightness. Surface with `RGBA16F_Extended_sRGB_linear` mode.
+Samsung monitor with VA matrix with a brightness of 1000 nit. Surface with `RGBA16F_Extended_sRGB_linear` mode.
 
 * An analog of ACES tonemapping is used, so blue turns into pink, and only then into white.
 
@@ -22,7 +22,7 @@ Samsung monitor with VA matrix with 1000 nit brightness. Surface with `RGBA16F_E
 
 ## Smartphone
 
-ASUS smartphone with AMOLED screen with 800 nit brightness. Surface with `RGBA16F_Extended_sRGB_linear` mode.
+ASUS smartphone with AMOLED screen with a brightness of 800 nit. Surface with `RGBA16F_Extended_sRGB_linear` mode.
 
 * Color value range 0..100 on monitor is approximately the same as range 0..24 on the smartphone.
 * Smartphone uses much simpler tonemapping which doesn't change the blue color.

AE/engine/docs/Profiling.md

@@ -1,30 +1,30 @@
-Builtin and external profiling & debugging tools.
+Built-in and external profiling & debugging tools.
 
 ## Task profiling
 
-Measure task begin/end time and draw graph.
+The profiler measures the start and end time of the task and builds a diagram for two frames.
 
 ![](img/TaskProfiler.jpg)
 
 
 ## Graphics profiling
 
-Measure pass begin/end time and draw graph.
+The profiler measures the start and end time of a render pass or a group of commands (compute, ray tracing, transfer), then builds a graph.
 
 ![](img/GraphicsProfiler.jpg)
 
 
 ## HW Counters
 
-Used hardware performance counters for ARM platform (Mali and Adreno).
+Used hardware performance counters for the ARM platform (Mali and Adreno).
 
 ![](img/ARM-HWCounters.jpg)
 
 
 ## Shader trace
 
 <details>
-<summary>Example of shader trace</summary>
+<summary>Example of shader trace (shader debugger)</summary>
 
 ```cpp
 //> gl_GlobalInvocationID: uint3 {8, 8, 0}
@@ -106,6 +106,7 @@ no source
 #### RenderDoc
 
  * Graphics debugging
+ * Shader debugging (requires `EShaderOpt::DebugInfo`)
  * Don't use for profiling!
  * [RenderDocApi](../src/platform/Utils/RenderDocApi.h) class for interaction
  * `IBaseContext::DebugMarker()`, `IBaseContext::PushDebugGroup()`, `IBaseContext::PopDebugGroup()` methods for interaction
@@ -123,6 +124,6 @@ no source
 #### Vulkan validation layers
 
  * Vulkan debugging
- * Synchronizations debugging
- * [EDeviceValidation](../src/graphics/Public/GraphicsCreateInfo.h) flags for interaction
+ * [Synchronizations debugging](https://www.lunarg.com/wp-content/uploads/2020/09/Final_LunarG_Guide_to_Vulkan-Synchronization_Validation_08_20.pdf)
+ * [EDeviceValidation](../src/graphics/Public/GraphicsCreateInfo.h#L17) flags for interaction
 

AE/engine/docs/ResourceCompilation.md

@@ -0,0 +1,187 @@
+# Resource compilation
+
+Engine specific:
+* All shaders must be precompiled and validated.
+* Engine uses only binary data, scripts and configs are allowed only at the resource compilation stage.
+* All graphics resources must have at least one [FeatureSet](FeatureSet.md), which is used for validation.
+* Engine guarantees that if a resource is compiled with FeatureSet and this FeatureSet is supported by the GPU then the resource is compatible with the GPU. 
+
+
+# Pipeline Compiler
+
+Some graphics resources such a shader binaries, pipeline, render pass, descriptor set layout and sampler descriptions are stored in the PipelinePack.<br/>
+Pipeline compiler converts these description from multiple scripts to a single binary file.
+
+Documentation for each script function is auto-generated and stored in '[pipeline_compiler.as](../shared_data/scripts/pipeline_compiler.as)', you can include this file in your script.
+
+**PipelineCompiler** is a standalone dynamic library with a single function:
+```
+extern "C" bool CompilePipelines (const PipelinesInfo *info);
+```
+
+## Render Pass
+
+Render pass description is divided on CompatibleRenderPass and RenderPass.
+
+**CompatibleRenderPass** - contains full render pass description but allows to change some states which doesn't break the render pass compatibility according to the [Vulkan spec](https://registry.khronos.org/vulkan/specs/1.3-extensions/html/vkspec.html#renderpass-compatibility)
+
+**RenderPass** - is a specialization of the CompatibleRenderPass which overrides small number of parameters:
+* Attachment layout (initial, final, in all subpasses)
+* Attachment load and store operations.
+
+## RenderTechnique
+
+* Contains graphics and compute pass sequence which should be executed in these order.
+* Contains render passes which is linked to the graphics passes.
+* Contains pipeline set for passes, they will be loaded immediately when RenderTechnique is loaded and destroyed when RenderTechnique is destroyed.
+* Contains relative resources: descriptor set layouts, pipeline layouts, samplers and other, which is used in render passes and pipelines.
+
+Note: In single PipelinePack pipelines will be shared across the RenderTechniques to avoid duplication and to minimize pipeline compilation time. But you should request pipeline for the specific RenderTechnique instead of using the same pipeline for multiple RenderTechniques
+
+### GraphicsPass
+
+* Used to execute single subpass of the render pass.
+* Sequence of graphics passes must be same as sequence of render pass subpasses.
+* May contains per-pass descriptor set.
+* Supports graphics, mesh and tile pipelines.
+
+### ComputePass
+
+* Can be used for compute dispatches or for ray tracing.
+* May contains per-pass descriptor set.
+
+
+## Pipelines
+
+### PipelineTemplate and PipelineSpecialization concept
+
+**PipelineTemplate** contains shaders, pipeline layout and some other parameters. Mutable parameters like a shader specialization constants will be defined in **PipelineSpecialization**.
+
+In runtime you can get already created PipelineSpecialization from RenderTechnique or create new PipelineSpecialization from PipelineTemplate using `IResourceManager::Create***Pipeline (const ***PipelineDesc &)`.
+
+There are PipelineTemplates:
+* ComputePipeline
+* GraphicsPipeline
+* MeshPipeline
+* RayTracingPipeline
+* TilePipeline
+
+There are PipelineSpecializations:
+* ComputePipelineSpec
+* GraphicsPipelineSpec
+* MeshPipelineSpec
+* TilePipelineSpec
+
+
+### GLSL shaders
+
+GLSL shaders are compatible with Vulkan and Metal backends. For Metal backend they are compiled to SPIRV, then using SPIRV-Cross they are converted to the MSL and then compiled to Metal bytecode. For Windows it requires [MetalTools for Windows](https://developer.apple.com/download/all/?q=metal%20developer%20tools%20for%20windows).
+
+**Shaders will have additional built-ins:**
+
+Macros for current shader type:<br/>
+`SH_VERT` - vertex shader<br/>
+`SH_TESS_CTRL` - tessellation control shader (*selects tessellation level*)<br/>
+`SH_TESS_EVAL` - tessellation evaluation shader (*executed for each vertex after tessellation*)<br/>
+`SH_GEOM` - geometry shader<br/>
+`SH_FRAG` - fragment (pixel) shader<br/>
+`SH_COMPUTE` - compute shader<br/>
+`SH_TILE` - tile shader<br/>
+`SH_MESH_TASK` - task shader (*also called amplification or object, generates mesh shader invocations*)<br/>
+`SH_MESH` - mesh shader<br/>
+`SH_RAY_GEN` - ray generation shader (*entry point for recursive ray tracing*)<br/>
+`SH_RAY_AHIT` - ray any hit (*executed when ray intersects with any triangle*)<br/>
+`SH_RAY_CHIT` - ray closest hit (*executed when ray tracer found nearest intersection*)<br/>
+`SH_RAY_MISS` - ray miss (*executed when no intersection is found*)<br/>
+`SH_RAY_INT` - ray intersection (*to write custom intersection function*)<br/>
+`SH_RAY_CALL` - callable shader (*can be used only in ray tracing shaders*)<br/>
+
+Extensions are added depends used FeatureSets in the shader. If feature in FeatureSet is `RequireTrue` then extension is marked as `required`.
+
+
+### MSL shaders
+
+**Shaders will have additional built-ins:**
+
+Always added:
+```
+#include <metal_stdlib>
+#include <simd/simd.h>
+using namespace metal;
+```
+
+If ray tracing feature is supported:
+```
+#include <metal_raytracing>
+using namespace raytracing;
+```
+
+If atomics are supported:
+```
+#include <metal_atomic>
+#define AE_HAS_ATOMICS 1
+```
+
+### AE-Style Shader Preprocessor
+
+In scrips use:
+```
+GlobalConfig cfg;
+cfg.SetPreprocessor( EShaderPreprocessor::AEStyle );
+``` 
+
+New scalar types: bool, byte, ubyte, short, ushort, int, uint, long, ulong, float, double, half.<br/>
+New vector types: same as scalar with 2, 3, 4 suffix, example: uint3.<br/>
+New matrix types: float, half, double with suffix 2x2 ... 4x4, example float3x4.<br/>
+
+GLSL-specific:<br/>
+`gl::` namespace for types.<br/>
+`gl.` namespace for global functions.<br/>
+See [aestyle.glsl.h](../shared_data/shaders/aestyle.glsl.h) for all types, this header can be used to enable auto-complete in IDE.
+
+MSL-specific:<br/>
+
+
+# Input Actions Packer
+
+**InputActionsBinding** is a standalone dynamic library with a single function:
+```
+extern "C" bool ConvertInputActions (const InputActionsInfo *info);
+```
+
+Converts multiple scripts with key & mouse bindings to a single binary file.
+
+Documentation for each script function is auto-generated and stored in '[input_actions.as](../shared_data/scripts/input_actions.as)', you can include this file in your script.
+
+
+# Asset Packer
+
+**AssetPacker** is a standalone dynamic library with a single function:
+```
+extern "C" bool PackAssets (const AssetInfo *info);
+```
+
+Documentation for each script function is auto-generated and stored in '[asset_packer.as](../shared_data/scripts/asset_packer.as)', you can include this file in your script.
+
+* Converts images to engine internal format.
+	- Supported crop, swizzle and other operations on image.
+	- Supported packing small images to the atlas.
+* Converts ttf fonts to engine internal format.
+	- Supported raster fonts - prerendered for specified resolution.
+	- Supported SDF and multichannel SDF fonts.
+
+
+# Offline Packer
+
+**OfflinePacker** is a console program which takes arguments:<br/>
+1 - path to the resource packing script.<br/>
+2 - (optional) path to the output directory, by default used current directory.
+
+Example:
+```
+OfflinePacker.exe "res_pack.as" "../compiled_resources"
+```
+
+Documentation for each script function is auto-generated and stored in '[offline_packer.as](../shared_data/scripts/offline_packer.as)', you can include this file in your script.
+
+**OfflinePacker** allows to write top-level script which runs pipeline compiler, input actions packer, asset packer and store result to an archives.