Ultraviolence Design Notes

[schwa]

Ultraviolence (UV)

Design Notes

SwiftUI style declarative framework for Metal Rendering.

Goals

TODO

Example code:

struct MyView: View {
  let geometry = [Cube(), Cone(), Sphere()]
  let vertexShader = VertexShader(named: "vertex_main")
  let fragmentShader = FragmentShader(named: "fragment_main")
  var body: some View {
      RenderView(pixelFormat: .bgra8Unorm, depthPixelFormat: .depth32Float)) {
        RenderPass {
          RenderPipeline(vertexShader: vertexShader, fragmentShader: fragmentShader, vertexDescriptor: vertexShader.makeDescriptor()) {
            ForEach(geometry) { geometry in
              Draw(geometry)
              .vertexUniform("color", Color.red)
            }
          }
        }
      }
    }
  }
}

Problems

1. Naming. We need a name for the View analog. I've been using RenderPass, but that's wrong. They are nodes in a graph butNode doesn't work because Node will be used elsewhere in the code - for now I'll be calling them UV.View...
2. Type safety. Unlike SwiftUI Views UV.View do not possess unlimited composablility. We need to emulate SwiftUI Views like Table where the children are of limited type.
3. Metal rendering code runs in two main phases "setup" and "render" (or "draw'). This is maybe badly represented by a single UV.View graph. Code running in setup time is expensive - things like compiling shaders, creating buffers, etc. Some of that setup can/should occur in the UV.View init methods. We may need an an onSetup style modifier
4. Global state. There's really now way to implement SwiftUI State without some global state - Swift property wrappers are very limited and can't peek into the parent object they're contained in. So we need to store the current object (or a stack of objects) in a global variable so the body of the property wrapper can access it.

Architecture

Users will be able to create renderers and GPGPU workflows by composing UV.Views together. Like SwiftUI there will be State, EnvironmentValue, Binding etc (again, naming will be a problem here), much of the UV api will be presented as a handful of UV.View objects and modifiers that can be composed together.

Like SwiftUI View, UV.View is a protocol that is intended to be implemented by a value-types. By combining UV.Views together the user can create a graph that describes the rendering pipeline (or pipelines).

A graph of nodes that mirrors the UV.View graph will be created. These nodes are reference types that contain and manage and respond to changes of the graph state. These nodes also contain data to detect if the UV.View they represent has changed and portions of the graph need to be recompiled.

The graph will be processed in two phases. The setup phase is represented by the init methods of the UV.View objects. This should happen only once or when state changes.

The render phase (this needs a better name, "draw" is the wrong term to use for GPGPU graphs, maybe "workload" phase?) is handled by walking the graph and building up the state only used at render/compute time. This render-time state is invisible to the UV.View objects.

Further Exploration

Tying in Metal Stitchable graphs is an interesting extension to this. Being able to stitch together shaders without having to write shader code may be very cool.