Roadmap

[lennart]

a small update on the progress on this project:

I did roughly two or three iterations of this project up to now with different results, in random order I implemented the following:

• caching for shaders hidden behind a flag (so you can do live coding of shaders in parallel with livecoding the patterns from tidal, essential for debugging as well…) (use --cache to cache shader programs)
• rendering each shader into a one of two framebuffer, then flip and pass in the just written framebuffer as a texture into the next shader (this refers to shaders being rendered at the same time, so blending could be done within the shaders, instead of glBlend)
• adding text rendering (freetype2 bitmap fonts, but I looked at and tried out freetype-gl which implements valve's distance field technique to get sharper fonts, this might be interesting)
• adding scribbling onto a separate layer (writing to framebuffer) via mouse or tablet input (no pressure sensing yet, this will be an effort but might be interesting)
• Update this is Daumenkino a simple Haskell/Tidal module to create a DSL to change the following parameters:
  • duration
  • red, green, blue, alpha
  • x,y,z,w
  • size
  • speed
  • blendmode
  • level (to change which of simultaneous layer is on top of which)
• Separation of layers into types: scribble, shader/bg, text

I added the haskell project onto github but I guess it's of no use right now, as I haven't settled on a Way to go with the interface from within tidal itself, yet.

The separation of layers into types wasn't a good idea, as the mental overhead of having to handle visuals and maybe audio (dirt/midi) will be giving you a headache anyway, so having to deal with separate streams for text, scribbling and background shaders will likely make your head explode…

Right now I am thinking of reducing this to just a single stream, like dirt does, and have scribbling and text be hidden behind params (like vowel, cutoff/resonance and others in dirt).

I guess I'll be removing text rendering for now and merge everything that works into master and release a bare-metal haskell package.

Conceptually text rendering makes me think of masking a background shader-layer by the text, in case of the rainbow effect this could create rainbow text…

Scribbling is something I'd like to think of as the loop input within tidal, I want to try out if one can constantly add vertices at the cursor position to a vertex buffer object (and loop over when it's full) and use the vertex input instead of the standard quad and have it passed to whatever shader you are currently using. I am actually quite anxious to see what this will look like and was thinking about allowing one to change the primitive type to be drawn, e.g. lines, triangles, polygons points. I prefer this variant over the framebuffer version as it would allow more control over the drawn scene, e.g. even apply different vertex shaders to the drawn vertices to achieve displacement mapping and so on.

Regarding support for older hardware, I try to target Desktop OpenGL 3.3+ and OpenGL ES 2+ (Raspberry Pi 2). I don't think anything below makes any sense.