Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renderer v1.0.0 #325

Merged
merged 104 commits into from
Jul 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
5c0f911
threadx removal WIP
wouterlucas May 23, 2024
3723c0a
Implement texture tracking in seperate class, rename renderer, make i…
wouterlucas May 27, 2024
49e6204
Fix inspector, fix shaders, fix textures
wouterlucas May 27, 2024
a198373
don't clear shader on unloadTexture
jfboeve May 28, 2024
82d2433
Purify Textures from the grip of ThreadX + More
frank-weindel May 31, 2024
017c70f
Fix RTT
wouterlucas May 31, 2024
4164347
Fix for texture memory clean up issue
frank-weindel Jun 4, 2024
b9f746b
Merge branch 'dev' into refactor/remove-threadx
frank-weindel Jun 4, 2024
95c6f2e
Fix example type errors
frank-weindel Jun 4, 2024
2913839
VRT: Fix hang
frank-weindel Jun 4, 2024
4b592d7
Remove internal Drivers and ThreadX support (#280)
wouterlucas Jun 4, 2024
f14f5c4
Fix initial texture loading from CoreNode
frank-weindel Jun 4, 2024
0722870
Fix initial texture loading from CoreNode (#290)
frank-weindel Jun 5, 2024
cab91f6
SubTextures now propagate renderability to parent Textures
frank-weindel Jun 5, 2024
6e497ae
Allow user to flipY in 'rtt' textures
frank-weindel Jun 5, 2024
61b9ef8
passing renderCoords from CoreNode to addQuad
jfboeve Jun 6, 2024
1c0437a
change checkRenderBounds function & simplify emitting process
jfboeve Jun 6, 2024
b0cb784
Remove INode
frank-weindel Jun 6, 2024
a707d46
Allow user to flipY in 'rtt' textures (#293)
frank-weindel Jun 6, 2024
a0e2ce8
Remove INode (#294)
frank-weindel Jun 7, 2024
6372e08
SubTextures now propagate renderability to parent Textures (#292)
frank-weindel Jun 7, 2024
831f032
RenderState/Viewport Event changes to DEV (#296)
wouterlucas Jun 8, 2024
d770e96
refactor(CoreTextureManager): Eliminate ctxTextureCache
frank-weindel Jun 12, 2024
2e75db9
refactor(CoreTextureManager): Eliminate ctxTextureCache (#299)
frank-weindel Jun 14, 2024
f271d6f
fix conditional block
jfboeve Jun 17, 2024
4536b46
passing renderCoords from CoreNode to addQuad (#295)
erikhaandrikman Jun 18, 2024
e2c7e1c
shaderProps animations
jfboeve Jun 19, 2024
126910d
removed shaderProps from CoreNodeWritableProps
jfboeve Jun 19, 2024
f803d44
refactor: Make ImageWorker work on older Tizen TVs
wouterlucas Jun 21, 2024
f0e2194
Add quadBuffer usage log
marcel-danilewicz-consult-red Jun 21, 2024
3b631a6
Implement Shader Controllers with generics for shader props
frank-weindel Jun 21, 2024
17d9fce
Rename *WritableProps to *Props
frank-weindel Jun 25, 2024
99e43e2
Dynamically typed Dynamic Shaders (first pass)
frank-weindel Jun 25, 2024
a0193d4
#304 Color setter cascade to subcomponents
M4tiz Jun 25, 2024
ad65e92
refactor: Make ImageWorker work on older Tizen TVs (#305)
frank-weindel Jun 25, 2024
f683898
Color setter cascade to subcomponents (#311)
frank-weindel Jun 25, 2024
50d56b8
Merge branch 'main' into dev
frank-weindel Jun 25, 2024
3748e8d
Merge branch 'dev' into feat/shader-controller
frank-weindel Jun 25, 2024
a211016
push wip
jfboeve Jun 26, 2024
38775b5
merge with shader-controller branch
jfboeve Jun 26, 2024
cb4bef9
Add v8 coverage
M4tiz Jun 17, 2024
bd2936a
working dynamic shader animations
jfboeve Jun 26, 2024
45e5086
Add API for checking quadBuffer usage
marcel-danilewicz-consult-red Jun 26, 2024
1f741ce
Remove cyclic logging of quadBuffer usage
marcel-danilewicz-consult-red Jun 26, 2024
363a910
optimize resolve defaults
jfboeve Jun 26, 2024
a0805bf
Upgrade to Vite 5
M4tiz Jun 21, 2024
a26008d
Add v8 coverage & upgrade to Vite 5 (#300)
frank-weindel Jun 26, 2024
1c42293
LRU-style Texture Memory Management
frank-weindel Jun 17, 2024
2b221cb
Address PR comments
frank-weindel Jun 18, 2024
ae5f6a6
LRU-style Texture Memory Management (#301)
frank-weindel Jun 26, 2024
8f4e93f
cleanup ShaderControllers, removed interface and changed to class
jfboeve Jun 27, 2024
1c8cef4
quadBufferUsage related code cleanup
marcel-danilewicz-consult-red Jun 27, 2024
82a552c
update test + upload vr snapshot
jfboeve Jun 28, 2024
d66eeb7
remove requestRender, added early returns when updating values
jfboeve Jul 1, 2024
35af177
removed spaces in snapshot names
jfboeve Jul 2, 2024
90f2c6e
scaleRotateTransform optimisation
elsassph Jul 2, 2024
d8f7116
Revert "quadBufferUsage related code cleanup"
marcel-danilewicz-consult-red Jul 3, 2024
ca35b69
Refactor getBufferInfo related code, change return type
marcel-danilewicz-consult-red Jul 3, 2024
953eee6
Only create scaleRotateTransform matrix when needed (#320)
frank-weindel Jul 3, 2024
510a91a
refactor + optimazations
jfboeve Jul 5, 2024
ae5215f
Merge branch 'main' into dev
frank-weindel Jul 5, 2024
e695432
Merge branch 'main' into dev
frank-weindel Jul 5, 2024
3d4f023
Implement Single Page Canvas Renderer, Free Texture on Canvas destroy.
wouterlucas May 18, 2024
50ad932
Implement bounds handling for Canvas Text
wouterlucas May 22, 2024
3ac9adf
Canvas2D Text: Use a managed child node to render text
frank-weindel Jun 27, 2024
2fdd7f9
Update text-canvas-font-no-metrics test snapshot
frank-weindel Jul 5, 2024
09f4705
Add missing copyright headers
frank-weindel Jul 5, 2024
be06ab7
Improve Canvas Text Renderer Performance (#319)
frank-weindel Jul 5, 2024
b2a2cf9
Merge branch 'dev' into feat/shader-controller-dyn
frank-weindel Jul 5, 2024
09f8d0f
Shader Animations w/ Dynamic Types for Shaders and Effects (#314)
frank-weindel Jul 5, 2024
113d744
1.0.0-rc.0
frank-weindel Jul 5, 2024
a3fd929
Make effect names optional
frank-weindel Jul 7, 2024
fa685ec
Export ShaderController types to API
frank-weindel Jul 7, 2024
bd1d64e
Fix: Destroy attached children when destroying a Node
frank-weindel Jul 7, 2024
db12659
Fix: Destroy attached children when destroying a Node (#328)
erikhaandrikman Jul 8, 2024
0d53c01
if no effectname continue to next iteration
jfboeve Jul 8, 2024
5ba63f8
Merge branch 'dev' into 286-add-quadbuffer-usage-log-dev
marcel-danilewicz-consult-red Jul 8, 2024
768c028
Make effect names optional (#326)
frank-weindel Jul 8, 2024
316532d
Export ShaderController types to API (#327)
frank-weindel Jul 8, 2024
94b194d
refactor usage of dynPropValuesMap, refactored loops
jfboeve Jul 8, 2024
2997fd4
Fix CoreNode.checkRenderProps, cleanup constructor
frank-weindel Jul 8, 2024
a418b9f
Export EffectDesc and related types to users
frank-weindel Jul 8, 2024
5f1b304
Refactored Core Animation after latest changes (#329)
frank-weindel Jul 8, 2024
8e3a736
Export EffectDesc and related types to users (#331)
frank-weindel Jul 8, 2024
f93b891
Fix CoreNode.checkRenderProps, cleanup constructor (#330)
frank-weindel Jul 8, 2024
e3f227a
1.0.0-rc.1
frank-weindel Jul 8, 2024
1dd83f5
Fix invalid quadBufferUsage calculation
marcel-danilewicz-consult-red Jul 9, 2024
cccb49d
Clamp the radius on the RoundedRectangle / RadiusEffect shader
M4tiz Jul 1, 2024
b8a07de
Clamp the radius on the RoundedRectangle / RadiusEffect shader (#317)
frank-weindel Jul 9, 2024
5e2bda6
VRT Dockerfile: Install the correct version of PNPM
frank-weindel Jul 9, 2024
70f6834
Export symbols necessary for custom shaders/effects/textures...
frank-weindel Jul 10, 2024
eed64a3
Add makeCacheKey to custom texture example
frank-weindel Jul 10, 2024
2e6d3a5
VRT Dockerfile: Install the correct version of PNPM (#332)
frank-weindel Jul 10, 2024
f00acf1
Export symbols necessary for custom shaders/effects/textures... (#334)
frank-weindel Jul 10, 2024
e3e874f
Add ability to query quad buffer usage info (#307)
frank-weindel Jul 10, 2024
f9ee041
1.0.0-rc.2
frank-weindel Jul 10, 2024
5cbb851
fix: Image worker broken with vite build
chiefcll Jul 10, 2024
edce167
fix: Image worker broken with vite build (#336)
frank-weindel Jul 10, 2024
c327639
1.0.0-rc.3
frank-weindel Jul 11, 2024
641d735
Add stage option to define quad buffer size (#335)
marcel-danilewicz-consult-red Jul 11, 2024
b7890d1
Use stage.options.quadBufferSize directly
marcel-danilewicz-consult-red Jul 12, 2024
554337d
Add stage option to define quad buffer size (#335) (#337)
frank-weindel Jul 12, 2024
0da5a3b
Rename TextureTypeMap back to TextureMap
frank-weindel Jul 15, 2024
a28a3f4
Rename TextureTypeMap back to TextureMap (#340)
frank-weindel Jul 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ releases
.tmp
.env
scripts
vitest.config.ts
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ node_modules
.idea
typedocs
dist
dist-cfg
dist-vitest
dist-*
examples/dist-tsc
visual-regression/failed-results
visual-regression/certified-snapshots/*-local
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ FROM mcr.microsoft.com/playwright:v1.39.0-jammy
# Set the working directory
WORKDIR /work

# Install PNPM
RUN npm install -g pnpm

# Copy the necessary files to the container
COPY .npmrc .npmrc
COPY package.json package.json

# Install PNPM
RUN corepack prepare && corepack enable

# Get pnpm to install the version of Node declared in .npmrc
RUN pnpm exec ls

Expand Down
151 changes: 24 additions & 127 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,145 +89,42 @@ See [docs/ManualRegressionTests.md].

See [RELEASE.md](./RELEASE.md)

## Main Space vs Core Space
## Installing Fonts

The Lightning 3 Renderer runs code in two logically seperate environments:
the **Main Space** and the **Core Space**.

Users of the Renderer will write most of their code for the Main Space using
the **Main API**. This is code that will always run on the browser's main thread
and includes initializing the Renderer, creating/modifying/destroying nodes,
controlling animations, etc.

The Core Space is where the actual rendering of each UI frame happens and is
mostly meant to be transparent to users of the Renderer. However, the Core Space
is where all of the code that must be tightly coupled to the rendering process
must be loaded and run. The Core Space is extendible by users by writing
**Core Extensions** via the **Core API**. This allows for users to develop their
own shaders, textures, text renderers, dynamic shader effects, and more. Fonts
used in an application must be loaded in this way too. The Core Space exists
seperately from the Main Space because it is allowed to execute on the page's
main thread OR a Web Worker thread. A **Core Driver** (see below) is used to
bridge the Main Space with the Core Space.

## Core Drivers

The Lightning 3 Renderer is designed to be able to use a single thread or
multiple web worker threads based on the configuration of a **Core Driver**.

A Core Driver essentially acts as a bridge between the Main and Core spaces
defined above.

The Renderer comes with two Core Drivers: the Main Core Driver for single
threaded rendering and the ThreadX Core Driver for multi-threaded rendering.

NOTE: The ThreadX Core Driver is experimental and even when the Renderer
graduates from beta may still not be ready for production use.

### Main Core Driver

The Main Core Driver renders your application on the web page's main thread.

It can be configured into the Renderer like so:

```ts
import { MainCoreDriver, RendererMain } from '@lightningjs/renderer';

const renderer = new RendererMain(
{
// App Config
},
'app', // App div ID
new MainCoreDriver(), // Main Render driver
);

// ...
```

### ThreadX Core Driver

The ThreadX Core Driver renders your application on a seperately spawned
Web Worker thread.

It can be configured into the Renderer like so:
Fonts can be installed into the Font Manager exposed by the Renderer's Stage.
There are two types of fonts that you can install, Web/Canvas2D fonts (WebTrFontFace)
and SDF fonts (SdfTrFontFace). Install that fonts that your applications needs
at start up so they are ready when your application is rendered.

```ts
import {
ThreadXCoreDriver,
RendererMain,
WebTrFontFace,
SdfTrFontFace,
} from '@lightningjs/renderer';

// The `@lightningjs/vite-plugin-import-chunk-url` Vite plugin is required for this:
import coreWorkerUrl from './common/CoreWorker.js?importChunkUrl';

const renderer = new RendererMain(
{
// App Config
appWidth: 1920,
appHeight: 1080,
// ...Other Renderer Config
},
'app', // App div ID
new ThreadXCoreDriver({
coreWorkerUrl,
});
'app', // id of div to insert Canvas.
);
```

## Core Extensions

To load fonts, and/or other custom code into the Core Space, you must write a
Core Extension and pass it via dynamically importable URL to the initialization
of the Renderer.

Just like with loading the ThreadX Core Web Worker for the ThreadX, you import
your core extension using the `@lightningjs/vite-plugin-import-chunk-url` plugin so that
it's code is bundled and loaded seperately from your main app's bundle.

You can write a Core Extension by extending the CoreExtension class from the
Core API like so:

```ts
import {
CoreExtension,
WebTrFontFace,
SdfTrFontFace,
type Stage,
} from '@lightning/renderer/core';

export default class MyCoreExtension extends CoreExtension {
async run(stage: Stage) {
// Load fonts into core
stage.fontManager.addFontFace(
new WebTrFontFace('Ubuntu', {}, '/fonts/Ubuntu-Regular.ttf'),
);

stage.fontManager.addFontFace(
new SdfTrFontFace(
'Ubuntu',
{},
'msdf',
stage,
'/fonts/Ubuntu-Regular.msdf.png',
'/fonts/Ubuntu-Regular.msdf.json',
),
);
}
}
```

And then in your application's main entry point you can import it using
`@lightningjs/vite-plugin-import-chunk-url`:

```ts
import coreExtensionModuleUrl from './MyCoreExtension.js?importChunkUrl';

// Set up driver, etc.
// Load fonts into renderer
renderer.stage.fontManager.addFontFace(
new WebTrFontFace('Ubuntu', {}, '/fonts/Ubuntu-Regular.ttf'),
);

// Initialize the Renderer
const renderer = new RendererMain(
{
// Other Renderer Config...
coreExtensionModule: coreExtensionModuleUrl,
},
'app',
driver,
renderer.stage.fontManager.addFontFace(
new SdfTrFontFace(
'Ubuntu',
{},
'msdf',
stage,
'/fonts/Ubuntu-Regular.msdf.png',
'/fonts/Ubuntu-Regular.msdf.json',
),
);
```
5 changes: 5 additions & 0 deletions docs/CustomShaderEffectTexture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Custom Shaders / Effects / Textures

For some examples on how to create custom shaders, effects and textures, see
the (custom-shader-effect-texture)[./example-projects/custom-shader-effect-texture]
example project.
44 changes: 22 additions & 22 deletions docs/ManualRegressionTests.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,49 +4,49 @@ In addition to the automated [Visual Regression Tests](../visual-regression/READ
there are some tests that should be run manually on embedded devices to ensure
the Renderer is working properly.

## Reference-Based Texture Memory Management Test
## Critical Texture Memory Cleanup Test

`?test=reference-texture-memory`
`?test=texture-cleanup-critical&monitor=true`

This test confirms the Renderer's ability to proactively garbage collect
textures that are, at the very least, likely to not be referenced anymore in
memory.
This test confirms the Renderer's ability to clean up textures when it never
has a chance to perform Idle Texture Cleanups.

Within an infinite loop, the test updates the texture of a single full screen
background node to a new random NoiseTexture in rapid succession.

**Expected Results**: The test runs for at least 1 hour without crashing.
**Expected Results**: The tests runs for at least 30 mins minutes without
crashing or the visible Nodes becoming black. The Memory Monitor shows loaded
textures reaching the Critical Threshold and then falling back to the target.

To confirm that the textures are being properly disposed of, you can use the Chrome Task Manager to monitor the GPU's memory usage:
To further confirm that the textures are being properly disposed of, you can use
the Chrome Task Manager to monitor the GPU's memory usage:

1. Click Window > Task Manager
2. Locate the "GPU Process"
3. Observe the "Memory Footprint" column
4. The value should eventually drop significantly toward a minimum and/or reach a
threadhold.
4. Like the Memory Monitor, the value should increase, and fall significantly
repeatedly.

By default, the ManualCountTextureUsageTracker is used to track texture usage. Also test the experimental FinalizationRegistryTextureUsageTracker instead, by setting the URL param "finalizationRegistry=true".
## Idle Texture Memory Cleanup Test

## Threshold-Based Texture Memory Management Test
`?test=texture-cleanup-idle&monitor=true`

`?test=threshold-texture-memory`

This test confirms the Renderer's ability to garbage collect textures from GPU VRAM
that are still referenced and assigned to Nodes but no longer visible within
the configured `boundsMargin` when the configured `txMemByteThreshold` is
exceeded.
This test confirms the Renderer's ability to clean up textures that are no longer
renderable (not in the configured `boundsMargin`) from GPU VRAM when the Renderer
becomes idle.

Within an infinite loop, this test generates a grid of Nodes with random NoiseTextures
assigned first completely visible on screen (for at least a frame) and then moves
them outside of the configured `boundsMargin` before repeating the loop.

**Expected Results**: The tests runs for at least XXXX minutes on an XXXX running WPE
without crashing or the visible Nodes becoming blank.
**Expected Results**: The tests runs for at least 30 mins minutes without
crashing or the visible Nodes becoming black. The Memory Monitor shows loaded
textures falling to the Target Threshold roughly every 5 seconds.

To test that the textures are being properly disposed of, you can use the Chrome Task Manager to monitor the GPU's memory usage:
To further test that the textures are being properly disposed of, you can use the Chrome Task Manager to monitor the GPU's memory usage:

1. Click Window > Task Manager
2. Locate the "GPU Process"
3. Observe the "Memory Footprint" column
4. The value should eventually drop significantly toward a minimum and/or reach a
threshold.
4. Like the Memory Monitor, the value should increase, and fall significantly
repeatedly.
21 changes: 21 additions & 0 deletions docs/example-projects/custom-shader-effect-texture/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<html>
<head>
<title>Renderer Browser Test</title>
<style>
body {
margin: 0;
padding: 0;
overflow: hidden;
}

#app {
display: inline-block;
background: #3677e0;
}
</style>
</head>
<body>
<div id="app"></div>
<script type="module" src="./src/index.ts"></script>
</body>
</html>
23 changes: 23 additions & 0 deletions docs/example-projects/custom-shader-effect-texture/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"name": "renderer-custom-shader",
"version": "1.0.0",
"description": "",
"main": "index.js",
"type": "module",
"scripts": {
"start": "vite --open --host",
"build": "tsc && vite build",
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"@lightningjs/renderer": "link:../../.."
},
"devDependencies": {
"@types/node": "^20.12.12",
"typescript": "^5.4.5",
"vite": "^5.2.11"
}
}
Loading
Loading