Optimize rendering for extremely large canvas (Copy Google Recorder UI)

[abretonc7s]

Hi,

I am trying to display a large number of items on a canvas (potentially 100k plus) but it crashes after a few thousands.
Instead, I want to try to optimize so that it only renders the items that are visible.
See this video of the google recorder that can display waveform for any audio size --> It can also get to hundred of thousands of items since they display 20 rectangles per second of audio.

Screen.Recording.2024-06-11.at.8.37.54.AM.mov

In my case if I try to display a big canvas, it crashes after a few thousand items and I get errors such as:

[Violation] 'message' handler took 7027ms
120[Violation] 'requestAnimationFrame' handler took <N>ms
[Violation] 'requestAnimationFrame' handler took 139ms
[Violation] 'requestAnimationFrame' handler took 133ms
[Violation] 'requestAnimationFrame' handler took 132ms
[Violation] 'requestAnimationFrame' handler took 133ms

So instead I think a workaround would be to have a fixed size canvas so that only visible items are rendered... But I havent found the correct way to implement.
Here is an example:

const Minimal = () => {
  const { width: screenWidth } = useWindowDimensions();
  const canvasWidth = screenWidth; // Canvas width is twice the screen width
  const styles = React.useMemo(
    () => getStyles(screenWidth, canvasWidth),
    [screenWidth, canvasWidth],
  );
  const translateX = useSharedValue(0);

  const panGesture = Gesture.Pan().onUpdate((event) => {
    translateX.value += event.translationX;
    console.log(`translateX: ${translateX.value}`);
  });

  return (
    <View style={styles.container}>
      <GestureDetector gesture={panGesture}>
        <View style={styles.canvasContainer}>
          <Canvas style={{ height: 300 }}>
            {Array.from({ length: 100 }, (_, index) => {
              return (
                <RoundedRect
                  key={"r" + index}
                  x={130 * (index + 1) + translateX.value}
                  y={64}
                  width={128}
                  height={128}
                  r={10}
                />
              );
            })}
          </Canvas>
        </View>
      </GestureDetector>
    </View>
  );
};

Here the translateX.value change is not detected in the main thread so it doesnt drag my rectangles.

What would you recommend to implement a similar experience as the demo?

[wcandillon]

have you looked into the atlas API? seems to be what you need: https://shopify.github.io/react-native-skia/docs/shapes/atlas/

[abretonc7s]

Thanks @wcandillon , I tried with Atlas:

import {
  Atlas,
  Canvas,
  Group,
  Rect,
  drawAsImage,
  rect,
  useRSXformBuffer,
} from "@shopify/react-native-skia";
import { set } from "lodash";
import React, { useEffect, useMemo, useState } from "react";
import {
  Platform,
  StyleSheet,
  Text,
  useWindowDimensions,
  View,
} from "react-native";
import { Gesture, GestureDetector } from "react-native-gesture-handler";
import { Button } from "react-native-paper";
import {
  useAnimatedStyle,
  useDerivedValue,
  useSharedValue,
} from "react-native-reanimated";

const getStyles = (screenWidth: number, canvasWidth: number) => {
  return StyleSheet.create({
    container: {
      flex: 1,
      justifyContent: "center",
      alignItems: "center",
    },
    scrollView: {
      width: "100%",
    },
    canvasContainer: {
      width: canvasWidth,
      height: 300,
      // backgroundColor: "#292a2d",
      justifyContent: "center",
      alignItems: "center",
      borderColor: "green",
      gap: 5,
      borderWidth: 1,
    },
    centeredLine: {
      position: "absolute",
      top: 0,
      bottom: 0,
      left: screenWidth / 2,
      width: 2,
      backgroundColor: "red",
    },
  });
};

const generateWaveform = (length: number) => {
  return Array.from({ length }, () => Math.random());
};

const RECT_WIDTH = 5;
const SPACE_BETWEEN_RECTS = 5;
const CANVAS_HEIGHT = 300;
type DISPLAY_TYPE = "group" | "atlas";

const imageSize = {
  width: RECT_WIDTH,
  height: CANVAS_HEIGHT,
};

const image = drawAsImage(
  <Rect rect={rect(0, 0, RECT_WIDTH, CANVAS_HEIGHT)} color="cyan" />,
  imageSize,
);

const Minimal = () => {
  const { width: screenWidth } = useWindowDimensions();
  const canvasWidth = screenWidth; // Canvas width is twice the screen width
  const styles = React.useMemo(
    () => getStyles(screenWidth, canvasWidth),
    [screenWidth, canvasWidth],
  );
  const translateX = useSharedValue(0);
  const [wavepoints, setWavepoints] = useState(generateWaveform(30000)); // Generate random waveform values
  const [type, setType] = useState<DISPLAY_TYPE>("atlas"); // Default to image type

  const panGesture = Gesture.Pan().onChange((event) => {
    translateX.value += event.changeX;
    console.log(`translateX: ${translateX.value}`);
  });

  const transform = useDerivedValue(() => {
    return [{ translateX: translateX.value }];
  });

  const loadData = async () => {
    try {
      const response = await fetch("/googlewaveform.json");
      const data = await response.json();
      setWavepoints(data[0]); // Update the wavepoints state with the fetched data
    } catch (error) {
      console.error("Error fetching waveform data:", error);
    }
  };

  const sprites = useMemo(
    () =>
      new Array(wavepoints.length)
        .fill(0)
        .map(() => rect(0, 0, RECT_WIDTH, CANVAS_HEIGHT)),
    [wavepoints.length],
  );

  const transforms = useRSXformBuffer(wavepoints.length, (val, i) => {
    "worklet";
    const height = wavepoints[i] * CANVAS_HEIGHT;
    const y = CANVAS_HEIGHT / 2 - height / 2;
    // console.log(`transforms: height: ${height} ${y}`)
    val.set(1, 0, (RECT_WIDTH + SPACE_BETWEEN_RECTS) * i + translateX.value, y);
  });

  return (
    <View style={styles.container}>
      <GestureDetector gesture={panGesture}>
        <View style={styles.canvasContainer}>
          <View style={{ flexDirection: "row", flexWrap: "wrap" }}>
            <Button
              onPress={() => {
                translateX.value = 0;
              }}
            >
              reset
            </Button>
            <Button
              onPress={() => {
                setType("atlas");
              }}
              mode={type === "atlas" ? "contained" : "outlined"}
            >
              ATLAS
            </Button>
            <Button
              onPress={() => {
                setType("group");
              }}
              mode={type === "group" ? "contained" : "outlined"}
            >
              GROUP
            </Button>
            {Platform.OS === "web" && (
              <Button onPress={loadData}>Load Remote Data</Button>
            )}
          </View>
          <Text>translareX: {translateX.value}</Text>
          <Text>Points: {wavepoints.length}</Text>
          {type === "group" ? (
            <Canvas style={{ height: 300, width: screenWidth, borderWidth: 1 }}>
              <Group transform={transform}>
                {wavepoints.map((amplitude, index) => {
                  const height = amplitude * CANVAS_HEIGHT; // Scale the height based on amplitude
                  const y = CANVAS_HEIGHT / 2 - height / 2; // Center the rectangle vertically

                  // skip 0 amplitude
                  if (height === 0) {
                    return null;
                  }
                  return (
                    <Rect
                      key={"r" + index}
                      x={(RECT_WIDTH + SPACE_BETWEEN_RECTS) * index}
                      y={y}
                      width={RECT_WIDTH}
                      height={height}
                    />
                  );
                })}
              </Group>
            </Canvas>
          ) : (
            <Canvas style={{ height: 300, width: screenWidth, borderWidth: 1 }}>
              <Atlas image={image} sprites={sprites} transforms={transforms} />
            </Canvas>
          )}
        </View>
      </GestureDetector>
    </View>
  );
};

export default Minimal;

Both seem to work, the issue currently is that I also need more advanced animation within the atlas. For example each rect should have color transition and height animation when first drawn. Anything you see that could improve rendering?

Thanks again!

[wcandillon]

I don't have the resources to investigate the code unfortunately. If you previous approach works better, please go for it. I just wanted to mention it as an interesting API if needed.

[abretonc7s]

Got it, thanks. Last question on the atlas API I may be missing:

const CustomRect = ({
  x: targetX,
  y: targetY,
  width,
  height: targetHeight,
  color,
}: any) => {
  const y = useSharedValue(targetY / 2);
  const height = useSharedValue(0);
  const x = useSharedValue(targetX);
  useEffect(() => {
    y.value = withTiming(targetY, { duration: 500 });
    height.value = withTiming(targetHeight, { duration: 500 });
    x.value = withTiming(targetX, { duration: 500 });
  });
  return <Rect width={width} x={x} y={y} height={height} color={color} />;
};

const image = drawAsImage(
  <CustomRect
    x={0}
    y={0}
    width={RECT_WIDTH}
    height={CANVAS_HEIGHT}
    color="cyan"
  />,
  imageSize,
);


  const transforms = useRSXformBuffer(wavepoints.length, (val, i) => {
    "worklet";
    const height = (wavepoints[i] * CANVAS_HEIGHT) / 2;
    const y = CANVAS_HEIGHT / 2 - height / 2;
    val.set(1, 0, (RECT_WIDTH + SPACE_BETWEEN_RECTS) * i + translateX.value, y);
  });

In the useRSXFormBuffer, I can edit properties around translation / scale but is there a way to change properties of the drawn image? For exemple, different points wavepoint[i] should have different colors but currently I can't find a way to edit property or have animation within the drawAsImage. I thought you may know of a way to get around it...

[wcandillon]

it's actually possible that the Atlas API may not work for this use case. The waveform size would need to be scale on the Y axis and the Atlas API doesn't support non uniform scaling. Sorry for the misleading suggestion

[abretonc7s]

Thanks that's what I thought, I will continue using Group and reduce the datapoints dynamically. The way google recorder can do it remains a mistery :)