Unable to makeImageSnapshot in original image size when Image dimensions are bigger than Canvas dimensions

[alarm109]

I'm trying to do a PhotoAdjustmentScreen, where we allow user to adjust Image to specific aspect ratio. I have Canvas component with Image inside it. And GestureHandler to move the Image around in Canvas to achieve desired position. And then save the Image in the maximum possible resolution.

Problem

When I'm trying to save image with from snapshot canvasRef.current?.makeImageSnapshot(). The width and height is 1179, because the screen width is 393 and aspect ratio 1:1. But the original image size is 3000x2000px and I can't save the image bigger than 1179.

What I'm trying to achieve

My goal is to make the canvas size the minimum size of image dimension in this case 2000px. So I would like to make canvas size 2000x2000px, but I want to be able to see everything on screen. How do I achieve this? I need to fit 2000x2000 size canvas on the screen when the maximum screen width is 393px.

Is it possible to make canvas the size of 2000x2000 and then scale it down to the screen width, so that I could see everything?
Maybe there is a better approach to achieve this?

An example showing how to add an Image to Canvas, display the whole Image on screen and just saving it in the same dimension as the original would be very helpful. 🙏

You can see my example code below.

export const PhotoAdjustAspectRatioCanvas = ({
  canvasHeight,
  canvasRef,
  canvasWidth,
  aspectRatio,
  photoUri,
}: Props) => {
  const matrix = useSharedValue(Skia.Matrix());

  const image = useImage(photoUri);

  const imageWidth = image?.width?.() ?? 1;
  const imageHeight = image?.height?.() ?? 1;

  let fitImageWidth = canvasWidth;
  let fitImageHeight = fitImageWidth * (imageHeight / imageWidth);

  if (fitImageHeight < canvasHeight) {
    fitImageHeight = canvasHeight;
    fitImageWidth = canvasHeight * (imageWidth / imageHeight);
  }

  return (
    <Container>
      <Canvas ref={canvasRef} style={{ flex: 1 }}>
        <Image
          image={image}
          fit="scaleDown"
          x={0}
          y={0}
          width={fitImageWidth}
          height={fitImageHeight}
          matrix={matrix}
        />
      </Canvas>

      <AdjustPhotoGestureHandler
        matrix={matrix}
        dimensions={rect(0, 0, fitImageWidth, fitImageHeight)}
        frame={{ x1: 0, x2: canvasWidth, y1: 0, y2: canvasHeight }}
      />
    </Container>
  );
};

[alarm109]

Okay, I found that you set Canvas size through style and setting width and height. But first I needed to divide the image size by 3. Because 1px is 3px in canvas. So I guess the 3000 was converted to 9000 and it was too much for canvas and the app crashed? Because when I set canvas width or height to 3000 the app crashes

[alarm109]

But the problem still stays the same. If I set the Canvas size bigger than device width then the content is offscreen. But I want to be able to see the content on screen.

[wcandillon]

I would definitely recommend to not use an on screen canvas bigger than what is visible and when needing to do the snapshot draw everything offscreen using the imperative API. Currently, you would need to redraw everything using a different API (declarative onscreen, imperative offscreen) but I'm sure there is a way to nicely structure your code to achieve the desired result. This looks interesting, keep us posted.

…

On Wed, Aug 30, 2023 at 3:06 PM Matas ***@***.***> wrote: But the problem still stays the same. If I set the Canvas size bigger than device width then the content is offscreen. But I want to be able to see the content on screen. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[alarm109]

Okay, I managed to get the result I want, but now I have another problem.

Current solution

So my solution looks similar to what it used to, I just create an offscreen canvas with the size of the original image. Then when the user is finished adjusting, I draw that image on the offscreen canvas and offset it by calculating the position of the image matrix.

  const originalImageSize = getSkiaImageSize(image);
  const { size: offscreenCanvasSize } = getDimensionsForAspectRatio(
    aspectRatio,
    originalImageSize,
  );
  const offscreenCanvas = Skia.Surface.MakeOffscreen(
    offscreenCanvasSize.width,
    offscreenCanvasSize.height,
  );

  const imageSizeScaledForCanvas = getScaledImageSizeForCanvas(
    originalImageSize,
    canvasSize,
  );


  const nextButtonPress = () => {
    if (image === null || offscreenCanvas === null) {
      return;
    }

    const canvas = offscreenCanvas.getCanvas();

    const translateX =
      originalImageSize.width *
      (imageMatrixValues.x / imageSizeScaledForCanvas.width);
    const translateY =
      originalImageSize.height *
      (imageMatrixValues.y / imageSizeScaledForCanvas.height);
    const src = rect(0, 0, originalImageSize.width, originalImageSize.height);
    const dest = rect(
      translateX,
      translateY,
      originalImageSize.width * imageMatrixValues.scale,
      originalImageSize.height * imageMatrixValues.scale,
    );
    canvas.drawImageRect(image, src, dest, Skia.Paint());
    offscreenCanvas.flush();

    const skiaImage = offscreenCanvas.makeImageSnapshot();

    if (skiaImage) {
      const base64 = skiaImage.encodeToBase64(ImageFormat.JPEG, 100);

      navigate(RouteName.PhotoEditor, {
        imageBase64: base64,
        containerSize,
        initialStickerKey,
      });
    }
  };

This current solution has a hacky way of taking the matrix values from the screen canvas. What I am doing is using useDerivedValue to check the changes of the image matrix and the I set imageTranslated state. @wcandillon My question to you, is there a better way to do this? Because when the user presses the button to continue, I run the code above and it runs on the JS thread, what is the best way to read matrix values and is it even possible to do it the correct way?

  const [imageTranslated, setImageTranslated] = useState({ x: 0, y: 0 });
  useDerivedValue(() => {
    const values = imageMatrix.value?.get?.();
    const translatedX = values?.[MatrixIndex.TransX] ?? 0;
    const translatedY = values?.[MatrixIndex.TransY] ?? 0;

    if (translatedX === 0 || translatedY === 0) {
      runOnJS(setImageTranslated)({ x: translatedX, y: translatedY });
    }
  }, []);

New problem

This adjustment screen is the first screen afterwards the screen lets you edit the photo by adding dynamic number of sticker and text elements, which are all movable, scalable, etc.

My current solution takes the image values and calculates the image size and position and draws it on the new offscreen canvas, pretty simple. But on the photo editor screen there will be not just one image, but there can be any number of elements. So if I try to repeat this tactic of redrawing every element from the screen canvas to the offscreen canvas I need an optimal way of reading dynamic number of matrix values.

P.S.

I noticed that when saving image to png the image size becomes 10mb, when the orginal image size is 2.5mb. When saving in jpg format the image size is 1.1mb. That it is pretty weird, because I would expect the png to be smaller than jpg. Also I tried saving as webp format, but always got png, don't know if this a bug or what, I was trying to do this on an iphone.

[cmkf01]

@alarm109 - did you come up with any ways to use .makeImageSnapshot() when the Image has been panned and zoomed? I'm not clear on how to adjust the rect argument with the matrix used to transform the image/ gesture handler. Hence, the snapshot does not reflect the the correct rect after the image is manipulated.

[alarm109]

My logic is pretty similar like in the sticker example. I would recommend for you to try to recreate this sticker example. Because .makeImageSnapshot() just takes a snapshot of the current canvas state. So when you zoom to the picure or the stickers it should reflect in the snapshot.