React Native - glb/gltf model works in emulator/usb connected device but fails in android apk

[dpsilvaa97]

So my problem is when I try to run the app in my device using the apk without being connected via usb debugging. Im using android. It works fine when using emulators even with the same version of my android, etc.. but it fails outside this env.

This is the code sample:

This works pretty well in the emulator or when my device is connected via usb debugging. But when I unplug my phone or try to install and run the apk this is what happens:

I believe it could be happening due to the different file systems. I have seen that the models are stored in \android\app\src\main\res\raw with the correct extension, which is '.glb' in this case, when I run the commands to generate the apk. It seems ok but for some reason it cant be found I believe.

I tried to change the code in terms of how to get the image.

With this attempt, it stills works fine in the emulator/usb but it fails in the apk in my device. But the error is the following one:

Any suggestion?

[dpsilvaa97]

Any information about this? :/

[asalaza6]

I get the same exact error. 'Could not load ...' > I think it has to do with an error with loading the glb models on native (obv)... Maybe it's an issue with react-native version, or another package version.

Have you found a solution on your end?

[dpsilvaa97]

I get the same exact error. 'Could not load ...' > I think it has to do with an error with loading the glb models on native (obv)... Maybe it's an issue with react-native version, or another package version.

Have you found a solution on your end?

Hey! Unfortunately not yet, I have tried different approaches of loading the model but didnt work. If I find out a workaround or something else I'll write here. If you get this to work, please share aswell :(

[dpsilvaa97]

@CodyJasonBennett I saw that you added some tags to it! Thanks for noticing! :)
Do you have some idea what could be causing it?

[CodyJasonBennett]

I haven't looked much into this specific case, but having require anywhere but in global scope has been historically problematic with Metro — it can only handle static dependencies.

[asalaza6]

@CodyJasonBennett So instead of require we should use another way of importing the gltf file like 'expo-asset'? If not could you point me in the right direction?

[dpsilvaa97]

@CodyJasonBennett So instead of require we should use another way of importing the gltf file like 'expo-asset'? If not could you point me in the right direction?

I have tried with import, require, expo-asset, .. but couldn't make it work :/ It seems that the apk itself can't find the model in its filesystem. Only when its running in debug mode via usb

[asalaza6]

I have tried with import, require, expo-asset, .. but couldn't make it work :/ It seems that the apk itself can't find the model in its filesystem. Only when its running in debug mode via usb

Same here, another alternative is to use Babylon JS Native, but that comes with it's own set of challenges. I'll try it this week

[CodyJasonBennett]

I meant to move the require/import to top-level rather than function body. I'll have to take a closer look when I get the chance.

As an aside, what challenges are you facing with Babylon Native? There is some overlap since a lot of issues depend on downstream fixes in react-native, some might be quicker to make use of there than here and vice versa.

[carmenchapa]

Came here from this discussion to say that I face the same issue with iOS: I get the model to render in development on device, and it crashes when push it to production (with eas update and Expo Go).
The strange part is that is happening with the exact same code as in CodyJasonBennett's example when I try to replace the model file with another, but the example one works.
I get the below console warnings on the iphone

[Error: Could not load 2: undefined)]

NSInvalidArgumentException: -[EXReactAppExceptionHandler handleSoftJSExceptionWithMessage:stack:exceptionId:extraDataAsJSON:]: unrecognized selector sent to instance 0x280d52c50

Error: Could not load 2: undefined)

This error is located at:
    in Unknown
    in p
    in Unknown
    in RCTView
    in Unknown
    in Unknown
    in Unknown
    in Unknown
    in RCTView
    in Unknown
    in RCTView
    in Unknown
    in b
    
    
    { [Error: Could not load 2: undefined)]
  componentStack: '\n    in Unknown\n    in p\n    in Unknown\n    in RCTView\n    in Unknown\n    in Unknown\n    in Unknown\n    in Unknown\n    in RCTView\n    in Unknown\n    in RCTView\n    in Unknown\n    in b',
  isComponentError: true }

[asalaza6]

I meant to move the require/import to top-level rather than function body. I'll have to take a closer look when I get the chance.

As an aside, what challenges are you facing with Babylon Native? There is some overlap since a lot of issues depend on downstream fixes in react-native, some might be quicker to make use of there than here and vice versa.

The main challenge is that in Babylon it forces you use a bare workflow vs a managed workflow. To integrate is a little different as the library uses different dependencies. It shouldn't be too bad and should be okay, but it's an inconvenience. I'll try to implement the 3d model with babylon this week

[dpsilvaa97]

Came here from this discussion to say that I face the same issue with iOS: I get the model to render in development on device, and it crashes when push it to production (with eas update and Expo Go). The strange part is that is happening with the exact same code as in CodyJasonBennett's example when I try to replace the model file another, but the example one works. I get the below console warnings on the iphone

[Error: Could not load 2: undefined)]

NSInvalidArgumentException: -[EXReactAppExceptionHandler handleSoftJSExceptionWithMessage:stack:exceptionId:extraDataAsJSON:]: unrecognized selector sent to instance 0x280d52c50

Error: Could not load 2: undefined)

This error is located at:
    in Unknown
    in p
    in Unknown
    in RCTView
    in Unknown
    in Unknown
    in Unknown
    in Unknown
    in RCTView
    in Unknown
    in RCTView
    in Unknown
    in b
    
    
    { [Error: Could not load 2: undefined)]
  componentStack: '\n    in Unknown\n    in p\n    in Unknown\n    in RCTView\n    in Unknown\n    in Unknown\n    in Unknown\n    in Unknown\n    in RCTView\n    in Unknown\n    in RCTView\n    in Unknown\n    in b',
  isComponentError: true }

So it works in production with the model from the example but fails when you change to another one?

[carmenchapa]

@dpsilvaa97 yes, exactly, and I could not find any other model that works, not even a simple cube exported from blended without textures, but I know nothing about 3D, maybe I'm missing something

[asalaza6]

I tried the example with same package versions in Android and get error in apk file, still works in simulator/expo.

[asalaza6]

@carmenchapa @dpsilvaa97 I actually found a workaround. I think there is a bug somewhere with the loading of the assets somehow @CodyJasonBennett, but I was able to load the model with a custom function. I found this custom function somewhere in another discussion. Here ->> expo/expo-three#273

This function will return your gltf model. You can use require('..path') for asset prop. I tested and works with expo & apk 😄

import { loadTextureAsync } from 'expo-three';
import { resolveAsync } from 'expo-asset-utils';
import { GLTF, GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';
import { decode } from 'base64-arraybuffer';
import * as FileSystem from "expo-file-system";
import THREE from 'three';

async function loadFileAsync({ asset, funcName }) {
    if (!asset)
        throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
    return (await resolveAsync(asset)).localUri ?? null;
}

export async function loadGLTFAsync({ asset, onAssetRequested }) {
    const uri = await loadFileAsync({
        asset,
        funcName: 'loadGLTFAsync',
    });

    if (!uri) return;

    const base64 = await FileSystem.readAsStringAsync(uri, {
        encoding: FileSystem.EncodingType.Base64,
    });

    const arrayBuffer = decode(base64);
    const loader = new GLTFLoader();

    return new Promise<GLTF>((resolve, reject) => {
        loader.parse(
            arrayBuffer,
            onAssetRequested,
            result => {
                resolve(result);
            },
            err => {
                reject(err);
            },
        );
    });
}

[CodyJasonBennett]

Awesome. We do a light progressive polyfill in Canvas to smoothen out compat in the threejs core.

react-three-fiber/packages/fiber/src/native/polyfills.ts

Lines 1 to 141 in bf66f08

	import * as THREE from 'three'
	import type { Asset } from 'expo-asset'
	// Check if expo-asset is installed (available with expo modules)
	let expAsset: typeof Asset | undefined
	try {
	expAsset = require('expo-asset')?.Asset
	} catch (_) {}
	/**
	* Generates an asset based on input type.
	*/
	function getAsset(input: string | number) {
	switch (typeof input) {
	case 'string':
	return expAsset!.fromURI(input)
	case 'number':
	return expAsset!.fromModule(input)
	default:
	throw new Error('R3F: Invalid asset! Must be a URI or module.')
	}
	}
	let injected = false
	export function polyfills() {
	if (!expAsset || injected) return
	injected = true
	// Don't pre-process urls, let expo-asset generate an absolute URL
	const extractUrlBase = THREE.LoaderUtils.extractUrlBase.bind(THREE.LoaderUtils)
	THREE.LoaderUtils.extractUrlBase = (url: string) => (typeof url === 'string' ? extractUrlBase(url) : './')
	// There's no Image in native, so create a data texture instead
	const prevTextureLoad = THREE.TextureLoader.prototype.load
	THREE.TextureLoader.prototype.load = function load(url, onLoad, onProgress, onError) {
	const texture = new THREE.Texture()
	// @ts-ignore
	texture.isDataTexture = true
	getAsset(url)
	.downloadAsync()
	.then((asset: Asset) => {
	texture.image = {
	data: asset,
	width: asset.width,
	height: asset.height,
	}
	texture.flipY = true
	texture.unpackAlignment = 1
	texture.needsUpdate = true
	onLoad?.(texture)
	})
	.catch(onError)
	return texture
	}
	// Fetches assets via XMLHttpRequest
	const prevFileLoad = THREE.FileLoader.prototype.load
	THREE.FileLoader.prototype.load = function (url, onLoad, onProgress, onError) {
	if (this.path) url = this.path + url
	const request = new XMLHttpRequest()
	getAsset(url)
	.downloadAsync()
	.then((asset) => {
	request.open('GET', asset.uri, true)
	request.addEventListener(
	'load',
	(event) => {
	if (request.status === 200) {
	onLoad?.(request.response)
	this.manager.itemEnd(url)
	} else {
	onError?.(event as unknown as ErrorEvent)
	this.manager.itemError(url)
	this.manager.itemEnd(url)
	}
	},
	false,
	)
	request.addEventListener(
	'progress',
	(event) => {
	onProgress?.(event)
	},
	false,
	)
	request.addEventListener(
	'error',
	(event) => {
	onError?.(event as unknown as ErrorEvent)
	this.manager.itemError(url)
	this.manager.itemEnd(url)
	},
	false,
	)
	request.addEventListener(
	'abort',
	(event) => {
	onError?.(event as unknown as ErrorEvent)
	this.manager.itemError(url)
	this.manager.itemEnd(url)
	},
	false,
	)
	if (this.responseType) request.responseType = this.responseType
	if (this.withCredentials) request.withCredentials = this.withCredentials
	for (const header in this.requestHeader) {
	request.setRequestHeader(header, this.requestHeader[header])
	}
	request.send(null)
	this.manager.itemStart(url)
	})
	return request
	}
	// Cleanup function
	return () => {
	THREE.LoaderUtils.extractUrlBase = extractUrlBase
	THREE.TextureLoader.prototype.load = prevTextureLoad
	THREE.FileLoader.prototype.load = prevFileLoad
	}
	}

Perhaps there's something going wrong with our usage or in loaders' load code paths. The latter seems unlikely with GLTFLoader specifically.

[carmenchapa]

@asalaza6 how do you use the function? and what do you pass for onAssetRequested?
Using it like this

const asset = require('./assets/asset.glb');
const result = await loadGLTFAsync({ asset })

gives me an error Promise constructor's argument is not a function

[asalaza6]

@carmenchapa onAssetRequested is an optional field, just a callback function after. Your code looks fine to me. I am not sure what's causing your issue. I'll just send you this screenshot snippet. The file prop is a require('..path')

[dpsilvaa97]

Thanks for sharing your knowledge! I have tried this approach but something is going wrong in the loader.parse, since it is not returning the promise. I placed console.logs inside both success and error functions of parse and none are showing up. I checked the value of arrayBuffer before enter the parse and its empty but the base64 its not.

I get this warning, and nothing the model is not loaded.

I tried to change de "decode" function to another one that I found. Which is this one:

And got this different warning. Also, the model is not lodead.

I believe the problem is within the encode/decode functions. Any suggestion?

[CodyJasonBennett]

Are you using GLTFLoader out of pmndrs/three-stdlib? It should allow you to run without checking navigator, try updating. If it doesn't then I can send a quick patch.

[dpsilvaa97]

Im using from this import:
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';

[asalaza6]

Hmm I'm not sure. It's really finnicky, I'm thinking maybe package versions, or implementation is a bit different. I'm not sure. How are you testing? Also make sure you're still using the native packages for other things like canvas, etc..

[dpsilvaa97]

I created a test component just like you just to see if it works, but it didnt

This component is being called here:

And this is where the file comes from:

The decode and resolveAsync functions are similiar to the ones in those libraries that you are using, I just got them individually from those libs just to see if it works to avoid adding unnecessary packages to the project.

I got this error:

[CodyJasonBennett]

Can you try with import { GLTFLoader } from 'three-stdlib' instead? three/examples aren't great for compat.

[asalaza6]

@dpsilvaa97 hmmm i get you, your packages can start to add up, here are my package versions if that's helpful

[dpsilvaa97]

import { GLTFLoader } from 'three-stdlib'

Alright!! The model appeared and it works in the apk aswell !! <3 Amazing guys thanks for the support and knowledge sharing!

[asalaza6]

Awesome! That's good.

[dpsilvaa97]

Does anyone has a suggestion about this?

In my previous solution that didnt work with the apk I was using useGLTF which returns "nodes". With that I could get access to the bones and the hand components.

With this loader the "nodes" is undefined. So far I could get the bones with this approach. But How do I get the Hand's geometry and skeleton to use in the SkinnedMesh?

[carmenchapa]

hey, thanks all for your sharings.

I can't make this work, I updated the libraries versions as in this comment, I'm using

import { GLTFLoader } from 'three-stdlib'

and still getting this error:

Not sure what my problem is. Perhaps this problem only happens on iOS ?

edit: there was a mistake in my code, forgot to remove one type annotation (I'm testing with plain js), the promise was never called. My apologies

This worked for me too! Uploaded the app with eas update and I can see the model in prod now 🎉
Thank you so much for the support and the workaround @CodyJasonBennett @asalaza6 Amazing!

[carmenchapa]

Hey @dpsilvaa97, did you find a solution for getting the nodes already?
I facing the same problem now, nodes are undefined

[CodyJasonBennett]

You can get nodes etc from useGraph. useLoader uses it internally in the onLoad callback.

[dpsilvaa97]

Hey @dpsilvaa97, did you find a solution for getting the nodes already? I facing the same problem now, nodes are undefined

Heyy! I solved by using the traverse function on the scene.

   scene.traverse((child) => {

   // here you can search for types, for example

   if (child.type === "Bone") //to get bones. I used this one to have access to the bones

   if (child.type === "SkinnedMesh") //to get the skinnedmesh itself. I used this one to get access to the model's geometry and skeleton  

   //you also have these ones, which I didnt use
   if(child.type==="Group")
   if(child.type==="Object3D")

   })

Hope it helps! :)

[carmenchapa]

You can get nodes etc from useGraph. useLoader uses it internally in the onLoad callback.

This worked for me. Thanks!

[dpsilvaa97]

@CodyJasonBennett should this issue be closed?

[farmketprocess]

Brother can you provide the file decode function and resolve async

i will be thankyou to you .

I am not able to load the model in react native so that why i need your help.

[YarooqH]

@farmketprocess you can take a look at the code posted by @carmenchapa's on this issue #2643

[CodyJasonBennett]

I've implemented patches for missing APIs in #2982 with fixes for Android APK. Codesandbox CI creates preview builds if you want to try ahead of release. I'll try to get this out shortly.

// package.json
{
  "dependencies": {
    "@react-three/fiber": "https://pkg.csb.dev/pmndrs/react-three-fiber/commit/0772cfaa/@react-three/fiber"
  }
}