Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

USDZ file loader #16005

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/dev/loaders/src/USD/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
/* eslint-disable import/no-internal-modules */
export * from "./usdLoadingOptions";
export * from "./usdFileLoader";
12 changes: 12 additions & 0 deletions packages/dev/loaders/src/USD/usdFileLoader.metadata.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// eslint-disable-next-line import/no-internal-modules
import type { ISceneLoaderPluginExtensions, ISceneLoaderPluginMetadata } from "core/index";

export const USDFileLoaderMetadata = {
name: "usd",

extensions: {
// eslint-disable-next-line @typescript-eslint/naming-convention
".usdz": { isBinary: true },
// eslint-disable-next-line @typescript-eslint/naming-convention
} as const satisfies ISceneLoaderPluginExtensions,
} as const satisfies ISceneLoaderPluginMetadata;
198 changes: 198 additions & 0 deletions packages/dev/loaders/src/USD/usdFileLoader.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
import type { ISceneLoaderPluginAsync, ISceneLoaderPluginFactory, ISceneLoaderAsyncResult, ISceneLoaderProgressEvent } from "core/Loading/sceneLoader";
import { registerSceneLoaderPlugin } from "core/Loading/sceneLoader";
import { USDFileLoaderMetadata } from "./usdFileLoader.metadata";
import { AssetContainer } from "core/assetContainer";
import type { Scene } from "core/scene";
import type { Nullable } from "core/types";
import type { AbstractMesh } from "core/Meshes/abstractMesh";
CedricGuillemet marked this conversation as resolved.
Show resolved Hide resolved
import { Mesh } from "core/Meshes/mesh";
import { VertexData } from "core/Meshes/mesh.vertexData";
import { RawTexture } from "core/Materials/Textures/rawTexture";
import { StandardMaterial } from "core/Materials/standardMaterial";
import { Texture } from "core/Materials/Textures/texture";
import { Engine } from "core/Engines/engine";
import { Constants } from "core/Engines/constants";
import type { USDLoadingOptions } from "./usdLoadingOptions";
import { _LoadScriptModuleAsync } from "../shared/tools.internal";

declare module "core/Loading/sceneLoader" {
// eslint-disable-next-line jsdoc/require-jsdoc
export interface SceneLoaderPluginOptions {
/**
* Defines options for the usd loader.
*/
[USDFileLoaderMetadata.name]: Partial<USDLoadingOptions>;
}
}

/**
* @experimental
* USD(z) file type loader.
* This is a babylon scene loader plugin.
*/
export class USDFileLoader implements ISceneLoaderPluginAsync, ISceneLoaderPluginFactory {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For new code, I would consider not having the plugin itself implement ISceneLoaderPluginFactory. You'll want to add a factory to packages\dev\loaders\src\dynamic.ts, so really you could follow the same pattern for the registerSceneLoaderPlugin side effect.

/**
* Defines the name of the plugin.
*/
public readonly name = USDFileLoaderMetadata.name;

private _assetContainer: Nullable<AssetContainer> = null;

private readonly _loadingOptions: Readonly<USDLoadingOptions>;
/**
* Defines the extensions the UDS loader is able to load.
* force data to come in as an ArrayBuffer
*/
public readonly extensions = USDFileLoaderMetadata.extensions;

/**
* Creates loader for USD files
* @param loadingOptions options for loading and parsing usdz files.
*/
constructor(loadingOptions: Partial<Readonly<USDLoadingOptions>> = USDFileLoader._DefaultLoadingOptions) {
this._loadingOptions = loadingOptions;
}

/** @internal */
createPlugin(): ISceneLoaderPluginAsync {
return new USDFileLoader();
}

private static readonly _DefaultLoadingOptions = {
usdLoaderUrl: "https://lighttransport.github.io/tinyusdz/",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default should be https://cdn.babylonjs.com (it is important so we can use the ability to change base URL)

} as const satisfies USDLoadingOptions;

/**
* Imports from the loaded USD data and adds them to the scene
* @param meshesNames a string or array of strings of the mesh names that should be loaded from the file
* @param scene the scene the meshes should be added to
* @param data the USD data to load
* @param rootUrl root url to load from
* @param onProgress callback called while file is loading
* @param fileName Defines the name of the file to load
* @returns a promise containing the loaded meshes, particles, skeletons and animations
*/
public async importMeshAsync(
meshesNames: any,
scene: Scene,
data: any,
rootUrl: string,
onProgress?: (event: ISceneLoaderProgressEvent) => void,
fileName?: string
): Promise<ISceneLoaderAsyncResult> {
return this._parse(meshesNames, scene, data, rootUrl).then((meshes) => {
return {
meshes: meshes,
particleSystems: [],
skeletons: [],
animationGroups: [],
transformNodes: [],
geometries: [],
lights: [],
spriteManagers: [],
};
});
}

private _initializeTinyUSDZAsync(): Promise<void> {
return _LoadScriptModuleAsync(
`
import Module from '${this._loadingOptions.usdLoaderUrl}/tinyusdz.js';
RaananW marked this conversation as resolved.
Show resolved Hide resolved
const returnedValue = await Module();
`
);
}

private _parse(meshesNames: any, scene: Scene, data: any, rootUrl: string): Promise<Array<AbstractMesh>> {
const babylonMeshesArray: Array<Mesh> = []; //The mesh for babylon

return this._initializeTinyUSDZAsync().then((tinyusdzModule: any) => {
const usd = new tinyusdzModule.TinyUSDZLoader(data);
const textures: { [key: string]: Texture } = {};
for (let i = 0; i < usd.numMeshes(); i++) {
const mesh = usd.getMesh(i);
const customMesh = new Mesh(`usdMesh-${i}`, scene);
customMesh._parentContainer = this._assetContainer;
const vertexData = new VertexData();
vertexData.positions = Array.from(mesh.points);
// flip position x instead of changing scaling.x
if (!scene.useRightHandedSystem) {
for (let positionIndex = 0; positionIndex < vertexData.positions.length; positionIndex += 3) {
vertexData.positions[positionIndex] *= -1;
}
}
vertexData.indices = Array.from(mesh.faceVertexIndices.reverse());
if (mesh.hasOwnProperty("texcoords")) {
vertexData.uvs = Array.from(mesh.texcoords);
}
if (mesh.hasOwnProperty("normals")) {
vertexData.normals = Array.from(mesh.normals);
} else {
vertexData.normals = [];
VertexData.ComputeNormals(vertexData.positions, vertexData.indices, vertexData.normals, { useRightHandedSystem: !scene.useRightHandedSystem });
}
vertexData.applyToMesh(customMesh);

const usdMaterial = usd.getMaterial(mesh.materialId);
if (usdMaterial.hasOwnProperty("diffuseColorTextureId")) {
const material = new StandardMaterial("usdMaterial", scene);
customMesh.material = material;

if (!textures.hasOwnProperty(usdMaterial.diffuseColorTextureId)) {
const diffTex = usd.getTexture(usdMaterial.diffuseColorTextureId);
const img = usd.getImage(diffTex.textureImageId);
const texture = new RawTexture(img.data, img.width, img.height, Engine.TEXTUREFORMAT_RGBA, scene, false, true, Texture.LINEAR_LINEAR);
material.diffuseTexture = texture;
textures[usdMaterial.diffuseColorTextureId] = texture;
material.sideOrientation = Constants.MATERIAL_ClockWiseSideOrientation;
} else {
material.diffuseTexture = textures[usdMaterial.diffuseColorTextureId];
}
}
babylonMeshesArray.push(mesh);
}
return babylonMeshesArray;
});
}

/**
* Load into an asset container.
* @param scene The scene to load into
* @param data The data to import
* @param rootUrl The root url for scene and resources
* @returns The loaded asset container
*/
public loadAssetContainerAsync(scene: Scene, data: string, rootUrl: string): Promise<AssetContainer> {
const container = new AssetContainer(scene);
this._assetContainer = container;

return this.importMeshAsync(null, scene, data, rootUrl)
.then((result) => {
result.meshes.forEach((mesh) => container.meshes.push(mesh));
// mesh material will be null before 1st rendered frame.
this._assetContainer = null;
return container;
})
.catch((ex) => {
this._assetContainer = null;
throw ex;
});
}

/**
* Imports all objects from the loaded OBJ data and adds them to the scene
* @param scene the scene the objects should be added to
* @param data the OBJ data to load
* @param rootUrl root url to load from
* @returns a promise which completes when objects have been loaded to the scene
*/
public loadAsync(scene: Scene, data: string, rootUrl: string): Promise<void> {
//Get the 3D model
return this.importMeshAsync(null, scene, data, rootUrl).then(() => {
// return void
});
}
}

// Add this loader into the register plugin
registerSceneLoaderPlugin(new USDFileLoader());
9 changes: 9 additions & 0 deletions packages/dev/loaders/src/USD/usdLoadingOptions.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/**
* Options for loading usdz files
*/
export type USDLoadingOptions = {
/**
* Defines where the usd loader module is
*/
usdLoaderUrl?: string;
};
1 change: 1 addition & 0 deletions packages/dev/loaders/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ export * from "./glTF/index";
export * from "./OBJ/index";
export * from "./STL/index";
export * from "./SPLAT/index";
export * from "./USD/index";
58 changes: 58 additions & 0 deletions packages/dev/loaders/src/shared/tools.internal.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/* eslint-disable @typescript-eslint/naming-convention */
/**
* This file is only for internal use only and should not be used in your code
*/

import { IsWindowObjectExist } from "core/Misc/domManagement";
import { Tools } from "core/Misc/tools";

let _UniqueResolveID = 13372024;

/**
* Load an asynchronous script (identified by an url) in a module way. When the url returns, the
* content of this file is added into a new script element, attached to the DOM (body element)
* @param scriptUrl defines the url of the script to load
* @param scriptId defines the id of the script element
* @returns a promise request object
* It is up to the caller to provide a script that will do the import and prepare a "returnedValue" variable
* @internal DO NOT USE outside of Babylon.js core
*/
export function _LoadScriptModuleAsync(scriptUrl: string, scriptId?: string): Promise<any> {
CedricGuillemet marked this conversation as resolved.
Show resolved Hide resolved
return new Promise((resolve, reject) => {
// Need a relay
let windowAsAny: any;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe type this so at least within this function it is less likely to have a type or type error? And use a map?

Suggested change
let windowAsAny: any;
let windowAsAny: { _LoadScriptModuleResolve?: Map<number, unknown> };

let windowString: string;

if (IsWindowObjectExist()) {
windowAsAny = window;
windowString = "window";
} else if (typeof self !== "undefined") {
windowAsAny = self;
windowString = "self";
} else {
reject(new Error("Cannot load script module outside of a window or a worker"));
return;
}

if (!windowAsAny._LoadScriptModuleResolve) {
windowAsAny._LoadScriptModuleResolve = {};
}
windowAsAny._LoadScriptModuleResolve[_UniqueResolveID] = resolve;

scriptUrl += `
${windowString}._LoadScriptModuleResolve[${_UniqueResolveID}](returnedValue);
${windowString}._LoadScriptModuleResolve[${_UniqueResolveID}] = undefined;
`;
_UniqueResolveID++;

Tools.LoadScript(
scriptUrl,
undefined,
(message, exception) => {
reject(exception || new Error(message));
},
scriptId,
true
);
});
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 5 additions & 0 deletions packages/tools/tests/test/visualization/config.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
{
"root": "https://cdn.babylonjs.com",
"tests": [
{
"title": "USDZ Import",
"playgroundId": "#BR98SG#1",
"referenceImage": "usdz_import.png"
},
{
"title": "NME Glow Manual",
"playgroundId": "#7QCYPB#320",
Expand Down