Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial work merge #1

Draft
wants to merge 80 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
1b6557d
Merging Initial Work
Oct 20, 2023
aa04f1b
create some test graphs and clean up code
Oct 23, 2023
765c858
add test graphs, update engine logic, switch rotation to quaternions
Oct 25, 2023
0983856
switch node ids to be implicit rather than explicit
Oct 30, 2023
2d2e249
add special floating point nodes and comparison math nodes
Oct 30, 2023
b13995c
add OnSelect experimental Node, fix bug with parsing CustomEvent rece…
Oct 31, 2023
d2fa307
fix bug in node traversal for non-unique node pointers and do sort ba…
Nov 2, 2023
6d2d101
Create LICENSE
mattmacf98 Nov 8, 2023
6a88cfc
create experimental start and stop animation nodes, fix bug with expo…
Nov 14, 2023
60e35f7
- Fixed registering all known pointers for large gltf scenes > 99 nodes
petermart Nov 14, 2023
ad90565
Merge pull request #3 from petermart/fix-babylon-decorator
mattmacf98 Nov 20, 2023
319af2b
add authoring tool node auto alignment, update world/animateTo and wo…
Nov 18, 2023
2daaa23
Merge branch 'initial-work-merge' of https://github.com/KhronosGroup/…
Nov 20, 2023
070f6c3
make world animate to cancellable
Nov 20, 2023
5a9b2a9
Checkpoint for Demo of world/animateTo and world/set
Nov 23, 2023
c4d3f9c
update failed out flows to be err instead
Nov 27, 2023
4c72410
world animate cubic bezier
Dec 26, 2023
41f54f8
update forLoop and DoN to be consistent with group conversation
Jan 8, 2024
ba253ff
add new math nodes for matrix, integers
Jan 9, 2024
72c2871
update throttle and delay node logic
Jan 29, 2024
2ff67ef
add isValid output value for get, refactor value evaluation logic to …
Feb 6, 2024
9dbe94a
Added some stuff from my fork.
petermart Feb 13, 2024
876457a
Added better auto organize. Disjoint graphs are now separated clearly.
petermart Feb 15, 2024
88cfa43
Merge pull request #5 from petermart/petermartin/better-auto-organize
mattmacf98 Feb 27, 2024
720f32b
Merge pull request #4 from petermart/petermartin/feburary-update
mattmacf98 Feb 27, 2024
cbd2024
sync flow nodes based on latest updates
Mar 5, 2024
cc2ace6
update throttle and delay node, fix issues with outValues structure …
Mar 11, 2024
bbb536e
update pointer paths to be pre-fixed with /, update world nodes to be…
Mar 27, 2024
ac8acc2
BUGFIX: sequence out flows not properly connecting to other node in f…
Mar 27, 2024
422f28a
remove delay node, create setDelay and cancelDelay with non-deletable…
Apr 17, 2024
9368b5a
add boolean logic cases to logical op nodes, implement Nan node, impl…
Apr 17, 2024
d8a1199
remove err flow from setDelay, fix logic for boolean XOR
Apr 29, 2024
1d70e8d
create combine and extract nodes
Apr 29, 2024
a39047e
create pointer interpolate node
Jun 10, 2024
e428f5d
add pointer for khr_materials_variant
Jun 13, 2024
89cf0b5
update lifecycle, custom event and animation nodes
Jul 2, 2024
87adc0c
change typing for float int and bool types to be singleton arrays
Jul 15, 2024
07c9e46
add activeCamera read only pointers, make it so pointer interpolation…
Aug 5, 2024
7a73cee
fix: make pointer interpolate come after gltf animations
Aug 5, 2024
7a5614c
frame graph on load, play glb asset on upload, throw unrecognized nod…
Aug 22, 2024
822d364
Fix: stopping or starting an undefined animation should not throw an …
hybridherbst Aug 26, 2024
f8f4e90
add materials and material pointers
Aug 28, 2024
da44b11
make Custom Event and Variable config selection a drop down
Aug 28, 2024
02fe9dc
add KHR_texture_Transform pointers
Aug 28, 2024
3389f88
Merge pull request #16 from needle-tools/fix/stopping-undefined-anima…
mattmacf98 Aug 28, 2024
2fc06ed
add float2 type for all math nodes
Aug 28, 2024
2ef2b2d
Merge branch 'initial-work-merge' of https://github.com/KhronosGroup/…
Aug 28, 2024
bf341b2
enable math extract nodes
Aug 29, 2024
6ecaef3
add output values from onTick, fix execution issue with how event que…
Aug 29, 2024
b930fac
clear inlined input value when connect input value via node sockets
Sep 5, 2024
5a6417a
properly select correct dropdown value for custom events and variable…
Sep 19, 2024
4438727
update graph if interactivity glb is loaded
Sep 19, 2024
4822bc6
implement KHR_selectability
Oct 30, 2024
a70b9c7
implement hoverability
Oct 30, 2024
ecffb79
add Node.js dependency to Prerequisites
hybridherbst Nov 6, 2024
9ef02e5
Merge pull request #25 from needle-tools/feature/readme-prerequisites
mattmacf98 Nov 13, 2024
fab1fde
add KHR_node_visibility
hybridherbst Nov 13, 2024
724cfb7
fix: import and export of KHR_interactivity should not have a "graph"…
hybridherbst Nov 13, 2024
26029f4
fix: sanitize graph layouting against routing issues in files
hybridherbst Nov 13, 2024
6e9b3d0
fix: adjust pointer/get and pointer/set to use "value" instead of "val"
hybridherbst Nov 13, 2024
000892d
feat: add "event/onSelect" log to make debugging easier
hybridherbst Nov 13, 2024
2a72ff9
fix: Sequence node only worked when inputs are named like array indic…
hybridherbst Nov 13, 2024
e9d8b76
fix: flow/setDelay uses "done", not "complete"
hybridherbst Nov 13, 2024
cf356d1
Merge pull request #29 from needle-tools/feature/various-fixes
mattmacf98 Nov 13, 2024
d78dd63
rename `customEvents` -> `events`
hybridherbst Nov 13, 2024
65e8d9d
properly get value in pointer/set, fix setting visibility
hybridherbst Nov 13, 2024
9e68d9b
fix: Receive event values should be initialized properly
hybridherbst Nov 14, 2024
123ebed
fix: pointer/interpolate uses "value", not "val"
hybridherbst Nov 18, 2024
dbfb738
create default camera which does autofitting by default
hybridherbst Nov 18, 2024
71774a1
Merge pull request #30 from needle-tools/fix/customEvents
mattmacf98 Nov 25, 2024
81efcbd
Merge pull request #31 from needle-tools/features/camera-fitting
mattmacf98 Nov 25, 2024
3034640
renamed val to value
pfcDorn Nov 20, 2024
8287e0c
fixed variable set and get, was not using "value" as input/output id
pfcDorn Nov 21, 2024
bd7534f
added more types to ADBE log
pfcDorn Nov 22, 2024
286d5ba
fix variable get/set and initial values for variables
hybridherbst Nov 22, 2024
8f25b3a
add math/clamp node
hybridherbst Nov 22, 2024
f4b75a8
implement some read-only pointers
Nov 25, 2024
1bb822a
Merge pull request #34 from needle-tools/features/get-set-value
mattmacf98 Nov 25, 2024
62c9ba3
add float4 to math nodes and fix some broken unit tests
Nov 25, 2024
de57398
Merge branch 'initial-work-merge' of https://github.com/KhronosGroup/…
Nov 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
node_modules
18 changes: 18 additions & 0 deletions .eslintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"root": true,
"parser": "@typescript-eslint/parser",
"plugins": [
"@typescript-eslint"
],
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended"
],
"rules": {
"@typescript-eslint/no-this-alias": "off"
},
"globals": {
"module": true
}
}
24 changes: 24 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.

# dependencies
/node_modules
/.pnp
.pnp.js

# testing
/coverage

# production
/build

# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
.idea

npm-debug.log*
yarn-debug.log*
yarn-error.log*
201 changes: 201 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

1. Definitions.

"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.

"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.

"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.

"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.

"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.

"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.

"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).

"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.

"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."

"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.

2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.

3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.

4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:

(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and

(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and

(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and

(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.

You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.

5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.

6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.

7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.

8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.

9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

END OF TERMS AND CONDITIONS

APPENDIX: How to apply the Apache License to your work.

To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright [yyyy] [name of copyright owner]

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
96 changes: 96 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# DCC React App for Authoring glTF Assets with KHR_interactivity Extension

This README provides an overview and instructions for using the DCC (Digital Content Creation) React app for authoring glTF assets with the new KHR_interactivity extension.

## Table of Contents
- [Introduction](#introduction)
- [Features](#features)
- [Getting Started](#getting-started)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Usage](#usage)
- [Advanced](#advanced)
- [Bringing Your Own Engine](#bringing-your-own-engine)
- [Extending types](#extending-types)
- [Adding Nodes to BasicBehaveEngine](#adding-nodes-to-basicbehaveengine)
- [Registering Pointer mappings](#registering-pointer-mappings)
- [W.I.P.](#wip)

## Introduction

This React-based Digital Content Creation (DCC) app is designed to streamline the process of authoring glTF assets with the KHR_interactivity extension. The KHR_interactivity extension allows you to add interactive features and behaviors to your 3D models, enhancing the user experience in 3D applications.

With this app, you can easily import, create, edit, and export glTF assets with KHR_interactivity extension support.

## Features

- **User-Friendly Interface**: The app provides an intuitive and user-friendly interface for creating and editing glTF assets.
- **KHR_interactivity Support**: Easily add and configure KHR_interactivity extension features to your 3D models.
- **Real-time Preview**: Preview your 3D model with interactivity features in real-time as you make changes.
- **Export Options**: Export your authored glTF assets with the KHR_interactivity extension included.
- **Engine Extensible**: The execution and authoring components are completely decoupled allowing for extending the app to use your own KHR_interactivity engine.

## Getting Started

Follow these instructions to set up and use the DCC React app for authoring glTF assets with KHR_interactivity extension.

### Installation

1. Install the required dependencies:

```bash
npm install --force
```
there is some dependency conflicts between the version of react that jest wants and the version used by reactflow, until that is fixed use the force flag for installation.

## Usage

1. Start the DCC React app:

```bash
npm start
```

2. Open your web browser and go to [http://localhost:3000](http://localhost:3000).

3. Use the app's interface to create or load your glTF asset. (There is a menu bar for adding custom events and variables on the right hand side, right-click the authoring view panel to bring up the add node modal)

4. Pick your engine (currently Logging or Babylon) and press play to see your graph in action. NOTE: if using the Babylon engine you will need to upload a glb first.

5. Use the Send Custom Event button to trigger custom events specified in your graph.

6. Once you're satisfied, export the glTF asset with the KHR_interactivity extension included.

## Advanced

This section is about how to add custom nodes and types to the authoring view and the execution engines as well as configuring the app to work with your own engine.

### Bringing your own engine
To add your own engine, you simply need to add a new [EngineType](./src/components/engineViews/EngineType.ts) and then add a tab and panel in [App.tsx](./src/App.tsx) to show a component built around your engine. The mechanism for communication between the Authoring View and the engines
is the behaveGraphRef which just stores a JSON of the current KHR_interactivity graph, your engine should ingest that JSON and run the graph.

### Extending types
Currently, types are located in the standard types array found in the [AuthoringNodeSpecs](./src/authoring/AuthoringNodeSpecs.ts) so to add a type just specify it there, for custom types, you should use signature "custom" and provide and extension for the structure (see AMZN_interactivity_string in the array as an example).
These custom types can then be added to the input value types arrays you want to use them in. For things like types of custom event values and variables, the authoring component will automatically check the graph's types array and allow you to pick your desired type via a dropdown.

### Adding Nodes to BasicBehaveEngine
The provided Basic BehaveEngine uses a decorator approach, so you behave nodes will be defined only once and the Logging and Babylon decorators will decorate certain touch points in the execution logic with their specific functionality. To create a new node...
1. add your new node's spec in [AuthoringNodeSpecs](./src/authoring/AuthoringNodeSpecs.ts)
2. add a new class in the [nodes directory](./src/BasicBehaveEngine/nodes) which extends BehaveEngineNode.
3. add your REQUIRED_VALUES and/or REQUIRED_CONFIGURATIONS arrays which will be used to validate the node.
4. In your constructor, run super() then set the instance name and finally run the needed validations (flows, values and configurations) for your node
5. override processNode, if your node has custom named out flows (i.e. in branch we have true and false instead of out) use this.processFlow, if not you can either do this.processFlow on the default out flow or run super.processNode
6. Finally, either in [BasicBehaveEngine](./src/BasicBehaveEngine/BasicBehaveEngine.ts) or in the decorator of your choice, call registerBehaveEngineNode with the namespace of your node and the node class name
7. If you want to do some complex behaviors, like async listeners look at [CustomEvent/Receive](./src/BasicBehaveEngine/nodes/customEvent/Receive.ts) it has a good example of how async nodes post their subsequent flows to the execution queue instead of invoking synchronously and shows how to set up an event listener so it is not triggered by another node but an event.

### Registering Pointer mappings
The KHR_interactivity spec allows for setting and getting properties of an object model, this logic of mapping an object path to a setter and getter is handled by the respective decorator (so a logging engine can set a path in a different way than a Babylon one to help with implementation specific mapping).
To add your own pointer, simply add the logic for your getter and setter to the registerKnownPointers function in the respective Decorators.


## W.I.P
1. [Author Types] Currently, the types used during authoring is a set array the tool should generate the types array based on the nodes currently in use
2. [Test Graphs] There are currently very few unit tests for entire test graphs asserting output and no JSON files of example test graphs to load and experiment with
3. [Continued New Node Support] The Spec is not finalized and there will be new nodes and updates to current nodes as it progresses
4. [Refactor Node Inheritance Tree] Currently, all nodes extend the base [BehaveEngineNode](./src/BasicBehaveEngine/BehaveEngineNode.ts), as the node library progresses there may be the opportunity to split the inheritance tree (e.x. a pure functional node group vs. a side effect node group)
5. [Subgraph Support] Subgraph support is currently being talked about, depending on if this will be done as a core piece or as an extension, the tool will need to be updated to allow authoring of subgraphs and the engine updated to support reading & running subgraphs.
Loading