Skip to content

Latest commit

 

History

History
88 lines (64 loc) · 4.93 KB

README.md

File metadata and controls

88 lines (64 loc) · 4.93 KB

ViPlanner Omniverse Extension

The ViPlanner Omniverse Extension offers a sophisticated testing environment for ViPlanner. Within NVIDIA Isaac Sim as a photorealistic simulator, this extension provides an assessment tool for ViPlanner's performance across diverse environments. The extension is developed using the Orbit Framework.

Remark

The extension for Matterport and Unreal Engine meshes with semantic information is currently getting updated to the latest Orbit version and will be available soon. An intermediate solution is given here.

Installation

To install the ViPlanner extension for Isaac Sim version 2023.1.1, follow these steps:

  1. Install Isaac Sim using the Orbit installation guide.
  2. Clone the orbit repo, checkout commit 477cd6b3f and link the viplanner extension. The specific commit is necessary as Orbit is under active development and the extension is not yet compatible with the latest version.
git clone [email protected]:NVIDIA-Omniverse/orbit.git
cd orbit
git checkout 477cd6b3f
cd source/extensions
ln -s {VIPLANNER_DIR}/omniverse/extension/omni.viplanner .
  1. TEMPORARY: To use Matterport with semantic information within Isaac Sim, a new extension has been developed as part of this work. Currently, all parts are getting updated to the latest Orbit version. A temporary solution that is sufficient for the demo script is available here. Please also clone and link it into orbit.
git clone [email protected]:pascal-roth/orbit_envs.git
cd orbit/source/extension
ln -s {ORBIT_ENVS}/extensions/omni.isaac.matterport .
  1. Then run the orbit installer script and additionally install ViPlanner in the Isaac Sim virtual environment.
./orbit.sh -i -e
./orbit.sh -p -m pip install -e {VIPLANNER_DIR}

Remark Also in orbit, it is necessary to comply with PEP660 for the install. This requires the following versions (as described here in detail)

Usage

A demo script is provided to run the planner in three different environments: Matterport, Carla, and NVIDIA Warehouse. In each scenario, the goal is represented as a movable cube within the environment.

To run the demo, download the model: [checkpoint] [config] and the environment files. Then adjust the paths (marked as ${USER_PATH_TO_USD}) in the corresponding config files.

Matterport

Config

To download Matterport datasets, please refer to the Matterport3D website. The dataset should be converted to USD format using Isaac Sim by executing the following steps:

  1. Import the .obj file (located under matterport_mesh) into Isaac Sim by going to File -> Import.
  2. Fix potential import setting such as Rotation and Scale. (Property Panel -> Transform -> Rotate:unitsResolve = 0.0; Scale:unitsResolve = [1.0, 1.0, 1.0])
  3. Export the scene as USD (File -> Save as).
./orbit.sh -p {VIPLANNER_DIR}/omniverse/standalone/viplanner_demo.py --scene matterport --model_dir {MODEL_DIR}

Carla

Download USD Link Download Texture Link Config

⚠️ Due to some code changes, the semantics are here not correctly received. We are working on a fix.

./orbit.sh -p {VIPLANNER_DIR}/omniverse/standalone/viplanner_demo.py --scene carla --model_dir {MODEL_DIR}

NVIDIA Warehouse

Download USD Link Config

./orbit.sh -p {VIPLANNER_DIR}/omniverse/standalone/viplanner_demo.py --scene warehouse --model_dir {MODEL_DIR}

Data Collection and Evaluation

Script for data collection and evaluation are getting updated to the latest Orbit version and will be available soon. If you are interested in the current state, please contact us.