Skip to content

Commit b3b95f3

Browse files
committed
[docs] update installation docs, add contribution doc
1 parent 1043744 commit b3b95f3

File tree

2 files changed

+235
-27
lines changed

2 files changed

+235
-27
lines changed

source/en/user_guide/internmanip/quick_start/installation.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,11 @@ chmod +x install.sh
244244
```bash
245245
./install.sh --all
246246
```
247+
> 💡 Tips:
248+
> Before installing genmanip, please ensure that Anaconda and Isaac Sim 4.5.0 are properly set up on your system. You can download the standalone version from [👉 Download Isaac Sim 4.5.0 (RC36)](https://download.isaacsim.omniverse.nvidia.com/isaac-sim-standalone%404.5.0-rc.36%2Brelease.19112.f59b3005.gl.linux-x86_64.release.zip)
249+
>
250+
> After downloading, extract the archive to a suitable directory (e.g., ~/tools/isaac-sim-4.5.0). You should set the path to your local Isaac Sim installation during running `install.sh`.
251+
247252

248253
#### Available Installation Options:
249254
```bash
@@ -291,6 +296,12 @@ source .venv/model/bin/activate
291296
deactivate
292297
```
293298

299+
> 🟡 Note: Unlike other environments that use venv, genmanip relies on Conda for environment management. You should always activate the environment using:
300+
> ```bash
301+
> conda activate genmanip
302+
> ```
303+
304+
294305
Optionally, users can customize the virtual environments directory path by passing the `--venv-dir {path}` option when executing `install.sh`.
295306
296307
```bash
@@ -361,6 +372,7 @@ When compiling C++ components (e.g., `building ManiSkill2_real2sim`), you might
361372
```
362373
363374
375+
364376
## Verification (WIP)
365377
366378
You can evaluate the pretrained **GR00t-N1** model on the `Simpler-Env` benchmark using a **client-server** architecture. This requires two separate terminal sessions:
@@ -419,3 +431,122 @@ If you prefer manual dataset preparation:
419431
- [CALVIN_ABC](https://huggingface.co/datasets/InternRobotics/InternData-Calvin_ABC)
420432
- [Google-Robot](https://huggingface.co/datasets/InternRobotics/InternData-fractal20220817_data)
421433
- [BridgeData-v2](https://huggingface.co/datasets/InternRobotics/InternData-BridgeV2)
434+
435+
436+
437+
## ⚠️ Troubleshooting
438+
439+
### 1. Conda Terms of Service Acceptance Error
440+
441+
If you encounter the following error during the genmanip installation:
442+
```bash
443+
CondaToNonInteractiveError: Terms of Service have not been accepted for the following channels:
444+
• https://repo.anaconda.com/pkgs/main
445+
• https://repo.anaconda.com/pkgs/r
446+
```
447+
Manually accept the Terms of Service for each affected channel by running these commands:
448+
```bash
449+
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
450+
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r
451+
```
452+
453+
454+
### 2. Tips for Slow or Unstable Networks
455+
456+
If you encounter errors such as timeouts or incomplete downloads, especially in network-restricted or low-bandwidth environments, we recommend the following approaches.
457+
458+
- Step 1: By default, `uv pip` uses relatively short HTTP timeouts. To extend the timeout, set the following environment variable before installation:
459+
```bash
460+
export UV_HTTP_TIMEOUT=600 # Timeout in seconds (10 minutes)
461+
```
462+
463+
- Step 2: Locate the package that failed to install. When running the `install.sh` script, if a package fails to install, identify the failing line in the output. This usually indicates which package wasn't downloaded properly.
464+
- Step 3: Dry-run to preview packages and versions. Use uv pip install --dry-run` to preview which packages (and exact versions) are going to be installed. For example:
465+
```bash
466+
uv pip install "git+https://github.com/NVIDIA/Isaac-GR00T.git#egg=isaac-gr00t[base]" --dry-run
467+
```
468+
This will list all packages along with their resolved versions, including those that might fail due to slow download.
469+
- Step 4: Activate your environment. Before manually installing the package, make sure you're in the correct virtual environment.
470+
```bash
471+
source .venv/{your_env_name}/bin/activate
472+
```
473+
- Step 5: Manually install the problematic package. After identifying the package and version, install it manually:
474+
```bash
475+
uv pip install torch==2.5.1 # Replace with your actual package and version
476+
```
477+
478+
<!---
479+
- By default, `uv pip` uses relatively short HTTP timeouts. To extend the timeout, set the following environment variable before installation:
480+
```bash
481+
export UV_HTTP_TIMEOUT=600 # Timeout in seconds (10 minutes)
482+
```
483+
- To ensure successful installation without network interruptions, you can download some large packages first and then install them locally:
484+
```bash
485+
uv pip download -d ./wheelhouse "some-large-package"
486+
uv pip install --no-index --find-links=./wheelhouse "some-large-package"
487+
```
488+
--->
489+
490+
### 3. `import simpler_env` Fails Due to Missing Vulkan Library
491+
492+
If you encounter the following error when trying to import simpler_env:
493+
```bash
494+
>>> import simpler_env
495+
Traceback (most recent call last):
496+
...
497+
ImportError: libvulkan.so.1: cannot open shared object file: No such file or directory
498+
```
499+
You can resolve this issue by installing the Vulkan runtime library via `apt`:
500+
```bash
501+
sudo apt update
502+
sudo apt install libvulkan1
503+
sudo ldconfig
504+
```
505+
506+
### 4. GCC Fails to Compile Due to Missing Dependencies
507+
508+
When compiling C++ components (e.g., `building ManiSkill2_real2sim`), you might encounter errors related to GCC or missing shared libraries. This guide walks you through how to resolve them without root/sudo permissions.
509+
510+
- Step 1: Use a modern GCC (recommended ≥ 9.3.0). Older system compilers (e.g., GCC 5.x or 7.x) may not support required C++ standards. It's recommended to switch to GCC 9.3.0 or newer:
511+
```bash
512+
export LD_LIBRARY_PATH=${PATH_TO}/gcc/gcc-9.3.0/lib64:${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
513+
export PATH=${PATH_TO}/gcc/gcc-9.3.0/bin:$PATH
514+
```
515+
> ⚠️ Note: Simply using a newer compiler might not be enough — it may depend on shared libraries that are not available on your system.
516+
- Step 2: Manually install required libraries. If you encounter errors like: `error while loading shared libraries: libmpc.so.2 (libmpfr.so.1, libgmp.so.3)`. If you do have `sudo` privileges, the easiest way is to install the required libraries system-wide using your system package manager.
517+
```bash
518+
sudo apt update
519+
sudo apt install gcc-9 g++-9
520+
```
521+
Or, you need to manually compile and install the following dependencies locally:
522+
```bash
523+
INSTALL_DIR=$HOME/local
524+
mkdir -p "$INSTALL_DIR"
525+
cd "$INSTALL_DIR"
526+
527+
wget https://ftp.gnu.org/gnu/gmp/gmp-6.2.1.tar.xz
528+
tar -xf gmp-6.2.1.tar.xz && cd gmp-6.2.1
529+
./configure --prefix="$INSTALL_DIR"
530+
make -j$(nproc)
531+
make install
532+
cd "$INSTALL_DIR"
533+
534+
wget https://www.mpfr.org/mpfr-current/mpfr-4.2.1.tar.xz
535+
tar -xf mpfr-4.2.1.tar.xz && cd mpfr-4.2.1
536+
./configure --prefix="$INSTALL_DIR" --with-gmp="$INSTALL_DIR"
537+
make -j$(nproc)
538+
make install
539+
cd "$INSTALL_DIR"
540+
541+
echo "📦 Installing MPC..."
542+
wget https://ftp.gnu.org/gnu/mpc/mpc-1.3.1.tar.gz
543+
tar -xf mpc-1.3.1.tar.gz && cd mpc-1.3.1
544+
./configure --prefix="$INSTALL_DIR" --with-gmp="$INSTALL_DIR" --with-mpfr="$INSTALL_DIR"
545+
make -j$(nproc)
546+
make install
547+
```
548+
- Step 3 (Optional): Fix Missing `.so` Versions. Sometimes you have the correct library version (e.g., `libgmp.so.10`), but GCC expects an older symlink name (e.g., `libgmp.so.3`). You can fix missing library versions with symlinks.
549+
- Step 4: Export the Library Path. Make sure the compiler can find your locally installed shared libraries:
550+
```bash
551+
export LD_LIBRARY_PATH=$HOME/local/lib:$LD_LIBRARY_PATH
552+
```

source/en/user_guide/internmanip/tutorials/environment.md

Lines changed: 104 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -42,12 +42,18 @@ eval_cfg = EvalCfg(
4242
eval_tasks=["task1", "task2", ...],
4343
res_save_path="path/to/save/results",
4444
is_save_img=False,
45-
camera_enable=CameraEnable(realsense=False, obs_camera=False, obs_camera_2=False),
46-
depth_obs=False,
45+
robot_type="franka",
4746
gripper_type="panda",
48-
env_num=1,
47+
franka_camera_enable=FrankaCameraEnable(
48+
realsense=False, obs_camera=False, obs_camera_2=False
49+
),
50+
aloha_split_camera_enable=AlohaSplitCameraEnable(
51+
top_camera=True, left_camera=True, right_camera=True
52+
),
53+
depth_obs=False,
4954
max_step=500,
5055
max_success_step=100,
56+
env_num=1,
5157
physics_dt=1/30,
5258
rendering_dt=1/30,
5359
headless=True,
@@ -63,12 +69,14 @@ eval_cfg = EvalCfg(
6369
| Parameter | Type | Default | Description |
6470
|-----------|------|---------|-------------|
6571
| `dataset_path` | str | None | If None, it will be automatically downloaded from Hugging Face dataset [URL](https://huggingface.co/datasets/OpenRobotLab/InternBench-M1) on the first run. |
66-
| `eval_tasks` | list[str] | ALL_EVAL_TASKS | Subset of 18 predefined tasks (validate against `internmanip.configs.env.genmanip_env.ALL_EVAL_TASKS`) |
72+
| `eval_tasks` | list[str] | ALL_EVAL_TASKS | The relative path to the task folder under dataset_path |
6773
| `res_save_path` | str | None | Evaluation results storage directory (disabled if None).|
6874
| `is_save_img` | bool | False | Enable per-step multi-camera image capture (requires disk space) |
69-
| `camera_enable` | CameraEnable | `CameraEnable(realsense=False, `<br>`obs_camera=False, obs_camera_2=False)` | Camera activation config:<br>• `realsense`: Ego-view gripper cam<br>• `obs_camera`: Fixed rear-view cam<br>• `obs_camera_2`: Fixed front-view cam |
75+
| `robote_type` | enum | "franka" | robot selection (`"franka"` or `"aloha_split"`) |
76+
| `gripper_type` | enum | "panda" | End effector selection **when robote_type is `franka`** (`"panda"` or `"robotiq"`) |
77+
| `franka_camera_enable` | FrankaCameraEnable | `FrankaCameraEnable(realsense=False, `<br>`obs_camera=False, `<br>`obs_camera_2=False)` | Camera activation config:<br>• `realsense`: Ego-view gripper cam<br>• `obs_camera`: Fixed rear-view cam<br>• `obs_camera_2`: Fixed front-view cam <br> Note that this only works **when robote_type is `franka`** |
78+
| `aloha_split_camera_enable` | AlohaSplitCameraEnable | `AlohaSplitCameraEnable(top_camera=False, `<br>`left_camera=False, `<br>`right_camera=False)` | Camera activation config:<br>• `top_camera`: Ego-view top head cam<br>• `left_camera`: Ego-view left gripper cam<br>• `right_camera`: Ego-view right gripper cam <br> Note that this only works **when robote_type is `aloha_split`** |
7079
| `depth_obs` | bool | False | Generate depth maps for active cameras |
71-
| `gripper_type` | enum | "panda" | End effector selection (`"panda"` or `"robotiq"`) |
7280
| `max_step` | int | 500 | Episode termination step threshold |
7381
| `headless` | bool | True | Disable GUI |
7482

@@ -85,17 +93,21 @@ eval_cfg = EvalCfg(
8593
> **Concurrency Note**: When `env_num > 1` or `ray_distribution.proc_num > 1`, environment outputs become multi-instance tensors. Agents must process batched observations and return batched actions.
8694
8795
## 3. I/O Specifications: Environment Outputs and Action Data Formats
96+
97+
### 3.1 when robote_type is **franka**
98+
8899
**Observation Structure**
100+
89101
```python
90102
observations: List[Dict] = [
91103
{
92-
"franka_robot": {
93-
"robot_pose": Tuple[array, array], # (position, oritention(quaternion))
104+
"robot": {
105+
"robot_pose": Tuple[array, array], # (position, oritention(quaternion: (w, x, y, z)))
94106
"joints_state": {
95107
"positions": array,
96108
"velocities": array
97109
},
98-
"eef_pose": Tuple[array, array], # (position, oritention(quaternion))
110+
"eef_pose": Tuple[array, array], # (position, oritention(quaternion: (w, x, y, z)))
99111
"sensors": {
100112
"realsense": {
101113
"rgb": array, # uint8 (480, 640, 3)
@@ -126,17 +138,17 @@ observations: List[Dict] = [
126138
]
127139
```
128140

129-
**Action Space Specifications**
130-
Agents must output `List[Union[ActionFormat1, ActionFormat2, ..., ActionFormat5]]` of the same length as the input observations.
141+
**Action Space Specifications**
142+
Agents must output `List[Union[List[float], dict]] = [action_1, action_2 , ...]` of the same length as the input observations.
131143
```python
132-
actions: List[Union[ActionFormat1, ActionFormat2, ..., ActionFormat5]] = [
133-
ActionFormat1,
134-
ActionFormat2,
144+
actions: List[Union[List[float], dict]] = [
145+
action_1,
146+
action_2,
135147
...
136148
]
137149
```
138150

139-
Supported action formats:
151+
The `action_x` supports any of the following formats:
140152

141153
**ActionFormat1**:
142154
```python
@@ -146,31 +158,96 @@ List[float] # (9,) or (13,) -> panda or robotiq
146158
```python
147159
{
148160
'arm_action': List[float], # (7,)
149-
'gripper_action': List[float], # (2,) or (9,) -> panda or robotiq
161+
'gripper_action': Union[List[float], int], # (2,) or (6,) -> panda or robotiq || -1 or 1 -> open or close
150162
}
151163
```
152164
**ActionFormat3**:
153165
```python
154166
{
155-
'arm_action': List[float], # (7,)
156-
'gripper_action': int, # -1 or 1 -> open or close
167+
'eef_position': List[float], # (3,) -> (x, y, z)
168+
'eef_orientation': List[float], # (4,) -> (quaternion: (w, x, y, z))
169+
'gripper_action': Union[List[float], int], # (2,) or (6,) -> panda or robotiq || -1 or 1 -> open or close
157170
}
158171
```
159-
**ActionFormat4**:
172+
173+
---
174+
175+
### 3.2 when robote_type is **aloha_split**
176+
177+
**Observation Structure**
178+
179+
```python
180+
observations: List[Dict] = [
181+
{
182+
"robot": {
183+
"robot_pose": Tuple[array, array], # (position, oritention(quaternion: (w, x, y, z)))
184+
"joints_state": {
185+
"positions": array,
186+
"velocities": array
187+
},
188+
"left_eef_pose": Tuple[array, array], # (position, oritention(quaternion: (w, x, y, z))) -> left gripper eef pose
189+
"right_eef_pose": Tuple[array, array], # (position, oritention(quaternion: (w, x, y, z))) -> right gripper eef pose
190+
"sensors": {
191+
"top_camera": {
192+
"rgb": array, # uint8 (480, 640, 3)
193+
"depth": array, # float32 (480, 640)
194+
},
195+
"left_camera": {
196+
"rgb": array,
197+
"depth": array,
198+
},
199+
"right_camera": {
200+
"rgb": array,
201+
"depth": array,
202+
},
203+
},
204+
"instruction": str,
205+
"metric": {
206+
"task_name": str,
207+
"episode_name": str,
208+
"episode_sr": int,
209+
"first_success_step": int,
210+
"episode_step": int
211+
},
212+
"step": int,
213+
"render": bool
214+
}
215+
},
216+
...
217+
]
218+
```
219+
220+
**Action Space Specifications**
221+
Agents must output `List[dict] = [action_1, action_2 , ...]` of the same length as the input observations.
222+
```python
223+
actions: List[dict] = [
224+
action_1,
225+
action_2,
226+
...
227+
]
228+
```
229+
230+
The `action_x` supports any of the following formats:
231+
232+
**ActionFormat1**:
160233
```python
161234
{
162-
'eef_position': List[float], # (3,) -> (x, y, z)
163-
'eef_orientation': List[float], # (4,) -> (quaternion)
164-
'gripper_action': List[float], # (2,) or (9,) -> panda or robotiq
235+
'left_arm_action': List[float], # (6,)
236+
'left_gripper_action': Union[List[float], int], # (2,) || -1 or 1 -> open or close
237+
'right_arm_action': List[float], # (6,)
238+
'right_gripper_action': Union[List[float], int], # (2,) || -1 or 1 -> open or close
165239
}
166240
```
167-
**ActionFormat5**:
241+
**ActionFormat2**:
168242
```python
169243
{
170-
'eef_position': List[float], # (3,) -> (x, y, z)
171-
'eef_orientation': List[float], # (4,) -> (quaternion)
172-
'gripper_action': int, # -1 or 1 -> open or close
244+
'left_eef_position': List[float], # (3,) -> (x, y, z)
245+
'left_eef_orientation': List[float], # (4,) -> (quaternion: (w, x, y, z))
246+
'left_gripper_action': Union[List[float], int], # (2,) || -1 or 1 -> open or close
247+
'right_eef_position': List[float], # (3,) -> (x, y, z)
248+
'right_eef_orientation': List[float], # (4,) -> (quaternion: (w, x, y, z))
249+
'right_gripper_action': Union[List[float], int], # (2,)|| -1 or 1 -> open or close
173250
}
174251
```
175-
252+
176253
**None Handling Protocol**: If observation element is `None` or invalid value, corresponding action must be `[]`.

0 commit comments

Comments
 (0)