About Thuman2.1 for icon

[QUANmoxiansheng]

run "python -m scripts.render_batch -debug -headless"
python -m scripts.render_batch -debug -headless
Start Rendering thuman2 with 36 views, 512x512 size.
Output dir: ./debug/thuman2_36views
Rendering types: ['light', 'normal', 'depth']
0%| | 0/1 [00:00<?, ?it/s]hands_componentsl:(45, 45)
data_struct.hands_componentsr[:num_pca_comps]:(12, 45)
Left Hand Pose: tensor([[ 0.0192, -0.0478, -0.6756, 0.0330, -0.0516, -0.6863, 0.0020, 0.0083,
-0.3404, -0.1276, -0.0921, -0.8259, -0.0590, 0.0388, -0.5376, -0.0335,
0.0615, -0.3179, -0.2578, -0.1713, -0.3081, -0.3131, -0.1107, -0.4532,
-0.0877, -0.0747, -0.3444, -0.1265, -0.1627, -0.5912, -0.0900, -0.0380,
-0.5762, -0.0611, 0.0226, -0.3637, 0.0073, 0.0445, 0.0866, 0.0435,
0.1607, 0.0659, 0.0282, 0.2099, -0.0075]])
Left Hand Pose: torch.Size([1, 45])
0%| | 0/1 [00:02<?, ?it/s]
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 74, in render_subject
rescale_fitted_body, joints = load_fit_body(
File "/home/b302003/zgq/ICON/lib/renderer/mesh.py", line 80, in load_fit_body
smpl_out = smpl_model(**model_forward_params)
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/b302003/zgq/ICON/lib/smplx/body_models.py", line 1220, in forward
left_hand_pose = torch.einsum('bi,ij->bj', [left_hand_pose, self.left_hand_components])
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/functional.py", line 373, in einsum
return einsum(equation, *_operands)
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/functional.py", line 378, in einsum
return _VF.einsum(equation, operands) # type: ignore[attr-defined]
RuntimeError: einsum(): subscript i has size 12 for operand 1 which does not broadcast with previously seen size 45
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 192, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 254, in
for _ in tqdm(
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/tqdm/std.py", line 1180, in iter
for obj in iterable:
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 865, in next
raise value
RuntimeError: einsum(): subscript i has size 12 for operand 1 which does not broadcast with previously seen size 45
I found that the structure of the hand pose in Thuman2.1 is torch Size([1, 45])
I modified mash.py：num_pca_comps=12→num_pca_comps=45
python -m scripts.render_batch -debug -headless
Start Rendering thuman2 with 36 views, 512x512 size.
Output dir: ./debug/thuman2_36views
Rendering types: ['light', 'normal', 'depth']
0%| | 0/1 [00:00<?, ?it/s]hands_componentsl:(45, 45)
data_struct.hands_componentsr[:num_pca_comps]:(12, 45)
Left Hand Pose: tensor([[ 0.0192, -0.0478, -0.6756, 0.0330, -0.0516, -0.6863, 0.0020, 0.0083,
-0.3404, -0.1276, -0.0921, -0.8259, -0.0590, 0.0388, -0.5376, -0.0335,
0.0615, -0.3179, -0.2578, -0.1713, -0.3081, -0.3131, -0.1107, -0.4532,
-0.0877, -0.0747, -0.3444, -0.1265, -0.1627, -0.5912, -0.0900, -0.0380,
-0.5762, -0.0611, 0.0226, -0.3637, 0.0073, 0.0445, 0.0866, 0.0435,
0.1607, 0.0659, 0.0282, 0.2099, -0.0075]])
Left Hand Pose: torch.Size([1, 45])
0%| | 0/1 [00:02<?, ?it/s]
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 74, in render_subject
rescale_fitted_body, joints = load_fit_body(
File "/home/b302003/zgq/ICON/lib/renderer/mesh.py", line 80, in load_fit_body
smpl_out = smpl_model(**model_forward_params)
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/b302003/zgq/ICON/lib/smplx/body_models.py", line 1220, in forward
left_hand_pose = torch.einsum('bi,ij->bj', [left_hand_pose, self.left_hand_components])
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/functional.py", line 373, in einsum
return einsum(equation, *_operands)
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/functional.py", line 378, in einsum
return _VF.einsum(equation, operands) # type: ignore[attr-defined]
RuntimeError: einsum(): subscript i has size 12 for operand 1 which does not broadcast with previously seen size 45
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 192, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 254, in
for _ in tqdm(
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/tqdm/std.py", line 1180, in iter
for obj in iterable:
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 865, in next
raise value
RuntimeError: einsum(): subscript i has size 12 for operand 1 which does not broadcast with previously seen size 45
I want to use Thuman2.1, what should I do?

[QUANmoxiansheng]

I modified mash.py：num_pca_comps=12→num_pca_comps=45
python -m scripts.render_batch -debug -headless
Start Rendering thuman2 with 36 views, 512x512 size.
Output dir: ./debug/thuman2_36views
Rendering types: ['light', 'normal', 'depth']
0%| | 0/1 [00:00<?, ?it/s]hands_componentsl:(45, 45)
data_struct.hands_componentsr[:num_pca_comps]:(45, 45)
Left Hand Pose: tensor([[ 0.0192, -0.0478, -0.6756, 0.0330, -0.0516, -0.6863, 0.0020, 0.0083,
-0.3404, -0.1276, -0.0921, -0.8259, -0.0590, 0.0388, -0.5376, -0.0335,
0.0615, -0.3179, -0.2578, -0.1713, -0.3081, -0.3131, -0.1107, -0.4532,
-0.0877, -0.0747, -0.3444, -0.1265, -0.1627, -0.5912, -0.0900, -0.0380,
-0.5762, -0.0611, 0.0226, -0.3637, 0.0073, 0.0445, 0.0866, 0.0435,
0.1607, 0.0659, 0.0282, 0.2099, -0.0075]])
Left Hand Pose: torch.Size([1, 45])
left_hand_pose:tensor([[ 0.0393, -0.0674, 0.1712, -0.0793, 0.0294, 0.2559, 0.0135, 0.0411,
-0.0908, 0.1081, 0.0022, 0.2151, -0.0212, -0.0391, 0.1139, 0.0069,
-0.0152, -0.0979, -0.2720, 0.0942, -0.0404, 0.1984, -0.0339, -0.1092,
-0.0774, -0.0360, -0.0559, -0.0490, 0.0692, 0.2110, 0.0424, 0.0240,
-0.1428, 0.0380, -0.0761, -0.1258, 0.1199, -0.0399, -0.1013, 0.0099,
0.0114, 0.0472, 0.0509, 0.1342, -0.1714]])
left_hand_pose:torch.Size([1, 45])
0%| | 0/1 [00:02<?, ?it/s]
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 74, in render_subject
rescale_fitted_body, joints = load_fit_body(
File "/home/b302003/zgq/ICON/lib/renderer/mesh.py", line 80, in load_fit_body
smpl_out = smpl_model(**model_forward_params)
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/b302003/zgq/ICON/lib/smplx/body_models.py", line 1263, in forward
vertices, joints = lbs(
File "/home/b302003/zgq/ICON/lib/smplx/lbs.py", line 207, in lbs
pose_offsets = torch.matmul(pose_feature, posedirs).view(batch_size, -1, 3)
RuntimeError: expected scalar type Double but found Float
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 192, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 254, in
for _ in tqdm(
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/tqdm/std.py", line 1180, in iter
for obj in iterable:
File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 865, in next
raise value
RuntimeError: expected scalar type Double but found Float

[QUANmoxiansheng]

I want to use Thuman2.1, what should I do?

[zjh21]

I am facing the same issue. After setting num_pca_comps to 45, I ran into double and float mismatch problems. I just tired to fix all these errors with .double(). Finally, there is a problem with key 'translation'. It seems that the correct key for THuman2.0 should be 'transl'. After that, the rendering process can be run. The results are correct for meshes from THuman2.0.

But I am still faced with incorrect camera problems. For the new meshes from THuman2.1, the rendered images seem to be pointed to the legs. I am still debuging...