Constraints for the grasped object in real life

[Promethees]

We are conducting an experiment using an external camera (Intel Realsense) to perform the grasping example with real world objects. However, whenever any unexpected characteristic was detected, the system threw out this error:

I wonder currently there is any restriction or limitation for the objects to be compatible with the grasping model that the grasp planner was trained for?

[Briancbn]

@Promethees There seems to be two error here in the log.

1. vtk viewer issue for the PCL Cloud viewer (which is ultimately causing the crash) @tanjpg Please help advise on this.

2. Unable to calculate grasp plan

   I would recommend testing the point cloud from RealSense directly with EMD (without EPD) first to ensure that there is no configuration issue on the EMD side. You can simply set the epd_enabled to false in the configuration file and the camera_parameters accordingly based on the RealSense output you have.

   If grasp planner is still not able to detect the grasp properly, I would suspect that due to the different setup, the passthrough filter is not working properly. This is to filter out the unnecessary points wrt the camera frame. If your camera setup is different, you need to change the passthrough filter configuration accordingly too.

I wonder currently there is any restriction or limitation for the objects to be compatible with the grasping model that the grasp planner was trained for?

There is no training involved the grasping algorithm implemented in EMD, as explained here, it is purely heuristic based. Take two-fingered gripper as an example, as long as there is a part of the object width that is smaller that the maximum grasping distance (2 * gripper_stroke), grasp planner should reach a solution for the object pointcloud.

However, the current grasp planner + execution is still limited mostly to a top down approach, but I believe @tanjpg can explain more on the limitation side.

[tanjpg]

@Promethees try setting the point_cloud_visualization parameter in the config file to false. This should prevent the viewer from launching.

For the Grasp Planner not planning, it can be due to the way the pointcloud is generated. Take a look at the General Parameters and Finger Parameters Documentation to see if some of the parameters can be tweaked.

Ideally, the Point Cloud Viewer would be a way to verify the tweak parameters work. Can I check if you are running EPD and EMD on the same machine? We also have faced occasional problems with the Pointcloud viewer when EPD is run on the same machine.

One other way as well is to show us your configuration file and we can check to see if there are any parameters can might be changed to improve the planning.

[Promethees]

Adjustments that I've tried to implement:

• Tweak epd_enabled to false, point_cloud_visualization to false
• Change PointCloud2 topic in rviz to
• Adjust the z component of parameter xyz of _d415 camera in the scene.urdf.xacro file for the execution launcher to be compatible with the real life setting. So that the "floor" my RealSense facing at can be placed right on the surface of the arm's base.

With these adjustment, everything works as expected, but only for a short while. After many trial, I figure this comes from the overload of RAM cause by the grasp_planning_node, which can be observed is shown below. Whenever the memory exploded, the grasp_planning_node died as a consequence.

I'm suspecting that this indicates potential memory leaks caused by grasp_planning_node.
P/s: in the figure given above, I'm running realsense_node in the leftmost window above, execution_node in the middle window above, and grasp_planning_node in the middle window below. The problem is whenever the grasp_planning_node was running, memory occupied kept rising up to the limit then killed everything.

[Briancbn]

We don't usually encounter this issue and we do run this for some prolong periods of time, I am suspecting disabling the viewer causes memory leak?
Let us try to reproduce it on our end and get back to you. Meanwhile, if you have time, maybe can also check if this issue occurs with the provided bag file.

[Promethees]

Hi @Briancbn, so far it seems that there's currently no rosbag files being provided in the repo, or rosbag folder under grasp_planner directory, as promised here...