iCub facial features in Gazebo

[patrickshas]

Brief description

Is there a way of using facial features of iCub in Gazebo?

Detailed context

I am just wondering if there is a way of utilizing the LED arrays for eyebrows and mouth for expression control in Gazebo?

Accompanying material

[pattacini]

Hi @patrickshas

I'm afraid we don't have the facial expressions in Gazebo yet unless anyone of our community has implemented them and is willing to contribute upstream.

cc @robotology/everyone

[traversaro]

Face expressions are not supported on the iCub model at the moment, but the relative plugin (gazebo_yarp_videotexture) is available and has been used for implementing facial expressions on the R1 robot see robotology/cer-sim#17 and robotology/cer-sim#18 .

As @pattacini said, I think at the moment there is no plan to work on this on the IIT side, but I opened a dedicated issue to track the implementation of this feature if someone in the community is interested in robotology/icub-models#37 .

[patrickshas]

Hi @pattacini and @traversaro , I could possibly work on this and contribute, since it’s relevant to my PhD research. A question: where can I find the specs and setup of the new generation face LEDs so I could get started? Thanks

[pattacini]

Nice to hear that @patrickshas 👍

Let me call for @marcoaccame and @Nicogene who worked on the implementation of the new expressions. They could possibly point you at the right direction.

[marcoaccame]

Hi @patrickshas, let us collect the relevant info and we shall soon come back to you.

[Nicogene]

Hi @patrickshas ,

here is the the rpc protocol of the real icub face expression:

set <part> <emotion>  set a specific emotion for a defined part   
        the available part are: mou, eli, leb, reb, all
        the available emotions are: neu, hap, sad, sur, ang, evi, shy, cun

set color <color>     set the color of the led
        the available colors are: black, white, red, lime, blue, yellow, cyan, magenta, silver, gray, maroon, olive, green, purple, teal, navy

set brig <brig>       set the brightness of the leds
        the brightness is defined by an iteger from 0 to 5, where 0 means led of

set mask (<col_leb> <m_name_leb> <brig_leb>) (<col_reb> <m_name_reb> <brig_reb>) (<col_mou> <m_name_mou> <brig_mou>) set color, bitmask and brightness for each part(leb, reb, mou)
        m_name stands for mask name and the availble bitmasks can be consulted/added in faceExpressions/emotions_rfe.ini

Here is some example of bitmasks:

bitmask_eyebrow_emotions 4
BM_EB0 (0 4 8 12 16)  //00011111h // surprised
BM_EB1 (1 5 9 13 17)  //00022222h // neutral, happy, shy
BM_EB2 (2 6 10 14 18) //00044444h // sad
BM_EB3 (3 7 11 15 19) //00044444h // angry

bitmask_mouth_emotions 5
BM_M0  (6 9 10 18)                     //00020300h // neutral
BM_M1  (0 2 6 9 10 12 14 18)           //00045645h // happy, evil
BM_M2  (4 5 6 9 10 16 17 18)           //00070670h // sad
BM_M3  (1 2 3 5 7 8 11 13 14 15 17 19) //000AE9AEh // surprised
BM_M4  (4 5 6 9 10 16 17 18)           //00070670h // angry

For example (6 9 10 18) means specify which that led 6, 9, 10 and 18 has to be set to on of that specific segment.

There are 3 segments, Left eybrow, right eyebrow and mouth as follow:
(⚠️ note that the enumeration is 1-based, the bitmasks are 0-based)

[aerydna]

Hi, remember that the implementation of the facial expression in the icub gazebo model actually requires a different approach wrt the implementation of the facial expression of R1. in R1 the operation to do was to generate the 3d model with a specific UV coordinates and using the videotexture plugin. Here to do a decent implementation is required to modify the icub model, cutting the led over the face (disconnecting the polygons) and developing a small plugin which is able to change the base color of the material. If i find some time in the next days i'll post an example of what i meant

[aerydna]

this is an example of how conceptually it should be implemented and how meshes should be split(prototyped in cinema 4d). the same exact thing is doable in gazebo
@traversaro

[jimingda]

Hi @aerydna

The icub model urdf version i use in gazebo is v2.5 currently, is there any method to integrate the same mesh of eye ball as you showed in the video above, or shall i update the model to a latest version.

Thanks in advance.

[traversaro]

Hi @jimingda, I think there are no updates on this beside the issues already linked in this discussion.

[aerydna]

I Guess It Is possibile to integrate my example to any model actually. What i did was a simple boolean subtraction between the icub face mesh and a mesh obtained by extruding some splines, obtained with the image of the icub facial parts posted here