A. Semantic similairity
- collect photgraphs and 50-100 word texts describing the texts
- do pretreatment of text if necessary (is there an advantage or not for pre-structured text?)
- (show the photo, user speaks, system transforms speech to text)
- perform semantic similarity analysis on texts
- perform semantic similarity analysis on images (for the moment we are not sure if this is possible)
- create semantic_similarity_matrix
- find trajectory in matrix to visit different similar photo pairs
- present sequence in a coheret interaction scenario
- use french language
- use Beam robot to present the interaction
B. Detection and interaction with person
- Detection and interaction with person1
- Recognize face from camera
- associate recognized face with photos that were provided
- associate face with other features (name, sex, hobbies)
- CHATBOT - simple spoken language interaction
C. Integration onto Beam - purely in telepresence mode: sharing screen and microphone
D. Scenario final: Two people come up system recognizes them then it finds in its database, two photos and stories that are similar from these people
Version simple: recognize one person, and tell a story using his pictures
Based on "semantic profile" the robot can propose/orient towards new activities
Biblio: Perception des patients/soignants sur les robots Attents des patients/soignants
Prepare le "pitch" pour le debut du datathon: macquette beam