Knife Game with DC Motor HAT and Solenoids
A mixed reality application of the knife game (https://en.wikipedia.org/wiki/Knife_game), demonstrating the impact of visual & audio perception on tactile perception.
The user puts their left hand's index finger in the case, and the experience starts. On the screen, the user can see an image of a virtual hand, with a virtual knife stabbing to the left and right of the virtual index finger. This is accompanied by an audible "tock...tock...tock", accompanied by the slight vibrations felt by the physical hand. Suddenly the knife on the screen stabs into the virtual finger, releasing virtual blood. At the same time the user feels a stab on the user's physical finger.
Here is a GIF of what the setup looked like in motion.
And here is a slow motion GIF of the solenoids working (they are too fast for a regular motion recording!).
This interaction has an animation happening on screen triggering something in the physical world. Because of this, Unity first connects to a Spacebrew server on the same laptop it is running on. This decision was taken in order to reduce the latency between the animation and the haptic feedback from the solenoids. Spacebrew communicates with the Raspberry Pi which is connected to the same network as the laptop. Finally, the Raspberry Pi triggers the solenoid every time it receives the appropriate message from Spacebrew.
1x Raspberry Pi incl. Power Cable 1x Stepper & DC Motor HAT 3x Solenoids 3x Male-Male Jumper Wires (to connect to ground) 3x Male-Male Jumper Wires (to connect to power) External Web Cam MDF to build case
Raspberry Pi 2.x Spacebrew (local version on computer) Unity 2017.3.0f3 Python 2.7 (NOT 3) Adafruit Motor HAT driver https://github.com/adafruit/Adafruit-Motor-HAT-Python-Library (documentation here: https://buildmedia.readthedocs.org/media/pdf/adafruit-motor-hat/latest/adafruit-motor-hat.pdf)
Use example code https://learn.adafruit.com/adafruit-dc-and-stepper-motor-hat-for-raspberry-pi?view=all
The scene was built on top of the base scene on the YUXI examples to take advantage of the Spacebrew connectivity.
Search for virtual hand and virtual knife on Google Poly (https://poly.google.com/). Animate the virtual knife by adding an Animator Component. Add an animation clip: record an animation clip of the knife going back-and-forth between the virtual hand's index finger, by recording in the Animation window and manually adjusting the knife's Position in the x, y, and z-axis on the Animation record window (https://www.youtube.com/watch?v=JeZkctmoBPw)
You will need to hard code the position (x,y and z coordinates) of the virtual assets (virtual hand and virtual knife). Do note there is a glitch: when Unity isn't played, the position of the virtual assets is not the same as when the Unity file is played. Because of this positioning discrepancy, we moved away from triggering the solenoids through collision detection on Unity and instead decided for triggering the solenoids via the animation itself, as shown below.
The code you are calling at each animation event is written on a script attached to the knife object. The code, described below, calls a function on the SpacebrewObject's Event script.
On the SpacebrewEvents script, attached to the SpacebrewObject, you add these functions, which are called by the previous animation.
On SpacebrewObject, you have to adjust the IP address of the server you're connecting to and create 3 publishers, one for each solenoid that you are activating.
It is important to pay attention to the names of the functions and the publishers. If you change the name of the publisher, make sure you also change the name that is being called by the function.
Laser cut MDF to fit three solenoids in an MDF box, which can fit a finger. Use black foam to block the casing entrance, to ensure that the index finger can only fit under the center solenoid. Ensure that the fiducial is visible to the webcam. For best effect, cover the entire setup such that the user cannot see their hands after putting it into the casing.