Knife Game with DC Motor HAT and Solenoids
A mixed reality application of the knife game (https://en.wikipedia.org/wiki/Knife_game), demonstrating the impact of visual & audio perception on tactile perception.
The user puts their left hand's index finger in the case, and the experience starts. On the screen, the user can see an image of a virtual hand, with a virtual knife stabbing to the left and right of the virtual index finger. This is accompanied by an audible "tock...tock...tock", accompanied by the slight vibrations felt by the physical hand. Suddenly the knife on the screen stabs into the virtual finger, releasing virtual blood. At the same time the user feels a stab on the user's physical finger.
Here is a GIF of what the setup looked like in motion.
And here is a slow motion GIF of the solenoids working (they are too fast for a regular motion recording!).
This interaction has an animation happening on screen triggering something in the physical world. Because of this, Unity first connects to a Spacebrew server on the same laptop it is running on. This decision was taken in order to reduce the latency between the animation and the haptic feedback from the solenoids. Spacebrew communicates with the Raspberry Pi which is connected to the same network as the laptop. Finally, the Raspberry Pi triggers the solenoid every time it receives the appropriate message from Spacebrew.
- 1x Raspberry Pi incl. Power Cable
- 1x Stepper & DC Motor HAT
- 3x Solenoids
- 3x Male-Male Jumper Wires (to connect to ground)
- 3x Male-Male Jumper Wires (to connect to power)
- External Web Cam
- MDF to build case
- Raspberry Pi 2.x
- Spacebrew (local version on computer)
- Unity 2017.3.0f3
- Python 2.7 (NOT 3)
- Adafruit Motor HAT driver https://github.com/adafruit/Adafruit-Motor-HAT-Python-Library (documentation here: https://buildmedia.readthedocs.org/media/pdf/adafruit-motor-hat/latest/adafruit-motor-hat.pdf)
Use example code https://learn.adafruit.com/adafruit-dc-and-stepper-motor-hat-for-raspberry-pi?view=all
See the Python script below. Documentation in comments (preceded by #). This script needs to be installed in the Pi (with the Motor HAT). To do that, we copied the script in our computer's compiler, logged into the Raspberry Pi (via Terminal, using ssh pi@Pi's IP address), used the nano command to create a new Python file (e.g. "nano TestScript.py"), pasted the script into the new file, and ran the script using the python command (e.g. "python TestScript.py") Note: please note that space and line indents are a critical part of Python's syntax. The code below will need to be refactored and formatted to properly compile.
import sys
import time
from pySpacebrew.spacebrew import Spacebrew
import RPi.GPIO as GPIO
#Setup Motor Hat
from Adafruit_MotorHAT import Adafruit_MotorHAT, Adafruit_DCMotor
import atexit
# create a default object, no changes to I2C address or frequency
mh = Adafruit_MotorHAT(addr=0x60)
# recommended for auto-disabling motors on shutdown!
def turnOffMotors():
mh.getMotor(1).run(Adafruit_MotorHAT.RELEASE)
mh.getMotor(2).run(Adafruit_MotorHAT.RELEASE)
mh.getMotor(3).run(Adafruit_MotorHAT.RELEASE)
atexit.register(turnOffMotors)
#Declare motors connected to the Motor Hat
myMotorLeft = mh.getMotor(1)
myMotorCenter = mh.getMotor(2)
myMotorRight = mh.getMotor(3)
myMotorLeft.setSpeed(150)
myMotorCenter.setSpeed(255)
myMotorRight.setSpeed(150)
# Setup Spacebrew with Publishers and Subscribers
brew = Spacebrew("D&S-YUXI_Solenoid", description="Python Solenoid Motor Hat", $
brew.addSubscriber("flipMotorLeft", "boolean")
brew.addSubscriber("flipMotorCenter", "boolean")
brew.addSubscriber("flipMotorRight", "boolean")
GPIO.setwarnings(False)
GPIO.setmode(GPIO.BCM)
GPIO.setup(18, GPIO.IN) #down is False
alreadySent = False # to 'debounce' the button
def handleBooleanLeft(value):
print("Received: "+str(value))
if (value == 'true' or str(value) == 'True'):
ActivateMotorLeft()
def handleBooleanCenter(value):
print("Received: "+str(value))
if (value == 'true' or str(value) == 'True'):
ActivateMotorCenter()
def handleBooleanRight(value):
print("Received: "+str(value))
if (value == 'true' or str(value) == 'True'):
ActivateMotorRight()
def ActivateMotorLeft():
print("Forward Left! ")
myMotorLeft.run(Adafruit_MotorHAT.BACKWARD)
print("Backward Left! ")
myMotorLeft.run(Adafruit_MotorHAT.RELEASE)
print("Sleeping Left")
def ActivateMotorCenter():
print("Forward Center! ")
myMotorCenter.run(Adafruit_MotorHAT.BACKWARD)
print("Backward Center! ")
myMotorCenter.run(Adafruit_MotorHAT.RELEASE)
print("Sleeping Center")
def ActivateMotorRight():
print("Forward Right! ")
myMotorRight.run(Adafruit_MotorHAT.BACKWARD)
print("Backward Right! ")
myMotorRight.run(Adafruit_MotorHAT.RELEASE)
print("Sleeping Right")
# for handling messages coming through spacebrew
brew.subscribe("flipMotorLeft", handleBooleanLeft)
brew.subscribe("flipMotorCenter", handleBooleanCenter)
brew.subscribe("flipMotorRight", handleBooleanRight)
try:
brew.start()
while True:
print("Staying alive")
time.sleep(0.1)
finally:
brew.stop()
The scene was built on top of the base scene on the YUXI examples to take advantage of the Spacebrew connectivity.
Search for virtual hand and virtual knife on Google Poly (https://poly.google.com/). Animate the virtual knife by adding an Animator Component. Add an animation clip: record an animation clip of the knife going back-and-forth between the virtual hand's index finger, by recording in the Animation window and manually adjusting the knife's Position in the x, y, and z-axis on the Animation record window (https://www.youtube.com/watch?v=JeZkctmoBPw)
You will need to hard code the position (x,y and z coordinates) of the virtual assets (virtual hand and virtual knife). Do note there is a glitch: when Unity isn't played, the position of the virtual assets is not the same as when the Unity file is played. Because of this positioning discrepancy, we moved away from triggering the solenoids through collision detection on Unity and instead decided for triggering the solenoids via the animation itself, as shown below.
The code you are calling at each animation event is written on a script attached to the knife object. The code, described below, calls a function on the SpacebrewObject's Event script.
On the SpacebrewEvents script, attached to the SpacebrewObject, you add these functions, which are called by the previous animation.
On SpacebrewObject, you have to adjust the IP address of the server you're connecting to and create 3 publishers, one for each solenoid that you are activating.
It is important to pay attention to the names of the functions and the publishers. If you change the name of the publisher, make sure you also change the name that is being called by the function.
Laser cut MDF to fit three solenoids in an MDF box, which can fit a finger. Use black foam to block the casing entrance, to ensure that the index finger can only fit under the center solenoid. Ensure that the fiducial is visible to the webcam. For best effect, cover the entire setup such that the user cannot see their hands after putting it into the casing.