Popping AR Balloons with Adafruit's Capacitive Touch Hat for Raspberry Pi

Hardware

1. Raspberry Pi 3 Model B
2. Adafruit's Capacitive Touch Hat
3. Balloons
4. Conductive fabric
5. Alligator Clips
6. Webcam
7. Printed out "Mixed Reality" Image

Software Used

1. Python - Code for the shield communicating with Raspberry Pi
2. Unity
3. Spacebrew

Workflow

Raspberry Pi and Cap Touch Hat

Getting the Cap Hat sensor to work

1. Solder the headers provided onto the shield and insert into the Raspberry Pi.
2. Enable i2C on Raspberry Pi.
3. Follow Adafruit's instructions on getting the libraries for the touch hat installed onto your Raspberry Pi.
4. Run Adafruit's "simplytest.py" and touch the sensor outputs on the board to make sure they're being read. You should see that when you touch each output (there are 12 total) - your terminal window should print out "[0] touched!" and when you let go, "[0] released!"

Connecting to Spacebrew Everytime a sensor input is touched, that data is sent to Spacebrew. We connected to balloons - one to pin 4 and another to pin 11 using alligator clips. You can connect anything, so long at it is conductive in nature (metals or even high water-content items like fruit).

Below are step-by-step instructions on how to get data from the capacitive touch hat to Spacebrew.

import sys
import time

#This is the Adafruit library for the shield
import Adafruit_MPR121.MPR121 as MPR121
from pySpacebrew.spacebrew import Spacebrew

First you need to import the appropriate libraries. Make sure the spacebrew library and Adafruit’s MPR121 library is in the same directory.

print('Adafruit MPR121 Capacitive Touch Sensor Test')

# Create MPR121 instance.
cap = MPR121.MPR121()

# Initialize communication with MPR121 using default I2C bus of device, and
# default I2C address (0x5A).  
if not cap.begin():
     print('Error initializing MPR121.  Check your wiring!')
     sys.exit(1)

Create an instance of MPR121, and initialize i2c communication between your raspberry pi and capacitive touch hat shield

# Setting up Spacebrew
# Create unique name (ex. "Juliana_Sareena") to identify your Raspberry Pi on Spacebrew
# The server IP and port are also unique
brew = Spacebrew("Juliana_Sareena", description="Popping VR Balloons", server="192.168.1.165", port=9000) 

# Two balloons - left and right - need unique publishers
brew.addPublisher("rightBalloon", "boolean")
brew.addPublisher("leftBalloon", "boolean")

To set up spacebrew, create a new instance (we called ours brew). Make sure you have a unique identifier, and description for what you’re trying to send. Your server IP address and part are also unique and should be changed

If you have more than one input (we have two balloons), then you need to publish each one to your Spacebrew variable (“brew”) in this case. We named our identifiers “right Balloon” and “leftBalloon.”

try:
    brew.start() #start the connection to Spacebrew
    print("Press Ctrl-C to quit.")
    time.sleep(0.5)
    while True:
        time.sleep(0.1) #delay prevents overloading Spacebrew with too much data
        if cap.is_touched(11):
            print('The right balloon is being touched!')
            brew.publish('rightBalloon',True)
            time.sleep(1)
        elif cap.is_touched(4):
            print('The left balloon is being touched!')
            brew.publish('leftBalloon',True)
            time.sleep(1)

finally:
    brew.stop()

The last piece of code is the loop function. We started the connection to Spacebrew with “brew.start()”. The cap.is_touched() calls a function in the MPR121 library, and the pin number of the touched input is passed. The function returns True if the input has been touched, and then the if statement sends a boolean to Spacebrew.

On the Spacebrew side - you can see from the screenshots below that our raspberry pi is now visible! And, when you touch either pins 4 or 11, it will show that the data has been sent to spacebrew. Huzzah!

Below is the entire code:

import sys
import time

#This is the Adafruit library for the shield
import Adafruit_MPR121.MPR121 as MPR121
from pySpacebrew.spacebrew import Spacebrew

print('Adafruit MPR121 Capacitive Touch Sensor Test')

# Create MPR121 instance.
cap = MPR121.MPR121()

# Initialize communication with MPR121 using default I2C bus of device, and
# default I2C address (0x5A).  
if not cap.begin():
     print('Error initializing MPR121.  Check your wiring!')
     sys.exit(1)

# Setting up Spacebrew
# Create unique name (ex. "Juliana_Sareena") to identify your Raspberry Pi on Spacebrew
# The server IP and port are also unique
brew = Spacebrew("Juliana_Sareena", description="Popping VR Balloons", server="192.168.1.165", port=9000) 

# Two balloons - left and right - need unique publishers
brew.addPublisher("rightBalloon", "boolean")
brew.addPublisher("leftBalloon", "boolean")

try:
    brew.start() #start the connection to Spacebrew
    print("Press Ctrl-C to quit.")
    time.sleep(0.5)
    while True:
        time.sleep(0.1) #delay prevents overloading Spacebrew with too much data
        if cap.is_touched(11):
            print('The right balloon is being touched!')
            brew.publish('rightBalloon',True)
            time.sleep(1)
        elif cap.is_touched(4):
            print('The left balloon is being touched!')
            brew.publish('leftBalloon',True)
            time.sleep(1)

finally:
    brew.stop()

Set Up Experiment Space

1. Place two black pieces of construction paper on a table
2. Tape the printed out anchor ( “Mixed Reality” with image of stones) onto the center of the paper Dimensions.
3. Blow up two balloons to a small-medium size size. On each, place the conductive fabric tape vertically along the length of the balloon.
4. Cut a slit at the end of the conductive fabric on each balloon, near the knotted base, and dog-ear the fabric (this is where you’ll be attaching the alligator clips)
5. Tape the balloons onto the black paper.
6. Set up a webcam over the experiment area

Connect RaspberryPi to Balloons

• Left Balloon: Connect an alligator clip from Pin 4 to the base of the balloon (clip it onto the dog-eared fabric).
• Right Balloon: Connect an alligator clip from Pin 11 to the base of the balloon (clip it onto the dog-eared fabric).

Create AR visuals in Unity

Create bubble visuals in Unity

1. Open MRHW_Proto001 file in Unity
2. Create a new scene with a Bubbly name
3. Create two new 3D sphere game objects, save as follows in the hierarchy:
4. Adjust size and texture to your liking
5. Toggle between "play" and "edit" modes to align 3D spheres to the balloons
6. Disable baseboard and other visible objects.

Define bubble behaviors

Connect bubble behaviors to Spacebrew

Connect Everything on Spacebrew