World On My Shoulders
Hardware
- 1 x Raspberry Pi 3 (Model B)
- 1 x Adafruit NeoPixel LED Strip with Alligator Clips - 30 LED/m
- 1 x Adafruit ADS1115 4-channel breakout boards
- 1 x Adafruit Fadecandy
Software
- Unity
- Processing
- Spacebrew
- Visual Studio Code (text editor)
- Mixed Reality Hardware Kit Library
- NeoPixel library
Assembling hardware and setting up the circuit
- Follow the circuit shown in Image 1 below
Connecting Raspberry Pi (“Pi”) to Spacebrew
-
Set up our Pi to enable access through Wi-Fi instead of USB cable
-
Install YUXI Mixed Reality Hardware Kit Library (“MRHKL”) onto our Pi
-
Log in to our Pi using the terminal via ssh:
ssh pi @[address our Pi received from wifi router] -
Install netatalk by entering:
sudo apt-get install netatalkto access our Pi folders through our computer.
Setting up Fadecandy Server to connect with NeoPixel
Using the terminal:
-
Install
sudo apt-get -y install git -
Retrieve the Fadecandy software from:
git clone git://github.com/scanlime/fadecandy -
Run the following:
cd fadecandy/servermake submodulesmake -
Once finished, enter:
sudo mv fcserver /usr/local/bin -
To start the fcserver program automatically when system boots, enter: @sudo nano /etc/rc.local.@
-
Copy and paste the following just above the final “exit 0” line:
/usr/local/bin/fcserver /usr/local/bin/fcserver.json >/var/log/fcserver.log 2>&1 & -
Create a new configuration file:
sudo nano /usr/local/bin/fcserver.json -
Copy and paste the following into the new file:
{
"listen": [null, 7890],
"verbose": true,
"color": {
"gamma": 2.5,
"whitepoint": [0.7, 0.7, 0.7]
},
"devices": [
"type": "fadecandy",
"serial": "[Serial Number]",
"map": [
[ 0, 0, 0, 0 ],
[ 0, 60, 64, 29 ],
[ 0, 120, 128, 29 ],
[ 0, 180, 192, 29 ],
[ 0, 240, 256, 29 ],
[ 0, 300, 320, 29 ],
[ 0, 360, 384, 29 ],
[ 0, 420, 448, 29 ]
}
Notes: Replace “X” with the number of LEDs your NeoPixel strip contains. Locate serial number by connecting our Fadecandy board to a USB port. A message with our serial number will show: USB device Fadecandy (Serial# TVUCPRXHXJQJOFKR, Version 1.07) attached. Copy this serial number to the serial line in the fcserver.json file
-
Reboot our Pi by entering
sudo reboot -
After rebooting, enter
tail -f /var/log/fcserver.logto verify that the Fadecandy server is running without problems.
Processing is a hub that communicates between the Spacebrew and Fadecandy servers. When a collision event is triggered in Unity, Unity publishes a value which is sent to the Fadecandy server through Processing (via the Spacebrew server, Processing and our Pi), thereby lighting up an LED. The Spacebrew server also has a “Slider Range Control Example” that allows for manual manipulation of the LEDs.
Fadecandy is connected with the Raspberry Pi to operate as a controller of the LEDs.
Spacebrew is a platform that allows communication between Processing and Unity and external controllers (e.g., Slider Range Example).
- Import the code below into Processing
import spacebrew.*;
// Update string server IP with our Spacebrew server IP
String server="192.168.1.191";
// Name of our subscriber in the Spacebrew server
String name="FadeCandyExample";
String description ="Client that sends and receives range messages. Range values go from 0 to 1023.";
// This is our start value of the remote slider on Spacebrew
int remote_slider_val = 512;
Spacebrew sb;
// Number of LEDS used in our project
int numLEDS = 29;
int incomingMappedValue;
// OPC is not a library that can be downloaded. A new tab must be created separately for the OPC code to run. The OPC code will be provided further below.
OPC opc;
PImage dot;
void setup()
{
size(800, 200);
frameRate(10);
// Instantiate the spacebrewConnection variable
sb = new Spacebrew( this );
// Declare our subscribers
sb.addSubscribe( "remote_slider", "range" );
// Connect!
sb.connect(server, name, description );
// Load a sample image
dot = loadImage("color-dot.png");
// Connect to the local instance of the fcserver, the Fadecandy server created earlier. Replace the below IP address with the IP address of our Raspberry Pi.
opc = new OPC(this, "192.168.1.128", 7890);
// Map one 30-LED strip to the center of the window
//opc.ledStrip(0, 30, width/2, height/2, width / 70.0, 0, false);
colorMode(HSB, 100);
}
void draw()
{
background(0);
}
void onRangeMessage( String name, int value ){
println("got range message " + name + " : " + value);
remote_slider_val = value;
// Take incoming range value that will be between 0 and 1023
// Map that value to the number of LEDS
incomingMappedValue = int(map(remote_slider_val, 0, 1023, 0, numLEDS));
// Call the draw with that number and light that many LEDS
drawToLEDS(incomingMappedValue);
print(incomingMappedValue);
}
void drawToLEDS(int _numOfLEDS) {
for (int i = 0; i < numLEDS; i++) {
if (i < _numOfLEDS) {
opc.setPixel(i, color(255, 255, 255));
} else {
opc.setPixel(i, color(0, 0, 0));
// When you haven't assigned any LEDs to pixels, you have to explicitly write them to the server. Otherwise, this happens automatically after draw().
opc.writePixels();
}
- In a new tab under the same sketch, import the following for the OPC:
/*
* Simple Open Pixel Control client for Processing,
* designed to sample each LED's color from some point on the canvas.
* Micah Elizabeth Scott, 2013
* This file is released into the public domain.
*/
import java.net.*;
import java.util.Arrays;
public class OPC implements Runnable
{
Thread thread;
Socket socket;
OutputStream output, pending;
String host;
int port;
int[] pixelLocations;
byte[] packetData;
byte firmwareConfig;
String colorCorrection;
boolean enableShowLocations;
OPC(PApplet parent, String host, int port)
this.host = host;
this.port = port;
thread = new Thread(this);
thread.start();
this.enableShowLocations = true;
parent.registerMethod("draw", this);
// Set the location of a single LED
void led(int index, int x, int y)
// For convenience, automatically grow the pixelLocations array. We do want this to be an array,
// instead of a HashMap, to keep draw() as fast as it can be.
if (pixelLocations == null) {
pixelLocations = new int[index + 1];
} else if (index >= pixelLocations.length) {
pixelLocations = Arrays.copyOf(pixelLocations, index + 1);
pixelLocations[index] = x + width * y;
// Set the location of several LEDs arranged in a strip.
// Angle is in radians, measured clockwise from +X.
// (x,y) is the center of the strip.
void ledStrip(int index, int count, float x, float y, float spacing, float angle, boolean reversed)
float s = sin(angle);
float c = cos(angle);
for (int i = 0; i < count; i++) {
led(reversed ? (index + count - 1 - i) : (index + i),
(int)(x + (i - (count-1)/2.0) * spacing * c + 0.5),
(int)(y + (i - (count-1)/2.0) * spacing * s + 0.5));
// Set the locations of a ring of LEDs. The center of the ring is at (x, y),
// with "radius" pixels between the center and each LED. The first LED is at
// the indicated angle, in radians, measured clockwise from +X.
void ledRing(int index, int count, float x, float y, float radius, float angle)
for (int i = 0; i < count; i++) {
float a = angle + i * 2 * PI / count;
led(index + i, (int)(x - radius * cos(a) + 0.5),
(int)(y - radius * sin(a) + 0.5));
// Set the location of several LEDs arranged in a grid. The first strip is
// at 'angle', measured in radians clockwise from +X.
// (x,y) is the center of the grid.
void ledGrid(int index, int stripLength, int numStrips, float x, float y,
float ledSpacing, float stripSpacing, float angle, boolean zigzag,
boolean flip)
float s = sin(angle + HALF_PI);
float c = cos(angle + HALF_PI);
for (int i = 0; i < numStrips; i++) {
ledStrip(index + stripLength * i, stripLength,
x + (i - (numStrips-1)/2.0) * stripSpacing * c,
y + (i - (numStrips-1)/2.0) * stripSpacing * s, ledSpacing,
angle, zigzag && ((i % 2) == 1) != flip);
// Set the location of 64 LEDs arranged in a uniform 8x8 grid.
// (x,y) is the center of the grid.
void ledGrid8x8(int index, float x, float y, float spacing, float angle, boolean zigzag,
boolean flip)
ledGrid(index, 8, 8, x, y, spacing, spacing, angle, zigzag, flip);
// Should the pixel sampling locations be visible? This helps with debugging.
// Showing locations is enabled by default. You might need to disable it if our drawing
// is interfering with your processing sketch, or if you'd simply like the screen to be
// less cluttered.
void showLocations(boolean enabled)
enableShowLocations = enabled;
// Enable or disable dithering. Dithering avoids the "stair-stepping" artifact and increases color
// resolution by quickly jittering between adjacent 8-bit brightness levels about 400 times a second.
// Dithering is on by default.
void setDithering(boolean enabled)
if (enabled)
firmwareConfig &= ~0x01;
else
firmwareConfig |= 0x01;
sendFirmwareConfigPacket();
// Enable or disable frame interpolation. Interpolation automatically blends between consecutive frames
// in hardware, and it does so with 16-bit per channel resolution. Combined with dithering, this helps make
// fades very smooth. Interpolation is on by default.
void setInterpolation(boolean enabled)
if (enabled)
firmwareConfig &= ~0x02;
else
firmwareConfig |= 0x02;
sendFirmwareConfigPacket();
// Put the Fadecandy onboard LED under automatic control. It blinks any time the firmware processes a packet.
// This is the default configuration for the LED.
void statusLedAuto()
firmwareConfig &= 0x0C;
sendFirmwareConfigPacket();
}
// Manually turn the Fadecandy onboard LED on or off. This disables automatic LED control.
void setStatusLed(boolean on)
firmwareConfig |= 0x04; // Manual LED control
if (on)
firmwareConfig |= 0x08;
else
firmwareConfig &= ~0x08;
sendFirmwareConfigPacket();
}
// Set the color correction parameters
void setColorCorrection(float gamma, float red, float green, float blue)
colorCorrection = "{ \"gamma\": " + gamma + ", \"whitepoint\": [" + red + "," + green + "," + blue + "]}";
sendColorCorrectionPacket();
// Set custom color correction parameters from a string
void setColorCorrection(String s)
colorCorrection = s;
sendColorCorrectionPacket();
// Send a packet with the current firmware configuration settings
void sendFirmwareConfigPacket()
if (pending == null) {
// We'll do this when we reconnect
return;
byte[] packet = new byte[9];
packet[0] = (byte)0x00; // Channel (reserved)
packet[1] = (byte)0xFF; // Command (System Exclusive)
packet[2] = (byte)0x00; // Length high byte
packet[3] = (byte)0x05; // Length low byte
packet[4] = (byte)0x00; // System ID high byte
packet[5] = (byte)0x01; // System ID low byte
packet[6] = (byte)0x00; // Command ID high byte
packet[7] = (byte)0x02; // Command ID low byte
packet[8] = (byte)firmwareConfig;
try {
pending.write(packet);
} catch (Exception e) {
dispose();
// Send a packet with the current color correction settings
void sendColorCorrectionPacket()
if (colorCorrection == null) {
// No color correction defined
return;
if (pending == null) {
// We'll do this when we reconnect
return;
byte[] content = colorCorrection.getBytes();
int packetLen = content.length + 4;
byte[] header = new byte[8];
header[0] = (byte)0x00; // Channel (reserved)
header[1] = (byte)0xFF; // Command (System Exclusive)
header[2] = (byte)(packetLen >> 8); // Length high byte
header[3] = (byte)(packetLen & 0xFF); // Length low byte
header[4] = (byte)0x00; // System ID high byte
header[5] = (byte)0x01; // System ID low byte
header[6] = (byte)0x00; // Command ID high byte
header[7] = (byte)0x01; // Command ID low byte
try {
pending.write(header);
pending.write(content);
} catch (Exception e) {
dispose();
// Automatically called at the end of each draw().
// This handles the automatic Pixel to LED mapping.
// If you aren't using that mapping, this function has no effect.
// In that case, you can call setPixelCount(), setPixel(), and writePixels()
// separately.
void draw()
if (pixelLocations == null) {
// No pixels defined yet
return;
if (output == null) {
return;
int numPixels = pixelLocations.length;
int ledAddress = 4;
setPixelCount(numPixels);
loadPixels();
for (int i = 0; i < numPixels; i++) {
int pixelLocation = pixelLocations[i];
int pixel = pixels[pixelLocation];
packetData[ledAddress] = (byte)(pixel >> 16);
packetData[ledAddress + 1] = (byte)(pixel >> 8);
packetData[ledAddress + 2] = (byte)pixel;
ledAddress += 3;
if (enableShowLocations) {
pixels[pixelLocation] = 0xFFFFFF ^ pixel;
writePixels();
if (enableShowLocations) {
updatePixels();
// Change the number of pixels in our output packet.
// This is normally not needed; the output packet is automatically sized
// by draw() and by setPixel().
void setPixelCount(int numPixels)
int numBytes = 3 * numPixels;
int packetLen = 4 + numBytes;
if (packetData == null || packetData.length != packetLen) {
// Set up our packet buffer
packetData = new byte[packetLen];
packetData[0] = (byte)0x00; // Channel
packetData[1] = (byte)0x00; // Command (Set pixel colors)
packetData[2] = (byte)(numBytes >> 8); // Length high byte
packetData[3] = (byte)(numBytes & 0xFF); // Length low byte
// Directly manipulate a pixel in the output buffer. This isn't needed
// for pixels that are mapped to the screen.
void setPixel(int number, color c)
int offset = 4 + number * 3;
if (packetData == null || packetData.length < offset + 3) {
setPixelCount(number + 1);
packetData[offset] = (byte) (c >> 16);
packetData[offset + 1] = (byte) (c >> 8);
packetData[offset + 2] = (byte) c;
// Read a pixel from the output buffer. If the pixel was mapped to the display,
// this returns the value we captured on the previous frame.
color getPixel(int number)
int offset = 4 + number * 3;
if (packetData == null || packetData.length < offset + 3) {
return 0;
return (packetData[offset] << 16) | (packetData[offset + 1] << 8) | packetData[offset + 2];
// Transmit our current buffer of pixel values to the OPC server. This is handled
// automatically in draw() if any pixels are mapped to the screen, but if you haven't
// mapped any pixels to the screen you'll want to call this directly.
void writePixels()
if (packetData == null || packetData.length == 0) {
// No pixel buffer
return;
if (output == null) {
return;
try {
output.write(packetData);
} catch (Exception e) {
dispose();
void dispose()
// Destroy the socket. Called internally when we've disconnected.
// (Thread continues to run)
if (output != null) {
println("Disconnected from OPC server");
socket = null;
output = pending = null;
public void run()
// Thread tests server connection periodically, attempts reconnection.
// Important for OPC arrays; faster startup, client continues
// to run smoothly when mobile servers go in and out of range.
for(;;) {
if(output == null) { // No OPC connection?
try { // Make one!
socket = new Socket(host, port);
socket.setTcpNoDelay(true);
pending = socket.getOutputStream(); // Avoid race condition...
println("Connected to OPC server");
sendColorCorrectionPacket(); // These write to 'pending'
sendFirmwareConfigPacket(); // rather than 'output' before
output = pending; // rest of code given access.
// pending not set null, more config packets are OK!
} catch (ConnectException e) {
dispose();
} catch (IOException e) {
dispose();
// Pause thread to avoid massive CPU load
try {
Thread.sleep(500);
catch(InterruptedException e) {
}
-
Create a GameObject in the form of a sphere (“Sphere”) .
-
Create a second GameObject in the form of a cube (“Cube”) and transform it to obtain properties of a floor, for the sphere to collide against.
-
Copy and paste the following code under our Cube by adding a new script under new component, named CollisionEvents:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class CollisionEvents : MonoBehaviour {
public GameObject spacebrewScript;
int collisionCount=0;
// Use this for initialization
void Start () {
}
public GameObject cloneScript;
void OnCollisionEnter(Collision collision){
// cloneScript.GetComponent<AddElements>().makeObject();
Debug.Log("collision started");
}
void OnCollisionStay (Collision collision){
Debug.Log("Stay occuring..");
}
void OnCollisionExit (Collision collision)
{
collisionCount += 8;
spacebrewScript.GetComponent<SpacebrewEvents>().sendMsg(collisionCount);
if(collision.gameObject.name == "Cube")
cloneScript.GetComponent<AddElements>().makeObject();
Debug.Log("Exit called...");
//}
}
// Update is called once per frame
void Update () {
}
}
Note: Do not change the title “CollisionEvents”. Otherwise, the code will not work.
[screenshot]
-
When “OnCollisionExit” (the “Event”) is triggered in Unity (e.g., where the sphere exits the plane), a signal is sent to Spacebrew with a value of 9 which translates to 1 LED light.
-
Using 30 LED lights, the equivalent of 1 LED light in the range 0-1023 is 36.
-
Due to the physics engine, the Event is triggered 4 times while the GameObject rolls outside of the plane.
-
To give physics to the Sphere and the Cube, add a new component titled Rigidbody.
-
Create an empty object named “Cloner”.
-
Copy and paste the following code under our Cloner object by adding a new script under new component, named AddElements:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class AddElements : MonoBehaviour {
public GameObject startGO;
public GameObject copyGO;
public void makeObject ()
{
//Debug.Log("Make object called");
Vector3 startPos;
Quaternion startRotation;
startPos = startGO.transform.position;
startRotation = startGO.transform.rotation;
Instantiate(copyGO,startPos,startRotation);
}
// Use this for initialization
void Start () {
}
// Update is called once per frame
void Update()
if (Input.GetKeyDown("space"))
makeObject();
Debug.Log("spacebar");
}
Note: Do not change the title “AddElements”. Otherwise, the code will not work.
- Under the Add Elements script, drag Cloner to the StartGo box and Sphere to the CopyGo box.
[Image]
[GIFs & Images]