four sensor canvas

my Ada Developers Academy capstone project

react app and arduino sketch for controlling a p5 canvas with RFID tokens on a 2x2 diamond shaped grid of RFID readers to activate the visuals on the computer screen with the base of each drawing being made by a machine learning model

how to start

gather parts & tools

• Arduino Uno R3 & USB cable (x1)
• Mifare RC522 RFID reader modules & header pins (x4)
• Mifare Classic 1K 13.56Mhz RFID sticker tags (x26)
• breadboard (x1)
• male-male jumper cables (x6)
• male-female jumper cables (x28)
• soldering station with eye protection and lead-free solder (lead-free is optional)
• material to build polygons to place sticker tags on/in (I used cardboard and hot glue and placed the sticker tags on the inside before sealing)

prepare arduino

upload the sketch and wire up the electronics

1. install MFRC522 Library v. 1.4.5 from Arduino IDE library manager
2. Wire arduino to RFID readers according to this diagram but with a 4th reader's SPI SS pin plugged into the Arduino's '6' pin as noted below the comment on line 35 in sketch.ino
3. solder headers onto the RFID readers well.

prepare RFID readers and tokens/tags

1. label / arrange the readers in a diamond shape and prepare their polygon controllers by putting RFID sticker tags onto each side of the polygons:

   • TOP / Reader 1 / 6 sided cube
   • LEFT / Reader 2 / 4 sided pyramid
   • RIGHT / Reader 3 / 4 sided pyramid
   • BOTTOM / Reader 4 / 12 sided thing

2. temporarily uncomment some lines in sketch.ino and upload it to the Arduino in order to set new UIDs to match up with readers:

   • uncomment line 95 in sketch.ino to confirm the readers are all connected and displaying their firmware versions (mfrc522[reader].PCD_DumpVersionToSerial();) If they aren't working, confirm your connections, wiring, and soldering are all good, and see these troubleshooting tips.
   • uncomment lines 193-194 to display the UIDs of your tokens (starting with Serial.print(buffer[i] < 0x10 ? " 0" : " ");)
   • replace the UIDs in tagarray beginning at line 45 to correspond to the UIDs displayed for your tokens. make sure to omit spaces and zeroes - ie convert "7C 0A 66 31" to "7CA6631"
   • recomment those lines

3. upload the sketch with the new UIDs onto the Arduino. The serial output from the Arduino IDE should now be a string message containing the reader position label and token number as tags are read.

connect arduino serial output to app

1. the arduino's serial output cannot be directly accessed by an app so it needs to be served, this can be easily done by installing the p5.serialcontrol gui app.
2. start serialcontrol with the usb port that the arduino is connected to (mine was "/dev/tty.usbmodem14201")
3. hardcode the name of that port into portName in line 150 of src/sketch.js.

run it

1. run npm install then npm start from root, open developer tools to see console output that should confirm that the arduino is connected and sketch-rnn models are loading from ml5.

📺 demo video with arduino input

💻 deployed version with dropdowns instead of arduino input

Notes

• This project was bootstrapped with Create React App.

• The Arduino sketch was built around @Annaane's Multiple RFID Readers Project

• Daniel Shiffman's Coding Train videos were extremely helpful in creating this project, especially this one about using ml5's sketch-rnn and this one about creating a flocking simulation in p5.

• p5.js, ml5.js, and p5.serialport.js are loaded via CDN links in index.html rather than managed by npm due to the way the libraries are structured and meant to be used.

• notes gist created while researching and creating project

• trello with what's next

• Sometimes I call it four sensor canvas, sometimes I call it four sensor palette 🤷🏽