CameraThing

This project is intended to be the epitome of a point-and-shoot camera. As soon as you take the photo it gets sent off to a cloud server which posts it to twitter with some (probably funny) captions generated by a tensorflow model (InceptionV3) trying to figure out what's in the photo you just took.

Check it out on twitter: @CameraThing.

Behold!

Details on the CameraThing can be found in /camera-thing.

Details on the cloud services on which the CameraThing depends can be found below, with more details in each service's subdirectory.

Cloud Services

The CameraThing is currently dependent upon the following cloud services:

1. An instance of the "tweeter" service, in found in /tweeter of this repo.
2. An instance of go-tensorflow-image-recognition, which has been added as a git submodule to this repo and was not written by me.

Local Development

The cloud services used by the CameraThing can be run locally with docker-compose like so:

git clone --recursive [email protected]:TheTeaCat/CameraThing.git
cd CameraThing
docker-compose up

Easy peasy!

The CameraThing simply queries the /tweet endpoint of the Tweeter service, which you can mock locally with the following curl command:

$ curl "localhost:8080/tweet?auth=dev&lat=53.36097329965131&long=-1.6899902029658576" -F 'image=@./test.jpg'
{"Tweet":"#Breakwater #Promontory #Seashore #GreyWhale #Wing (53.36097,-1.68999)"}

You do not need to have twitter credentials to test the tweeter service. For more details, see the docs in /tweeter.

Deployment

Details on how to deploy the cloud services.

Heroku

The cloud services are dockerised and could easily be deployed to Heroku. Relevant docs for doing this can be found here: https://devcenter.heroku.com/categories/deploying-with-docker

Digitalocean/VPS

You can deploy to a digitalocean droplet or other VPS by SSHing into it, installing docker, then doing:

git clone --recursive [email protected]:TheTeaCat/CameraThing.git
cd CameraThing
docker-compose up -d

You can then update by doing:

cd CameraThing
git pull
docker-compose down
docker-compose up -d

To view the logs:

docker-compose logs -f -t

An example of healthy log output:

Attaching to camerathing_tweeter_1, camerathing_imagerec_1
imagerec_1  | 2021-05-17T11:24:37.145636320Z Listening on port 8080...
tweeter_1   | 2021-05-17T11:24:38.711815035Z go: downloading github.com/drswork/go-twitter v0.0.0-20190721142740-110a39637298
tweeter_1   | 2021-05-17T11:24:38.759193581Z go: downloading github.com/dghubble/oauth1 v0.7.0
tweeter_1   | 2021-05-17T11:24:38.822071433Z go: downloading github.com/nfnt/resize v0.0.0-20180221191011-83c6a9932646
tweeter_1   | 2021-05-17T11:24:39.038935342Z go: downloading github.com/cenkalti/backoff v2.1.1+incompatible
tweeter_1   | 2021-05-17T11:24:39.070894538Z go: downloading github.com/dghubble/sling v1.3.0
tweeter_1   | 2021-05-17T11:24:39.366430315Z go: downloading github.com/google/go-querystring v1.0.0
tweeter_1   | 2021-05-17T11:24:42.286375568Z 2021/05/17 11:24:42 Tweeter service started.
tweeter_1   | 2021-05-17T11:24:42.286493822Z 2021/05/17 11:24:42 Creating recogniser instance...
tweeter_1   | 2021-05-17T11:24:42.286568766Z 2021/05/17 11:24:42 Creating tweeter instance...
tweeter_1   | 2021-05-17T11:24:42.514809485Z 2021/05/17 11:24:42 Logged into twitter as: Camera Thing
tweeter_1   | 2021-05-17T11:24:42.514976972Z 2021/05/17 11:24:42 Creating endpoint instances...
tweeter_1   | 2021-05-17T11:24:42.515112458Z 2021/05/17 11:24:42 Attaching endpoint handlers...
tweeter_1   | 2021-05-17T11:24:42.515290175Z 2021/05/17 11:24:42 Using port 8080...
tweeter_1   | 2021-05-17T11:24:42.515419248Z 2021/05/17 11:24:42 Listening and serving!
tweeter_1   | 2021-05-17T11:45:43.037054362Z 2021/05/17 11:45:43       [/tweet] - Request @ /tweet!
tweeter_1   | 2021-05-17T11:45:43.501037730Z 2021/05/17 11:45:43       [/tweet] - Longitude and latitude provided, using geolocation
tweeter_1   | 2021-05-17T11:45:45.266833446Z 2021/05/17 11:45:45       [TWEETER] - Successfully tweeted: Mongoose? Wombat? Wallaby? Weasel? Chesapeake Bay Retriever? (53.36097,-1.68999)
tweeter_1   | 2021-05-17T11:45:45.266990972Z 2021/05/17 11:45:45 [201] [/tweet] - Successfully tweeted: Mongoose? Wombat? Wallaby? Weasel? Chesapeake Bay Retriever? (53.36097,-1.68999)
tweeter_1   | 2021-05-17T11:46:40.255168229Z 2021/05/17 11:46:40       [/tweet] - Request @ /tweet!
tweeter_1   | 2021-05-17T11:46:40.458978395Z 2021/05/17 11:46:40       [/tweet] - No longitude or latitude provided, not using geolocation
tweeter_1   | 2021-05-17T11:46:41.901819054Z 2021/05/17 11:46:41       [TWEETER] - Successfully tweeted: Mongoose? Wombat? Wallaby? Weasel? Chesapeake Bay Retriever? 
tweeter_1   | 2021-05-17T11:46:41.901948053Z 2021/05/17 11:46:41 [201] [/tweet] - Successfully tweeted: Mongoose? Wombat? Wallaby? Weasel? Chesapeake Bay Retriever? 

Future work

If I don't immediately disassemble my CameraThing to make something else, I might like to:

1. Connect all the modules properly (not using jumper wires).
2. Build a better housing for it - the camera doesn't sit straight stuck to the side of a cardboard box so it's hard to frame photos up properly. If it was mounted properly then it would make it much easier to register the position of the camera by feel like you can do with a regular camera.
3. Use a github action for continuous deployment of the cloud services.
4. Make the cloud services save the images to a filesystem and a database with their InceptionV3 labels and geolocations, then implement a searching service so I can make a webpage that allows you to search the images by their InceptionV3 labels and see them all on a map.
5. Expand the IO of the ESP32 so I can use a SIM800L and a GPS featherwing at the same time.
6. Swap out the InceptionV3 tensorflow model for a more accurate, or at least funnier one.
7. Do some more image processing server-side to correct for white balance in particular.
8. Make the CameraThing send me text messages when it successfully makes a tweet containing a link to the tweet that was created.

Acknowledgements

This video, by "bitluni", was super helpful in figuring out how to get the OV7670 working.

esp32-camera, the library I used to interface with the OV7670 which also provides a utility to perform JPEG compression.

This video, by Rui Santos, was also helpful in figuring out how to use the SIM800L module.

TinyGSM, the library I used for the SIM800L module.

Tin Rabzelj and contributors, for creating go-tensorflow-image-recognition.

Also, thanks to my flatmate Edward Hails for the nice button.