Photo taking/tagging/submitting from within OwnTracks app

[jpmens]

Would it be technically possible to take photographs from within the OwnTracks iOS app (Android would follow suit if feasable) which are then either submitted via MQTT (or HTTP if that is the selected transport) tagged in such a way (EXIF data?) that there can be a direct association between the picture and the OwnTracks location?

I'm thinking along the lines of:

• open OwnTracks app
• take photo
• photo is tagged and transmitted via MQTT to broker (i.e. then to Recorder), maybe as topic /pic where it is stored
• payload contains base64-encoded photograph, location, current tag and/or current PoI if set.

This might be quite a bit of work, but possibly a neat feature to have, and in fact somebody at #geomobNL recently actually asked for this.

[ckrey]

Yes!

First thoughts:

We could either take a photograph or select an existing photograph from the photo library. This would be similar to the process to create a card. With the photograph we could require a "POI" to be entered.

As the next step, we would

• either send a location message which includes a thumbnail of the image "img":<base64-encoded-thumbnail> and treat is as we do currently with PoI data. We could display the thumbnails on the map... In this case, I would recommend to resize the image to a smaller size e.g. 192x192 pixels as we do for cards.

• Or we could send the full size image to a special subtopic (e.g. owntracks/<user>/<device>/img). The payload of this message would be the same as a location message, except a "img":<base64-encoded-full-size-image> key/value pair.

• or somehow link the full size image information to the location message

[jpmens]

or somehow link the full size image information to the location message

Is it possible to manipulate / augment the EXIF data in a taken photograph with, say, a designated UUID which is could be submitted with the location message to later associate location with the image?

[ckrey]

I can extract the original filename e.g. IMG_1255 and put this into the location message.
How would you transmit the actual photo? Via the app or the way you transfer your photos normally?
If you do it via OwnTracks, we could create an UUID and use it both in the location and the image playload.

[jpmens]

I suspect the taken photograph would remain on the iPhone storage which it should.

Its filename or UUID would be transported in a _type:photo message to the broker at an owntracks/<user>/<device>/img ideally with the current location in it (lat, lon should suffice).

I would suggest the _type:photo has an encoding:base64 tag (maybe one day distinct encodings will be possible) and an "img":<base64-encoded>" photo.

[ckrey]

POC:

Result:

{
  "tst": "2024-11-26T08:26:09.475577+0000",
  "topic": "owntracks/ck00/iPhoneSE/image",
  "qos": 0,
  "retain": 0,
  "payloadlen": 4342432,
  "payload": {
    "_type": "image",
...
    "image": "/9j/4TorRXhpZgAATU0AKgAAAAgADAEPAAIAAAAGAAAAngEQAAIAAAAbAAAApAESAAMAAAABAAYAAAEaAAUAAAABAAAAwAEbAAUAAAABAA
    "imageName": "IMG_1333",
...
    "lat": 51.196003,
    "lon": 6.688493,
    "poi": "Gans",
    "t": "u",
    "tid": "SE",
...
  }

base64-encoded .jpeg includes EXIF data, etc.:

But... recorder does not like long payloads (.rec file is correct):

stderr: "invalid strptime format on \/n9aAIvLHr+n\/16ABI+eT\/..."

[jpmens]

It looks a bit as though curry is on me tomorrow evening!

[jpmens]

stderr: "invalid strptime format

probably due to falling through as we don't yet have code for _type: image

[jpmens]

So, after discussion, I think we've settled on:

• _type: location message will contain poi as well as base64-encoded thumbnail of an image selected from the photo library.
• in addition the payload will contain the photo's filename as generated by iOS' camera
• Recorder will extract the thumbnail and store it so it can be viewed from the file system if so desired, similarly to how we currently process faces in a CARD.

That way, people can shoot a photo, then upload a POI location with a thumbnail of the photo. In theory, the thumbnail can be displayed on the map and hopefully Android and Frontend will at some stage have the capabilities of displaying the thumbnail for the location likewise.

[ckrey]

Example location message payload with new optional image and imageName elements:

{
  "_type": "location",
  "acc": 5,
  "alt": 0,
  "cog": 282,
  "conn": "w",
  "image":"/9j/4AAQSkZJRgABAQAASABIAAD/4QCARXhpZgAATU0AKgAAAAgABQESAAMAAAABAAEAAAE..."
  "imageName": "IMG_0003",
  "lat": 37.324615,
  "lon": -122.02494,
  "m": 1,
  "poi": "Waterfall",
  "t": "u",
  "tid": "PH",
  "tst": 1733319012,
  "vel": 14
}

[ckrey]

UI:

[ckrey]

Result:

[jpmens]

That looks Supercalifragilisticexpialidocious.

[jpmens]

I think I might have uncovered a small bug.

• publish POI with image (or without) in simulator|
• then, without moving, publish another (with or without image)
• on the map I can click my way through the individual avatars and find both images
• but in Friends details, I see first POI description only (the latter are not shown but have been published over MQTT)

Different tst

[jpmens]

$ ./ocat -S JP --user=jane --device=phone --format json --fields=tst,poi
{
 "count": 5,
 "locations": [
  {
   "tst": 1734273515
  },
  {
   "tst": 1734273515,
   "poi": "No tulips at this restaurant"
  },
  {
   "tst": 1734274523,
   "poi": "fabulous rijsttafel"
  },
  {
   "tst": 1734274523
  },
  {
   "tst": 1734274523,
   "poi": "p1"
  }
 ]
}

[ckrey]

Friends Details? What is displayed there with the publishes above?
N.B. The last 3 timestamps are identical. This means only the fabulous rijstafel is stored/displayed

[jpmens]

Which is what I would have expected to see but didn't.

$ tst 1734273515
1734273515 Sun Dec 15 15:38:35 2024

my simulator still (right now) shows the previous POI:

[jpmens]

$ ./ocat -S JP --user=jane --device=phone --format json --fields=tst,created_at,poi
{
 "count": 5,
 "locations": [
  {
   "tst": 1734273515,
   "created_at": 1734274023
  },
  {
   "tst": 1734273515,
   "created_at": 1734274109,
   "poi": "No tulips at this restaurant"
  },
  {
   "tst": 1734274523,
   "created_at": 1734274552,
   "poi": "fabulous rijsttafel"
  },
  {
   "tst": 1734274523,
   "created_at": 1734274617
  },
  {
   "tst": 1734274523,
   "created_at": 1734274625,
   "poi": "p1"
  }
 ]
}

[ckrey]

I cannot reproduce the problem. Did you set the location in Xcode Simulator via Features/Location/Custom Location ?
I see you access Friends Details from the Friends tab, not from the map. Please note the Friends Details view is not updated but shows the last waypoint at the time you selected it from the friends table view. To update the view, return to the friends table view and then open the Friends Details again.

[jpmens]

Did you set the location in Xcode Simulator via `Features/Location/Custom Location` ?

Yes, I did. I will try to reproduce and will report back.

[jpmens]

The good news and the bad news in one go: I cannot reproduce this, not even when I access Friends Details from the Friends tab. I do not know what I did to make that happen, so we'll ignore it.

I'm sorry about the noise caused.