Working on update for macOS Sonoma

[RhetTbull]

I'm working on updating osxphotos for macOS Sonoma. The bad news is that osxphotos will not currently work on Sonoma. The good news is that the Photos database changes appear to be minor and I think the update won't be too hard. The biggest change will be in code that handles faces as the foreign key columns have changed names. That will require some rework of the code to handle both old and new database schemas.

The entire Sonoma Photos schema is on the wiki here and the diff from Ventura is here.

To run Sonoma, I used the excellent VirtualBuddy. I could not get Sonoma beta installed though as my host OS is Ventura 13.4.1 so I followed these instructions inside a Ventura VM to enable beta updates then upgraded the Ventura VM to Sonoma.

Here are the major differences in Photos 9 (Sonoma) from Photos 8 (Ventura) databases that affect osxphotos:

ZADDITIONALASSETATTRIBUTES

• A few columns are reordered
• ZSNOWDAYSNOWPLOWIDENTIFIER is deleted

ZASSET

• New column ZICONICSCORE FLOAT

ZCLOUDMASTER

A few columns are reordered

ZDETECTEDFACE

• ZASSETFORFACE replaces ZASSET

• ZPERSONFORFACE replaces ZPERSON

• New columns:

  • ZGAZERECTSTRING VARCHAR
  • ZDURATION FLOAT
  • ZASSETFORTEMPORALDETECTEDFACES INTEGER
  • ZPERSONFORTEMPORALDETECTEDFACES INTEGER
  • ZASSETFORTORSO INTEGER
  • ZVUOBSERVATIONID INTEGER
  • ZSTARTTIME FLOAT
  • ZGAZEANGLE FLOAT
  • ZPERSONFORTORSO INTEGER
  • ZGAZECONFIDENCE FLOAT

ZEXTENDEDATTRIBUTES

• Several columns reordered

ZGENERICALBUM

• Several columns reordered

ZINTERNALRESOURCE

• ZORIENTATION deleted
• ZQUALITYSORTVALUE deleted

ZMEMORY

• A few columns reordered

ZMOMENT

• A few columns reordered

ZPERSON

• A couple columns reordered

• New columns:

  • ZASSETSORTORDER INTEGER
  • ZMDID VARCHAR

ZPHOTOSHIGHLIGHT

• New columns:

  • ZCLOUDDELETESTATE INTEGER
  • ZADAPTIVEVISIBILITYSTATE INTEGER
  • ZLASTENRICHMENTDATE TIMESTAMP
  • ZFAVORITE INTEGER
  • ZADAPTIVEMERGEPOLICY INTEGER
  • ZADAPTIVEKEYASSETSHAREDUSEREDITED INTEGER
  • ZADAPTIVEKEYASSETPRIVATEUSEREDITED INTEGER
  • ZADAPTIVEKEYASSETPRIVATE INTEGER
  • ZVERBOSESMARTDESCRIPTIONUSEREDITED VARCHAR
  • ZADAPTIVEKEYASSETSHARED INTEGER
  • ZCLOUDLOCALSTATE INTEGER

[RhetTbull]

Update: I have this working! The only discrepancy I see now is a difference in unknown faces in the test.

[RhetTbull]

Version v0.60.8 adds alpha support for macOS Sonoma. If you're using Sonoma I would welcome any feedback you have on osxphotos!

[ZarK]

How about support for Spatial Video captured by the new iPhone 15 Pro models? Is that something that will only be added later in X.1 maybe so they could keep it secret for this long?

[RhetTbull]

From a guy who still uses an iPhone SE (I like the small size & the home button), what's spatial video? ;-) Seriously though, I imagine the video will appear in the library like normal but have a separate sub-type as do slow-mo, time-lapse, etc. If it's a new file format then there may need to be a code change to osxphotos to detect the files. Until I have access to a Sonoma test library that contains a spatial video I won't be able to test this or make any code changes.

[RhetTbull]

The only reference to "spatial" in the database schema at the moment is ZADDITIONALASSETATTRIBUTES.ZSPATIALOVERCAPTUREGROUPIDENTIFIER but this has been around for a while (and I'm not sure what it's used for). My guess is spatial video will simply be a new sub-type of video when shown in the library.

[ZarK]

It’s 3D video that can be viewed on the upcoming Apple Vision Pro, that the new iPhone 15 Pro models can capture.

Yes I would expect it to be something similar to Live Photos so you can watch both a still of it and the video on a normal screen as well. Will be interesting to see what it exports as - hopefully a regular video plus something more.

[ZarK]

Spatial overcapture might be some depth information used by Portrait photos so you can edit the effect afterwards? That will be a “default” now when iPhone identity a portrait, regardless of using portrait mode.

[ZarK]

Here is a sample file of Spatial Video recorded by IPhone 15 Pro (not my file, shared in /r/VisionPro): https://www.dropbox.com/scl/fi/fbckljpkqy6ts0f4omvwd/Vision-Sample.MOV?rlkey=azkjbmhidydm855e75o88foj7&dl=0

Here is a document from Apple describing the format: https://developer.apple.com/av-foundation/HEVC-Stereo-Video-Profile.pdf

[RhetTbull]

Thanks!

[RhetTbull]

@ZarK osxphotos doesn't need to process the actual file, only the metadata about the file. Do you have any of these files in your personal library? If so, could you select one of them in Photos then run this command:

osxphotos debug-dump photos --selected > spatial.txt

Then send the spatial.txt file to me at [email protected]

[ZarK]

If I use your exact command, I get this error:

osxphotos debug-dump photos --selected > spatial.txt
Usage: osxphotos debug-dump [OPTIONS]
Try 'osxphotos debug-dump -h' for help.

Error: Got unexpected extra argument (photos)

If I remove photos as argument, I just get an empty file:

osxphotos debug-dump --selected > spatial.txt
Using last opened Photos library: /Volumes/Photos/Photos Library.photoslibrary
2023-12-17 20:53:35,831 - root - WARNING - photosdb.py - 157 - WARNING: This module has only been tested with macOS versions [10.12, 10.13, 10.14, 10.15, 10.16, 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 12.0, 12.1, 12.2, 12.3, 12.4, 12.5, 12.6, 12.7, 13.0, 13.1, 13.2, 13.3, 13.4, 13.5, 13.6, 14.0, 14.1]: you have Darwin, OS version: 14.2

cat spatial.txt
Opening database: /Volumes/Photos/Photos Library.photoslibrary
Done; took 39.67 seconds

[RhetTbull]

Sorry, I think the command should be

osxphotos debug-dump --dump photos --selected > spatial.txt

[ZarK]

That did something, spatial.txt sent on email.