Clarification ask: timewarp with --timezone 'named timezone' and --match-time: seems --match-time is ignored when using a 'named timezone' #1777

oPromessa · 2025-01-08T14:07:03Z

Before submitting a bug report, please ensure you are running the most recent version of osxphotos and that the bug is reproducible on the latest version

$  osxphotos --version
osxphotos, version 0.69.2
Python 3.12.7 (main, Oct 11 2024, 17:54:25) [Clang 14.0.0 (clang-1400.0.29.202)]
Python executable: /Users/XXX/Documents/GitHub/osxphotos/venv/bin/python
macOS 12.7.6, x86_64

Describe the bug

Possibly not a bug, maybe an enhancement or I'misreading the timewarp scenarios with --timezone 'named timezone' vs UTCOffset and --match-time: for me it seems --match-time option is ignored when using a 'named timezone'.

Side Comment: it also seems "--inspect" reports the timestamp shifted and UTC Offset incorrectly and not accounting for DST in test 1. and 3. It should show +02:00 and not +01:00 for UTC Offset.

From help timewarp --timezone we have this note also:

Note: when a named timezone is
provided, daylight savings time will be
considered when adjusting the time; it will
not be considered when a UTC offset is
provided. For list of valid IANA timezone
names, see https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

Test	initial date_time on Photos	initial --inspect	--timezone	--match-time	final --inspect	final date_time on Photos	Comment
1	2017-06-24 21:26:51 GMT+1	2017-06-24 21:26:51+0100, +0100, Europe/Lisbon	'Europe/Madrid'	True	2017-06-24 21:26:51+0100, +0100, Europe/Madrid	2017-06-24 22:26:51 CEST	Preferred to have CEST but timestamp is shifted
2	2017-06-24 21:26:51 GMT+1	2017-06-24 21:26:51+0100, +0100, Europe/Lisbon	UTC Offset +02:00	True	2017-06-24 21:26:51+0200, +0200, GMT+0200	2017-06-24 21:26:51 GMT+2	Correct timestamp and timezone. Would like it to be CST
3	2017-06-24 21:26:51 GMT+1	2017-06-24 21:26:51+0100, +0100, Europe/Lisbon	'Europe/Madrid'	False	2017-06-24 21:26:51+0100, +0100, Europe/Madrid	2017-06-24 22:26:51 CEST	Same as case 1. Preferred to have CEST but timestamp is shifted

To Reproduce

1. Using 'named timezone' --match-time

Test 1 detail

$ osxphotos timewarp -i                              
filename, uuid, photo time (local), photo time, timezone offset, timezone name, date added (local)
IMG_20170624_212651.JPG, 8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D, 2017-06-24 21:26:51+0100, 2017-06-24 21:26:51+0100, +0100, Europe/Lisbon, 2017-09-02 22:27:16+0100

$ osxphotos timewarp --timezone 'Europe/Madrid' --verbose --timestamp --match-time  --force
2025-01-08 13:16:15.728033 -- Skipping date/time update for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D), already matches new timezone Europe/Madrid
2025-01-08 13:16:15.783004 -- Updated timezone for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D) from Europe/Lisbon, offset=3600 to Europe/Madrid, offset=3600
2025-01-08 13:16:15.784026 -- Done.

$ osxphotos timewarp -i                                                                    
filename, uuid, photo time (local), photo time, timezone offset, timezone name, date added (local)
IMG_20170624_212651.JPG, 8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D, 2017-06-24 21:26:51+0100, 2017-06-24 21:26:51+0100, +0100, Europe/Madrid, 2017-09-02 22:27:16+0100

In Photos in the Photo Info (enabling display of timezone on the Mac Regional Settings): 2017-06-24 22:26:51 CEST

2. Using 'UTC Offset' --match-time

Test 2 detail

$ osxphotos timewarp -i                                                                   
filename, uuid, photo time (local), photo time, timezone offset, timezone name, date added (local)
IMG_20170624_212651.JPG, 8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D, 2017-06-24 21:26:51+0100, 2017-06-24 21:26:51+0100, +0100, Europe/Lisbon, 2017-09-02 22:27:16+0100

$ osxphotos timewarp --timezone '+02:00' --verbose --timestamp --match-time  --force       
2025-01-08 13:34:17.978847 -- Adjusted local date/time for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D) to 2017-06-24 20:26:51 to match previous time 2017-06-24 21:26:51 but in new timezone GMT+0200.
2025-01-08 13:34:17.982241 -- Photo date/time is now 2017-06-24 20:26:51 in new timezone GMT+0200.
2025-01-08 13:34:18.043436 -- Updated timezone for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D) from Europe/Lisbon, offset=3600 to GMT+0200, offset=7200
2025-01-08 13:34:18.044173 -- Done.

$ NOk.2017 [1102]> osxphotos timewarp -i                                                             
filename, uuid, photo time (local), photo time, timezone offset, timezone name, date added (local)
IMG_20170624_212651.JPG, 8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D, 2017-06-24 20:26:51+0100, 2017-06-24 21:26:51+0200, +0200, GMT+0200, 2017-09-02 22:27:16+0100

In Photos in the Photo Info (enabling display of timezone on the Mac Regional Settings): 2017-06-24 21:26:51 GMT+2

3. Using 'named timezone'

Test 3 detail

$ osxphotos timewarp -i                                        
filename, uuid, photo time (local), photo time, timezone offset, timezone name, date added (local)
IMG_20170624_212651.JPG, 8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D, 2017-06-24 21:26:51+0100, 2017-06-24 21:26:51+0100, +0100, Europe/Lisbon, 2017-09-02 22:27:16+0100

$ osxphotos timewarp --timezone 'Europe/Madrid' --verbose --timestamp --force              
2025-01-08 13:40:48.517488 -- Updated timezone for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D) from Europe/Lisbon, offset=3600 to Europe/Madrid, offset=3600
2025-01-08 13:40:48.518577 -- Done.

$ osxphotos timewarp -i                                                      
filename, uuid, photo time (local), photo time, timezone offset, timezone name, date added (local)
IMG_20170624_212651.JPG, 8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D, 2017-06-24 21:26:51+0100, 2017-06-24 21:26:51+0100, +0100, Europe/Madrid, 2017-09-02 22:27:16+0100

In Photos in the Photo Info (enabling display of timezone on the Mac Regional Settings): 2017-06-24 22:26:51 CEST

Expected behavior
Would like to use --timezone 'named timezone' (respecting DST) and being able to use --match-time to keep the same timestamp.

Screenshots
N/A

Desktop (please complete the following information):

$ sw_vers 
ProductName:	macOS
ProductVersion:	12.7.6
BuildVersion:	21H1320

Additional context
Add any other context about the problem here.

The text was updated successfully, but these errors were encountered:

oPromessa · 2025-01-08T14:22:59Z

A possible workaround would be to use --time-delta DELTA with -01 hour for Test 1 case or Test 3 (as it seems --match-time is ignored).

Result: In Photos is okay! timewarp -i still shows the shifted time with UTCOffset +01:00 as I think is not considering DST.

In Photos in the Photo Info (enabling display of timezone on the Mac Regional Settings): 2017-06-24 21:26:51 CEST

$ osxphotos timewarp -i                                                                      
filename, uuid, photo time (local), photo time, timezone offset, timezone name, date added (local)
IMG_20170624_212651.JPG, 8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D, 2017-06-24 21:26:51+0100, 2017-06-24 21:26:51+0100, +0100, Europe/Lisbon, 2017-09-02 22:27:16+0100

$ osxphotos timewarp --timezone 'Europe/Madrid' --verbose --timestamp --time-delta '-01 hours'  --force
2025-01-08 14:18:52.141249 -- Updated date/time for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D) from: 2017-06-24 21:26:51+01:00 to 2017-06-24 20:26:51+01:00
2025-01-08 14:18:52.202521 -- Updated timezone for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D) from Europe/Lisbon, offset=3600 to Europe/Madrid, offset=3600
2025-01-08 14:18:52.203266 -- Done.

$ osxphotos timewarp -i                                                                                
filename, uuid, photo time (local), photo time, timezone offset, timezone name, date added (local)
IMG_20170624_212651.JPG, 8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D, 2017-06-24 20:26:51+0100, 2017-06-24 20:26:51+0100, +0100, Europe/Madrid, 2017-09-02 22:27:16+0100

RhetTbull · 2025-01-08T16:53:58Z

I'll take a look. Likely a bug. I completely rewrote this part of the code a few months ago and it was extremely tricky to get right. My least favorite topic in programming is working with timezones!

oPromessa · 2025-01-08T17:47:17Z

I get you! 😆

I guess not, but would time warp be able to operate over a specific photo (either with --uuid or --name or --album)?

Or could I import timewarp from a python app and provide one by one the pics?

Hmmm. I guess a combination of show and timewarp could do the trick! Hmmm. No. Show does not actually select the image.

RhetTbull · 2025-01-08T18:06:47Z

but would time warp be able to operate over a specific photo (either with --uuid or

Yes, you can use --uuid with timewarp as well as -uuid-from-file

oPromessa · 2025-01-08T18:16:33Z

::embarrassing:: 😃 not to have fully read the doc and looking for --uuid.

The initial description for timewarp through me off: "Changes will be applied to all photos currently selected in Photos."

What I see happening is that timewarp runs over the --uuid specified IDs plus the selected pics as it shows me the total pics tone

$ osxphotos timewarp  --uuid 70D3637B-1321-4AB5-93FC-6C4EB214BA23 --timezone 'Europe/Madrid' --time-delta '-3600 seconds'
⚠️  About to process 3 photos with timewarp. This will directly modify your 
Photos library database using undocumented features. While this functionality 
(...)

But now that I know about it ... I jus make sure not to select any pic in Photos!

... and it will make my life SOOO MUCH EASIER to update thousands of wrong timezones pics!

RhetTbull · 2025-01-08T19:18:42Z

Yes, I should update the help text. Also, I think if --uuid is used then the current selection should not be used. I'll open a new issue for this.

I've been thinking about how to make batch-edit and timewarp (and any future commands that operate on selected photos) more useful. One idea I'm thinking of is something like this to use osxphotos query:

osxphotos query --query-options --uuid-only | osxphotos timewarp --uuid-from-file -

This would pipe a list of uuids to timewarp for processing. This avoids having to clutter timewarp with all the query options. It would require a new option for osxphotos query that prints only uuid. (This can be accomplished today with osxphotos query --print "{uuid}" --quiet but that's cumbersome). Open to feedback on improving this approach.

oPromessa · 2025-01-09T19:30:26Z

This would pipe a list of uuids to timewarp for processing. This avoids having to clutter timewarp with all the query options. It would require a new option for osxphotos query that prints only uuid. (This can be accomplished today with osxphotos query --print "{uuid}" --quiet but that's cumbersome). Open to feedback on improving this approach.

Yes, I'm currently going on an Herculean task to go thru my whole library to set Location and appropriate Timezone for all pics.

A. Based on cli_example_2.py I've built a program which:
1. Lists info from:
  - Photos info like: edited (check also for .AAE transformation) , path, datetime, timezone/offset/dst, Location
  - Exiftool dict in osxphotos: datetime, timezone/offset/dst, Location
2. Complements it with:
  - Using TimezoneFinder, pytz: determines Timezone/Offset/DST based on Location
  - and then generates a status by comparing Locations and timezone info to determine appropriate status: Ok, NOk, NotAvail (when no Location info exist)
3. and writes a .CSV file
B. Use mlr miller csv tool to process the .CSV and, for Example:
- find pics with Status == NOk
- gets the uuid
- and generates an osxphotos command line like "osxphtos timewarp --uuid {} --timezone {} ...""
- also to add a set a pics to an Album add-to-album with --uuids and then run add-locations to a set of pics
- on occasion I also have a sequence of exiftool to view all dates and then a set of parameters to correct all the dates (including timezones) on the files themselves.
C. Which I then pass on to xargs and bash to execute any automated changes.
D. Other cases I add the pics to an album which require manual adjustment.

oPromessa · 2025-01-15T22:21:19Z

I'll take a look. Likely a bug. I completely rewrote this part of the code a few months ago and it was extremely tricky to get right. My least favorite topic in programming is working with timezones!

Did a few more tests and I thing issue is with DST.

Lisbon TZ with DST offset is 3600
Madrid TZ without DST is 3600 -- but with DST is 7200

I guess on this line one should calculate the delta taking DST into consideration. Today I'm not feeling capable of working it out 😸 -- and there maybe side impacts I'm not aware! Do you agree this is the place? Some guidance from ou and maybe I can give it a try !!

osxphotos/osxphotos/photodates.py

Lines 209 to 215 in 56a6601

    
           old_timezone_offset = PhotoTimeZone(library_path=library_path).get_timezone(photo)[ 
        
               0 
        
           ] 
        
           delta = old_timezone_offset - new_timezone.offset 
        
           photo_date = photo.date 
        
           new_photo_date = update_datetime( 
        
               dt=photo_date, time_delta=datetime.timedelta(seconds=delta)

$ osxphotos timewarp --timezone 'Europe/Madrid' --verbose --timestamp --match-time  --force
2025-01-08 13:16:15.728033 -- Skipping date/time update for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D), already matches new timezone Europe/Madrid
2025-01-08 13:16:15.783004 -- Updated timezone for photo IMG_20170624_212651.JPG (8984A3C1-EFD5-4AB2-9DB4-B39D32C48E3D) from Europe/Lisbon, offset=3600 to Europe/Madrid, offset=3600
2025-01-08 13:16:15.784026 -- Done.

RhetTbull · 2025-01-16T03:01:48Z

I think I see the problem. It's not actually in this part of the code. The old timezone offset is determined from the Photos database so that is considered "truth data". The new timezone offset is determined by the Timezone class.

The Timezone for the new timezone is created here:

osxphotos/osxphotos/photodates.py

Lines 300 to 301 in 56a6601

    
           tz_new = Timezone(tz_name) 
        
           update_photo_time_for_new_timezone(library_path, photo, tz_new, verbose)

The offset gets computed here (but doesn't account for DST, it returns just the offset for the standard timezone):

osxphotos/osxphotos/photodates.py

Lines 209 to 212 in 56a6601

    
           old_timezone_offset = PhotoTimeZone(library_path=library_path).get_timezone(photo)[ 
        
               0 
        
           ] 
        
           delta = old_timezone_offset - new_timezone.offset

What's needed is to add here an offset_for_date(self, date: datetime.datetime) -> int: method that returns the offset for a given date. It should call the secondsFromGMTForDate_ method of NSTimeZone. For consistency, it should also include an offset_str_for_date(self, date: datetime.datetime)->str: method.

osxphotos/osxphotos/timezones.py

Lines 38 to 68 in 56a6601

    
           class Timezone: 
        
               """Create Timezone object from either name (str) or offset from GMT (int)""" 
        
               def __init__(self, tz: Union[str, int]): 
        
                   with objc.autorelease_pool(): 
        
                       self._from_offset = False 
        
                       if isinstance(tz, str): 
        
                           # the NSTimeZone methods return nil if the timezone is invalid 
        
                           self.timezone = Foundation.NSTimeZone.timeZoneWithAbbreviation_( 
        
                               tz 
        
                           ) or Foundation.NSTimeZone.timeZoneWithName_(tz) 
        
                           if not self.timezone: 
        
                               raise ValueError(f"Invalid timezone: {tz}") 
        
                       elif isinstance(tz, int): 
        
                           self.timezone = Foundation.NSTimeZone.timeZoneForSecondsFromGMT_(tz) 
        
                           self._from_offset = True 
        
                       else: 
        
                           raise TypeError("Timezone must be a string or an int") 
        
                       self._name = self.timezone.name() 
        
               @property 
        
               def name(self) -> str: 
        
                   return self._name 
        
               @property 
        
               def offset(self) -> int: 
        
                   return self.timezone.secondsFromGMT() 
        
               @property 
        
               def offset_str(self) -> str: 
        
                   return format_offset_time(self.offset)

RhetTbull · 2025-01-16T12:17:17Z

@oPromessa I'm going to need to fix this for another branch I'm working on to fix import time / timezone (#849 and #1661) so I'll update the fix for DST here as well.

RhetTbull added bug Something isn't working cli Pertains to the command line interface labels Jan 8, 2025

RhetTbull mentioned this issue Jan 8, 2025

Fix timewarp and batch-edit so that --uuid option prevents processing of current selection. #1778

Closed

oPromessa mentioned this issue Jan 26, 2025

Look at using tzfpy to restore timezone name when importing #1788

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clarification ask: timewarp with --timezone 'named timezone' and --match-time: seems --match-time is ignored when using a 'named timezone' #1777

Clarification ask: timewarp with --timezone 'named timezone' and --match-time: seems --match-time is ignored when using a 'named timezone' #1777

oPromessa commented Jan 8, 2025 •

edited

Loading

oPromessa commented Jan 8, 2025 •

edited

Loading

RhetTbull commented Jan 8, 2025

oPromessa commented Jan 8, 2025

RhetTbull commented Jan 8, 2025

oPromessa commented Jan 8, 2025 •

edited

Loading

RhetTbull commented Jan 8, 2025

oPromessa commented Jan 9, 2025 •

edited

Loading

oPromessa commented Jan 15, 2025

RhetTbull commented Jan 16, 2025

RhetTbull commented Jan 16, 2025

Clarification ask: timewarp with --timezone 'named timezone' and --match-time: seems --match-time is ignored when using a 'named timezone' #1777

Clarification ask: timewarp with --timezone 'named timezone' and --match-time: seems --match-time is ignored when using a 'named timezone' #1777

Comments

oPromessa commented Jan 8, 2025 • edited Loading

1. Using 'named timezone' --match-time

In Photos in the Photo Info (enabling display of timezone on the Mac Regional Settings): 2017-06-24 22:26:51 CEST

2. Using 'UTC Offset' --match-time

In Photos in the Photo Info (enabling display of timezone on the Mac Regional Settings): 2017-06-24 21:26:51 GMT+2

3. Using 'named timezone'

In Photos in the Photo Info (enabling display of timezone on the Mac Regional Settings): 2017-06-24 22:26:51 CEST

oPromessa commented Jan 8, 2025 • edited Loading

In Photos in the Photo Info (enabling display of timezone on the Mac Regional Settings): 2017-06-24 21:26:51 CEST

RhetTbull commented Jan 8, 2025

oPromessa commented Jan 8, 2025

RhetTbull commented Jan 8, 2025

oPromessa commented Jan 8, 2025 • edited Loading

RhetTbull commented Jan 8, 2025

oPromessa commented Jan 9, 2025 • edited Loading

oPromessa commented Jan 15, 2025

RhetTbull commented Jan 16, 2025

RhetTbull commented Jan 16, 2025

oPromessa commented Jan 8, 2025 •

edited

Loading

oPromessa commented Jan 8, 2025 •

edited

Loading

oPromessa commented Jan 8, 2025 •

edited

Loading

oPromessa commented Jan 9, 2025 •

edited

Loading