Throw for zzzz (and beyond) in parse_datetime #11331
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
|
This pull request was exported from Phabricator. Differential Revision: D64797504 |
|
This pull request was exported from Phabricator. Differential Revision: D64797504 |
kevinwilfong
pushed a commit
to kevinwilfong/velox
that referenced
this pull request
Oct 24, 2024
Summary: Pull Request resolved: facebookincubator#11331 This diff throws a user exception for JODA's zzzz (or more) patterns (all equivalent) in Presto's parse_datetime function. JODA does not support parsing time zone long names (specified by the pattern zzzz) so neither should Velox to be consistent. Previously we were treating any number of 'z's as representing a time zone abbreviation in the pattern which is completely wrong. Reviewed By: amitkdutta Differential Revision: D64797504
kevinwilfong
force-pushed
the
export-D64797504
branch
from
b20a022 to
5b23dfe
Compare
kevinwilfong
requested review from
assignUser and
majetideepak
as code owners
|
This pull request was exported from Phabricator. Differential Revision: D64797504 |
kevinwilfong
pushed a commit
to kevinwilfong/velox
that referenced
this pull request
Oct 24, 2024
Summary: Pull Request resolved: facebookincubator#11331 This diff throws a user exception for JODA's zzzz (or more) patterns (all equivalent) in Presto's parse_datetime function. JODA does not support parsing time zone long names (specified by the pattern zzzz) so neither should Velox to be consistent. Previously we were treating any number of 'z's as representing a time zone abbreviation in the pattern which is completely wrong. Reviewed By: amitkdutta Differential Revision: D64797504
kevinwilfong
force-pushed
the
export-D64797504
branch
from
5b23dfe to
251addf
Compare
assignUser
reviewed
Oct 28, 2024
CMakeLists.txt
Outdated
Comment on lines
484
to
485
| set_source(ICU) | ||
| resolve_dependency(ICU COMPONENTS i18n) |
There was a problem hiding this comment.
See my comment in #11330 please move this to the root cmake file and keep all components that are used in the boost file (these calls are cached).
added 5 commits
October 29, 2024 09:39
…or#11283) Summary: The format_datetime currently outptus the time zone id if for 3 or fewer 'z' characters in the format string. However, the JODA library, which this is based on, does this for 3 or more 'Z' characters. This diff fixes this, as well as adds support for a single 'Z' (which outputs the same thing as 'ZZ' just without the colon). So 'Z' is fully supported for any number of characters. To be more explicit: https://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html From the JODA docs: `'Z' outputs offset without a colon (-0800), 'ZZ' outputs the offset with a colon(-08:00), 'ZZZ' or more outputs the zone id(America/Los_Angeles).` And not clearly explained in the docs, but from experimentation: `'z', 'zz', or 'zzz' outputs the abbreviation of the time zone(PDT), and 'zzzz' or more outputs the time zone name(Pacific Daylight Time)` Currently DateTimeFormatter treats 'zzzz' or more like JODA treats 'ZZZ' or more. This diff marks 'zzzz' or more as unsupported (we can implement that in a future change), and moves that logic under 'ZZZ' or more to be consistent. It also implements 'Z' (previously only 'ZZ' was implemented in DateTimeFormatter). Reviewed By: bikramSingh91 Differential Revision: D64500193
Summary: This diff adds support for JODA's ZZZ pattern in Presto's parse_datetime function. This is used to parse time zone IDs (called "time zone names" in the tz library, but this means something else in JODA). I borrowed the algorithm from JODA to ensure it matches Presto Java's behavior. The idea is to greedily consume the longest substring that matches a known time zone. I borrowed their algorithm which is to break the set of known time zones into a list of those without a prefix (without the '/' character) and lists of suffixes for those with prefixes. This limits the number of strings that need to be compared. I modified it slightly to pre-sort these lists by size descending, so we don't have to necessarily compare every string, but can stop early if we find a match. One other change is I added a get_time_zone_names function to our copy of the tz library. I tried calling get_tzdb() from DateTimeFormatter directly and accessing its zones member to get the names, but for some reason after get_tzdb() returns every time_zone in zones (except the first one) has a string name_ that has nullptr for its data after get_tzdb() returns. I spent a good amount of time trying to figure out why, but couldn't figure it out, so I gave up and added this helper method (for whatever reason everything is fine as long as it's done in the tz.cpp file). If anyone has pointers as to what's going on I'm happy to investigate further, I'd much rather use the existing get_tzdb function if I can. Reviewed By: bikramSingh91 Differential Revision: D64708598
Summary: This diff adds support for JODA's z, zz, zzz patterns (all equivalent) in Presto's forma_datetime function. This is used to format time zone abbreviations. Reviewed By: pedroerp Differential Revision: D64774281
…or#11330) Summary: This diff adds support for JODA's zzzz (or more) patterns (all equivalent) in Presto's format_datetime function. This is used to format long time zone names. Long time zone names are not available from the IANA time zone database, so we can't use the tz library to generate these. Fortunately, unicode provides some utilities to generate these. Reviewed By: pedroerp Differential Revision: D64795407
Summary: This diff throws a user exception for JODA's zzzz (or more) patterns (all equivalent) in Presto's parse_datetime function. JODA does not support parsing time zone long names (specified by the pattern zzzz) so neither should Velox to be consistent. Previously we were treating any number of 'z's as representing a time zone abbreviation in the pattern which is completely wrong. Reviewed By: amitkdutta, pedroerp Differential Revision: D64797504
kevinwilfong
force-pushed
the
export-D64797504
branch
from
251addf to
1d48ef6
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64797504 |
pedroerp
approved these changes
Oct 29, 2024
|
This pull request has been merged in 3bcc5f3. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This diff throws a user exception for JODA's zzzz (or more) patterns (all equivalent) in
Presto's parse_datetime function.
JODA does not support parsing time zone long names (specified by the pattern zzzz)
so neither should Velox to be consistent. Previously we were treating any number of
'z's as representing a time zone abbreviation in the pattern which is completely
wrong.
Differential Revision: D64797504