Skip to content

Commit

Permalink
feat(function): Fix Presto URL functions to more closely match Presto…
Browse files Browse the repository at this point in the history
… Java's behavior (facebookincubator#11488)

Summary:

Today the Presto URL functions rely on a regex to extract features of a URL.  Due to the limitations of
regexes this is not sufficient to validate that a URL is indeed valid. This leads to a number of cases
where Presto Java will return NULL due to an invalid URL and Velox will return some substring.

To address this I've implemented a parser for URIs based on the RFC 3986 spec which can both
validate a URL and extract features of it.

While testing this change I noticed a number of other discrepancies between Presto Java's and
Velox's implementations of these UDFs (mostly related to unescaping/decoding URLs or portions
thereof) that had been missed, likely due to the noise from the different handling of invalid URLs.

Those are addressed in this diff as well so that I could effectively test it.

Reviewed By: kgpai

Differential Revision: D65695961
  • Loading branch information
Kevin Wilfong authored and facebook-github-bot committed Nov 21, 2024
1 parent df3266c commit 72fc7e1
Show file tree
Hide file tree
Showing 5 changed files with 1,103 additions and 171 deletions.
1 change: 1 addition & 0 deletions velox/functions/prestosql/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ velox_add_library(
TransformKeys.cpp
TransformValues.cpp
TypeOf.cpp
URIParser.cpp
URLFunctions.cpp
VectorArithmetic.cpp
WidthBucketArray.cpp
Expand Down
Loading

0 comments on commit 72fc7e1

Please sign in to comment.