Skip to content

Commit

Permalink
Address comments
Browse files Browse the repository at this point in the history
  • Loading branch information
mohsaka committed Sep 9, 2024
1 parent a0e4692 commit c4dc2bf
Show file tree
Hide file tree
Showing 5 changed files with 35 additions and 36 deletions.
4 changes: 2 additions & 2 deletions velox/docs/develop/types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -148,15 +148,15 @@ Supported range of milliseconds is [0xFFF8000000000000L, 0x7FFFFFFFFFFFF]
store timezone ID. Supported range of timezone ID is [1, 1680].
The definition of timezone IDs can be found in ``TimeZoneDatabase.cpp``.

IPADDRESS represents an IPV6 or IPV4 formatted IPV6 address. Its physical
IPADDRESS represents an IPv6 or IPv4 formatted IPv6 address. Its physical
type is HUGEINT. The format that the address is stored in is defined as part of `(RFC 4291#section-2.5.5.2) <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.5.5.2>`_
As Velox is run on Little Endian systems and the standard is network byte(Big Endian)
order, we reverse the bytes to allow for masking and other bit operations
used in IPADDRESS/IPPREFIX related functions. This type can be used to
create IPPREFIX networks as well as to check IPADDRESS validity within
IPPREFIX networks.

IPPREFIX represents an IPV6 or IPV4 formatted IPV6 address along with a one byte
IPPREFIX represents an IPv6 or IPv4 formatted IPv6 address along with a one byte
prefix length. Its physical type is VARBINARY but has a fixed length of 17 bytes.
The format that the address is stored in is defined as part of `(RFC 4291#section-2.5.5.2) <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.5.5.2>`_.
The prefix length is stored in the last byte of the VARBINARY array.
Expand Down
56 changes: 28 additions & 28 deletions velox/docs/functions/presto/conversion.rst
Original file line number Diff line number Diff line change
Expand Up @@ -700,50 +700,50 @@ is the number of whole days in the interval, HH is then number of hours between
From IPADDRESS
^^^^^^^^^^^^^^

Casting from IPADDRESS to VARCHAR returns a string formatted as x.x.x.x for IPV4 formatted IPV6 addresses.
For all other IPV6 addresses it will be formatted in compressed alternate form IPV6 defined in `RFC 4291#section-2.2 <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.2>`_
Casting from IPADDRESS to VARCHAR returns a string formatted as x.x.x.x for IPv4 formatted IPv6 addresses.
For all other IPv6 addresses it will be formatted in compressed alternate form IPv6 defined in `RFC 4291#section-2.2 <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.2>`_

IPV4:
IPv4:

::

SELECT cast(ipaddress '1.2.3.4' as varchar); -- '1.2.3.4'

IPV6:
IPv6:

::

SELECT cast(ipaddress '2001:0db8:0000:0000:0000:ff00:0042:8329' as varchar); -- '2001:db8::ff00:42:8329'
SELECT cast(ipaddress '0:0:0:0:0:0:13.1.68.3' as varchar); -- '::13.1.68.3'

IPV4 mapped IPV6:
IPv4 mapped IPv6:

::

SELECT cast(ipaddress '::ffff:ffff:ffff' as varchar); -- '255.255.255.255'

From IPPrefix
^^^^^^^^^^^^^^
From IPPREFIX
^^^^^^^^^^^^^

Casting from IPPREFIX to VARCHAR returns a string formatted as *x.x.x.x/<prefix-length>* for IPV4 formatted IPV6 addresses.
Casting from IPPREFIX to VARCHAR returns a string formatted as *x.x.x.x/<prefix-length>* for IPv4 formatted IPv6 addresses.

For all other IPV6 addresses it will be formatted in compressed alternate form IPV6 defined in `RFC 4291#section-2.2 <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.2>`_
For all other IPv6 addresses it will be formatted in compressed alternate form IPv6 defined in `RFC 4291#section-2.2 <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.2>`_
followed by */<prefix-length>*. `[RFC 4291#section-2.3] <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.3>`_

IPV4:
IPv4:

::

SELECT cast(ipprefix '1.2.0.0/16' as varchar); -- '1.2.0.0/16'

IPV6:
IPv6:

::

SELECT cast(ipprefix '2001:db8::ff00:42:8329/128' as varchar); -- '2001:db8::ff00:42:8329/128'
SELECT cast(ipprefix '0:0:0:0:0:0:13.1.68.3/32' as varchar); -- '::/32'

IPV4 mapped IPV6:
IPv4 mapped IPv6:

::

Expand All @@ -755,24 +755,24 @@ Cast to VARBINARY
From IPADDRESS
^^^^^^^^^^^^^^

Returns the IPV6 address as a 16 byte varbinary string in network byte order.
Returns the IPv6 address as a 16 byte varbinary string in network byte order.

Internally, the type is a pure IPv6 address. Support for IPv4 is handled using the IPv4-mapped IPv6 address range `(RFC 4291#section-2.5.5.2) <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.5.5.2>`_.
When creating an IPADDRESS, IPv4 addresses will be mapped into that range.

IPV6:
IPv6:

::

SELECT cast(ipaddress '2001:0db8:0000:0000:0000:ff00:0042:8329' as varbinary); -- 0x20010db8000000000000ff0000428329

IPV4:
IPv4:

::

SELECT cast('1.2.3.4' as ipaddress); -- 0x00000000000000000000ffff01020304

IPV4 mapped IPV6:
IPv4 mapped IPv6:

::

Expand Down Expand Up @@ -1102,12 +1102,12 @@ From VARCHAR
^^^^^^^^^^^^

To cast a varchar to IPAddress input string must be in the form of either
IPV4 or IPV6.
IPv4 or IPv6.

For IPV4 it must be in the form of:
For IPv4 it must be in the form of:
x.x.x.x where each x is an integer value between 0-255.

For IPV6 it must follow any of the forms defined in `RFC 4291#section-2.2 <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.2>`_.
For IPv6 it must follow any of the forms defined in `RFC 4291#section-2.2 <https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.2>`_.

Full form:

Expand Down Expand Up @@ -1149,16 +1149,16 @@ Invalid examples:
From VARBINARY
^^^^^^^^^^^^^^

To cast a varbinary to IPAddress it must be either IPV4(4 Bytes)
or IPV6(16 Bytes) in network byte order.
To cast a varbinary to IPAddress it must be either IPv4(4 Bytes)
or IPv6(16 Bytes) in network byte order.

IPV4:
IPv4:

::

[01, 02, 03, 04] -> 1.2.3.4

IPV6:
IPv6:

::

Expand All @@ -1170,7 +1170,7 @@ When creating an IPADDRESS, IPv4 addresses will be mapped into that range.
When formatting an IPADDRESS, any address within the mapped range will be formatted as an IPv4 address.
Other addresses will be formatted as IPv6 using the canonical format defined in `RFC 5952 <https://datatracker.ietf.org/doc/html/rfc5952.html>`_.

IPV6 mapped IPV4 address:
IPv6 mapped IPv4 address:

::

Expand Down Expand Up @@ -1212,9 +1212,9 @@ The IPPREFIX string must be in the form of *<ip_address>/<ip_prefix>* as defined
The IPADDRESS portion of the IPPREFIX follows the same rules as casting
`IPADDRESS to VARCHAR <#ipaddress-varchar>`_.

The prefix portion must be <= 32 if the IP is an IPV4 address or <= 128 for an IPV6 address.
As with IPADDRESS, any IPV6 address in the form of an IPV4 mapped IPV6 address will be
interpreted as an IPV4 address. Only the canonical(smallest) IP address will be stored
The prefix portion must be <= 32 if the IP is an IPv4 address or <= 128 for an IPv6 address.
As with IPADDRESS, any IPv6 address in the form of an IPv4 mapped IPv6 address will be
interpreted as an IPv4 address. Only the canonical(smallest) IP address will be stored
in the IPPREFIX.

Examples:
Expand Down Expand Up @@ -1242,7 +1242,7 @@ From IPADDRESS
^^^^^^^^^^^^^^

Returns an IPPREFIX where the prefix length is the length of the entire IP Address.
Prefix length for IPV4 is 32 and for IPV6 it is 128.
Prefix length for IPv4 is 32 and for IPv6 it is 128.

Examples:

Expand Down
2 changes: 1 addition & 1 deletion velox/functions/prestosql/TypeOf.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ std::string typeName(const TypePtr& type) {
if (isHyperLogLogType(type)) {
return "HyperLogLog";
} else if (isIPPrefixType(type)) {
return "ipaddress";
return "ipprefix";
}
return "varbinary";
case TypeKind::TIMESTAMP:
Expand Down
8 changes: 4 additions & 4 deletions velox/functions/prestosql/types/IPAddressType.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -221,14 +221,14 @@ class IPAddressCastOperator : public exec::CastOperator {
const SelectivityVector& rows,
BaseVector& result) {
auto* flatResult = result.as<FlatVector<int128_t>>();
const auto* ipaddresses = input.as<SimpleVector<StringView>>();
const auto* prefixes = input.as<SimpleVector<StringView>>();

context.applyToSelectedNoThrow(rows, [&](auto row) {
const auto intAddr = ipaddresses->valueAt(row);
const auto prefix = prefixes->valueAt(row);
int128_t addrResult = 0;
folly::ByteArray16 addrBytes;

memcpy(&addrBytes, intAddr.data(), kIPAddressBytes);
memcpy(&addrBytes, prefix.data(), kIPAddressBytes);
std::reverse(addrBytes.begin(), addrBytes.end());

memcpy(&addrResult, &addrBytes, kIPAddressBytes);
Expand Down Expand Up @@ -259,7 +259,7 @@ class IPAddressCastOperator : public exec::CastOperator {
if (v6Addr.isIPv4Mapped()) {
result.data()[kIPAddressBytes] = kIPV4Bits;
} else {
result.data()[kIPAddressBytes] = (unsigned char)kIPV6Bits;
result.data()[kIPAddressBytes] = kIPV6Bits;
}

result.finalize();
Expand Down
1 change: 0 additions & 1 deletion velox/functions/prestosql/types/IPPrefixType.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,6 @@ class IPPrefixCastOperator : public exec::CastOperator {
context.applyToSelectedNoThrow(rows, [&](auto row) {
const auto intAddr = ipaddresses->valueAt(row);
folly::ByteArray16 addrBytes;
std::string s;

memcpy(&addrBytes, intAddr.data(), kIPAddressBytes);
folly::IPAddressV6 v6Addr(addrBytes);
Expand Down

0 comments on commit c4dc2bf

Please sign in to comment.