Skip to content

Conversation

@tswast
Copy link
Contributor

@tswast tswast commented Oct 31, 2025

Rationale for this change

I'm working on a system that supports picosecond precision timestamps, and I'm interested in interoperability with the Arrow ecosystem. See #47848

What changes are included in this PR?

This PR only adds the new enum value TimeUnit::PICO. This enum value is unused for now, but I plan to introduce a timestamp128 type in the future which supports this unit type.

Are these changes tested?

I have updated and run the C++ test suite.

root@dc20d61d62a2:/github.com/apache/arrow/cpp/build# 
...
100% tests passed, 0 tests failed out of 91

Label Time Summary:
arrow-compute-tests    = 128.41 sec*proc (15 tests)
arrow-tests            =  65.74 sec*proc (38 tests)
arrow_acero            = 120.93 sec*proc (14 tests)
arrow_dataset          = 103.61 sec*proc (13 tests)
arrow_substrait        =   0.98 sec*proc (1 test)
filesystem             =   1.97 sec*proc (1 test)
parquet-tests          =  88.40 sec*proc (10 tests)
unittest               = 508.07 sec*proc (91 tests)

Total Test time (real) =  50.20 sec

Are there any user-facing changes?

This PR includes breaking changes to public APIs.

Adding a new enum variable did break some tests that assumed that all TimeUnit::values() are valid for use in the timestamp (64-bit) data type.

This enum value is unused for now, but I plan to introduce a
timestamp128 type in the future which supports this.
@tswast tswast requested a review from wgtmac as a code owner October 31, 2025 20:42
@github-actions
Copy link

⚠️ GitHub issue #47848 has been automatically assigned in GitHub to PR creator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change isn't necessary for this particular PR, but it will be for timestamp128 support. Let me know if you want me to break this out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, maybe it is, because otherwise cpp/src/arrow/array/diff.cc won't compile. I needed to be able to handle all unit enum values, and I don't think it makes sense for picoseconds to be stored in anything smaller than int128.

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Oct 31, 2025
This enum value is unused for now, but I plan to introduce a
timestamp128 type in the future which supports this.
@tswast
Copy link
Contributor Author

tswast commented Oct 31, 2025

Looks like I probably need to adjust some linker stuff on Windows? Lots of errors like this:

"C:\Program Files\Git\usr\bin\ccache.exe" C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1444~1.352\bin\Hostx64\x64\cl.exe  /nologo /TP -DARROW_EXPORTING -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_WITH_TIMING_TESTS -D_CRT_SECURE_NO_WARNINGS -D__SSE2__ -D__SSE4_1__ -D__SSE4_2__ -ID:\a\arrow\arrow\build\cpp\src -ID:\a\arrow\arrow\cpp\src -ID:\a\arrow\arrow\cpp\src\generated /D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING /W3 /EHsc /wd5105 /bigobj /utf-8 /W3 /wd4365 /wd4267 /wd4838 /wd4800 /wd4996 /wd4065 /arch:SSE4.2  /Ob0 /Od /RTC1 /WX -std:c++17 -MDd -Zi /showIncludes /Fosrc\arrow\CMakeFiles\arrow_array_shared.dir\array\diff.cc.obj /Fdsrc\arrow\CMakeFiles\arrow_array_shared.dir\ /FS -c D:\a\arrow\arrow\cpp\src\arrow\array\diff.cc
D:\a\arrow\arrow\cpp\src\arrow/util/int128_internal.h(26): fatal error C1083: Cannot open include file: 'boost/multiprecision/cpp_int.hpp': No such file or directory

@wgtmac
Copy link
Member

wgtmac commented Nov 2, 2025

This seems to be a spec change. Perhaps it is better to discuss this on the [email protected] per the process https://arrow.apache.org/docs/format/Changing.html#discussion-and-voting-process

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants