Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-43956: [C++][Format] Add initial Decimal32/Decimal64 implementations #43957

Merged
merged 56 commits into from
Sep 30, 2024
Merged
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
50b57b6
GH-43956: [C++][Format] Add initial Decimal32/Decimal64 implementations
zeroshade Sep 4, 2024
12896a9
fix linting
zeroshade Sep 4, 2024
5e125e0
fixing a build issue
zeroshade Sep 4, 2024
64aad89
add not implemented for new decimal types with pyarrow
zeroshade Sep 5, 2024
f2a46dd
make conversion explicit instaed of implicit
zeroshade Sep 5, 2024
ed3ff00
deprecate old decimal, add smallest_decimal, fix tests
zeroshade Sep 5, 2024
00caa8a
fix linting
zeroshade Sep 5, 2024
1684403
fix implicit cast
zeroshade Sep 5, 2024
9b28a0e
update from feedback
zeroshade Sep 5, 2024
6ba84d5
fix build issue
zeroshade Sep 5, 2024
a510534
enable decimal32/64 integration testing
zeroshade Sep 9, 2024
67512e5
python lint
zeroshade Sep 9, 2024
02aea22
Update cpp/src/arrow/integration/json_internal.cc
zeroshade Sep 9, 2024
db59d6d
Update cpp/src/arrow/ipc/metadata_internal.cc
zeroshade Sep 9, 2024
ebed805
updates from feedback
zeroshade Sep 10, 2024
b95d9ba
pre-commit and fix
zeroshade Sep 10, 2024
f4b3a81
rebase and fix
zeroshade Sep 10, 2024
5d82818
ran pre-commit
zeroshade Sep 10, 2024
315579d
Update cpp/src/arrow/type_fwd.h
zeroshade Sep 10, 2024
5bef850
updates from feedback and propagation
zeroshade Sep 10, 2024
53cf289
more updates from feedback
zeroshade Sep 10, 2024
87a7445
pre-commit linting
zeroshade Sep 10, 2024
7ad7dbc
convert away from deprecated function
zeroshade Sep 10, 2024
202df3b
pre-commit
zeroshade Sep 10, 2024
683a789
fix test
zeroshade Sep 10, 2024
ccbe6ca
changes from feedback for swapendian
zeroshade Sep 10, 2024
91e3c13
fix unit tests
zeroshade Sep 10, 2024
f47a1f4
Update cpp/src/arrow/array/builder_dict.h
zeroshade Sep 11, 2024
86aa26c
fixed test issue in parquet
zeroshade Sep 11, 2024
34493f7
Update cpp/src/arrow/engine/substrait/expression_internal.cc
zeroshade Sep 11, 2024
f71303c
Update cpp/src/arrow/compute/kernels/hash_aggregate.cc
zeroshade Sep 11, 2024
f276a9e
updates from feedback
zeroshade Sep 11, 2024
3b6d74a
lint and windows build fix
zeroshade Sep 11, 2024
7ea13c4
Update cpp/src/arrow/integration/json_internal.cc
zeroshade Sep 16, 2024
295b4c8
Update cpp/src/arrow/util/basic_decimal.h
zeroshade Sep 16, 2024
a8c0c75
updates from feedback and comments
zeroshade Sep 16, 2024
46c84f1
remove commented out code
zeroshade Sep 16, 2024
4973864
ran pre-commit for linting
zeroshade Sep 16, 2024
b7d2bf3
sumtype and accumulator type for decimals should be consistent
zeroshade Sep 16, 2024
512d464
simplify check
zeroshade Sep 16, 2024
8f46e50
linting
zeroshade Sep 16, 2024
ae3d8a2
Add tests for Decimal32 and Decimal64
zeroshade Sep 18, 2024
bf9eb74
linting
zeroshade Sep 18, 2024
2fb271a
remove abs from FromReal, only constexpr in C++23 and newer
zeroshade Sep 18, 2024
b44888d
simplify a bunch of tests with a generic typed_test
zeroshade Sep 19, 2024
e2957a9
use FromRealApprox
zeroshade Sep 19, 2024
980d6fb
static_cast instead of implicit cast
zeroshade Sep 19, 2024
2a3e5c4
remove special cases, adjust tests
zeroshade Sep 19, 2024
c723754
Update cpp/src/arrow/util/decimal.cc
zeroshade Sep 24, 2024
5382eb4
more updates from comments
zeroshade Sep 24, 2024
9fda783
add reference to issue for decimal32 approx
zeroshade Sep 24, 2024
af8c722
make RoundedRightShift a no-op
zeroshade Sep 30, 2024
48639e3
fix tests
zeroshade Sep 30, 2024
b110605
avoid ASAN issue
zeroshade Sep 30, 2024
1d97e27
fix ubsan test
zeroshade Sep 30, 2024
39032f2
fix ubsan
zeroshade Sep 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
ran pre-commit for linting
zeroshade committed Sep 30, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 4973864712e10fb9acc056254051ce80eac0fc52
6 changes: 4 additions & 2 deletions cpp/src/arrow/type_traits.cc
Original file line number Diff line number Diff line change
@@ -72,12 +72,14 @@ int RequiredValueAlignmentForBuffer(Type::type type_id, int buffer_index) {
case Type::MAP: // Same as LIST
case Type::INTERVAL_MONTHS: // Stored as int32_t*
case Type::INTERVAL_DAY_TIME: // Stored as two contiguous 32-bit integers
case Type::DECIMAL32: // May be cast to SmallBasicDecimal* which requires alignment of 4
case Type::DECIMAL32: // May be cast to SmallBasicDecimal* which requires alignment
// of 4
return 4;
case Type::INT64:
case Type::UINT64:
case Type::DOUBLE:
case Type::DECIMAL64: // May be cast to SmallBasicDecimal* which requires alignment of 8
case Type::DECIMAL64: // May be cast to SmallBasicDecimal* which requires alignment
// of 8
case Type::DECIMAL128: // May be cast to GenericBasicDecimal* which requires
// alignment of 8
case Type::DECIMAL256: // May be cast to GenericBasicDecimal* which requires
6 changes: 2 additions & 4 deletions cpp/src/arrow/util/basic_decimal.h
Original file line number Diff line number Diff line change
@@ -422,9 +422,7 @@ class ARROW_EXPORT BasicDecimal64 : public SmallBasicDecimal<int64_t> {
}

/// \brief Absolute value (in-place)
BasicDecimal64& Abs() {
return *this < 0 ? Negate() : *this;
}
BasicDecimal64& Abs() { return *this < 0 ? Negate() : *this; }

/// \brief Absolute value
static BasicDecimal64 Abs(const BasicDecimal64& in) {
@@ -481,7 +479,7 @@ class ARROW_EXPORT BasicDecimal64 : public SmallBasicDecimal<int64_t> {
value_ &= right.value_;
return *this;
}

/// \brief Shift left by the given number of bits.
BasicDecimal64& operator<<=(uint32_t bits);

4 changes: 2 additions & 2 deletions cpp/src/arrow/util/decimal.h
Original file line number Diff line number Diff line change
@@ -121,7 +121,7 @@ class ARROW_EXPORT Decimal32 : public BasicDecimal32 {

/// \brief Convert to a signed integer
template <typename T, typename = internal::EnableIfIsOneOf<T, int32_t, int64_t>>
Result<T> ToInteger() const {
Result<T> ToInteger() const {
return static_cast<T>(value_);
}

@@ -233,7 +233,7 @@ class ARROW_EXPORT Decimal64 : public BasicDecimal64 {

/// \brief Convert to a signed integer
template <typename T, typename = internal::EnableIfIsOneOf<T, int32_t, int64_t>>
Result<T> ToInteger() const {
Result<T> ToInteger() const {
return static_cast<T>(value_);
}