Skip to content

Commit

Permalink
[VL] Daily Update Velox Version (2024_03_21) (#5064)
Browse files Browse the repository at this point in the history
Velox changes:
```
64f9618c3 by Wei He, Fix exception thrown in custom result verifier in WindowFuzzer (9189)
3db2a9ad8 by Nicholas Ormrod, Deshim //folly:dynamic to //folly/json:dynamic in velox (8931)
43a552715 by Sergey Pershin, Add Task:driverCounts() (9166)
7e8105a0b by rui-mo, Fix gatherFromTimestampBuffer (9170)
5186d0bc9 by joey.ljy, Fix the CMakeLists.txt to sort the cpp files in alphabetical order (9164)
6f09b8fe5 by Richard Barnes, Remove unused exception parameter from velox/common/base/tests/StatusTest.cpp (9180)
b8c752abc by Richard Barnes, Enable `-Wmismatched-tags` in velox/PACKAGE (9181)
2c044edfc by Richard Barnes, Remove unused variables in velox/tpch/gen/dbgen/text.cpp (9142)
e574117a4 by Richard Barnes, Remove unused variables in velox/connectors/hive/tests/HiveDataSinkTest.cpp
f99cf2858 by Masha Basmanova, Extract CppToType and CastTypeChecker from Type.h (9161)
491c46fa3 by Krishna Pai, Fix build on ubuntu and gcc 11 (9163)
469e2da6c by Jacob Wujciak-Jens, Move Linux build, adapters and presto fuzzer to gha (8734)
443c72dc2 by joey.ljy, Unregister the UDTs after use in ArrayViewTest and ArrayWriterTest (9175)
35fd4b757 by Ma, Rong, Add make_ym_interval Spark function (8887)
fe0c1fd52 by Jialiang Tan, Threshold based spilling removal (9069)
07d638c1b by 8dukongjian, Fix regr_r2 failure in AggregationFuzzer (9173)
8981498a9 by Masha Basmanova, Replace fromKindToScalerType with createScalarType (9162)
cd2523fac by Kevin Wilfong, Fix hang in MultiThreadedTaskCursor when Task fails (9134)
5e077909f by xiaoxmeng, Table scan cleanup (9156)
21491d74a by Kevin Wilfong, Fix NPE caused by testingRunArbitration (9137)
1d95a474a by Kevin Wilfong, Fix bug in PrestoVectorSerializers when serializing empty Vectors with complex types (9157)
29d0f317d by Wei He, Fix window fuzzer test (9153)
```
Co-authored-by: yan ma <[email protected]>
  • Loading branch information
GlutenPerfBot authored Mar 21, 2024
1 parent 42c6be8 commit f51b7ab
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 11 deletions.
10 changes: 0 additions & 10 deletions cpp/velox/compute/WholeStageResultIterator.cc
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,6 @@ const std::string kSpillStrategyDefaultValue = "auto";
const std::string kAggregationSpillEnabled = "spark.gluten.sql.columnar.backend.velox.aggregationSpillEnabled";
const std::string kJoinSpillEnabled = "spark.gluten.sql.columnar.backend.velox.joinSpillEnabled";
const std::string kOrderBySpillEnabled = "spark.gluten.sql.columnar.backend.velox.orderBySpillEnabled";
const std::string kAggregationSpillMemoryThreshold =
"spark.gluten.sql.columnar.backend.velox.aggregationSpillMemoryThreshold";
const std::string kJoinSpillMemoryThreshold = "spark.gluten.sql.columnar.backend.velox.joinSpillMemoryThreshold";
const std::string kOrderBySpillMemoryThreshold = "spark.gluten.sql.columnar.backend.velox.orderBySpillMemoryThreshold";
const std::string kMaxSpillLevel = "spark.gluten.sql.columnar.backend.velox.maxSpillLevel";
const std::string kMaxSpillFileSize = "spark.gluten.sql.columnar.backend.velox.maxSpillFileSize";
const std::string kMinSpillRunSize = "spark.gluten.sql.columnar.backend.velox.minSpillRunSize";
Expand Down Expand Up @@ -507,12 +503,6 @@ std::unordered_map<std::string, std::string> WholeStageResultIterator::getQueryC
std::to_string(veloxCfg_->get<bool>(kJoinSpillEnabled, true));
configs[velox::core::QueryConfig::kOrderBySpillEnabled] =
std::to_string(veloxCfg_->get<bool>(kOrderBySpillEnabled, true));
configs[velox::core::QueryConfig::kAggregationSpillMemoryThreshold] = std::to_string(
veloxCfg_->get<uint64_t>(kAggregationSpillMemoryThreshold, 0)); // spill only when input doesn't fit
configs[velox::core::QueryConfig::kJoinSpillMemoryThreshold] =
std::to_string(veloxCfg_->get<uint64_t>(kJoinSpillMemoryThreshold, 0)); // spill only when input doesn't fit
configs[velox::core::QueryConfig::kOrderBySpillMemoryThreshold] =
std::to_string(veloxCfg_->get<uint64_t>(kOrderBySpillMemoryThreshold, 0)); // spill only when input doesn't fit
configs[velox::core::QueryConfig::kMaxSpillLevel] = std::to_string(veloxCfg_->get<int32_t>(kMaxSpillLevel, 4));
configs[velox::core::QueryConfig::kMaxSpillFileSize] =
std::to_string(veloxCfg_->get<uint64_t>(kMaxSpillFileSize, 20L * 1024 * 1024));
Expand Down
2 changes: 1 addition & 1 deletion ep/build-velox/src/get_velox.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
set -exu

VELOX_REPO=https://github.com/oap-project/velox.git
VELOX_BRANCH=2024_03_20
VELOX_BRANCH=2024_03_21
VELOX_HOME=""

#Set on run gluten on HDFS
Expand Down

0 comments on commit f51b7ab

Please sign in to comment.