Skip to content

Commit

Permalink
Extend expression fuzzer test to support decimal
Browse files Browse the repository at this point in the history
  • Loading branch information
rui-mo committed Apr 3, 2024
1 parent f0f5986 commit b5d686a
Show file tree
Hide file tree
Showing 31 changed files with 964 additions and 84 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/scheduled.yml
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,7 @@ jobs:
--seed ${RANDOM} \
--enable_variadic_signatures \
--velox_fuzzer_enable_complex_types \
--velox_fuzzer_enable_decimal_type \
--lazy_vector_generation_ratio 0.2 \
--velox_fuzzer_enable_column_reuse \
--velox_fuzzer_enable_expression_reuse \
Expand Down Expand Up @@ -346,6 +347,7 @@ jobs:
--duration_sec 3600 \
--enable_variadic_signatures \
--velox_fuzzer_enable_complex_types \
--velox_fuzzer_enable_decimal_type \
--velox_fuzzer_enable_column_reuse \
--velox_fuzzer_enable_expression_reuse \
--max_expression_trees_per_step 2 \
Expand Down Expand Up @@ -465,6 +467,7 @@ jobs:
--enable_variadic_signatures \
--lazy_vector_generation_ratio 0.2 \
--velox_fuzzer_enable_column_reuse \
--velox_fuzzer_enable_decimal_type \
--velox_fuzzer_enable_expression_reuse \
--max_expression_trees_per_step 2 \
--retry_with_try \
Expand Down
7 changes: 7 additions & 0 deletions velox/docs/develop/scalar-functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -998,6 +998,13 @@ element of the array and returns a new array of the results.

The signature of a function that handles DECIMAL types can additionally take
variables and constraints to represent the precision and scale values.
The variables of input decimal types store the input precisions and scales.
Their names begin with an incrementing prefix starting from 'a', and followed
by '_precision' or '_scale'. Variables of output decimal types store the output
precision and scale. Their names begin with 'r', and followed by '_precision'
or '_scale'. When there is only one input decimal type, and the output type
holds the same precision and scale with the input type, the variables could be
named as 'precision' and 'scale'.
The constraints are evaluated using a type calculator built from Flex and Bison
tools. The decimal arithmetic addition function has the following signature:

Expand Down
3 changes: 3 additions & 0 deletions velox/docs/develop/testing/fuzzer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,8 @@ Below are arguments that toggle certain fuzzer features in Expression Fuzzer:

* ``--velox_fuzzer_enable_complex_types``: Enable testing of function signatures with complex argument or return types. Default is false.

* ``--velox_fuzzer_enable_decimal_type``: Enable testing of function signatures with decimal argument or return type. Default is false.

* ``--lazy_vector_generation_ratio``: Specifies the probability with which columns in the input row vector will be selected to be wrapped in lazy encoding (expressed as double from 0 to 1). Default is 0.0.

* ``--velox_fuzzer_enable_column_reuse``: Enable generation of expressions where one input column can be used by multiple subexpressions. Default is false.
Expand Down Expand Up @@ -246,6 +248,7 @@ An example set of arguments to run the expression fuzzer with all features enabl
--enable_variadic_signatures
--lazy_vector_generation_ratio 0.2
--velox_fuzzer_enable_complex_types
--velox_fuzzer_enable_decimal_type
--velox_fuzzer_enable_expression_reuse
--velox_fuzzer_enable_column_reuse
--retry_with_try
Expand Down
20 changes: 0 additions & 20 deletions velox/expression/ReverseSignatureBinder.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -18,27 +18,7 @@

namespace facebook::velox::exec {

bool ReverseSignatureBinder::hasConstrainedIntegerVariable(
const TypeSignature& type) const {
if (type.parameters().empty()) {
auto it = variables().find(type.baseName());
return it != variables().end() && it->second.isIntegerParameter() &&
it->second.constraint() != "";
}

const auto& parameters = type.parameters();
for (const auto& parameter : parameters) {
if (hasConstrainedIntegerVariable(parameter)) {
return true;
}
}
return false;
}

bool ReverseSignatureBinder::tryBind() {
if (hasConstrainedIntegerVariable(signature_.returnType())) {
return false;
}
return SignatureBinderBase::tryBind(signature_.returnType(), returnType_);
}

Expand Down
4 changes: 0 additions & 4 deletions velox/expression/ReverseSignatureBinder.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,6 @@ class ReverseSignatureBinder : private SignatureBinderBase {
}

private:
/// Return whether there is a constraint on an integer variable in type
/// signature.
bool hasConstrainedIntegerVariable(const TypeSignature& type) const;

const TypePtr returnType_;
};

Expand Down
36 changes: 36 additions & 0 deletions velox/expression/tests/ArgumentGenerator.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#pragma once

#include "velox/core/ITypedExpr.h"
#include "velox/expression/tests/utils/FuzzerToolkit.h"

namespace facebook::velox::test {

class ExpressionFuzzer;

class ArgumentGenerator {
public:
virtual ~ArgumentGenerator() = default;

// Generates function arguments of the specified signature.
virtual std::vector<core::TypedExprPtr> generate(
ExpressionFuzzer* expressionFuzzer,
const CallableSignature& input,
int32_t maxNumVarArgs) = 0;
};

} // namespace facebook::velox::test
Loading

0 comments on commit b5d686a

Please sign in to comment.