Skip to content

Commit

Permalink
feat(planbuilder): Accept schema for tableWriter() (#11829)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: #11829

When creating a TableWriter node, allowing the client to specify a
schema different from the default (the output of the previous operator).

Reviewed By: Yuhta

Differential Revision: D67102795

fbshipit-source-id: d48ab68385ae8db4381fcfd7575a904957b23a36
  • Loading branch information
pedroerp authored and facebook-github-bot committed Dec 12, 2024
1 parent 3e4233e commit 98b1497
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 3 deletions.
5 changes: 3 additions & 2 deletions velox/exec/tests/utils/PlanBuilder.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -443,9 +443,10 @@ PlanBuilder& PlanBuilder::tableWrite(
const std::unordered_map<std::string, std::string>& serdeParameters,
const std::shared_ptr<dwio::common::WriterOptions>& options,
const std::string& outputFileName,
const common::CompressionKind compressionKind) {
const common::CompressionKind compressionKind,
const RowTypePtr& schema) {
VELOX_CHECK_NOT_NULL(planNode_, "TableWrite cannot be the source node");
auto rowType = planNode_->outputType();
auto rowType = schema ? schema : planNode_->outputType();

std::vector<std::shared_ptr<const connector::hive::HiveColumnHandle>>
columnHandles;
Expand Down
5 changes: 4 additions & 1 deletion velox/exec/tests/utils/PlanBuilder.h
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,8 @@ class PlanBuilder {
/// only be specified in non-bucketing write.
/// @param compressionKind Compression scheme to use for writing the
/// output data files.
/// @param schema Output schema to be passed to the writer. By default use the
/// output of the previous operator.
PlanBuilder& tableWrite(
const std::string& outputDirectoryPath,
const std::vector<std::string>& partitionBy,
Expand All @@ -499,7 +501,8 @@ class PlanBuilder {
const std::unordered_map<std::string, std::string>& serdeParameters = {},
const std::shared_ptr<dwio::common::WriterOptions>& options = nullptr,
const std::string& outputFileName = "",
const common::CompressionKind = common::CompressionKind_NONE);
const common::CompressionKind = common::CompressionKind_NONE,
const RowTypePtr& schema = nullptr);

/// Add a TableWriteMergeNode.
PlanBuilder& tableWriteMerge(
Expand Down

0 comments on commit 98b1497

Please sign in to comment.