Support MAP type in derived column creation during segment reload

[xiangfu0]

Support add a derived MAP type column from existing JSON or String field.
Reload table/segment will create the new data and index
Sample derived column could be:

{
          "columnName": "event_properties_map",
          "transformFunction": "jsonStringToMap(event_properties)"
}

[Copilot]

Pull Request Overview

This pull request adds support for derived MAP type columns during segment reload. The change enables creation of MAP columns from existing JSON or STRING fields through function transformations, with proper handling during table/segment reload.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
ExpressionTransformerTest.java	Adds test for JSON-to-MAP transformation and updates assertions to use assertNull
BaseDefaultColumnHandler.java	Implements MAP column handling in derived column creation with proper stats collection and forward index creation
MapColumnPreIndexStatsCollector.java	Enhances MAP stats collection to handle various value types (boolean, object, etc.) and convert them to appropriate storage types
PinotDataType.java	Adds BYTES conversion support and serialization/deserialization methods for MAP type
FunctionUtils.java	Adds MAP type detection for Map class instances

[codecov-commenter]

Codecov Report

❌ Patch coverage is 44.59459% with 41 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.17%. Comparing base (62ec58c) to head (ef947ce).
⚠️ Report is 24 commits behind head on master.

Files with missing lines	Patch %	Lines
...or/impl/stats/MapColumnPreIndexStatsCollector.java	61.22%	16 Missing and 3 partials ⚠️
...loader/defaultcolumn/BaseDefaultColumnHandler.java	16.66%	15 Missing ⚠️
...a/org/apache/pinot/common/utils/PinotDataType.java	0.00%	5 Missing ⚠️
...rg/apache/pinot/common/function/FunctionUtils.java	0.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #17113      +/-   ##
============================================
- Coverage     63.17%   63.17%   -0.01%     
  Complexity     1428     1428              
============================================
  Files          3104     3114      +10     
  Lines        183488   183994     +506     
  Branches      28127    28196      +69     
============================================
+ Hits         115916   116235     +319     
- Misses        58613    58776     +163     
- Partials       8959     8983      +24     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-11	63.14% <44.59%> (+<0.01%)	⬆️
java-21	63.13% <44.59%> (-0.02%)	⬇️
temurin	63.17% <44.59%> (-0.01%)	⬇️
unittests	63.16% <44.59%> (-0.01%)	⬇️
unittests1	56.07% <22.97%> (-0.11%)	⬇️
unittests2	33.66% <44.59%> (+0.06%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

[Copilot]

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

[Copilot]

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

[Copilot]

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

[Copilot]

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.