Preserve constant values across union operations

[gokselk]

Which issue does this PR close?

Closes #13804.

Rationale for this change

Currently, DataFusion doesn't preserve constant values across union operations even when both sides have the same constant value. This change enables better optimization by tracking and preserving constant values when they match.

What changes are included in this PR?

• Added value: Option<ScalarValue> field to ConstExpr
• Added methods to get/set constant values
• Modified union operation logic to preserve matching constant values
• Updated equality comparison for ConstExpr
• Added tests for constant value preservation in unions

Are these changes tested?

Yes, added new test case test_union_constant_value_preservation that verifies constant value preservation across unions.

Are there any user-facing changes?

No user-facing changes. This is an internal optimization improvement.

[gokselk]

cc: @berkaysynnada @ozankabak

[berkaysynnada]

I have just one suggestion, otherwise LGTM

[alamb]

Thanks @gokselk and @berkaysynnada

I suggest we try to write an end to end sqllogictest for this query too.

[ozankabak]

I wonder if we should change across_partitions to an enum; i.e.

enum PartitionValues {
    Uniform(Option<ScalarValue>),
    Heterogenous(Vec<Option<ScalarValue>>)
}

with Uniform meaning that all partitions have the same value given in the payload (if known), and Heterogenous meaning partitions can have different constant values (each of which is given in the vector, if known).

[berkaysynnada]

This appears as SortPreservingMergeExec before these changes 👍🏻

[berkaysynnada]

I'm planning to merge this PR once the CI is all green