Add Variant Type

[SpencerTorres]

Summary

Implement Variant column type. Partially resolves #1430.
closes #1195

Implementation

This implementation adds 2 major types to the module:

• column.Variant - the column implementation for (de)serialization
• clickhouse.Variant - a container to hold variant values (optional for (de)serialization). This type also has the ability to provide a preferred type in cases where it is ambiguous to existing column type detection (such as Array(UInt8) vs String)

column.Variant

Serialization

// Variant(Array(Map(String, String)), Array(UInt8), Bool, Int64, String)
batch, err := conn.PrepareBatch(ctx, "INSERT INTO test_variant (c)")
require.NoError(t, err)
require.NoError(t, batch.Append(int64(42))) // Accepts primitives
require.NoError(t, batch.Append(clickhouse.NewVariantWithType("test", "String"))) // Accepts Variants with type preference
require.NoError(t, batch.Append(true))
require.NoError(t, batch.Append(clickhouse.NewVariant([]uint8{0xA, 0xB, 0xC}).WithType("Array(UInt8)"))) 
require.NoError(t, batch.Append(nil)) // Accepts nil
require.NoError(t, batch.Append([]map[string]string{{"key1": "val1"}, {"key2": "val2"}})) // Accepts complex types

When values are appended via col.AppendRow(), the input v interface{} type is checked. If it is nil, a Null discriminator is appended. If it is a clickhouse.Variant with a preferred type, then the specified column type will be appended along with its matching discriminator. The underlying column's AppendRow function is re-used so that we don't need to re-implement its logic.

As a catch-all, the input value will be tested against each column type until it succeeds. For example, Variant(Bool, Int64, String) will try to append as bool, int64, then string. If a value does not fit into any column type, it will return an error.

Sometimes types will conflict. Due to alphabetical sorting of the type, Array(UInt8) would be used before String since Array allows for string input. I have researched different solutions to this, including a type priority system, but it would be complex to implement. For now it is easiest to let the user simply input NewVariantWithType(int64(42), "Int64") or NewVariant(int64(42)).WithType("Int64") if they want a specific type within the variant. For complex types like maps, reflection will be used if a type isn't specified.

After all rows are appended, the Native format is used to serialize the data into the buffer. First with serializationVersion, then the uint8 array for discriminators, then each column's Encode function is re-used as usual (similar to Tuple).

Deserialization

The Native format deserializes the discriminators and builds a set of offsets for each column. This allows for storing multiple columns with mixed lengths. When the user wants to read a row, we can index into the correct row of each column to get the corresponding type.

In practice this looks like this:

var row clickhouse.Variant // Scan into variant

require.True(t, rows.Next())
err = rows.Scan(&row)
require.NoError(t, err)
require.Equal(t, int64(42), row.Any())

Or, if you know your types ahead of time, you can also scan directly into it:

var i int64 // Scan directly into int64
require.True(t, rows.Next())
err = rows.Scan(&i)
require.NoError(t, err)
require.Equal(t, int64(84), i)

This pattern works by simply calling the underlying column's ScanRow function. It is safest to scan into Variant however.
If you need to switch types on Variant for your own type detection, you can use variantRow.Any() to return any.
You can also switch on ClickHouse type strings by using variantRow.Type().

clickhouse.Variant

clickhouse.Variant is simply a wrapper around any that also includes an optional string to represent the preferred ClickHouse type. It implements stdlib sql interfaces such as driver.Value and Scan. If you need to access the underlying value you can use Any(). This type can be constructed with the clickhouse.NewVariant(v) function, or with clickhouse.NewVariantWithType(v, "Array(String)") if you need to provide a preferred type.

The clickhouse.Variant type should be used in structs and when scanning from column.Variant. It can also be used for insertion, although a variant type may be required if there's overlap between types.

You can use the preferred type for insertion when the Variant column has types that overlap. For example if you had Variant(Array(UInt8), String), a Go string would be inserted as an Array(UInt8). If you wanted to force this to be a ClickHouse String, you could use clickhouse.NewVariantWithType(v, "String") to provide the preferred type. If the preferred type is not present in the Variant, the row will fail to append to the block. Types can be added on an existing Variant by calling exampleVariant.WithType(t string), which will return a new clickhouse.Variant with the preferred type set.

Checklist

Delete items not relevant to your PR:

• Unit and integration tests covering the common scenarios were added

[jkaflik]

1st review

[jkaflik]

Overall LGTM. Let's push ongoing conversations before merging.

[SpencerTorres]

Updated to resolve all review notes. Thanks! 👍

I will apply these changes to the other PRs too