[WIP][Spark] Python DeltaTableBuilder API for Identity Columns

[c27kwan]

Which Delta project/connector is this regarding?

• Spark
• Standalone
• Flink
• Kernel
• Other (fill in here)

Description

This PR is part of #1959

In this PR, we extend the addColumn interface in DeltaTableBuilder to allow for Identity Columns creation.

Resolves #1072

How was this patch tested?

New tests.

Does this PR introduce any user-facing changes?

We introduce three new parameters to the addColumn method: generatedAlwaysAsIdentity, identityStart, and identityStep, which can be used to specify Identity Columns that are GENERATED BY DEFAULT and GENERATED ALWAYS.

Interface

def addColumn(
        self,
        colName: str,
        dataType: Union[str, DataType],
        nullable: bool = True,
        generatedAlwaysAs: Optional[str] = None,
        generatedAlwaysAsIdentity: Optional[bool] = None,
        identityStart: Optional[int] = None,
        identityStep: Optional[int] = None,
        comment: Optional[str] = None,
) -> "DeltaTableBuilder"

Example Usage

 DeltaTable.create()
    .tableName("tableName")
    .addColumn("id", dataType=LongType(), generatedAlwaysAsIdentity=true, identityStart = 0, identityStep = 1)
    .execute()

[c27kwan]

This is being replaced by #3404. Closing.