Ensure arrow field's nullable flag matches the schema column

[steinitzu]

Description

Updates nullability of fields in arrow schema as part of normalize_py_arrow_item.
Fixes issue that affects at least sql database source when pandas backend and bigquery destination is used. Any resource yielding dataframes when pk/non-nullable columns are defined has the same problem, e.g.

import dlt
import pandas as pd


@dlt.resource(primary_key="col1")
def some_data():
    yield pd.DataFrame({"col1": [1, 2, 3], "col2": ["a", "b", "c"]})

# Fails due to schema mismatch
info = dlt.pipeline(full_refresh=True, destination="bigquery").run(some_data)
print(info)

Fails with:

Provided Schema does not match Table serene-anagram-189621:dlt_ipython_dataset_20240531072948.some_data. 
Field col1 has changed mode from REQUIRED to NULLABLE

Related Issues

Somewhat related to dlt-hub/verified-sources#430 , part of the reason bigquery fails

[netlify]

✅ Deploy Preview for dlt-hub-docs canceled.

Name	Link
🔨 Latest commit	e3d46e9
🔍 Latest deploy log	https://app.netlify.com/sites/dlt-hub-docs/deploys/665a29f068cf450008966e61

[rudolfix]

LGTM! is this PR still draft?

[steinitzu]

LGTM! is this PR still draft?

@rudolfix no just didn't have time to wait for all tests to run.
You can merge if it looks good to you 🙂