Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev: Pangea v1alpha: Do Not Merge. #2071

Closed
wants to merge 49 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
1b234c8
Adds ExternalCatalogDatasetOptions to Dataset
chalmerlowe Sep 4, 2024
8029213
adds ExternalCatalogTableOptions class and assorted content
chalmerlowe Sep 5, 2024
0992bbf
modifies argument names to snake_case
chalmerlowe Sep 11, 2024
45ddd89
replaces dtype placeholder with parameter names
chalmerlowe Sep 11, 2024
1411460
updates the inclusion of value in properties to use repr version
chalmerlowe Sep 11, 2024
20ee950
updates another inclusion of value in properties to use repr version
chalmerlowe Sep 11, 2024
bee33ef
updates type check via isinstance() or None
chalmerlowe Sep 11, 2024
15acfb3
Merge branch 'main' into add-pangea-classes
chalmerlowe Sep 11, 2024
ee69f24
adds tests related to ExternalCatalogDatasetOptions
chalmerlowe Sep 12, 2024
aeab931
Merge branch 'main' into add-pangea-classes
chalmerlowe Sep 12, 2024
f9d657b
adds test suite for ExternalCatalogTableOptions and minor tweaks else…
chalmerlowe Sep 12, 2024
89896a3
corrects Error type of failing test
chalmerlowe Sep 19, 2024
c452459
forgive me... a wild mess of tests, tweaks, etc
chalmerlowe Sep 26, 2024
199e903
Updates isinstance_or_raise, refines ExternalCatalogDatasetOptions in…
chalmerlowe Oct 2, 2024
e238ba0
Updates ExternalCatalogTableOptions and associated tests
chalmerlowe Oct 2, 2024
5fc89ae
Tweaks several docstrings
chalmerlowe Oct 2, 2024
68d04f0
Adds content related to ForeignTypeInfo
chalmerlowe Oct 2, 2024
2a5774e
add new classes and tests
chalmerlowe Oct 3, 2024
cbd08c5
Merge branch 'main' into add-pangea-classes
chalmerlowe Oct 3, 2024
0fcf424
Update tests/unit/test_schema.py
chalmerlowe Oct 3, 2024
d7698d2
Update google/cloud/bigquery/_helpers.py
chalmerlowe Oct 11, 2024
43dc45e
updates logic and tests related to _isinstance_or_raise'
chalmerlowe Oct 11, 2024
4f117a7
updates from_api_repr and a number of tests and cleans up miscellaneo…
chalmerlowe Oct 11, 2024
defa38c
Update google/cloud/bigquery/_helpers.py
chalmerlowe Oct 14, 2024
14d1bd8
Most recent round of tweaks and experiments
chalmerlowe Oct 30, 2024
1b7ba09
Updates from futures import annotation.
chalmerlowe Nov 1, 2024
79bbeb2
Updates from_api_repr() and external_config tests
chalmerlowe Nov 4, 2024
d71d904
Updates external_catalog_dataset functions in dataset.py and tests.
chalmerlowe Nov 4, 2024
b0a7fb1
Adds fixtures, tests, corrections to classes and tests
chalmerlowe Nov 6, 2024
d0d96fa
Updates comments and addes a to_api_repr test
chalmerlowe Nov 6, 2024
16e2c2c
Merge branch 'main' into add-pangea-classes
chalmerlowe Nov 12, 2024
116de78
Revises test for additional clarity
chalmerlowe Nov 13, 2024
5d0b7d6
Merge branch 'add-pangea-classes' into pangea-v1alpha
chalmerlowe Nov 15, 2024
e912133
chore: merge contents of main that were in add-pangea-classes into pa…
chalmerlowe Nov 15, 2024
0960426
Updates test_dataset to account for coverage
chalmerlowe Nov 15, 2024
8e5af84
Removes single line comment
chalmerlowe Nov 15, 2024
b67dda2
chore: syncing with main (#2067)
chalmerlowe Nov 18, 2024
236455c
fix: Updates mypy and pytype annotations, etc (#2072)
chalmerlowe Nov 22, 2024
9fd8854
feat: adds two from_api_tests: StorageDesc & SerDeInfo (#2074)
chalmerlowe Nov 25, 2024
8162a2c
feat: adds ForeignTypeInfo test (#2076)
chalmerlowe Nov 25, 2024
74beca6
Merge branch 'main' into pangea-v1alpha
chalmerlowe Nov 25, 2024
48c8cc6
feat: Adds attributes to SchemaField (#2077)
chalmerlowe Nov 26, 2024
1fcbc09
fix: updates tests on getters to resolve coverage issues (#2080)
chalmerlowe Dec 4, 2024
07bc30a
Merge branch 'main' into pangea-v1alpha
chalmerlowe Dec 6, 2024
bb5c06c
Merge branch 'main' into pangea-v1alpha
chalmerlowe Dec 10, 2024
23eb8eb
Merge branch 'main' into pangea-v1alpha
chalmerlowe Dec 30, 2024
c4bd724
Update google/cloud/bigquery/schema.py
chalmerlowe Jan 2, 2025
94a39b4
Update schema.py
chalmerlowe Jan 2, 2025
100708d
Fix: add roundingmode as str test (#2098)
chalmerlowe Jan 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: adds ForeignTypeInfo test (#2076)
* likely gonna delete this branch cause TestTableSchema may not be necessary

* Adds ForeignTypeInfo test_from_api_repr

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* fixes merge conflict

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
chalmerlowe and gcf-owl-bot[bot] authored Nov 25, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 8162a2cc653fa4958e693c4c5b8c149032626278
74 changes: 16 additions & 58 deletions google/cloud/bigquery/schema.py
Original file line number Diff line number Diff line change
@@ -24,7 +24,6 @@
from google.cloud.bigquery import standard_sql
from google.cloud.bigquery._helpers import (
_isinstance_or_raise,
_from_api_repr,
_get_sub_prop,
)
from google.cloud.bigquery.enums import StandardSqlTypeNames
@@ -501,6 +500,7 @@ def _to_schema_fields(schema):
sequence is not a :class:`~google.cloud.bigquery.schema.SchemaField`
instance or a compatible mapping representation of the field.
"""

for field in schema:
if not isinstance(field, (SchemaField, collections.abc.Mapping)):
raise ValueError(
@@ -598,61 +598,6 @@ def to_api_repr(self) -> dict:
return answer


class TableSchema:
"""Schema of a table

Args:
fields (Optional[list]): Describes the fields in a table.
foreignTypeInfo (Optional[str]): Specifies metadata of the foreign data type
definition in field schema.
"""

def __init__(
self, fields: Optional[list] = None, foreign_type_info: Optional[str] = None
):
self._properties: Dict[str, Any] = {}
self.fields = fields
self.foreign_type_info = foreign_type_info

@property
def fields(self) -> Any:
"""Describes the fields in a table."""

return self._properties.get("fields")

@fields.setter
def fields(self, value: list, dtype: str) -> None:
value = _isinstance_or_raise(value, list, none_allowed=True)
self._properties["fields"] = value

@property
def foreign_type_info(self) -> Any:
"""Optional. Specifies metadata of the foreign data type definition in
field schema (TableFieldSchema.foreign_type_definition)."""

return self._properties.get("foreignTypeInfo")

@foreign_type_info.setter
def foreign_type_info(self, value: str, dtype: str) -> None:
if not isinstance(value, str):
raise ValueError(
f"Pass {value} as a '{repr(dtype)}'." f"Got {type(value)}."
)
self._properties["foreignTypeInfo"] = value

def to_api_repr(self) -> dict:
"""Build an API representation of this object.

Returns:
Dict[str, Any]:
A dictionary in the format used by the BigQuery API.
"""
return copy.deepcopy(self._properties)

def from_api_repr(self, resource):
return _from_api_repr(self, resource)


class ForeignTypeInfo:
"""Metadata about the foreign data type definition such as the system in which the
type is defined.
@@ -687,8 +632,21 @@ def to_api_repr(self) -> dict:
"""
return copy.deepcopy(self._properties)

def from_api_repr(self, resource):
return _from_api_repr(self, resource)
@classmethod
def from_api_repr(cls, resource):
"""Factory: constructs an instance of the class (cls)
given its API representation.

Args:
resource (Dict[str, Any]):
API representation of the object to be instantiated.

Returns:
An instance of the class initialized with data from 'resource'.
"""
config = cls()
config._properties = copy.deepcopy(resource)
return config


class StorageDescriptor:
20 changes: 17 additions & 3 deletions tests/unit/test_schema.py
Original file line number Diff line number Diff line change
@@ -16,7 +16,7 @@
from google.cloud.bigquery.standard_sql import StandardSqlStructType
from google.cloud.bigquery.schema import (
PolicyTagList,
# ForeignTypeInfo,
ForeignTypeInfo,
StorageDescriptor,
SerDeInfo,
)
@@ -1121,8 +1121,6 @@ class TestForeignTypeInfo:

@staticmethod
def _get_target_class():
from google.cloud.bigquery.schema import ForeignTypeInfo

return ForeignTypeInfo

def _make_one(self, *args, **kw):
@@ -1160,6 +1158,22 @@ def test_to_api_repr(self, type_system, expected):
result = self._make_one(type_system=type_system)
assert result.to_api_repr() == expected

def test_from_api_repr(self):
"""GIVEN an api representation of a ForeignTypeInfo object (i.e. resource)
WHEN converted into a ForeignTypeInfo object using from_api_repr() and
displayed as a dict
THEN it will have the same representation a ForeignTypeInfo object created
directly (via _make_one()) and displayed as a dict.
"""
resource = {"typeSystem": "TYPE_SYSTEM_UNSPECIFIED"}

expected = self._make_one(type_system="TYPE_SYSTEM_UNSPECIFIED")

klass = self._get_target_class()
result = klass.from_api_repr(resource)

assert result.to_api_repr() == expected.to_api_repr()


@pytest.fixture
def _make_storage_descriptor():