Skip to content

Commit

Permalink
Merge pull request #3320 from cal-itp/update_make_bq_safe
Browse files Browse the repository at this point in the history
Fixes a few scenarios i've seen when parsing airtable and excel for BQ
  • Loading branch information
vevetron authored Apr 3, 2024
2 parents e02c558 + d4a95f6 commit 75c175a
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 1 deletion.
6 changes: 6 additions & 0 deletions packages/calitp-data-infra/calitp_data_infra/storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,13 @@ def get_fs(gcs_project="", **kwargs):
def make_name_bq_safe(name: str):
"""Replace non-word characters.
See: https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#identifiers.
Add underscore if starts with a number. Also sometimes excel has columns names that are
all numbers, not even strings of numbers (ノ゚0゚)ノ~
"""
if type(name) != str:
name = str(name)
if name[:1].isdigit():
name = "_" + name
return str.lower(re.sub("[^\w]", "_", name)) # noqa: W605


Expand Down
2 changes: 1 addition & 1 deletion packages/calitp-data-infra/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "calitp-data-infra"
version = "2024.2.12rev1"
version = "2024.4.1"
description = "Shared code for developing data pipelines that process Cal-ITP data."
authors = ["Andrew Vaccaro <[email protected]>"]

Expand Down

0 comments on commit 75c175a

Please sign in to comment.