{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":356376019,"defaultBranch":"main","name":"public-datasets-pipelines","ownerLogin":"GoogleCloudPlatform","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2021-04-09T19:17:21.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/2810941?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1725402488.0","currentOid":""},"activityList":{"items":[{"before":"4a5c9e21b973b885bf1eaea0d389be49709b3745","after":null,"ref":"refs/heads/dependabot/pip/cryptography-42.0.4","pushedAt":"2024-09-03T22:28:08.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"}},{"before":null,"after":"619365763fe5aeb4b1eef31afd08886bc2010193","ref":"refs/heads/dependabot/pip/cryptography-43.0.1","pushedAt":"2024-09-03T22:28:01.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"Chore(deps-dev): Bump cryptography from 38.0.1 to 43.0.1\n\nBumps [cryptography](https://github.com/pyca/cryptography) from 38.0.1 to 43.0.1.\n- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)\n- [Commits](https://github.com/pyca/cryptography/compare/38.0.1...43.0.1)\n\n---\nupdated-dependencies:\n- dependency-name: cryptography\n dependency-type: indirect\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"Chore(deps-dev): Bump cryptography from 38.0.1 to 43.0.1"}},{"before":"8e71e5d1e437a59ad7eace148928fa50620c6765","after":"57b890b5eb50e67816b044359aa5949024ae9af9","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-30T05:01:45.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"08b1529c984aac94966503f2cbca2834dcf2eae8","after":"a8eb762dd62be5a592e5bc3d985572e2363abc66","ref":"refs/heads/main","pushedAt":"2024-08-30T05:01:08.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"fix: Migration of DAG to v2-2024 environment - relevant changes (#786)","shortMessageHtmlLink":"fix: Migration of DAG to v2-2024 environment - relevant changes (#786)"}},{"before":"5d4c607041a16f2ef8629995ac252602729d89fa","after":"8e71e5d1e437a59ad7eace148928fa50620c6765","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-29T16:35:54.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"2f81461da38f4ecc44bb3ccfb43e30c730738e51","after":"08b1529c984aac94966503f2cbca2834dcf2eae8","ref":"refs/heads/main","pushedAt":"2024-08-29T16:35:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Fix: Resolved issue in GSOD_BY_YEAR where files with 402 errors (#783)\n\n* Fix: Resolved issue in GSOD_BY_YEAR where files with 402 errors were outputting to the source file and breaking the process.\r\n\r\n* fix: Modify sequence of pipelines in DAG.","shortMessageHtmlLink":"Fix: Resolved issue in GSOD_BY_YEAR where files with 402 errors (#783)"}},{"before":"4385b2a0460db79c9b8d7bedfb596bc1b277cf7e","after":"5d4c607041a16f2ef8629995ac252602729d89fa","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-23T14:20:24.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"920332fd864985b723be5daa0fdd460292670742","after":"2f81461da38f4ecc44bb3ccfb43e30c730738e51","ref":"refs/heads/main","pushedAt":"2024-08-23T14:19:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Fix: Rolling back to previous implementation. Modified for v2-2024 environment. (#777)\n\n* fix: Re-engineered.\r\n\r\n* fix: Modified schedule to be daily.\r\n\r\n* fix: Resolve schedule.\r\n\r\n* fix: Rolling back to previous implementation. Modified for v2-2024 environment.\r\n\r\n* fix: Lint issues.\r\n\r\n* fix: Lint issues.","shortMessageHtmlLink":"Fix: Rolling back to previous implementation. Modified for v2-2024 en…"}},{"before":"e580a1a5c999ca4268e5d7253d4e40cd8381c077","after":"4385b2a0460db79c9b8d7bedfb596bc1b277cf7e","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-19T16:57:38.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"5007547d5e469857c681da94a6dc5af288360bc2","after":"920332fd864985b723be5daa0fdd460292670742","ref":"refs/heads/main","pushedAt":"2024-08-19T16:57:07.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Fix: Re-engineer and Migrate Hacker News DAG (#762)\n\n* fix: Re-engineered and migrated hacker news to new 2024 environment.\r\n\r\n* fix: Added download source file.\r\n\r\n* fix: Added annotation to docker and included copy from source to destination.","shortMessageHtmlLink":"Fix: Re-engineer and Migrate Hacker News DAG (#762)"}},{"before":"bbca31e2ba5166535f5ae059047fe4fff0bc3d0a","after":"e580a1a5c999ca4268e5d7253d4e40cd8381c077","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-09T16:27:27.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"f62d4f51647a49c65d3474a211b6d586a8629400","after":"5007547d5e469857c681da94a6dc5af288360bc2","ref":"refs/heads/main","pushedAt":"2024-08-09T16:26:59.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Fix: Iowa liquor sales fix 20230828 (#623)\n\n* Fix: Resolved directory exists error in pipeline.yaml libraries IO dag.\r\n\r\n* Fix: Resolves lint errors in both noaa and thelook_ecommerce dataset deployments.\r\n\r\n* Fix: Resolved BQ job query configuration issue.\r\n\r\n* Fix: Pipeline chronology issue.\r\n\r\n* fix: Not production ready. Issue arrose with data type difference between float and int in specific column in source data.","shortMessageHtmlLink":"Fix: Iowa liquor sales fix 20230828 (#623)"}},{"before":"53ff85d71cbec4e4b13f4e5d0ab4f81aefb34f18","after":"bbca31e2ba5166535f5ae059047fe4fff0bc3d0a","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-09T16:24:31.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"42caa82c611cc33dadb278220cedef4c6d4d0d00","after":"f62d4f51647a49c65d3474a211b6d586a8629400","ref":"refs/heads/main","pushedAt":"2024-08-09T16:23:58.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Feat: Onboard Clemson Dice Traffic Vision Dataset (#475)\n\n* Feat: Onboard US Climate Normals. Not ready for production.\r\n\r\n* fix: further update to pipeline.yaml. Untested.\r\n\r\n* fix: Not Ready For Production.\r\n\r\n* Fix: File copies work. Not production Ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Requires testing. Not production Ready.\r\n\r\n* fix: Further testing. Not production ready.\r\n\r\n* fix: Further testing. Not production ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Testing with hourly.access. Not production ready\r\n\r\n* fix: Not Ready For Production. Extending code to process historical data.\r\n\r\n* fix: Further testing. Not Production Ready.\r\n\r\n* fix: Further testing. Added pipelines for daily/monthly/annualseasonal. Not Production Ready.\r\n\r\n* fix: First Round of AF testing.\r\n\r\n* fix: Not Ready for Production\r\n\r\n* fix: Fixes to pipeline variables for AF testing.\r\n\r\n* fix: fixed file schemas.\r\n\r\n* fix: Newly revised architecture. Builds schema files on-the-fly.\r\n\r\n* fix: Tested dynamic creation of table plus load. Not production Ready.\r\n\r\n* fix: Tested with looping constructs, however, the source files can differ in structure between a prefix-filetype within the same source subfolder, which furthers inconsistency of source file structure; therefore this implementation as-is fails when iterating through some append load process.\r\n\r\n* fix: Use of GCSToBigQueryOperator within python.\r\n\r\n* fix: Identifies exception when schema for file is different than others of same prefix type (and essentially the table itself), although does not yet handle it.\r\n\r\n* fix: Extended use of load to bq by creating an alternative table for source files with alternative schemas\r\n\r\n* fix: Testing with normals-hourly appears to be stable.\r\n\r\n* fix: Appears to work locally. Pre-testing in AF.\r\n\r\n* fix: Resolved dependencies.\r\n\r\n* fix: Issues exist with folder specification. Not production Ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Not Production Ready. Works for one file in each prefix. Needs to be expanded to process all files for that prefix.\r\n\r\n* fix: Removing gsutil etc as it is problemmatic in code. Not Ready For Production.\r\n\r\n* fix: Not Ready For Production.\r\n\r\n* fix: resolving issues with regex and re.match.\r\n\r\n* fix: further changes and testing. Downloads individual files but not production Ready at this point.\r\n\r\n* fix: Not fully Tested. Not production Ready.\r\n\r\n* fix: Further Testing.\r\n\r\n* fix: Further Testing.\r\n\r\n* Further testing.\r\n\r\n* fix: resolve breaking due to >2 schema differences within file prefix group.\r\n\r\n* Further testing.\r\n\r\n* Tested locally.\r\n\r\n* fix: Testing in Dev local, ready for AF.\r\n\r\n* feat: First attempt.\r\n\r\n* fix: resolved load issue.\r\n\r\n* fix: runs in production.\r\n\r\n* fix: removed unwanted checkin to this branch.\r\n\r\n* fix: resolve pre-pend issue.\r\n\r\n* fix: resolve pre-pend issue.\r\n\r\n* fix: resolve pre-pend issue.\r\n\r\n* fix: resolve pre-pend issue. Also, correct reference to folder that files go to.\r\n\r\n* fix: GoogleCloudStorageToGoogleCloudStorage operator fills logs when excessive number of copies are involved. In this case 219k+. Therefore changed out for gsutil in a BashOperator.\r\n\r\n* fix: Resolved transfer_zip_files to CloudDataTransferServiceToGCSOperator\r\n\r\n* fix: Added create directories.\r\n\r\n* fix: Re-engineered to utilize python solution instead of operators.\r\n\r\n* fix: Re-engineered solution to use python.\r\n\r\n* fix: resolved path issues.\r\n\r\n* fix: Extended memory and CPU.\r\n\r\n* fix: Resolved Ephemeral storage.\r\n\r\n* fix: Continue further architecture mods - creation of batch metadata files to contain paths to files to load later.\r\n\r\n* fix: 1st attempt at reading batch data files.\r\n\r\n* fix: Testing.\r\n\r\n* fix: Testing.\r\n\r\n* fix: Testing again.\r\n\r\n* Testing.\r\n\r\n* fix: Added google.api to requirements.txt\r\n\r\n* fix: Added retrying module.\r\n\r\n* fix: attempting to use gsutil in python to transfer from source to public datasets bucket. However, an error occurs because gsutil is not available in cluster that airflow/DAG runs on. GCSToGCS operator copy fills up the logs and for some reason bails out when there are around 15k files transferred. Needed is a solution. This is not production ready.\r\n\r\n* Fix: Lint Issues.\r\n\r\n* Fix: Lint Issues.","shortMessageHtmlLink":"Feat: Onboard Clemson Dice Traffic Vision Dataset (#475)"}},{"before":null,"after":"3d9b2ea8effdae6fc45ee8306920c422d8fa3edc","ref":"refs/heads/dependabot/pip/datasets/us_climate_normals/pipelines/_images/run_csv_transform_kub/apache-airflow-2.9.3","pushedAt":"2024-08-09T16:22:06.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"chore(deps): bump apache-airflow\n\nBumps [apache-airflow](https://github.com/apache/airflow) from 2.2.5 to 2.9.3.\n- [Release notes](https://github.com/apache/airflow/releases)\n- [Changelog](https://github.com/apache/airflow/blob/main/RELEASE_NOTES.rst)\n- [Commits](https://github.com/apache/airflow/compare/2.2.5...2.9.3)\n\n---\nupdated-dependencies:\n- dependency-name: apache-airflow\n dependency-type: direct:production\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"chore(deps): bump apache-airflow"}},{"before":"d1a6f2f18008f5bd656fd6fa94595380beef9d54","after":"53ff85d71cbec4e4b13f4e5d0ab4f81aefb34f18","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-09T16:21:20.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"fae330630babb1af0b1286dad6c172277c5ef1a7","after":"42caa82c611cc33dadb278220cedef4c6d4d0d00","ref":"refs/heads/main","pushedAt":"2024-08-09T16:20:55.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Feat: Onboard US Climate Normals Dataset (#446)\n\n* Feat: Onboard US Climate Normals. Not ready for production.\r\n\r\n* fix: further update to pipeline.yaml. Untested.\r\n\r\n* fix: Not Ready For Production.\r\n\r\n* Fix: File copies work. Not production Ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Requires testing. Not production Ready.\r\n\r\n* fix: Further testing. Not production ready.\r\n\r\n* fix: Further testing. Not production ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Testing with hourly.access. Not production ready\r\n\r\n* fix: Not Ready For Production. Extending code to process historical data.\r\n\r\n* fix: Further testing. Not Production Ready.\r\n\r\n* fix: Further testing. Added pipelines for daily/monthly/annualseasonal. Not Production Ready.\r\n\r\n* fix: First Round of AF testing.\r\n\r\n* fix: Not Ready for Production\r\n\r\n* fix: Fixes to pipeline variables for AF testing.\r\n\r\n* fix: fixed file schemas.\r\n\r\n* fix: Newly revised architecture. Builds schema files on-the-fly.\r\n\r\n* fix: Tested dynamic creation of table plus load. Not production Ready.\r\n\r\n* fix: Tested with looping constructs, however, the source files can differ in structure between a prefix-filetype within the same source subfolder, which furthers inconsistency of source file structure; therefore this implementation as-is fails when iterating through some append load process.\r\n\r\n* fix: Use of GCSToBigQueryOperator within python.\r\n\r\n* fix: Identifies exception when schema for file is different than others of same prefix type (and essentially the table itself), although does not yet handle it.\r\n\r\n* fix: Extended use of load to bq by creating an alternative table for source files with alternative schemas\r\n\r\n* fix: Testing with normals-hourly appears to be stable.\r\n\r\n* fix: Appears to work locally. Pre-testing in AF.\r\n\r\n* fix: Resolved dependencies.\r\n\r\n* fix: Issues exist with folder specification. Not production Ready.\r\n\r\n* fix: Not Production Ready.\r\n\r\n* fix: Not Production Ready. Works for one file in each prefix. Needs to be expanded to process all files for that prefix.\r\n\r\n* fix: Removing gsutil etc as it is problemmatic in code. Not Ready For Production.\r\n\r\n* fix: Not Ready For Production.\r\n\r\n* fix: resolving issues with regex and re.match.\r\n\r\n* fix: further changes and testing. Downloads individual files but not production Ready at this point.\r\n\r\n* fix: Not fully Tested. Not production Ready.\r\n\r\n* fix: Further Testing.\r\n\r\n* fix: Further Testing.\r\n\r\n* Further testing.\r\n\r\n* fix: resolve breaking due to >2 schema differences within file prefix group.\r\n\r\n* Further testing.\r\n\r\n* Tested locally.\r\n\r\n* fix: Testing in Dev local, ready for AF.\r\n\r\n* Fix: Issues with Lint and also regular expression issues.\r\n\r\n* Fix: Issues with Lint.\r\n\r\n* Fix: Issues with Lint.\r\n\r\n* Fix: Resolved dag id name.\r\n\r\n* Fix: Resolved dag id name.\r\n\r\n* Fix: Resolved dag id name.","shortMessageHtmlLink":"Feat: Onboard US Climate Normals Dataset (#446)"}},{"before":"00d89e052704077689ba21d8027f53f56379cc5b","after":"d1a6f2f18008f5bd656fd6fa94595380beef9d54","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-09T16:18:49.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"23228617e3860b9e2dde8b52a76b3183a18a7954","after":"fae330630babb1af0b1286dad6c172277c5ef1a7","ref":"refs/heads/main","pushedAt":"2024-08-09T16:18:21.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Feat: Onboard Visual Questions & Answers Dataset (#405)\n\n* feat: Initial version. Not production ready.\r\n\r\n* fix: A little more code and testing. Not production ready.\r\n\r\n* fix: Now loads annotations. Not production ready.\r\n\r\n* fix: Further work. Not production ready.\r\n\r\n* fix: Loads annotations and questions. Not production ready.\r\n\r\n* fix: Black hook issues. Not production ready.\r\n\r\n* fix: Further work. Not production ready.\r\n\r\n* fix: More code testing - checkin. Not production ready.\r\n\r\n* fix: More code testing - checkin. Not production ready.\r\n\r\n* fix: More code changes. Not production ready.\r\n\r\n* fix: some refactoring. not production ready.\r\n\r\n* fix: Tested questions detail file works as intended.\r\n\r\n* fix: working on testing complementary pairs. does not yet work.\r\n\r\n* fix: Resolved complimentary pairs output structure.\r\n\r\n* fix: Added code to load data for annotations and questions. Not production ready.\r\n\r\n* fix: Added code to load data for annotations. Incomplete product.\r\n\r\n* fix: Fixed upload to gcs function. Tested with annotations and questions.\r\n\r\n* fix: Added and tested complementary pairs load process\r\n\r\n* fix: Code to download and upload to GCS the images. Not production ready.\r\n\r\n* fix: Testing.\r\n\r\n* fix: removed development code unintended.\r\n\r\n* fix: Expanding ephimeral storage in order to resolve limitations\r\n\r\n* fix: Resolve flake and black hooks.\r\n\r\n* fix: Switched single quotes for double in pipeline.yaml","shortMessageHtmlLink":"Feat: Onboard Visual Questions & Answers Dataset (#405)"}},{"before":"e91d0fcc326954b1213987c9a10f56899cfe2c0f","after":"00d89e052704077689ba21d8027f53f56379cc5b","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-09T16:16:26.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"b86a51dd411b452dfd05e3e5e12e147e0d94d3db","after":"23228617e3860b9e2dde8b52a76b3183a18a7954","ref":"refs/heads/main","pushedAt":"2024-08-09T16:15:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"feat: Onboard Census Bureau International Dataset (#425)\n\n* feat: Onboard Census Bureau International Dataset\r\n\r\n* fix: Resolved black hook issues.\r\n\r\n* Fix: Lint issues with a couple of files.","shortMessageHtmlLink":"feat: Onboard Census Bureau International Dataset (#425)"}},{"before":"d312bc89fd2026faf267db2f2f3a099dbbf43fcb","after":"e91d0fcc326954b1213987c9a10f56899cfe2c0f","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-09T16:13:54.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"e8f0240a24ab6e0b8198c08f672f89245c74f6ae","after":"b86a51dd411b452dfd05e3e5e12e147e0d94d3db","ref":"refs/heads/main","pushedAt":"2024-08-09T16:13:21.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Fix: Resolve Terraform Configs For Carbon-Free Energy calculator (#404)\n\n* feat: Onboard cfe_calculator. Not production ready.\r\n\r\n* fix: Next revision, not production ready\r\n\r\n* fix: Not production ready.\r\n\r\n* fix: Resolved dag name.\r\n\r\n* fix: Resolved dag name.\r\n\r\n* fix: Resolve dag id.\r\n\r\n* fix: Resolve github checks on dataset.yaml for this project.\r\n\r\n* fix: Resolve github checks on dataset.yaml for this project.\r\n\r\n* fix: Resolve github checks on dataset.yaml for this project.\r\n\r\n* fix: Resolve github checks on dataset.yaml for this project.\r\n\r\n* fix: Resolve github checks on dataset.yaml for this project.\r\n\r\n* fix: Resolve github checks on dataset.yaml for this project.\r\n\r\n* fix: Resolve github checks on dataset.yaml for this project.\r\n\r\n* Delete provider.tf\r\n\r\n* fix: Terraform mods.","shortMessageHtmlLink":"Fix: Resolve Terraform Configs For Carbon-Free Energy calculator (#404)"}},{"before":"8b058edd4744eb3d622fa9b0735d2d6c381da602","after":"d312bc89fd2026faf267db2f2f3a099dbbf43fcb","ref":"refs/heads/release-please--branches--main","pushedAt":"2024-08-08T14:17:39.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"release-please[bot]","name":null,"path":"/apps/release-please","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/40688?s=80&v=4"},"commit":{"message":"chore(main): release 6.0.0","shortMessageHtmlLink":"chore(main): release 6.0.0"}},{"before":"fbd5ec2eab2aed0d3fbc4fb7ce4964b3f32f96b0","after":"e8f0240a24ab6e0b8198c08f672f89245c74f6ae","ref":"refs/heads/main","pushedAt":"2024-08-08T14:17:09.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"nlarge-google","name":"Nicholas Large","path":"/nlarge-google","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/84149918?s=80&v=4"},"commit":{"message":"Fix: Fixes to dags in new v2-2024 environment (#760)\n\n* fix: yaml issue\r\n\r\n* fix: Flake linter issues\r\n\r\n* fix: Flake black issue\r\n\r\n* fix: Resolutions to configurations and some code in order to move to new v2-2024 environment.\r\n\r\n* Fix: Fixes to census_bureau_acs post push to v2-2024 environment.\r\n\r\n* fix: Invalid reference to variable when timeout occurs during GSOD_BY_YEAR pipeline execution.\r\n\r\n* fix: Miscellaneous fixes.\r\n\r\n* fix: Move GSOD and Storms by year to end of process.\r\n\r\n* Fix: Resolve KubernetesPodOperator issues and reduce code size.\r\n\r\n* Fix: Chicago Taxi Trips now runs E2E successfully.\r\n\r\n* Fix: Lint Issues\r\n\r\n* Fix: Comment out section that breaks due to the fact that structural changes in the 'resouce limit' code section of pipeline.yaml in airflow has changed causing issues with validation of KubernetesPodOperator.\r\n\r\n* Fix: Comment out code that breaks due to the fact that structural changes in the 'resouce limit' code section of pipeline.yaml in airflow has changed causing issues with validation of KubernetesPodOperator.\r\n\r\n* Fix: Comment out code that breaks due to the fact that structural changes in the 'resouce limit' code section of pipeline.yaml in airflow has changed causing issues with validation of KubernetesPodOperator. Resolve import issue.\r\n\r\n* Fix: Comment out code that breaks due to the fact that structural changes in the 'resouce limit' code section of pipeline.yaml in airflow has changed causing issues with validation of KubernetesPodOperator. Resolve import issue.","shortMessageHtmlLink":"Fix: Fixes to dags in new v2-2024 environment (#760)"}},{"before":null,"after":"6d61d8977c2b46896218105174995281c5e65e7b","ref":"refs/heads/dependabot/pip/grpcio-1.53.2","pushedAt":"2024-07-30T11:16:11.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"chore(deps-dev): bump grpcio from 1.50.0 to 1.53.2\n\nBumps [grpcio](https://github.com/grpc/grpc) from 1.50.0 to 1.53.2.\n- [Release notes](https://github.com/grpc/grpc/releases)\n- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)\n- [Commits](https://github.com/grpc/grpc/compare/v1.50.0...v1.53.2)\n\n---\nupdated-dependencies:\n- dependency-name: grpcio\n dependency-type: indirect\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"chore(deps-dev): bump grpcio from 1.50.0 to 1.53.2"}},{"before":null,"after":"00398d978964de5c7bae7250a25a5c18df7cff79","ref":"refs/heads/dependabot/pip/setuptools-70.0.0","pushedAt":"2024-07-15T17:24:17.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"chore(deps-dev): bump setuptools from 65.5.0 to 70.0.0\n\nBumps [setuptools](https://github.com/pypa/setuptools) from 65.5.0 to 70.0.0.\n- [Release notes](https://github.com/pypa/setuptools/releases)\n- [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst)\n- [Commits](https://github.com/pypa/setuptools/compare/v65.5.0...v70.0.0)\n\n---\nupdated-dependencies:\n- dependency-name: setuptools\n dependency-type: indirect\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"chore(deps-dev): bump setuptools from 65.5.0 to 70.0.0"}},{"before":null,"after":"6b8351f4abe52e222b9224af20a84d8a003f6988","ref":"refs/heads/dependabot/pip/zipp-3.19.1","pushedAt":"2024-07-09T18:03:44.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"chore(deps-dev): bump zipp from 3.10.0 to 3.19.1\n\nBumps [zipp](https://github.com/jaraco/zipp) from 3.10.0 to 3.19.1.\n- [Release notes](https://github.com/jaraco/zipp/releases)\n- [Changelog](https://github.com/jaraco/zipp/blob/main/NEWS.rst)\n- [Commits](https://github.com/jaraco/zipp/compare/v3.10.0...v3.19.1)\n\n---\nupdated-dependencies:\n- dependency-name: zipp\n dependency-type: indirect\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"chore(deps-dev): bump zipp from 3.10.0 to 3.19.1"}},{"before":null,"after":"0a3b69b948c616f2daefd8dcf0ceee53645a96d1","ref":"refs/heads/dependabot/pip/certifi-2024.7.4","pushedAt":"2024-07-05T23:11:36.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"chore(deps-dev): bump certifi from 2022.9.24 to 2024.7.4\n\nBumps [certifi](https://github.com/certifi/python-certifi) from 2022.9.24 to 2024.7.4.\n- [Commits](https://github.com/certifi/python-certifi/compare/2022.09.24...2024.07.04)\n\n---\nupdated-dependencies:\n- dependency-name: certifi\n dependency-type: indirect\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"chore(deps-dev): bump certifi from 2022.9.24 to 2024.7.4"}},{"before":"313093caad66faf9e8fb36013d385567c320aa5a","after":null,"ref":"refs/heads/dependabot/pip/urllib3-1.26.18","pushedAt":"2024-06-17T23:50:37.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAErBQzOwA","startCursor":null,"endCursor":null}},"title":"Activity · GoogleCloudPlatform/public-datasets-pipelines"}