Skip to content

Latest commit

 

History

History
644 lines (567 loc) · 86 KB

CHANGELOG.md

File metadata and controls

644 lines (567 loc) · 86 KB
Raw

Changelog

1.0.2-beta (2024-10-24)

Bug Fixes

1.0.1-beta (2024-10-24)

Bug Fixes

  • update build versions for Java and Python (#1005) (33bff2d)

1.0.0-beta (2024-10-24)

⚠ BREAKING CHANGES

  • Use spark bigtable connector instead of hbase connector (#1004)
  • Use spark bigtable connector instead of hbase (#989)
  • GCS to BigTable multiple column family ingestion (#958)

Features

  • 965 upgrade dataproc serverless version to 1.2 python (#995) (817ec52)
  • GCS to BigTable multiple column family ingestion (#958) (d92c02e)
  • Initial commit for Spark runtime update to 1.2 (#963) (c53dc88)
  • Mongo To BigQuery java template (#954) (671aa21)
  • Spanner JDBC driver can use postgresql dialect (#957) (4b022fb)
  • Use spark bigtable connector instead of hbase (#989) (98f0caa)
  • Use spark bigtable connector instead of hbase connector (#1004) (3f13e3c)

Bug Fixes

0.5.0-beta (2024-03-04)

Features

  • Add arguments to run MongToBQ template (f0bfd4c)
  • Add integration test for MongoToBigQuery Python template (906e26f)
  • Add preliminary documentation for MongoToBigQuery template (99146fa)
  • added support for running python templates on an existing dataproc cluster (bed0d62)
  • Built java KafkaToBQDstream template for Dataproc (220cd0e)
  • Built Java KafkaToGCSDstream template for Dataproc (d6c6232)
  • Connection to MongoDB established via the template (8ba9e18)
  • Initialise MongoToBQ Python template test file (1345dfc)
  • MongoToBigQuery template initiated with config to run from main.py (4178ec9)
  • MongoToBQ template writes successfully to destination table (21f44ba)

Bug Fixes

  • Add Usage details of MongoToBQ template to README (3cb21f1)
  • integration test for MongoToBigQuery python (955db75)
  • removed dupliacted code for read and write operations (b3e8f98)
  • removed dupliacted code for read and write operations (9d5c5f4)

Documentation

  • Add recommended jar versions to run MongoToGCS template (#871) (3d1b729)
  • added instructions for CLUSTER mode in python README (ecb2955)
  • added jenkins status for python cluster integration tests (73967d1)
  • added links in README for KafkaToBQDstream template (bfc076c)
  • added links in README for KafkaToGCSDstream template (8def950)
  • adding blog for cassandra to gcs (5ab3342)
  • adding blog for cassandra to gcs (47a478b)
  • adding blogpost for MongodbToGCS (2960f71)
  • adding updates in a few python templates (1597691)
  • clarify serverless and cluster modes, improve template execution instructions (7a8e529)
  • enhanced instructions for better clarity on serverless and cluster modes (5ae1999)
  • hyperlink blog guide for RedshiftToGCS java template (4dcf0a5)
  • updating GCS to Cloud Storage in Java templates (426b772)
  • updating gcs to cloud storage in remaining python templates (ec9925b)

0.4.0-beta (2023-08-07)

Features

  • 756 postgre sql to big query (#805) (121bf93)
  • add bq_dataset_region parameter (a9647ad)
  • Add ORACLE_SCHEMA to parameterize scripts (969e314)
  • added init file (3273fe7)
  • Added Paramterized script file (2951e3a)
  • HiveToBiqquery parameterize script (2eec146)
  • include logging (ca1f9be)
  • include logging (2714b7f)
  • include logging (cf0a81c)
  • service account as env variable (9c93710)
  • service account as env variable (57292f9)
  • update nb constants & script name (cb07530)
  • update nb with IS_PARAMETERIZED flag (b4dcd35)
  • update notebook constants (5c9bdc2)
  • update working dir (ea6a52b)
  • update working dir for parameterization (5840c88)
  • Updated notebook to handle parameters (08dd1c4)
  • updated readme for parametrized script (82ff6ad)
  • updated readme for parametrized script (afac2a6)
  • updated run_notebook (c84883f)

Bug Fixes

  • add log_output (09dc88a)
  • black formatting (fcab888)
  • correct pip install cmd (f9c17c5)
  • correct project id variable (5c61b94)
  • correct sqlachemy execute (14cb3d3)
  • Deal with upgraded google_cloud_pipeline_components.experimental vs v1 (4551fab)
  • Deal with upgraded google_cloud_pipeline_components.experimental vs v1 (2304179)
  • deletes gcs folder created after all tests run (66e3000)
  • except statement (b88758a)
  • Fix merge conflict (deb3a43)
  • Fix merge from main (5a75ae0)
  • Fix merge from main (6ad8fe6)
  • Fix merge from main (2dd4be5)
  • Fix merge from main (fcc8ca6)
  • Fix merge from main (90a65ec)
  • ignored some tests in general template (ae74321)
  • Implement de-duplication (b1c9f8a)
  • implement de-duplication changes (1aeef92)
  • implement de-duplication changes (02c2508)
  • implement get_common_args function (6525dc5)
  • implement get_env_vars in base script (0734fb7)
  • integration test - change custom container image (#810) (8943508)
  • make subnet optional variable (64c0e68)
  • make subnet optional variable (294eea5)
  • modified except block (8871b25)
  • move max_parallelism from common_args (1f88912)
  • Notebook MYSQLTABLE_LIST to MYSQL_TABLE_LIST (5abe159)
  • Notebook MYSQLTABLE_LIST to MYSQL_TABLE_LIST (e22fb72)
  • oracle table list parsing corrected (5f33fb4)
  • oracle table list parsing corrected (7f1e913)
  • raise exception on job failure (e5b1a99)
  • remove common args from script (fc125f2)
  • remove duplicate get_env_var (73bad48)
  • remove log_level from nb_parameters (634d926)
  • remove log_level from nb_parameters (fe49799)
  • remove project id constant (e9173bb)
  • removed duplicate arugment (e0c3262)
  • removed pubsub output project parameter (24a9d18)
  • removed pubsub output project parameter (838c23a)
  • removed unused outputProjectID (5f106c4)
  • removing version parameter while running with cluster (d9faf34)
  • renamed init file (aa60431)
  • Resolve conflicting changes (8ab7e0d)
  • small fixes in notebook (155c5c8)
  • update project id variable (97e58f1)
  • update PROJECT_ID to PROJECT (1a06e52)
  • update scipt_name with oracletopostgres (5dac5d4)
  • update spark to 3.3.2 (60757dd)
  • updated indentation (306e4e5)
  • updated log_output (0b2a510)
  • updated notebook with tag parameters (df007b4)
  • updated notebook_constants (c4aaaa7)
  • updated pom.xml to avoid commons-cli conflict in cluster mode (40cc8f7)
  • updated postgres arg in notebook_constants (5900066)
  • updated sql connection (8c5680d)
  • updated sql query error for boolean value (a71e7f1)
  • updated to UUID GCS staging location (a6fef9e)

Documentation

  • added Oracle to Postgres notebook blogpost link to README files (281bde5)
  • Change Dataflow to Dataproc (#815) (74cee01)
  • Corrected typos in Snowflake README (78e9d42)
  • Corrected typos in Snowflake README (ab60e0e)
  • removed PubSubLiteToGCS from README files (a5d4c06)

0.3.0-beta (2023-05-12)

Features

  • 655 pubsublitetobigtable java template - Fixes for changes requested (aa591d8)
  • 655 pubsublitetobigtable java template - Fixes for changes requested (1000dde)
  • 655 pubsublitetobigtable java template - Fixes for changes requested (656b485)
  • 655 pubsublitetobigtable java template - Fixes for Jenkins file and other changes requested (f187870)
  • add init to util (42e0019)
  • add parameterize_script (2d11bcc)
  • added not_parameterized flag (713be50)
  • added papermill script (987cd21)
  • added template constants (bfca34d)
  • argument parsing commit (7c682a9)
  • correct args (39cda79)
  • created the new notebook Oracle to Postgres along with readme file (#561) (15b7cc6)
  • formatting (d736ae6)
  • moved files (dbfded2)
  • remove try exception (abb7e33)
  • update arg description (41728fe)
  • update arguments and description (01d2b57)
  • update parameter variable (fa5ba31)
  • update requirements (5af1c8d)
  • update requirements.txt (35f5cd3)
  • update requirements.txt and change constants (95e3b14)
  • update write mode (27ca094)
  • updated constants (4e5ae31)
  • updated notebook to be parameterized (c6578ee)
  • updated parameterize_script (aa3bf83)
  • updated script (f46c711)

Bug Fixes

  • correct output mode (cf7c864)
  • Exclude GRPC jar file from Spanner notebooks (60a27c5)
  • execute validateInput from main (f403fad)
  • formatting (5b7cf0e)
  • formatting (c6f86e1)
  • improved usage of validateInput (49b55d6)
  • improved usage of validateInput for CassandraToBQ (77f54f1)
  • improved usage of validateInput for GCSToSpanner (ae24e62)
  • included output mode in function (11e0b97)
  • KAFKA to GCS changes (a50a634)
  • Patch to handle tags in cloudbuild.yaml (#751) (d757278)
  • remove validateInput call from templates (dc8a407)
  • remove validateInput call in psl2bt & fix ut (38f11c8)
  • remove validateOrThrow call from of() (beb3d9d)
  • update KAFKA to GCS (ddc174b)
  • update parameter parsing (cde18e9)
  • update README.md (eae609e)
  • update README.md (c88b041)
  • update README.md (a42a177)
  • update README.md (0349419)
  • update template constants (a66b82e)
  • update unit tests (c280eda)
  • update validateInput() and unit test (20b9991)
  • update working directory based on execution (99b0cc7)
  • updated grpc version, corrected working directory and fixed sqlalchemy breaking changes (3ee71cd)
  • updated unit test cases for CassandraToBQ (c1b2ef7)
  • updated unit test cases for GCSToSpanner (19a052b)
  • updated unit test cases for RedshiftToGCS (038e5e8)
  • updated validateInput method in CassandaToGCS (601910a)
  • updating the link of notebook oracle to postgres in read me files (7a37457)

Documentation

  • added CassandratoGCS blogpost link to README files (766b789)
  • added integration test badges (2fd5108)
  • fix JDBC To Spanner property name (fcde248)

0.2.0-beta (2023-04-11)

Features

  • Add all supported Spark CSV options to BigQuery and Cassandra README files (33539b0)
  • Add all supported Spark CSV options to GCS reader Python templates (5c38e65)
  • Add all supported Spark CSV options to GCS source README file (0001a40)
  • Add all supported Spark CSV options to README files (a689f61)
  • Add all supported Spark CSV write options to GCS reader Python templates (a213aa9)
  • add integration test for JDBCToJDBC (5f16cde)
  • Add JUnit test for DataplexGCStoBQ template (6d19473)
  • Add JUnit test for DataplexGCStoBQ template (52745b8)
  • Added automation bots to execute Jenkins build jobs (#712) (806119f)
  • Cater for legacy CSV options in text_to_bigquery.py (517c163)
  • GCStoMongo : fixed typos and variable values in utc (d65faa0)
  • GCStoMongo new template - updated template name in print statement (1ab03d5)
  • Refine README entries for new CSV options (34ae430)
  • Small refactor to add spark options functions (100b7aa)

Bug Fixes

  • Add validation for additional input fields to the template (dbaa408)
  • Add validation for additional input fields to the template (2685c2c)
  • added processing time as template property (57b4d61)
  • build and integration test breakage (a0781db)
  • container name for GCSTOBIGTABLE (a6f876f)
  • fixed unit tests (5972eda)
  • formatting (65c08e9)
  • Made timeout value configurable via templateProperty (7dafc64)
  • minor fix (cec0ea0)
  • minor fix (0410575)
  • minor fix (ba94365)
  • Modified subscription name (c8dc953)
  • modified timeout property name (87e5aa0)
  • reduce number of imports (72eb0e7)
  • remove extra changes from rebase errors (5108afe)
  • remove extra changes from rebase errors (4adc445)
  • remove sensitive information from logs (4eda3f1)
  • removed unwanted imports (cbbb7f7)
  • revert "hive_ddl_extractor_enhancement" (78632f1)
  • revert "hive_ddl_extractor_enhancement" (4581468)
  • update Dataproc version to 1.1 (2627351)
  • update parameters to dummy values (4e0a3d7)
  • update parameters to dummy values (d803cf1)
  • Update release-please-config.json (#703) (7a344bd)
  • updated README (194cc96)
  • updated tests (a6eeade)
  • updated the ad-hoc folder path in Jenkins integration-test-python job (fa8eec1)
  • Updated the ad-hoc folder path in Jenkins job (#724) (3a14176)
  • Updated the bots to skip release-please branch (#717) (9fbd66c)
  • upgrade dataproc version (c8bf1bf)

Documentation

0.2.0 (2023-03-17)

Features

  • Add csv as a valid input format for GCSTOJDBC Java template (4710ade)
  • Add fetchsize attribute to Oracle based Python notebooks (57e70e2)
  • Add fetchsize attributes to template.properties file (fa09b3e)
  • Add fetchsize attributes to template.properties file (d399307)
  • Add JDBC fetchsize attribute to Java templates (93fafc3)
  • Add JDBC fetchsize attribute to Python templates (0f9114c)
  • Add sessionInitStatement to JDBC read Java templates (b7bb8e1)
  • Add sessionInitStatement to JDBC read Python templates (6198dfa)
  • Add sessionInitStatement to Oracle reading Python notebooks (34702d6)
  • Added a new feature to extract HIVE table DDLs from Hive Metastore (3b10378)
  • Added Java Mongo to GCS new Template (a3c9237)
  • Added KafkaToPubSub template (617e6e9)
  • Added optional arguments to make DDL compatible with Spark Tables and BQ Translation API (c897566)
  • Added spark log level config for HIVE and HBASE templates (640c177)
  • Added spark log level for GCS & S3 templates (8ba0f9c)
  • Added Spark Log Level support for BQ and Cassandra Templates (055ec1a)
  • Added spark log level support for Kafka & PubSub templates (9878dac)
  • Added spark log level support for other templates (51ae1d3)
  • added SPARK_LOG_LEVEL variable (3b0b4ee)
  • Adding spark log level property (d9f204d)
  • Built Python CASSANDRAtoGCS template for Dataproc (97a90fc)
  • Built Python CASSANDRAtoGCS template for Dataproc (552745a)
  • Created Kafka-To-GCS Python Template and added Documentations (de3f374)
  • Fix input data source logic (8f33fdd)
  • Integrated log.level to JDBC templates (ca59e29)
  • New python template for kafka to bq (c6ce462)
  • New template for Kafka To GCS (ec6822c)
  • S3ToBigQuery change read logic and tests (c3c59d8)
  • S3TOBIGQUERY constants (b4bc63b)
  • S3TOBIGQUERY tests (8232217)
  • S3TOBIGQUERY Update README.md (3d521f9)
  • Update README.md (6bfd85c)

Bug Fixes

  • #398 (d122413)
  • #447 (0720102)
  • 447 - Added video links in the readme (ec75083)
  • 448 - Added video links in the readme (3b984f5)
  • 448 - Updated video links in the readme (7f8fd8a)
  • add batchInsertSize to template.properties (267d8bf)
  • add catalogName to templates.properties (bb7593a)
  • add default mode to template.properties (854bfcf)
  • add Spark runtime version to fix #577 (0bd33d6)
  • add Spark runtime version to Java file (83a1057)
  • added default value for kafka.starting.offset (1d0a298)
  • added output mode in template (23fc1b1)
  • Argument name for temp bucket (3b2cefa)
  • avoid unbound input_data (db5d5f6)
  • BIGQUERYTOGCS Configurable write mode (f4408f8)
  • BIGQUERYTOGCS default write mode (9be23eb)
  • BIGQUERYTOGCS Updated README.md (f8f125d)
  • BIGQUERYTOGCS Updated README.md (1740f7d)
  • cassandra_to_bigquery accepts either input query or both keyspace and table (a65ba01)
  • CassandraToBQ accepts either input query or both keyspace and table (ff6b968)
  • CassandraToBQ catalog name to default to casscon (c5d22dd)
  • CassandraToGCS accepts either input query or both keyspace and table (b963901)
  • changed error log (2156a62)
  • Correct ojdbc jar file name (24c27a0)
  • corrected links in readme (3990e76)
  • Corrected Template Constants (26fcf11)
  • corrected validateInput method (e8bb138)
  • Enabled partition level writes in HIVE2GCS template (f7e109e)
  • enabled SPARK_PROPERTIES in start.sh (b0be39d)
  • fixed formatting (7ec60f9)
  • Fixing Kafka2Bq template script (e83388b)
  • formatting (ffb98ef)
  • formatting issues (76debf7)
  • GCSTOBIGQUERY Configurable write mode (613d7e4)
  • GCSTOBIGQUERY Updated README.md (eae7da6)
  • GCSTOBIGTABLE Updated README.md (6ef6802)
  • GCSTOGCS Updated README.md (85672b3)
  • GCSTOJDBC Updated README.md (46048d6)
  • HBASETOGCS Updated README.md (3c208aa)
  • HIVETOBIGQUERY Updated README.md (44e262a)
  • HIVETOGCS Updated README.md (8ea621b)
  • JDBC_URL in Oracle notebooks to use DB service format (ac0b769)
  • JDBCTOGCS add input query as argument (a8d2242)
  • made kafka.message.format a required property (652f7c2)
  • minor changes (e7e0621)
  • minor changes (d9b829b)
  • minor fixes (beb4976)
  • Modification done for requested changes (9c96551)
  • Modified Dependencies in pom.xml (c171712)
  • MongoToGCS java template test (03a2852)
  • MONGOTOGCS Updated README.md (0ec9550)
  • MSSQL to POSTGRES replaced == with = (d7cc235)
  • options method to read (b27c66e)
  • partition write fix in HIVE2GCS template (3adf2b2)
  • read saveModeString (c8ad354)
  • Redact sensitive keys (68b36ed)
  • Removed import error (75a8b54)
  • removed umwanted files (3149c34)
  • Removed unnecessary Properties (1595cf1)
  • removing kafka2gcs separate issue (7f2e22a)
  • revert "hive_ddl_extractor_enhancement" (0d93f47)
  • Revert "Removed import error" (51a9c2c)
  • revert build breaking change (649429e)
  • revert partition level writes (a138158)
  • S3TOBIGQUERY Configurable write mode (631f948)
  • S3TOBIGQUERY minor fixes (0a61f50)
  • simplified check conditions in SnowflakeToGCS (c3ce1a2)
  • snowflake_to_gcs accepts either query or db, schema and table (9b5d502)
  • SnowflakeToGCS accepts either query or db, schema and table (df0ba77)
  • Spotless apply for HiveToGCS (ead0f40)
  • supports parquet format (9d12f60)
  • Template should accept either query or input table (5605b67)
  • TEXTTOBIGQUERY Updated README.md (0e93243)
  • update Dataproc version to 1.1 (3133db1)
  • Update Oracle Client installation to be dynamic, previously hardcoded values were out of date (9b925b8)
  • Update python-publish.yml (#675) (9fe458d)
  • update README.md (4c7775f)
  • update README.md (7837071)
  • Update README.md (2b8b53a)
  • Update README.md (d7f10a3)
  • Update README.md (2cfb64c)
  • Update README.md (b2cb4cd)
  • update unit tests (1744547)
  • updated (a62686c)
  • updated Cassandra templates' logic (ce24124)
  • Updated kokoro release pypi security keystore (#489) (79e0691)
  • Updated pom.xml (8be6939)
  • updated readme (c9e209d)
  • updated README (e2dd8b2)
  • updated README (bdeac39)
  • updated README (dfb25a1)
  • Updated README (1a849b0)
  • Updated README (dde6a80)
  • updated README.md (8525410)
  • updated README.md (c788ca3)
  • updated README.md (d0854fc)
  • Updated required properties for input_keyspace and input_table (6dba293)
  • Updated start.sh for wrong attribute name (ae6bc22)
  • upgrade dataproc version (ae8a648)
  • upgraded dataproc version to 1.1 (f31319c)
  • Use ORACLE_URL so SQLAlchemy can connect to Oracle DB service (70a07fb)
  • WORDCOUNT remove duplicate save command and unused property (1e819a6)
  • WORDCOUNT remove input format property (e7a426f)

Documentation

  • Add description of sessioninitstatement to Java JDBC readme file (1a82c31)
  • Add fetchsize to Java template JDBC readme (cfbef9f)
  • Add Oracle examples (with fetchsize) to JDBC readme (4dab429)
  • Add reWriteBatchedInserts=True to PostgreSQL target examples and recommendation to README files (b24552e)
  • added links in readme for cassandra to gcs (d8db7c7)
  • Added links in README for Mongo To GCS (fb2d3e5)
  • Added Spark Log Level in README.md (3ee051b)
  • blogost for Hbase to GCS (e24a77a)
  • Change example for JDBC_SESSION_INIT_STATEMENT in JDBC readme (8127054)
  • Fix merge conflict (c43e3c4)
  • Fix some typos (1045184)
  • Fix some typos (291cc91)
  • fixed incorrect README links (fe8a2bb)
  • minor fix (17e40b8)
  • Published article for MsSqltoBQ Notebook and added hyperlinks (05e87fd)
  • Release automation (#505) (d12338b)
  • test (8b8e200)
  • update branding name to cloud storage (d8b5206)
  • Updated all README for new template (3a1368e)
  • updated README (c4c4ca8)
  • Updated README (cd1d255)
  • Updating the docs screenshots link (#507) (ee00765)

0.1.0 (2023-03-10)

Features

  • Add csv as a valid input format for GCSTOJDBC Java template (4710ade)
  • Add fetchsize attribute to Oracle based Python notebooks (57e70e2)
  • Add fetchsize attributes to template.properties file (fa09b3e)
  • Add fetchsize attributes to template.properties file (d399307)
  • Add JDBC fetchsize attribute to Java templates (93fafc3)
  • Add JDBC fetchsize attribute to Python templates (0f9114c)
  • Add sessionInitStatement to JDBC read Java templates (b7bb8e1)
  • Add sessionInitStatement to JDBC read Python templates (6198dfa)
  • Add sessionInitStatement to Oracle reading Python notebooks (34702d6)
  • Added a new feature to extract HIVE table DDLs from Hive Metastore (3b10378)
  • Added Java Mongo to GCS new Template (a3c9237)
  • Added KafkaToPubSub template (617e6e9)
  • Added optional arguments to make DDL compatible with Spark Tables and BQ Translation API (c897566)
  • Added spark log level config for HIVE and HBASE templates (640c177)
  • Added spark log level for GCS & S3 templates (8ba0f9c)
  • Added Spark Log Level support for BQ and Cassandra Templates (055ec1a)
  • Added spark log level support for Kafka & PubSub templates (9878dac)
  • Added spark log level support for other templates (51ae1d3)
  • added SPARK_LOG_LEVEL variable (3b0b4ee)
  • Adding spark log level property (d9f204d)
  • Built Python CASSANDRAtoGCS template for Dataproc (97a90fc)
  • Built Python CASSANDRAtoGCS template for Dataproc (552745a)
  • Created Kafka-To-GCS Python Template and added Documentations (de3f374)
  • Fix input data source logic (8f33fdd)
  • Integrated log.level to JDBC templates (ca59e29)
  • New python template for kafka to bq (c6ce462)
  • New template for Kafka To GCS (ec6822c)
  • S3ToBigQuery change read logic and tests (c3c59d8)
  • S3TOBIGQUERY constants (b4bc63b)
  • S3TOBIGQUERY tests (8232217)
  • S3TOBIGQUERY Update README.md (3d521f9)
  • Update README.md (6bfd85c)

Bug Fixes

  • #398 (d122413)
  • add batchInsertSize to template.properties (267d8bf)
  • add catalogName to templates.properties (bb7593a)
  • add default mode to template.properties (854bfcf)
  • add Spark runtime version to fix #577 (0bd33d6)
  • add Spark runtime version to Java file (83a1057)
  • added default value for kafka.starting.offset (1d0a298)
  • added output mode in template (23fc1b1)
  • Argument name for temp bucket (3b2cefa)
  • avoid unbound input_data (db5d5f6)
  • BIGQUERYTOGCS Configurable write mode (f4408f8)
  • BIGQUERYTOGCS default write mode (9be23eb)
  • BIGQUERYTOGCS Updated README.md (f8f125d)
  • BIGQUERYTOGCS Updated README.md (1740f7d)
  • cassandra_to_bigquery accepts either input query or both keyspace and table (a65ba01)
  • CassandraToBQ accepts either input query or both keyspace and table (ff6b968)
  • CassandraToBQ catalog name to default to casscon (c5d22dd)
  • CassandraToGCS accepts either input query or both keyspace and table (b963901)
  • changed error log (2156a62)
  • Correct ojdbc jar file name (24c27a0)
  • corrected links in readme (3990e76)
  • Corrected Template Constants (26fcf11)
  • corrected validateInput method (e8bb138)
  • Enabled partition level writes in HIVE2GCS template (f7e109e)
  • Fixing Kafka2Bq template script (e83388b)
  • formatting (ffb98ef)
  • formatting issues (76debf7)
  • GCSTOBIGQUERY Configurable write mode (613d7e4)
  • GCSTOBIGQUERY Updated README.md (eae7da6)
  • GCSTOBIGTABLE Updated README.md (6ef6802)
  • GCSTOGCS Updated README.md (85672b3)
  • GCSTOJDBC Updated README.md (46048d6)
  • HBASETOGCS Updated README.md (3c208aa)
  • HIVETOBIGQUERY Updated README.md (44e262a)
  • HIVETOGCS Updated README.md (8ea621b)
  • JDBC_URL in Oracle notebooks to use DB service format (ac0b769)
  • JDBCTOGCS add input query as argument (a8d2242)
  • made kafka.message.format a required property (652f7c2)
  • minor changes (e7e0621)
  • minor changes (d9b829b)
  • minor fixes (beb4976)
  • Modification done for requested changes (9c96551)
  • Modified Dependencies in pom.xml (c171712)
  • MONGOTOGCS Updated README.md (0ec9550)
  • MSSQL to POSTGRES replaced == with = (d7cc235)
  • options method to read (b27c66e)
  • partition write fix in HIVE2GCS template (3adf2b2)
  • read saveModeString (c8ad354)
  • Redact sensitive keys (68b36ed)
  • Removed import error (75a8b54)
  • removed umwanted files (3149c34)
  • Removed unnecessary Properties (1595cf1)
  • removing kafka2gcs separate issue (7f2e22a)
  • Revert "Removed import error" (51a9c2c)
  • S3TOBIGQUERY Configurable write mode (631f948)
  • S3TOBIGQUERY minor fixes (0a61f50)
  • simplified check conditions in SnowflakeToGCS (c3ce1a2)
  • snowflake_to_gcs accepts either query or db, schema and table (9b5d502)
  • SnowflakeToGCS accepts either query or db, schema and table (df0ba77)
  • Spotless apply for HiveToGCS (ead0f40)
  • supports parquet format (9d12f60)
  • Template should accept either query or input table (5605b67)
  • TEXTTOBIGQUERY Updated README.md (0e93243)
  • update Dataproc version to 1.1 (3133db1)
  • Update Oracle Client installation to be dynamic, previously hardcoded values were out of date (9b925b8)
  • update README.md (4c7775f)
  • update README.md (7837071)
  • Update README.md (2b8b53a)
  • Update README.md (d7f10a3)
  • Update README.md (2cfb64c)
  • Update README.md (b2cb4cd)
  • update unit tests (1744547)
  • updated Cassandra templates' logic (ce24124)
  • Updated kokoro release pypi security keystore (#489) (79e0691)
  • Updated pom.xml (8be6939)
  • updated readme (c9e209d)
  • updated README (e2dd8b2)
  • updated README (bdeac39)
  • updated README (dfb25a1)
  • Updated README (1a849b0)
  • Updated README (dde6a80)
  • updated README.md (8525410)
  • updated README.md (c788ca3)
  • updated README.md (d0854fc)
  • Updated required properties for input_keyspace and input_table (6dba293)
  • Updated start.sh for wrong attribute name (ae6bc22)
  • upgrade dataproc version (ae8a648)
  • upgraded dataproc version to 1.1 (f31319c)
  • Use ORACLE_URL so SQLAlchemy can connect to Oracle DB service (70a07fb)
  • WORDCOUNT remove duplicate save command and unused property (1e819a6)
  • WORDCOUNT remove input format property (e7a426f)

Documentation

  • Add description of sessioninitstatement to Java JDBC readme file (1a82c31)
  • Add fetchsize to Java template JDBC readme (cfbef9f)
  • Add Oracle examples (with fetchsize) to JDBC readme (4dab429)
  • Add reWriteBatchedInserts=True to PostgreSQL target examples and recommendation to README files (b24552e)
  • added links in readme for cassandra to gcs (d8db7c7)
  • Added links in README for Mongo To GCS (fb2d3e5)
  • Added Spark Log Level in README.md (3ee051b)
  • Change example for JDBC_SESSION_INIT_STATEMENT in JDBC readme (8127054)
  • Fix merge conflict (c43e3c4)
  • Fix some typos (1045184)
  • Fix some typos (291cc91)
  • Release automation (#505) (d12338b)
  • update branding name to cloud storage (d8b5206)
  • Updated all README for new template (3a1368e)
  • updated README (c4c4ca8)
  • Updated README (cd1d255)
  • Updating the docs screenshots link (#507) (ee00765)