v1.0.3-beta.1

The 1.0.3-beta release has an issue with GCS on JDBC 3.13.18, so we revert #334 and then create a new release on top

What's Changed

• SNOW-663704 Send hex-encoded min/max for BINARY by @sfc-gh-lsembera in #329
• Remove JAXB by @sfc-gh-lsembera in #332
• NO-SNOW: revert https://github.com/snowflakedb/snowflake-kafka-connector/pull/546/files as it's causing failure on GCS by @sfc-gh-tzhang in #334
• V1.0.3-beta.1 release by @sfc-gh-tzhang in #335

Full Changelog: v1.0.3-beta...v1.0.3-beta.1

v1.0.3-beta

This release contains a few bug fixes and improvements for Snowpipe Streaming:

• [Improvement] In order to increase timestamp parsing performance, the number of accepted timestamp formats has been limited. Please see the Javadoc for SnowflakeStreamingIngestChannel#insertRows for more details.
• [Improvement] Add column name to data validation error messages
• [Improvement] Convert empty variant strings into variant column to NULL
• [Improvement] Avoid datetime parsing wherever possible to improve performance
• [Improvement] Allow inf, -inf and nan strings into FLOAT columns
• [Improvement] Add telemetry to track the end2end latency
• [Fixes] Fix small parquet binary max length inconsistency with Arrow
• [Improvement] Add option to use parquet file writer per channel and then merge them into one per chunk
• [Improvement] Improve build latency for high throughput case with many channels
• [Fix] Fix various issues with unicode strings
• [Fix] Ingest SDK Does Not Honor http.nonProxyHosts JVM Argument
• [Improvement] Enable Parquet file by default and enforce single rowcount Parquet files

This release contains a few bug fixes and improvements for Snowpipe:

• [Fix] SimpleIngestIT runtime created database

What's Changed

• SNOW-710291 Allow inf, -inf and nan strings into FLOAT columns by @sfc-gh-lsembera in #306
• SNOW-684267 Add column name to data validation error messages by @sfc-gh-asen in #309
• SNOW-711338 Convert empty variant strings into Variant column to NULL by @sfc-gh-asen in #310
• SNOW-710401 Avoid datetime parsing wherever possible by @sfc-gh-lsembera in #311
• SNOW-701710: [Client side] Add telemetry to track the end2end latency for every BDEC by @sfc-gh-tzhang in #295
• SNOW-690793 use parquet file writer per channel and then merge them i… by @sfc-gh-gdoci in #299
• @no-snow fix small parquet binary max length inconsistency with Arrow by @sfc-gh-gdoci in #319
• SNOW-715618: Improve build latency for high throughput case with many channels by @sfc-gh-tzhang in #321
• SNOW-663621 Fix problems with unicode strings by @sfc-gh-lsembera in #300
• Enable Parquet data type tests by @sfc-gh-lsembera in #313
• [SNOW-726993] Ingest SDK Does Not Honor http.nonProxyHosts JVM Argument by @sfc-gh-wfateem in #325
• SNOW-731571 SimpleIngestIT runtime created database by @sfc-gh-lozhang in #327
• SNOW-620624 enforce single rowcount Parquet files by @sfc-gh-gdoci in #324
• SNOW-733699 make Parquet the default blob format version instead of Arrow by @sfc-gh-gdoci in #322
• SNOW-711299 New date/time processing by @sfc-gh-lsembera in #320
• NO-SNOW: various fixes/improvements in the SDK by @sfc-gh-tzhang in #330
• V1.0.3-beta release by @sfc-gh-tzhang in #326

New Contributors

• @sfc-gh-asen made their first contribution in #309
• @sfc-gh-wfateem made their first contribution in #325
• @sfc-gh-lozhang made their first contribution in #327

Full Changelog: v1.0.2-beta.7...v1.0.3-beta

v1.0.2-beta.7

This release contains a few bug fixes and improvements for Snowpipe Streaming:

• [Bug Fix] Fix incorrect behaviors for table with quoted columns
• [Bug Fix] Fix binary data type for Parquet
• [Bug Fix] Fix incorrect behaviors when sending telemetry data to Snowflake
• [Bug Fix] Fix incorrect behaviors for TIMESTAMP fractional part
• [Improvement] Remove circular dependency on channel object
• [Improvement] Add filename in Parquet MD to support

What's Changed

• SNOW-637237 Test opening several thousand channels in parallel by @sfc-gh-lsembera in #294
• @snow SNOW-657667 Streaming ingest: factor out channel from the buffer by @sfc-gh-azagrebin in #285
• SNOW-696155 fix binary values for Parquet by @sfc-gh-gdoci in #292
• SNOW-647377 Support quoted column names by @sfc-gh-lsembera in #293
• SNOW-703928 add two system properties for IT tests by @sfc-gh-gdoci in #296
• @snow SNOW-705508 Exclude slf4j-reload4j from Hadoop dep by @sfc-gh-azagrebin in #301
• NO-SNOW: fix client telemetry logic by @sfc-gh-tzhang in #297
• SNOW-706682 Log retry-related messages as INFO by @sfc-gh-lsembera in #302
• SNOW-705594: fix ConcurrentModificationException on the telemetry hashmap by @sfc-gh-tzhang in #303
• V1.0.2-beta.7 release by @sfc-gh-tzhang in #305
• SNOW-707520 Fix incorrect fractional part calculation of TIMESTAMP by @sfc-gh-lsembera in #304
• SNOW-704566 Add filename in Parquet extra MD by @sfc-gh-kkloudas in #298

Full Changelog: v1.0.2-beta.6...v1.0.2-beta.7

v1.0.2-beta.6

This release contains a few bug fixes and improvements for Snowpipe Streaming:

• [Improvement] Add parquet file support, this will be our default file format in the future
• [Improvement] Add support to specify a memory limit per client instead of using the system max memory
• [Improvement] Remove a bunch of unused code and setup code coverage

This release contains a few bug fixes and improvements for both Snowpipe and Snowpipe Streaming:

• [Improvement] Improve the HTTP retry logic

What's Changed

• SNOW-470176: [Snyk] Security upgrade com.fasterxml.jackson.core:jackson-databind from 2.13.2.1 to 2.13.4 by @sfc-gh-snyk-sca-sa in #237
• @snow SNOW-633432 Parquet file generator by @sfc-gh-azagrebin in #200
• SNOW-667402:Semgrep workflow update by @sfc-gh-hpathak in #234
• NO-SNOW: add support to reset a rowIndex in the error response whenever needed by @sfc-gh-tzhang in #241
• Revert "@snow SNOW-633432 Parquet file generator (#200)" by @sfc-gh-gdoci in #249
• Introduce parquet file generator by @sfc-gh-gdoci in #251
• Revert "Introduce parquet file generator (#251)" by @sfc-gh-tzhang in #269
• [Snyk] Security upgrade org.apache.arrow:arrow-compression from 9.0.0 to 10.0.0 by @snyk-bot in #271
• SNOW-637237 Thread-safe HTTP client initialisation by @sfc-gh-lsembera in #277
• SNOW-684474: Support adding a memory limit through the config file by @sfc-gh-tzhang in #279
• @snow SNOW-633432 Parquet file generator (fixed dep) by @sfc-gh-azagrebin in #280
• SNOW-637237 Retry for 2 minutes since the first failure by @sfc-gh-lsembera in #281
• Revert "@snow SNOW-612516 Streaming ingest: add random file reader to scan single columns, format version 2 by @sfc-gh-azagrebin in #282
• NO-SNOW: enable code coverage check in the SDK by @sfc-gh-tzhang in #283
• @snow no-snow Streaming ingest: always send BDEC version as server side supports this by @sfc-gh-azagrebin in #284
• @snow no-snow Fix more parquet dependencies causing vulnerabilities by @sfc-gh-azagrebin in #287
• V1.0.2-beta.6 release by @sfc-gh-tzhang in #286

New Contributors

• @sfc-gh-hpathak made their first contribution in #234
• @snyk-bot made their first contribution in #271

Full Changelog: v1.0.2-beta.5...v1.0.2-beta.6

v1.0.2-beta.5

Please Note: This is a release on top of v1.0.2-beta.4
This release contains a few bug fixes and improvements for Snowpipe Streaming

[Improvement] Improve and fix a few data type validation logic
[Improvement] Add throttling logic on direct memory based on uploading queue size
[Improvement] Add basic logic to support Parquet file format, this work is still in progress and we're still using Arrow
[Improvement] Improve exception handling logic

What's Changed

• PRODSEC-1924 rm whitesource legacy files by @sfc-gh-jfan in #217
• @snow SNOW-657206 SteamingIngestIT: add random suffix to DB name to fix concurrent CI runs by @sfc-gh-azagrebin in #222
• @snow SNOW-633432 Refactor RowBuffer and Flusher API by @sfc-gh-azagrebin in #202
• @snow SNOW-633432 Refactor tests to support various BDEC format versions by @sfc-gh-azagrebin in #209
• SNOW-470176: [Snyk] Fix for 2 vulnerabilities by @sfc-gh-snyk-sca-sa in #225
• SNOW-655614 Data type validation: Boolean by @sfc-gh-lsembera in #228
• SNOW-655614 Data type validation: Date and Time by @sfc-gh-lsembera in #229
• SNOW-655614 Data type validation: NUMBER and REAL by @sfc-gh-lsembera in #230
• SNOW-655614 Data type validation: String and Binary by @sfc-gh-lsembera in #231
• SNOW-616916: throttle the insertRows API when the flush task queue size reaches a certain threshold by @sfc-gh-tzhang in #215
• NO-SNOW: add stacktrace during blob building failure by @sfc-gh-tzhang in #232
• SNOW-655614 Data type validation: Semi-structured types by @sfc-gh-lsembera in #233
• V1.0.2-beta.5 release by @sfc-gh-tzhang in #235

New Contributors

• @sfc-gh-snyk-sca-sa made their first contribution in #225
• @sfc-gh-lsembera made their first contribution in #228

Full Changelog: v1.0.2-beta.4...v1.0.2-beta.5

v1.0.2-beta.4

Please Note: This is a release on top of v1.0.2-beta.3
This release contains a few bug fixes and improvements for Snowpipe Streaming

• [Bug Fix] Check max column length for Binary data type
• [Bug Fix] Fix ARRAY data type with String elements
• [Improvement] Support running IT tests as part of merge gate
• [Improvement] Update the InsertError response to include extra or missing column information

What's Changed

• Run streaming ingest integration tests on build by @sfc-gh-mnaides in #207
• SNOW-644959: Return all the extra columns in the insertRow response by @sfc-gh-tzhang in #210
• SNOW-637927: Check max column length for Binary data type by @sfc-gh-tzhang in #206
• SNOW-649753: Update InsertError to include extra/missing column info for KC schema evolution by @sfc-gh-tzhang in #212
• SNOW-644956: fix string data type in ARRAY by @sfc-gh-tzhang in #211
• V1.0.2-beta.4 release by @sfc-gh-tzhang in #213

Full Changelog: v1.0.2-beta.3...v1.0.2-beta.4

v1.0.2-beta.3

Please Note: This is a release on top of v1.0.2-beta.2
This release contains a few bug fixes and improvements for Snowpipe Streaming:

• [Bug Fix] Fix default min/max value in EP for REAL column
• [Bug Fix] Fix register retry logic
• [Bug Fix] Support ARRAY data type
• [Bug Fix] Fix timestamp used when generating the blob
• [Improvement] Support column level compression of the blob (not enabled yet)
• [Improvement] Support client telemetry

This release contains a few bug fixes and improvements for both Snowpipe and Snowpipe Streaming:

• [Improvement] Add HTTP connection pooling manager for better HTTP connection management

What's Changed

• V1.0.2-beta.2 release by @sfc-gh-tzhang in #190
• prodsec-1251 minor update on jira token by @sfc-gh-jfan in #192
• SNOW-619777, SNOW-606454: fix default min/max value in EP for real column by @sfc-gh-tzhang in #194
• NO-SNOW http connection pooling manager by @sfc-gh-japatel in #193
• NO-SNOW check the retry logic with millis instead of ms and seconds by @sfc-gh-japatel in #197
• SNOW-626104: Support ARRAY data type in the SDK by @sfc-gh-tzhang in #196
• NO-SNOW: fix logging format to NOT add a new line by @sfc-gh-tzhang in #198
• @snow SNOW-612516 Streaming ingest: add random file reader to scan single columns, format version 2 by @sfc-gh-azagrebin in #191
• SNOW-356156: support client telemetry through client_telemetry_v by @sfc-gh-tzhang in #195
• @snow SNOW-634840 Disable sending blob_version to server side until supported in prod by @sfc-gh-azagrebin in #201
• SNOW-593873 SNOW-636695 Round timestamp to seconds when creating blob name by @sfc-gh-kkloudas in #203
• V1.0.2-beta.3 release by @sfc-gh-tzhang in #204

New Contributors

• @sfc-gh-kkloudas made their first contribution in #203

Full Changelog: v1.0.2-beta.2...v1.0.2-beta.3

v0.10.8

Fix on top of v0.10.7

• Use this version if you are considering using 0.10.7
• All fixes from 0.10.7 plus a fixes a bug in it
  • Fix IllegalStateException found while testing

v0.10.4-beta.3

Full Changelog: v0.10.4-beta.2...v0.10.4-beta.3

Built on top of 0.10.4-beta.2

• SNOW-585421 Add PoolingHttpClientConnectionManager as the connection manager
• Fix IllegalStateException found while testing

v1.0.2-beta.2

Please Note: This is a release on top of 1.0.2-beta.1

It contains a few bug fixes and improvements for Snowpipe Streaming:

• [Bug Fix] Fix the issue when the account_name has a hyphen
• [Bug Fix] Fix a channel.close timeout issue due to high upload latency
• [Bug Fix] Fix wrong name separator used to build the file path for different platform like Windows
• [Bug Fix] Fix the column stats for collated strings, this will require server side fix to be deployed in prod
• [Bug Fix] Add more debug logging in order to debug a memory exception that comes from the Arrow library
• [Improvement] Improve the README and EXAMPLE for Snowpipe Streaming
• [Improvement] Upgrade Apache Arrow library to the latest version

What's Changed

• NO-SNOW: accept account names with hyphen by @sfc-gh-tzhang in #174
• NO-SNOW: A few small fixes on Snowpipe Streaming PrPr by @sfc-gh-tzhang in #175
• no-snow Add readme and call offsetToken API in Example file by @sfc-gh-japatel in #177
• NO-SNOW: Update Arrow library to 7.0.0 to support VectorSchemaRootAppender by @sfc-gh-tzhang in #180
• SNOW-592855, SNOW-606515: fix channel.close time out issue due to high upload latency by @sfc-gh-tzhang in #182
• NO-SNOW: fix test failure due to time zone difference by @sfc-gh-tzhang in #183
• NO-SNOW: debug memory issue by @sfc-gh-tzhang in #185
• PRODSEC-1251 Whitesource Transition to Snyk by @sfc-gh-jfan in #170
• SNOW-610062: use "/" as name separator instead of using Paths.get by @sfc-gh-tzhang in #187
• SNOW-608483 fix column stats for collated strings by @sfc-gh-mnaides in #186
• NO-SNOW: improve README and Snowpipe Streaming Example by @sfc-gh-tzhang in #189

Full Changelog: v1.0.2-beta.1...v1.0.2-beta.2