CHANGELOG.md

Changelog

48.0.0 (2023-10-18)

Full Changelog

Breaking changes:

• Evaluate null_regex for string type in csv (now such values will be parsed as Null rather than "") #4942 [arrow] (haohuaijin)
• fix(csv)!: infer null for empty column. #4910 [arrow] (kskalski)
• feat: log headers/trailers in flight CLI (+ minor fixes) #4898 [arrow] [arrow-flight] (crepererum)
• fix(arrow-json)!: include null fields in schema inference with a type of Null #4894 [arrow] (kskalski)
• Mark OnCloseRowGroup Send #4893 [parquet] (devinjdangelo)
• Specialize Thrift Decoding (~40% Faster) (#4891) #4892 [parquet] (tustvold)
• Make ArrowRowGroupWriter Public and SerializedRowGroupWriter Send #4850 [parquet] (devinjdangelo)

Implemented enhancements:

• Allow schema fields to merge with Null datatype #4901 [arrow]
• Add option to FlightDataEncoder to always send dictionaries #4895 [arrow] [arrow-flight]
• Rework Thrift Encoding / Decoding of Parquet Metadata #4891 [parquet]
• Plans for supporting Extension Array to support Fixed shape tensor Array #4890
• Implement Take for UnionArray #4882 [arrow]
• Check precision overflow for casting floating to decimal #4865 [arrow]
• Replace lexical #4774 [arrow]
• Add read access to settings in csv::WriterBuilder #4735 [arrow]
• Improve the performance of "DictionaryValue" row encoding #4712 [arrow] [arrow-flight]

Fixed bugs:

• Should we make blank values and empty string to None in csv? #4939 [arrow]
• [FlightSQL] SubstraitPlan structure is not exported #4932 [arrow] [arrow-flight]
• Loading page index breaks skipping of pages with nested types #4921 [parquet]
• CSV schema inference assumes Utf8 for empty columns #4903 [arrow]
• parquet: Field Ids are not read from a Parquet file without serialized arrow schema #4877 [parquet]
• make_primitive_scalar function loses DataType Internal information #4851 [arrow]
• StructBuilder doesn't handle nulls correctly for empty structs #4842 [arrow]
• NullArray::is_null() returns false incorrectly #4835 [arrow]
• cast_string_to_decimal should check precision overflow #4829 [arrow]
• Null fields are omitted by infer_json_schema_from_seekable #4814 [arrow]

Closed issues:

• Support for reading JSON Array to Arrow #4905 [arrow]

Merged pull requests:

• Assume Pages Delimit Records When Offset Index Loaded (#4921) #4943 [parquet] (tustvold)
• Update pyo3 requirement from 0.19 to 0.20 #4941 [arrow] (crepererum)
• Add FileWriter schema getter #4940 [arrow] (haixuanTao)
• feat: support parsing for parquet writer option #4938 [parquet] (fansehep)
• Export SubstraitPlan structure in arrow_flight::sql (#4932) #4933 [arrow] [arrow-flight] (amartins23)
• Update zstd requirement from 0.12.0 to 0.13.0 #4923 [parquet] [arrow] (dependabot[bot])
• feat: add method for async read bloom filter #4917 [parquet] (hengfeiyang)
• Minor: Clarify rationale for FlightDataEncoder API, add examples #4916 [arrow] [arrow-flight] (alamb)
• Update regex-syntax requirement from 0.7.1 to 0.8.0 #4914 [arrow] (dependabot[bot])
• feat: document & streamline flight SQL CLI #4912 [arrow] [arrow-flight] (crepererum)
• Support Arbitrary JSON values in JSON Reader (#4905) #4911 [arrow] (tustvold)
• Cleanup CSV WriterBuilder, Default to AutoSI Second Precision (#4735) #4909 [arrow] (tustvold)
• Update proc-macro2 requirement from =1.0.68 to =1.0.69 #4907 [arrow] [arrow-flight] (dependabot[bot])
• chore: add csv example #4904 [arrow] (fansehep)
• feat(schema): allow null fields to be merged with other datatypes #4902 [arrow] (kskalski)
• Update proc-macro2 requirement from =1.0.67 to =1.0.68 #4900 [arrow] [arrow-flight] (dependabot[bot])
• Add option to FlightDataEncoder to always resend batch dictionaries #4896 [arrow] [arrow-flight] (alexwilcoxson-rel)
• Fix integration tests #4889 (tustvold)
• Support Parsing Avro File Headers #4888 (tustvold)
• Support parquet bloom filter length #4885 [parquet] (letian-jiang)
• Replace lz4 with lz4_flex Allowing Compilation for WASM #4884 [parquet] [arrow] (tustvold)
• Implement Take for UnionArray #4883 [arrow] (avantgardnerio)
• Update tonic-build requirement from =0.10.1 to =0.10.2 #4881 [arrow] [arrow-flight] (dependabot[bot])
• parquet: Read field IDs from Parquet Schema #4878 [parquet] (Samrose-Ahmed)
• feat: improve flight CLI error handling #4873 [arrow] [arrow-flight] (crepererum)
• Support Encoding Parquet Columns in Parallel #4871 [parquet] (tustvold)
• Check precision overflow for casting floating to decimal #4866 [arrow] (viirya)
• Make align_buffers as public API #4863 [arrow] (viirya)
• Enable new integration tests (#4828) #4862 (tustvold)
• Faster Serde Integration (~80% faster) #4861 [arrow] (tustvold)
• fix: make_primitive_scalar bug #4852 [arrow] (JasonLi-cn)
• Update tonic-build requirement from =0.10.0 to =0.10.1 #4846 [arrow] [arrow-flight] (dependabot[bot])
• Allow Constructing Non-Empty StructArray with no Fields (#4842) #4845 [arrow] (tustvold)
• Refine documentation to Array::is_null #4838 [arrow] (alamb)
• fix: add missing precision overflow checking for cast_string_to_decimal #4830 [arrow] (jonahgao)

* This Changelog was automatically generated by github_changelog_generator