From 67262b8df747a21ad356997d9a4af1a7cf96b65d Mon Sep 17 00:00:00 2001 From: kindly Date: Sat, 10 Feb 2024 00:49:16 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20@=20kindly/f?= =?UTF-8?q?latterer@272c982ec49cbf3f41a41681ea0badd6992ed0fa=20?= =?UTF-8?q?=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- _sources/changelog.md.txt | 5 + changelog.html | 545 +++++++++++++++++++------------------- objects.inv | Bin 2868 -> 2894 bytes searchindex.js | 2 +- 4 files changed, 285 insertions(+), 267 deletions(-) diff --git a/_sources/changelog.md.txt b/_sources/changelog.md.txt index 71bd17e..3a76c04 100644 --- a/_sources/changelog.md.txt +++ b/_sources/changelog.md.txt @@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file. and this project adheres to [Semantic Versioning](http://semver.org/). +## [0.19.14] - 2024-02-10 + +### Fixed +- Large xlsx cell values being truncated panic when multi threading. + ## [0.19.13] - 2024-01-26 ### Fixed diff --git a/changelog.html b/changelog.html index 1456e55..57dfab7 100644 --- a/changelog.html +++ b/changelog.html @@ -202,83 +202,92 @@

Change LogSemantic Versioning.

-

[0.19.13] - 2024-01-26#

+

[0.19.14] - 2024-02-10#

Fixed#

    -

  • Large xlsx cell values being truncated causing panic if in unicode char.

    • +

  • Large xlsx cell values being truncated panic when multi threading.

-

[0.19.12] - 2023-12-03#

-
-

New#

+

[0.19.13] - 2024-01-26#

+
+

Fixed#

    -

  • Upgrade deps, low_memory option for API

    • +

  • Large xlsx cell values being truncated causing panic if in unicode char.

-
-

[0.19.10] - 2023-10-02#

-

New#

+

[0.19.12] - 2023-12-03#

+
+

New#

    -

  • Upgrade deps, better build times due to latest duckdb

    • +

  • Upgrade deps, low_memory option for API

-

[0.19.8] - 2023-06-21#

+

[0.19.10] - 2023-10-02#

New#

    -

  • arrays_as_table option added to convert all arrays to their own table.

    • +

  • Upgrade deps, better build times due to latest duckdb

-

[0.19.6] - 2023-06-21#

+

[0.19.8] - 2023-06-21#

-

Fixed#

+

New#

    -

  • Errors get raised for postgresql conversion.

    • +

  • arrays_as_table option added to convert all arrays to their own table.

-

[0.19.5] - 2023-06-07#

+

[0.19.6] - 2023-06-21#

Fixed#

    -

  • Parquet naming of headers where incorrect for dates.

    • +

  • Errors get raised for postgresql conversion.

-

[0.19.4] - 2023-05-26#

+

[0.19.5] - 2023-06-07#

Fixed#

    -

  • Allow multiple files while downloading from s3

    • -

  • Stop detecting floats where precision is too low.

    • +

  • Parquet naming of headers where incorrect for dates.

-

[0.19.3] - 2023-05-11#

+

[0.19.4] - 2023-05-26#

Fixed#

    -

  • CSV output to S3 broken in some cases.

    • -

  • Stop csv directory being made when using S3

    • +

  • Allow multiple files while downloading from s3

    • +

  • Stop detecting floats where precision is too low.

-

[0.19.1] - 2023-05-10#

+

[0.19.3] - 2023-05-11#

-

New#

+

Fixed#

+
    +

  • CSV output to S3 broken in some cases.

    • +

  • Stop csv directory being made when using S3

    • +
+
+
+
+

[0.19.1] - 2023-05-10#

+
+

New#

  • JSON Input sources from STDIN, HTTP, S3 and allow all inputs to be GZIPed if have .gz ending.

  • Command line now accepts multiple files from any source.

    • @@ -288,37 +297,37 @@

    New#

-
-

[0.18.0] - 2023-03-17#

-
-

New#

+
+

[0.18.0] - 2023-03-17#

+
+

New#

  • Better type guessing for database inserts.

  • no_link option that removes _link fields in the output.

-
-

[0.17.1] - 2023-01-07#

-
-

Fixed#

+
+

[0.17.1] - 2023-01-07#

+
+

Fixed#

  • Truncate cell that is larger than xlsx allows.

  • Allow more rows in xlsx in non threaded mode.

-
-

[0.17.0] - 2022-12-03#

-
-

New#

+
+

[0.17.0] - 2022-12-03#

+
+

New#

  • Web Assembly version of libflatterer. Available to use here https://lite.flatterer.dev/.

  • Upgrade to vue 3 and vite for web frontend.

-
-

Fixed#

+
+

Fixed#

  • Ignore blank lines in json lines files

  • Better errors when too many files are open

    • @@ -326,47 +335,47 @@

    Fixed#<

-
-

[0.16.2] - 2022-10-25#

-
-

New#

-
    -

  • Support python 3.11

    • -
-
-
-

[0.16.2] - 2022-10-16#

+

[0.16.2] - 2022-10-25#

-

Fixed#

+

New#

    -

  • Error not writing larger XLSX files

    • +

  • Support python 3.11

-

[0.16.1] - 2022-10-16#

+

[0.16.2] - 2022-10-16#

Fixed#

    -

  • Cors for web api

    • +

  • Error not writing larger XLSX files

-

[0.16.0] - 2022-07-27#

+

[0.16.1] - 2022-10-16#

-

New#

+

Fixed#

    -

  • Local web interface for exploring flatterer features flatterer --web.

    • +

  • Cors for web api

-

[0.15.0] - 2022-07-27#

+

[0.16.0] - 2022-07-27#

New#

    +

  • Local web interface for exploring flatterer features flatterer --web.

    • +
+
+
+
+

[0.15.0] - 2022-07-27#

+
+

New#

+

  • evolve option for sqlite and postgres. Can add data to existing tables and will alter tables if new fields are needed.

  • drop option for sqlite.

@@ -379,28 +388,28 @@

Changed -

[0.14.2] - 2022-07-27#

-
-

New#

+
+

[0.14.2] - 2022-07-27#

+
+

New#

  • Postgres connection from environment variable

-
-

[0.14.1] - 2022-07-23#

-
-

Fixed#

+
+

[0.14.1] - 2022-07-23#

+
+

Fixed#

  • sql_script option to export scripts that for sqlite and postgres to make output backward compatable with earlier versions.

-
-

[0.14.0] - 2022-07-22#

-
-

New#

+
+

[0.14.0] - 2022-07-22#

+
+

New#

  • pushdown option. Copy data from top level objects down to child (one-to-many) tables. This is useful if the data has its own keys (such as id fields) that you want to exist in the related tables. Also useful for denormalizing the data so querying on a common field, requires less joining.

  • postgres option. Export to postgres database by supplying a connection string.

    • @@ -415,193 +424,193 @@

    Removed -

    [0.13.2] - 2022-06-24#

    -
    -

    New#

    +
    +

    [0.13.2] - 2022-06-24#

    +
    +

    New#

    • files option, so multiple files can be supplied at once.

    -
    -

    [0.13.1] - 2022-05-27#

    +
    +

    [0.13.1] - 2022-05-27#

    • Threads option now can output xlsx

    -
    -

    [0.13.0] - 2022-04-28#

    -
    -

    New#

    +
    +

    [0.13.0] - 2022-04-28#

    +
    +

    New#

    • Threads option, so that can be run on all cores. Works best with ndjson input.

    • Parquet export option.

    -
    -

    Changed#

    -
      -

    • BREAKING: json-lines option renamed to ndjson

      • -

    • New json-stream option that works in same way as the old json-lines option and accepts concatonated json.

      • -
    -
    -
    -
    -

    [0.12.12] - 2022-04-15#

    Changed#

      -

    • Datapackage output uses correct date type

      • -

    • Lists of strings are now escaped the same way as optional quoted CSVs

      • +

    • BREAKING: json-lines option renamed to ndjson

      • +

    • New json-stream option that works in same way as the old json-lines option and accepts concatonated json.

    -

    [0.12.11] - 2022-04-09#

    +

    [0.12.12] - 2022-04-15#

    Changed#

      -

    • Clearer errors when error happens in rust. BREAKING CHANGE, if catching certain error types in python these may have changed.

      • -

    • datapackage output now has forign keys.

      • +

    • Datapackage output uses correct date type

      • +

    • Lists of strings are now escaped the same way as optional quoted CSVs

    -

    [0.12.10] - 2022-03-21#

    +

    [0.12.11] - 2022-04-09#

    -

    Fixed#

    +

    Changed#

      -

    • Python decimal converted to float not string.

      • +

    • Clearer errors when error happens in rust. BREAKING CHANGE, if catching certain error types in python these may have changed.

      • +

    • datapackage output now has forign keys.

    -

    [0.12.9] - 2022-03-07#

    +

    [0.12.10] - 2022-03-21#

    Fixed#

      -

    • SQLite export lower memory use

      • +

    • Python decimal converted to float not string.

    -

    [0.12.8] - 2022-03-01#

    +

    [0.12.9] - 2022-03-07#

    -

    Changed#

    +

    Fixed#

      -

    • SQLite export has indexes and foreign key contraints.

      • +

    • SQLite export lower memory use

    -

    [0.12.7] - 2022-02-27#

    +

    [0.12.8] - 2022-03-01#

    -

    Fixed#

    +

    Changed#

      -

    • main_table_name was number caused exception

      • +

    • SQLite export has indexes and foreign key contraints.

    -

    [0.12.6] - 2022-02-26#

    +

    [0.12.7] - 2022-02-27#

    Fixed#

      -

    • list of JSON strings supplied to flatten fixed.

      • -

    • datapackage.json named correctly.

      • +

    • main_table_name was number caused exception

    -

    [0.12.4] - 2022-02-18#

    +

    [0.12.6] - 2022-02-26#

    -

    Changed#

    +

    Fixed#

      -

    • flatten python function now accepts iterator

      • -

    • Docs for flatten

      • -

    • Tests for flatten

      • -

    • iterator_flatten now deprecated as it is just a subset of `flatten’

      • +

    • list of JSON strings supplied to flatten fixed.

      • +

    • datapackage.json named correctly.

    -

    [0.12.3] - 2022-02-15#

    +

    [0.12.4] - 2022-02-18#

    -

    Fixed#

    +

    Changed#

      -

    • More lenient if tmp directory can not be deleted.

      • +

    • flatten python function now accepts iterator

      • +

    • Docs for flatten

      • +

    • Tests for flatten

      • +

    • iterator_flatten now deprecated as it is just a subset of `flatten’

    -

    [0.12.1] - 2022-02-15#

    +

    [0.12.3] - 2022-02-15#

    Fixed#

      -

    • Preview option in python CLI and library.

      • +

    • More lenient if tmp directory can not be deleted.

    -

    [0.12] - 2022-02-02#

    +

    [0.12.1] - 2022-02-15#

    -

    New#

    +

    Fixed#

      -

    • SQLite export option

      • +

    • Preview option in python CLI and library.

    -

    [0.11] - 2022-01-25#

    +

    [0.12] - 2022-02-02#

    New#

      +

    • SQLite export option

      • +
    +
    +
    +
    +

    [0.11] - 2022-01-25#

    +
    +

    New#

    +

    • Support top level object. All list of objects are streamed and top level object data saved in main table.

    • New yaglish parser for both json stream and arrays.

    • Library has schema_guess function to tell if data is a JSON Stream or has an array of object.

    -
    -

    Fixed#

    +
    +

    Fixed#

    • Empty objects do not make a line in output.

    -
    -

    [0.10.1] - 2021-12-22#

    -
    -

    New#

    +
    +

    [0.10.1] - 2021-12-22#

    +
    +

    New#

    • ctrlc support added.

    -
    -

    Changed#

    +
    +

    Changed#

    • Logging output improved.

    • Traceback not shown for CLI use.

    -
    -

    Fixed#

    +
    +

    Fixed#

    • Occurences where output folder not being deleted.

    -
    -

    [0.10] - 2021-12-22#

    -
    -

    New#

    +
    +

    [0.10] - 2021-12-22#

    +
    +

    New#

    • tables.csv input in order to control tab names. Tables File option

    • Beginning to use logging.

    -
    -

    Changed#

    +
    +

    Changed#

    • Better handling of long excel sheet names names. See https://github.com/kindly/flatterer/issues/12

    • field_type no longer required in fields.csv.

      • @@ -610,8 +619,8 @@

      Changed

    • More human readable error messages.

    -
    -

    Fixed#

    +
    +

    Fixed#

    • Bad characters in XLSX stripped and raise warning.

    • Check limits on XLSX files and raise error if found.

      • @@ -620,51 +629,51 @@

      Fixed#<

    -
    -

    [0.9] - 2021-12-12#

    -
    -

    Fixed#

    +
    +

    [0.9] - 2021-12-12#

    +
    +

    Fixed#

    • Removed unwrap on channel send, to remove possible panic.

    -
    -

    Changed#

    +
    +

    Changed#

    • Table ordering of output in JSON input order. Making xlsx and fields.csv table order reflect the input data.

    • Lib has new FlatFiles::new_with_dafualts() to make using the library less verbose.

    • Use insta for more tests.

    -
    -

    New#

    +
    +

    New#

    • Lib has preview option, meaning CSV output will optionally only show specified number of lines.

    -
    -

    [0.8] - 2021-12-01#

    -
    -

    Changed#

    +
    +

    [0.8] - 2021-12-01#

    +
    +

    Changed#

    • Paths to data in sqlite and postgres start at root of output.

    • Clippy for linting and insta for tests.

    -
    -

    [0.7.1] - 2021-12-01#

    -
    -

    Changed#

    +
    +

    [0.7.1] - 2021-12-01#

    +
    +

    Changed#

    • Do less work when just exporting Metadata.

    • Minor speedup due to not using format so much.

    -
    -

    [0.7] - 2021-11-24#

    +
    +

    [0.7] - 2021-11-24#

    Added#

      @@ -674,10 +683,10 @@

      Added#

    -
    -

    [0.6.2] - 2021-11-24#

    -
    -

    Fixed#

    +
    +

    [0.6.2] - 2021-11-24#

    +
    +

    Fixed#

    • Change to pypi metadata

    • Tests run in action

      • @@ -685,28 +694,28 @@

      Fixed#<

    -
    -

    [0.6.1] - 2021-11-24#

    -
    -

    Fixed#

    +
    +

    [0.6.1] - 2021-11-24#

    +
    +

    Fixed#

    • Regression in speed due to new error handling

    -
    -

    [0.6] - 2021-11-23#

    -
    -

    Changed#

    +
    +

    [0.6] - 2021-11-23#

    +
    +

    Changed#

    • New error handling using anyhow, giving errors more context.

    -
    -

    [0.5] - 2021-11-22#

    -
    -

    Added#

    +
    +

    [0.5] - 2021-11-22#

    +
    +

    Added#

    -
    -

    [0.4] - 2021-11-20#

    -
    -

    Added#

    +
    +

    [0.4] - 2021-11-20#

    +
    +

    Added#

    • Postgresql and sqlite scripts to load CSV data into databases.

    • Wheel builds for Windows and MacOS, automatically published using github actions.

    -
    -

    [0.3] - 2021-10-25#

    -
    -

    Added#

    +
    +

    [0.3] - 2021-10-25#

    +
    +

    Added#

    @@ -791,202 +800,206 @@

    Added#<
    • Change Log
        -
      • [0.19.13] - 2024-01-26
          +
        • [0.19.14] - 2024-02-10
          • -
        • [0.19.12] - 2023-12-03
            -
          • New
            • +
          • [0.19.13] - 2024-01-26
            • -
          • [0.19.10] - 2023-10-02
              -
            • New
              • +
            • [0.19.12] - 2023-12-03
              • -
            • [0.19.8] - 2023-06-21
                +
              • [0.19.10] - 2023-10-02
                • -
              • [0.19.6] - 2023-06-21
                  -
                • Fixed
                  • +
                • [0.19.8] - 2023-06-21
                  • -
                • [0.19.5] - 2023-06-07
                    +
                  • [0.19.6] - 2023-06-21
                    • -
                  • [0.19.4] - 2023-05-26
                      +
                    • [0.19.5] - 2023-06-07
                      • -
                    • [0.19.3] - 2023-05-11
                        +
                      • [0.19.4] - 2023-05-26
                        • -
                      • [0.19.1] - 2023-05-10
                          -
                        • New
                          • +
                        • [0.19.3] - 2023-05-11
                          • -
                        • [0.18.0] - 2023-03-17
                            +
                          • [0.19.1] - 2023-05-10
                            • -
                          • [0.17.1] - 2023-01-07
                              -
                            • Fixed
                              • +
                            • [0.18.0] - 2023-03-17
                              • -
                            • [0.17.0] - 2022-12-03
                                -
                              • New
                                • -
                              • Fixed
                                • +
                              • [0.17.1] - 2023-01-07
                                • -
                              • [0.16.2] - 2022-10-25
                                  -
                                • New
                                  • +
                                • [0.17.0] - 2022-12-03
                                  • -
                                • [0.16.2] - 2022-10-16
                                    -
                                  • Fixed
                                    • +
                                  • [0.16.2] - 2022-10-25
                                    • -
                                  • [0.16.1] - 2022-10-16
                                      +
                                    • [0.16.2] - 2022-10-16
                                      • -
                                    • [0.16.0] - 2022-07-27
                                        -
                                      • New
                                        • +
                                      • [0.16.1] - 2022-10-16
                                        • -
                                      • [0.15.0] - 2022-07-27
                                          +
                                        • [0.16.0] - 2022-07-27
                                          • -
                                        • [0.14.2] - 2022-07-27
                                            +
                                          • [0.15.0] - 2022-07-27
                                            • -
                                          • [0.14.1] - 2022-07-23
                                              -
                                            • Fixed
                                              • +
                                            • [0.14.2] - 2022-07-27
                                              • -
                                            • [0.14.0] - 2022-07-22
                                                -
                                              • New
                                                • -
                                              • Removed
                                                • +
                                              • [0.14.1] - 2022-07-23
                                                • -
                                              • [0.13.2] - 2022-06-24
                                                  +
                                                • [0.14.0] - 2022-07-22
                                                  • -
                                                • [0.13.1] - 2022-05-27
                                                  • -
                                                • [0.13.0] - 2022-04-28
                                                    -
                                                  • New
                                                    • -
                                                  • Changed
                                                    • +
                                                  • [0.13.2] - 2022-06-24
                                                    • -
                                                  • [0.12.12] - 2022-04-15
                                                      +
                                                    • [0.13.1] - 2022-05-27
                                                      • +
                                                    • [0.13.0] - 2022-04-28
                                                      • -
                                                    • [0.12.11] - 2022-04-09
                                                        +
                                                      • [0.12.12] - 2022-04-15
                                                        • -
                                                      • [0.12.10] - 2022-03-21
                                                          -
                                                        • Fixed
                                                          • +
                                                        • [0.12.11] - 2022-04-09
                                                          • -
                                                        • [0.12.9] - 2022-03-07
                                                            +
                                                          • [0.12.10] - 2022-03-21
                                                            • -
                                                          • [0.12.8] - 2022-03-01
                                                              -
                                                            • Changed
                                                              • +
                                                            • [0.12.9] - 2022-03-07
                                                              • -
                                                            • [0.12.7] - 2022-02-27
                                                                -
                                                              • Fixed
                                                                • +
                                                              • [0.12.8] - 2022-03-01
                                                                • -
                                                              • [0.12.6] - 2022-02-26
                                                                  +
                                                                • [0.12.7] - 2022-02-27
                                                                  • -
                                                                • [0.12.4] - 2022-02-18
                                                                    -
                                                                  • Changed
                                                                    • +
                                                                  • [0.12.6] - 2022-02-26
                                                                    • -
                                                                  • [0.12.3] - 2022-02-15
                                                                      -
                                                                    • Fixed
                                                                      • +
                                                                    • [0.12.4] - 2022-02-18
                                                                      • -
                                                                    • [0.12.1] - 2022-02-15
                                                                        +
                                                                      • [0.12.3] - 2022-02-15
                                                                        • -
                                                                      • [0.12] - 2022-02-02
                                                                          -
                                                                        • New
                                                                          • +
                                                                        • [0.12.1] - 2022-02-15
                                                                          • -
                                                                        • [0.11] - 2022-01-25
                                                                            +
                                                                          • [0.12] - 2022-02-02
                                                                            • -
                                                                          • [0.10.1] - 2021-12-22
                                                                              -
                                                                            • New
                                                                              • -
                                                                            • Changed
                                                                              • -
                                                                            • Fixed
                                                                              • +
                                                                            • [0.11] - 2022-01-25
                                                                              • -
                                                                            • [0.10] - 2021-12-22
                                                                                -
                                                                              • New
                                                                                • -
                                                                              • Changed
                                                                                • -
                                                                              • Fixed
                                                                                • +
                                                                              • [0.10.1] - 2021-12-22
                                                                                • -
                                                                              • [0.9] - 2021-12-12
                                                                                  +
                                                                                • [0.10] - 2021-12-22
                                                                                  • -
                                                                                • [0.8] - 2021-12-01
                                                                                    -
                                                                                  • Changed
                                                                                    • +
                                                                                  • [0.9] - 2021-12-12
                                                                                    • -
                                                                                  • [0.7.1] - 2021-12-01
                                                                                      +
                                                                                    • [0.8] - 2021-12-01
                                                                                      • -
                                                                                    • [0.7] - 2021-11-24
                                                                                        -
                                                                                      • Added
                                                                                        • +
                                                                                      • [0.7.1] - 2021-12-01
                                                                                        • -
                                                                                      • [0.6.2] - 2021-11-24
                                                                                          -
                                                                                        • Fixed
                                                                                          • +
                                                                                        • [0.7] - 2021-11-24
                                                                                          • -
                                                                                        • [0.6.1] - 2021-11-24
                                                                                            +
                                                                                          • [0.6.2] - 2021-11-24
                                                                                            • -
                                                                                          • [0.6] - 2021-11-23
                                                                                              -
                                                                                            • Changed
                                                                                              • +
                                                                                            • [0.6.1] - 2021-11-24
                                                                                              • -
                                                                                            • [0.5] - 2021-11-22
                                                                                                -
                                                                                              • Added
                                                                                                • +
                                                                                              • [0.6] - 2021-11-23
                                                                                                • -
                                                                                              • [0.4] - 2021-11-20 diff --git a/objects.inv b/objects.inv index 9d19a9de5d7c76ee6bc41e9c55e05550a31067cb..dac3c606c4eb6ea61f4add6c0b86a5458dc7173d 100644 GIT binary patch delta 2679 zcmV--3W)Wz7S0xs@_$+XS+AV+e=XbUl}1SCO-xTRR}M~9P}gR7H=v+v0o6#QG(@@= zB>zVs!&o5Q9?AcWkPcuv)H1_RGFJ|yI^%`vZDQk|?K%WDa5Z+_bGhD_+b`F**9pa4 zYc5zX*Bf(Pz$MM#$C~Rx?t~q@H(A$LSdXx-52-Ol)yDM=YJWVUhJfmZ6jhrxgj6>i zQ6or=s14JGjSOluqQ=0z)}(uBuZ^L1t;gP#Kq|Rby<3Sv9j(<{1J`n?-SPuL^;T=> zS?*}Jygm>D+OEE@s2~o43A=iI7=*U#Q&d+`K@@PcA=U3K5YgwWFFnhL7KkR!SKonD zkDz+7qB;iE8-GzYmMpXALR5w;s_YG9?A?t=y)bV3By4cm>F-=irJ`e%B3bhec*_EF>^y$K$nUbpQ zs?>(mczvOIwnI^!L{Q_)T9w+6n)GW8n2ji^%(4==R)4NGrd}4yuE+!?(A0jhLT!w7 zFW3e9VkUC+4;6e@QNN}_Xf}R>qL_!h*Dw%R73K~M_IN?mpUmreWOzGM%CceW zcO-NAKy2=XiELyGFqvXZE`_$B9})W0xfI6)fZW3X2attFg6mUz892B;75xEWLGty1 zhwD?1?})G;^`MRj`%$HDOst=J+Y^V=`j5p65dc!+3;#QzO1i#>s8U|lsSvCZ*=UmYCQA{Xg?Mv@0 zn$lZ-CzBfpX42!=YP*19*VNg0*MAj=2@IiY(C9h!xjUUZ`>`nVc9vrZ-H8T!AhSl# zX;RKjnTh8hy~Mq~0_x|g;)bMcKbE|gJ-yW#maVWg-Hi8WPvq<0*5_s??m&#NHSGT? zPQz(0T5|N)e_wBa#-Zy@?Ys&fwp;7BhlkC(_C3%OS9U8`Ft2Zl%`IC-H-F!oc@mJ3 z;m>yuFplT4D|oN~%T0oYf?riz2?1p5%~ROe)`PNLR#b?%7~b-~S7q_NNt;)=x2j8S z?&h^SFlWD!eOg>p_f5|Hj(N^0t+KSWv$RVeoAfx`0f@ttKB^&04q2br&=)ZTXO*FY z%%B8hR6)&?xDy-R@HAl{IDc<}bDQpS1SH45@xNEcI-5AsPp`Hfkmml%P87EL@^G(D zJg$wnQ(aQ{PBhqq`H@9wwac}Fc#GLI>1%5<6L;zNp({hmNb-YdTWkj&cpE}@WnoX; zw1MkoKHglbo1a1qs|EwYKc7+RYg|7e0r&!3dIc^H@JN7%0|FA@<9~pN1OzyMHe!eZ zJlcp64xo(~;{e)-2@W8Qh`RxK11|2!9=QiD?#Mp52QGdGK79+gBm3kYxVR(xxpnKRd`0wy zV0&a`%tcx@t>4&)WxeBOzhk}cEyypbvZ!(<&P-ABA8hwVve%-AVVUV_{sYmXY&)gh zf!n1U799q8Bq}Pp4eQ``WE~@J+pz@kNFkM!pl!u91eNu(eSgf4b@OVcC(^)E(l_<9 z^;Ps)?^vhBJKw`8j%iUzAG0f#ydwCauCj5- zZp=X_-saCNv1NPtNe@fQy}SLye~lZfYEJrLOV7vVGzD*KKE)3eC6FjQmc&6R?#*)` zGX9~I0F7f`UVqw+&ZsfkP!!9?b-`aZ$FxJ&XbS3g1F{K;B6QmPls0KsV}Ujh6cM)7 z%@L7Bx$qBB6i`%CSdhC~I*=$L40_#m`zCMgc6Z2+DTW?MZdBXainal5@~2|>)xv}+ ziH-aM~RXb8ek|CxF%+Ui93V#vRlDiOsr$NDWVQrxKC6ywsfEHQ1Fw57(5h$g*%0tW}#(p!z{E5ZWd?Z{SZatLdPRY z$Ax}J6n~HljZ2h~^VTI(3MDX3Amo7E6&;>BnsSn3J&2Zp`&qKAr72Emjt8Z-KTC!a z*9NG=(?mJG?FLDqEGAga^H{tf+J=}cI5wg&Vv%Jql#=Q6s0{87tO`T(D=qOLf3IW` zY`*uFUFyBJY-l<0GNOGNOaK(b=*R6pp#uN#aDPg7KbR{`lu%F^NXl=Ypku{5AXa9G z_-fRv11=;lSL_l(;Qdb{w8VrZSZ4q8oOg(_me_o_|NYzN2SZus+K9Fm+1za(e!c&^ zoznkVPJIEL;SuP`UVVg@204sJe!R|#x?zqx++Jk}&_YKUzmOxSWYhAl9p!d;9A6sK=(k?|B27|u{0lI015N0d5+ zxRAUYM$}8$z99Ny_>Y_IA6I%0|D1fLO!6?r;uy)fKyR{Qny?WM2GUj96^MGDlWwrs20~ zjg;S|rOdf0r-k^yxBD9{CQ^Z_?q+TkjEg+Lx@DJFZJsu}!$3pyDKpCP4Nn}N^Y75o ltSM#w#rm(SE^i*wosi`wWGIkskL3SGNCz+-a+$tlt{g~p#tYTk#Kt|_bqH***4TB=<$7aozg*v5Cj{4z zH5aUx>y5cC;F4zWL(O#|cft|daE_` zEO)e9ULObnZCBq_R1gQjgk8Np3_{!WDe78LK@@PcA=U3K5YgwWCq2uD7KkR!SKonD zkD`W(>KIgSM1OT5m9nenE2?WyM|KU?Z=%ZG3I-M!QNy=U zq{izD)w3Ov8VPEAS*ubTQj>nI0kaWBm04B-*UHt#)PKuj*%g_{eDzyJcNa7x7K7Z|`{EQJtA1yUj+DKvn?C<+}&q!NlmJfcEaSvli; zLLd6em~kwbF3ODFgcMF0;XqhZk{N0&JTfv%%29$EzX>T$ju-_d0O*i)uZZsi0-b^X zq*<0{C`L+r;D0AnZR)0yU|yH>|7c#9boFFPSARBa{U&2h9f-}n){u>C0VY$7$)(T+ z^aDblI+o&)0FYbg;Q(qI{Rr2mw$e{vpXW}M?Iw@!hTeN8WZcM z-pR!NwEko9LIi-6_`?5AsFJQ9BdV0w2M(f4W*Z)&Ou2gyAn~{ zdgS}l87HLz-}9#R$e^ECXOkc6Cn8PKGWYWLD$AeP;OUrG9sJ-5y5%R=Pkjf};nRLU z7iHF(vkvvO(BH++kT6`tJO)Z4ODgMKTG~a`c4=7>NK^KPw=C5fWB2=_+LJ&jb$=s& zul9LWR9XIFs_9;7-fMn&)r+ZZ01{uywCnOFZ&udV^atnvV|#aNJ?d_$%vYJeq{ma4 z+y86p%AB*unL?jsLlAq;1#KV2ghJN7^vj7!*uYc@BVY@F6_xi-++K4;VC57)qgFToZS(H}0Tq}t8kWG`m zwl*_ymwq3*GNgJF-vi0loloetddzc07v64a8%}A@>gtA&1;PJc1n3 zdUyaiGv>>h`XnDYEus&lKCx^0P--D4f2zx;oJN^)pG9i|_gLht;G9SR4H%UABCcf) z2thT=EHQ)_|HA)5wSPlio@|?2x30=pL|+KDM^?sMq-E3kjg45=J8t$n*8ARq{Guw0 zDre%%6gB_Bc5ftmEqWN1nXcwP5G~5KQ`#N4UAke>VUS0nqN3Zd4sJ)*G2*rzOAwC~ zQb`HgR$N0+SwGvy{8%@yc6uTWJSBZoKU-f#pY@J)TDJf*(qL zWo_w?)-7v8D1W~rb|TBm{8)5(CQIZh8<*_H9E9R+{>&0vwwIsuu(aH}+fV%0xUs6{ zq#w5Qd~8lr@V4et{7_K>iNa$^9F*eTJO?7ad0TL{(!;_X!ULKUs*u zLm^nWQ@CjsS_U`FLc8E*aTeYWQ8X@eJfd`5=yyZ`xzM;o898rVGNn)g;{-wu*j>@# zsiP?;Ie*rJXc@SlCCgfx;)LdSP-^?LWH@ncfI2))l;hiOkQB;dg5^Aq#S5Zsh{=Lu zBN`(XSq4KXnNE+&;O@YxFf_l?5)bnCN+!YPdvDpL-h0c2mJ=@{+NZ$;KtYUt-2M|P z@DC5CboYa~(nJXbm4T%E_6a&xyaQrohKR35y?;93Lh^FOE+GWo|1?5NOjv?t_CL>g zhbU`_&4>HnzkPl%ly$C+Xls$p-S*+v`_J1c{h#I37tl%0MRarNP~So)xqp^5bCI9r zGBOvox%2?AViz-f8S0B?P$$D+4%s`Lg*91;jc}E~_V=6XhtFe8(x!&kR>FiWS7X=$ zLw_pV#fU(0`qma1Pf>y44CNtNo-lYssZ)px$;)9xy_D?>qA!O3xY_=3rT6g9$!E%x z59l)0z5*Zfh_8I=L1ujHQ{Q*hdDnVUa_g}H@Lx1cu#&nAs{fUtRGNIx-%zAgW~b-w zkXK!?OUM7eK||)n@5PA4Ml5q=rDhs_n^x9H`CVGdoSSl5h!1?bztLhM6{zZN=2pSD z$OEifc6rt2X|p>FG(?{=qa5Gx#Nj#r4lT`^Qs!T*|GMh(<}uv~DZU)`V;8#9 (CLI Only) ": [[5, "help-small-cli-only-small"]], "Output Formats": [[5, "output-formats"]], "CLI Usage": [[5, "cli-usage"], [5, "id1"], [5, "id3"], [5, "id5"], [5, "id7"], [5, "id9"], [5, "id11"], [5, "id13"], [5, "id15"], [5, "id17"], [5, "id19"], [5, "id21"], [5, "id23"], [5, "id25"], [5, "id27"], [5, "id29"], [5, "id31"], [5, "id33"], [5, "id35"], [5, "id37"], [5, "id39"], [5, "id41"], [5, "id43"], [5, "id45"], [5, "id47"]], "Python Usage": [[5, "python-usage"], [5, "id2"], [5, "id4"], [5, "id6"], [5, "id8"], [5, "id10"], [5, "id12"], [5, "id14"], [5, "id16"], [5, "id18"], [5, "id20"], [5, "id22"], [5, "id24"], [5, "id26"], [5, "id28"], [5, "id30"], [5, "id32"], [5, "id34"], [5, "id36"], [5, "id38"], [5, "id40"], [5, "id42"], [5, "id44"], [5, "id46"], [5, "id48"], [5, "id49"], [5, "id50"]], "Main Table Name": [[5, "main-table-name"]], "Pushdown Fields": [[5, "pushdown-fields"]], "No Link Fields": [[5, "no-link-fields"]], "Path to JSON Array": [[5, "path-to-json-array"]], "New Line Delemited JSON (NDJSON)": [[5, "new-line-delemited-json-ndjson"]], "JSON Stream": [[5, "json-stream"]], "JSON Path Filter": [[5, "json-path-filter"]], "Force": [[5, "force"]], "Postgres Schema": [[5, "postgres-schema"]], "Evolve Tables": [[5, "evolve-tables"]], "Drop Tables": [[5, "drop-tables"]], "Fields File": [[5, "fields-file"]], "Only Fields": [[5, "only-fields"]], "Tables File": [[5, "tables-file"]], "Only Tables": [[5, "only-tables"]], "Inline One To One": [[5, "inline-one-to-one"]], "Arrays as Table": [[5, "arrays-as-table"]], "Schema": [[5, "schema"]], "Table Prefix": [[5, "table-prefix"]], "Path Separator": [[5, "path-separator"]], "Schema Titles": [[5, "schema-titles"]], "Stats": [[5, "stats"]], "Preview": [[5, "preview"]], "Threads": [[5, "threads"]], "Sql Scripts": [[5, "sql-scripts"]], "Low Memory (api only)": [[5, "low-memory-api-only"]], "Outputs": [[6, "outputs"]], "CSV": [[6, "csv"]], "XLSX": [[6, "xlsx"]], "SQLITE": [[6, "sqlite"]], "POSTGRES": [[6, "postgres"]], "PARQUET": [[6, "parquet"]], "fields.csv": [[6, "fields-csv"]], "tables.csv": [[6, "tables-csv"]], "Postgresql Files": [[6, "postgresql-files"]], "postgresql_schema.sql": [[6, "postgresql-schema-sql"]], "postgresql_load.sql": [[6, "postgresql-load-sql"]], "Sqlite Files.": [[6, "sqlite-files"]], "sqlite_schema.sql": [[6, "sqlite-schema-sql"]], "sqlite_load.sql": [[6, "sqlite-load-sql"]], "datapackage.json": [[6, "datapackage-json"]], "URL": [[7, "url"]], "Region and authentication.": [[7, "region-and-authentication"]], "Example": [[7, "example"]], "Usage Note": [[7, "usage-note"]], "Web Interface": [[8, "web-interface"]]}, "indexentries": {}}) \ No newline at end of file +Search.setIndex({"docnames": ["changelog", "development", "index", "inputs", "library", "options", "outputs", "s3", "web"], "filenames": ["changelog.md", "development.md", "index.md", "inputs.md", "library.md", "options.md", "outputs.md", "s3.md", "web.md"], "titles": ["Change Log", "Development", "Flatterer. Making JSON flatterer", "Input Sources", "Python Library", "Option Reference", "Outputs", "S3", "Web Interface"], "terms": {"all": [0, 1, 3, 4, 5, 6, 8], "notabl": 0, "thi": [0, 1, 2, 3, 4, 5, 6, 7], "project": 0, "document": 0, "file": [0, 4, 7, 8], "adher": 0, "semant": 0, "version": [0, 5, 7], "larg": [0, 4, 5], "xlsx": [0, 2, 4, 5], "cell": 0, "valu": [0, 5, 8], "being": [0, 5], "truncat": [0, 6], "panic": 0, "when": [0, 2, 5, 6, 7, 8], "multi": 0, "thread": 0, "caus": [0, 4, 5], "unicod": 0, "char": 0, "upgrad": 0, "dep": 0, "low_memori": [0, 5], "option": [0, 2, 4, 6, 7, 8], "api": 0, "better": 0, "build": 0, "time": [0, 2, 5, 6], "due": 0, "latest": [0, 7], "duckdb": 0, "arrays_as_t": [0, 5], "convert": [0, 2], "arrai": [0, 2, 4], "own": [0, 2, 5, 6], "tabl": [0, 4], "error": [0, 5], "get": [0, 1, 2, 3, 5, 8], "rais": [0, 5], "postgresql": 0, "convers": [0, 4], "parquet": [0, 2, 5, 7], "name": [0, 2, 4, 6], "header": [0, 5, 6], "where": [0, 2, 4, 5, 6], "incorrect": 0, "date": [0, 2, 4, 5, 6], "allow": [0, 5, 6], "multipl": [0, 5], "while": 0, "download": [0, 5], "from": [0, 2, 3, 4, 5, 6, 7], "s3": [0, 2], "stop": [0, 5], "detect": 0, "float": 0, "precis": 0, "i": [0, 2, 3, 4, 5, 6, 7], "too": [0, 2], "low": 0, "csv": [0, 4, 5, 7], "output": [0, 1, 3, 7], "broken": 0, "some": [0, 2, 5, 6], "case": [0, 4, 5], "directori": [0, 3, 4, 5, 6], "made": [0, 6], "us": [0, 2, 3, 4, 5, 6, 7], "json": [0, 3, 4, 7], "input": [0, 2, 4, 5, 6, 7], "sourc": [0, 1, 2, 4], "stdin": [0, 7], "http": [0, 5, 7], "gzipe": [0, 3], "have": [0, 2, 5, 6], "gz": [0, 3], "end": [0, 2, 3, 5], "command": [0, 2, 3, 4, 5, 6, 8], "line": [0, 2, 3, 4, 6, 8], "now": [0, 4], "accept": [0, 7, 8], "ani": [0, 5, 8], "stream": [0, 2, 4, 7], "path": [0, 3, 4, 6, 7], "queri": [0, 5], "filter": 0, "object": [0, 2, 4, 5], "stat": 0, "produc": [0, 5, 6], "statist": [0, 5], "ouput": 0, "type": [0, 2, 5, 6], "guess": [0, 2, 6], "databas": [0, 2, 4, 5, 6], "insert": [0, 2, 5], "no_link": [0, 5], "_link": [0, 2, 4, 5, 6], "field": [0, 4, 7], "larger": 0, "than": [0, 2, 5], "more": [0, 2, 4, 5], "row": [0, 5], "non": [0, 4, 5], "mode": [0, 5, 6], "web": [0, 3, 5], "assembli": 0, "libflatter": 0, "avail": 0, "here": 0, "lite": 0, "flatter": [0, 1, 3, 4, 5, 6, 7, 8], "dev": 0, "vue": 0, "vite": 0, "frontend": 0, "ignor": 0, "blank": 0, "mani": [0, 5, 6], "ar": [0, 2, 3, 4, 5, 6, 7], "open": [0, 4, 8], "could": [0, 2, 4, 5], "support": [0, 2, 7], "python": [0, 1, 3], "write": 0, "cor": 0, "local": [0, 5, 7], "interfac": 0, "explor": [0, 2, 8], "featur": [0, 2, 8], "evolv": 0, "sqlite": [0, 2, 4, 5], "postgr": [0, 2], "can": [0, 2, 3, 4, 5, 6, 7, 8], "add": [0, 4, 5], "data": [0, 2, 4, 5, 6, 7, 8], "exist": [0, 5], "alter": [0, 5], "need": [0, 1, 2, 4, 5, 6, 7, 8], "drop": 0, "No": 0, "Will": [0, 5], "temp": 0, "space": [0, 4, 5], "suppli": [0, 4, 5], "switch": 0, "connect": [0, 3, 5, 7], "environ": [0, 1, 5, 7, 8], "variabl": [0, 1, 5, 7, 8], "sql_script": [0, 5, 6], "export": [0, 5, 7], "script": [0, 6], "make": [0, 4, 5], "backward": 0, "compat": 0, "earlier": 0, "pushdown": 0, "copi": [0, 5, 6], "top": [0, 5], "level": [0, 5], "down": [0, 5], "child": [0, 5], "one": [0, 2, 5, 6], "ha": [0, 2, 5], "its": [0, 2, 5, 6], "kei": [0, 2, 4, 5], "id": [0, 2, 4, 5, 6], "you": [0, 2, 3, 4, 5], "want": [0, 2, 4, 5], "relat": [0, 2, 5, 6], "also": [0, 2, 3, 4, 5, 7], "denorm": 0, "so": [0, 2, 4, 5, 8], "common": [0, 5], "requir": [0, 2, 5, 6, 7], "less": [0, 5], "join": [0, 2, 5], "string": [0, 4, 5], "schema": [0, 6, 7], "choos": [0, 5], "If": [0, 2, 3, 4, 5], "alreadi": [0, 5], "longer": 0, "actual": 0, "creat": [0, 1, 5, 6], "onc": [0, 1, 2, 7], "run": [0, 1, 4, 5, 6, 8], "core": [0, 5], "work": [0, 2, 3, 4, 5], "best": [0, 2, 5], "ndjson": [0, 3], "break": 0, "renam": [0, 2], "same": [0, 2, 3, 4, 5, 6], "wai": [0, 5, 7, 8], "old": [0, 5], "concaton": [0, 5], "datapackag": [0, 2, 5], "correct": [0, 5], "list": [0, 2, 4, 5], "escap": [0, 5], "quot": 0, "clearer": 0, "happen": 0, "rust": [0, 1, 2], "catch": 0, "certain": [0, 5], "mai": [0, 2, 5], "forign": 0, "decim": 0, "lower": [0, 5], "memori": [0, 2, 4], "index": 0, "foreign": [0, 2], "contraint": 0, "main_table_nam": [0, 5], "wa": [0, 2, 5], "number": [0, 2, 4, 5, 6], "except": [0, 2], "flatten": [0, 2, 4, 5, 7, 8], "correctli": 0, "function": [0, 7], "iter": [0, 4], "doc": [0, 7], "test": [0, 1], "iterator_flatten": 0, "deprec": 0, "just": [0, 5], "subset": 0, "lenient": 0, "tmp": 0, "delet": [0, 5], "preview": 0, "cli": 0, "librari": [0, 3], "save": [0, 5], "main": [0, 4], "yaglish": 0, "parser": [0, 2], "both": [0, 5], "schema_guess": 0, "tell": 0, "an": [0, 2, 4, 5], "empti": [0, 5], "do": [0, 2, 4, 5], "ctrlc": 0, "improv": [0, 5], "traceback": 0, "shown": 0, "occur": 0, "folder": [0, 5], "order": [0, 2, 5], "control": 0, "tab": [0, 5], "begin": 0, "handl": 0, "long": [0, 2, 3], "excel": [0, 5], "sheet": [0, 2, 5, 6], "see": [0, 2, 3, 4, 5], "github": [0, 1], "com": [0, 3, 5, 7], "kindli": 0, "issu": [0, 4, 5], "field_typ": [0, 2, 4, 5, 6], "redund": 0, "_link_": 0, "main_tabl": 0, "human": 0, "readabl": 0, "messag": [0, 5], "bad": 0, "charact": [0, 5], "strip": 0, "warn": [0, 4, 5], "check": [0, 5], "limit": [0, 6], "found": [0, 6], "lot": [0, 2, 5], "edg": 0, "instead": [0, 5], "leav": 0, "unus": 0, "partial": 0, "unwrap": 0, "channel": 0, "send": 0, "possibl": [0, 4], "reflect": 0, "lib": 0, "flatfil": 0, "new_with_dafualt": 0, "verbos": 0, "insta": 0, "mean": [0, 2, 5, 7], "onli": [0, 2, 4, 7], "show": [0, 4, 5], "specifi": [0, 2, 5, 7], "start": [0, 5], "root": [0, 5], "clippi": 0, "lint": 0, "metadata": [0, 2, 4, 6], "minor": 0, "speedup": 0, "format": [0, 2, 3, 7], "much": [0, 2], "abil": 0, "field_titl": [0, 2, 4, 5], "column": [0, 2, 5], "head": [0, 2, 5, 6], "titl": [0, 2, 4, 6], "out": [0, 2, 5], "jsonschema": [0, 5], "speed": [0, 5], "smartstr": 0, "smallvec": 0, "pypi": 0, "action": 0, "changelog": 0, "regress": 0, "anyhow": 0, "give": [0, 5], "context": 0, "prefix": 0, "namespec": 0, "separ": [0, 2], "_": [0, 2, 5], "them": 0, "load": [0, 2, 5, 6], "wheel": [0, 1, 2], "window": [0, 2, 8], "maco": [0, 2], "automat": [0, 8], "publish": 0, "inlin": 0, "One": 0, "item": [0, 4, 5], "treat": [0, 3, 5], "sub": [0, 2, 5], "pleas": 1, "contribut": 1, "stabl": 1, "toolchain": [1, 2], "virtualenv": 1, "clone": 1, "repo": 1, "virtual": 1, "ve": 1, "bin": 1, "activ": 1, "maturin": 1, "tool": [1, 2, 3, 4, 6], "extens": [1, 2, 5, 6], "pip": [1, 2], "releas": 1, "anylinux": [1, 2], "3": [1, 2, 4, 5, 6], "6": [1, 2, 4, 5], "distribut": 1, "sudo": 1, "docker": 1, "t": [1, 5], "To": [1, 8], "v": 1, "pwd": 1, "io": 1, "set": [1, 4, 5, 7, 8], "enviro": [1, 5], "flatterer_log": 1, "trace": 1, "full": [1, 3, 5], "debug": 1, "opinion": 2, "which": [2, 4, 5, 6, 7], "tri": 2, "analysi": [2, 7], "receiv": 2, "structur": [2, 5, 6], "deepli": 2, "nest": 2, "well": 2, "hard": 2, "determin": [2, 5], "what": [2, 5], "contain": [2, 4, 5, 6], "even": [2, 5], "after": [2, 5], "know": 2, "how": [2, 3], "part": 2, "pipelin": 2, "aim": 2, "first": [2, 4, 5], "go": [2, 4, 5, 6], "face": 2, "abov": [2, 3, 4, 5], "problem": 2, "It": [2, 4, 5, 6], "up": [2, 5], "your": [2, 4, 5], "hand": 2, "written": [2, 5], "benefit": 2, "over": [2, 5, 6], "most": [2, 5, 7], "approach": 2, "fast": 2, "bind": 2, "eas": 2, "10x": 2, "faster": 2, "effici": 2, "custom": 2, "practic": 2, "been": 2, "learnt": 2, "countless": 2, "gener": [2, 4, 5], "parent": [2, 5], "greater": 2, "binari": 2, "linux": 2, "x64": 2, "univers": 2, "x86": 2, "On": 2, "other": [2, 4, 5], "platform": [2, 4, 5, 6], "sai": [2, 5], "like": [2, 5, 6, 7], "game": [2, 3, 4, 5, 6], "1": [2, 4, 5, 6, 8], "A": [2, 4, 5, 6, 8], "released": [2, 4, 5, 6], "2015": [2, 4, 5], "01": [2, 4, 5], "xbox": 2, "playstat": 2, "rate": 2, "code": 2, "e": [2, 4, 5, 7], "everyon": [2, 4], "2": [2, 4, 5, 6, 7], "b": [2, 4, 5], "2016": [2, 4, 5], "pc": [2, 5], "games_dir": [2, 3, 4, 6], "refer": [2, 4], "detail": 2, "addit": [2, 5], "inform": [2, 4, 7], "configur": [2, 3, 8], "import": [2, 4, 5, 6], "By": [2, 5], "follow": [2, 5, 6, 7, 8], "tree": 2, "rating_cod": [2, 4, 6], "rating_nam": [2, 4, 6], "0": [2, 4, 5, 8], "special": 2, "primari": 2, "uniqu": [2, 5], "per": [2, 6], "promot": 2, "relationship": [2, 5, 6], "properti": [2, 5], "therefor": [2, 5], "_link_gam": [2, 5, 6], "everi": [2, 5], "easili": [2, 6], "would": [2, 5], "_link_platform": [2, 5], "_link__": 2, "table_nam": [2, 4, 5, 6], "referenc": 2, "about": [2, 4, 5, 6], "field_nam": [2, 4, 5, 6], "count": [2, 4, 5, 6], "text": [2, 4, 5, 6], "The": [2, 3, 4, 5, 6, 7], "initi": 2, "edit": [2, 5], "rerun": 2, "transform": 2, "new_games_dir": 2, "f": [2, 4, 5, 6], "myfield": [2, 5], "rearrang": [2, 5], "remov": [2, 5, 6], "flag": 2, "includ": [2, 5], "tabular": [2, 6], "datapackg": [2, 6], "spec": [2, 6], "retriev": 3, "varieti": 3, "below": 3, "exampl": [3, 4, 5], "thei": [3, 5], "rel": 3, "cat": [3, 6], "my": [3, 5], "bucketnam": 3, "bucket": [3, 7], "assum": [3, 5], "compress": 3, "select": [3, 5], "last": 3, "argument": [3, 4], "rest": 3, "games2": [3, 4], "games3": 3, "As": [3, 4, 5], "each": [3, 5, 6], "variou": 3, "For": [3, 4, 5], "new": [3, 6], "delimit": [3, 5], "repres": [4, 5, 6], "howev": [4, 5, 7], "dict": 4, "list_of_object": 4, "complic": [4, 5], "feed": 4, "ijson": 4, "def": 4, "array_item_gener": 4, "fixtur": 4, "basic": [4, 6], "rb": 4, "manipul": 4, "modifi": [4, 5], "yield": 4, "befor": [4, 5], "process": [4, 5], "byte": 4, "interpret": 4, "true": [4, 5, 6, 7], "describ": 4, "print": 4, "fiel": 4, "5": 4, "table_titl": [4, 5, 6], "develop": 4, "db": [4, 5, 6], "4": [4, 5], "7": 4, "_link_main": 4, "8": 4, "9": 4, "10": [4, 5], "11": 4, "similar": 4, "locat": 4, "fals": [4, 5], "locatin": 4, "dataset": [4, 5], "put": [4, 5, 6], "rating_co": 4, "system": [4, 5], "temporari": [4, 7], "g": [4, 5, 7], "look": [5, 6], "output_directori": 5, "base": 5, "nocsv": 5, "default": [5, 8], "noxlsx": 5, "nosqlit": 5, "noparquet": 5, "d": 5, "n": 5, "m": 5, "without": [5, 6, 7, 8], "p": 5, "j": 5, "o": 5, "l": 5, "singl": 5, "y": 5, "alwai": [5, 7], "": 5, "url": 5, "seper": 5, "denot": 5, "within": [5, 6], "h": 5, "given": 5, "slug": 5, "underscore_slug": 5, "w": 5, "integ": 5, "amount": [5, 6], "final": 5, "result": 5, "cpu": 5, "fit": 5, "exit": 5, "input_fil": 5, "user": [5, 6], "pass": 5, "host": [5, 6, 8], "dbname": 5, "should": 5, "In": 5, "env": 5, "database_url": 5, "my_env_var": 5, "envir": 5, "inputfil": 5, "ouput_dir": 5, "defin": [5, 7], "extra": 5, "appear": [5, 6], "ram": 5, "4gb": 5, "game_id": 5, "platforms_requir": 5, "platforms_nam": 5, "platforms_id": 5, "push": 5, "trust": 5, "suffici": 5, "back": 5, "togeth": 5, "analys": 5, "delimet": 5, "fastest": 5, "pars": 5, "jl": 5, "sometim": 5, "call": 5, "doe": [5, 6], "json_stream": 5, "goessner": 5, "net": 5, "articl": 5, "jsonpath": 5, "express": 5, "particular": 5, "evalu": 5, "against": 5, "null": 5, "film": 5, "second": 5, "care": 5, "json_path": 5, "logic": 5, "condit": 5, "either": 5, "OR": 5, "myschema": 5, "postgres_schema": 5, "differ": 5, "come": 5, "rule": 5, "conflict": 5, "coerc": 5, "origin": 5, "kept": 5, "still": 5, "id_prefix": 5, "random": 5, "ad": 5, "across": 5, "recommend": 5, "conatain": 5, "Not": 5, "complet": 5, "parallel": 5, "safe": 5, "two": 5, "try": 5, "currupt": 5, "retri": 5, "loos": 5, "chang": 5, "unwant": 5, "miss": 5, "match": 5, "eventu": 5, "easiest": 5, "sure": 5, "behaviour": 5, "exclud": 5, "only_field": 5, "tables_nam": 5, "basi": 5, "tabless": 5, "only_t": 5, "ever": 5, "associ": 5, "With": 5, "under": 5, "inline_one_to_on": 5, "encod": 5, "remot": 5, "server": [5, 6], "otherwis": 5, "mytabl": 5, "myprefix_": 5, "myprefix_myt": 5, "namespac": 5, "table_prefix": 5, "myobject_myfield": 5, "myobject": 5, "theres": 5, "my_object": 5, "my_field": 5, "my_object_my_field": 5, "confus": 5, "might": 5, "expect": 5, "clash": 5, "fix": 5, "whatev": 5, "___": 5, "my_object__my_field": 5, "__": 5, "path_separo": 5, "take": 5, "modif": 5, "slugifi": 5, "charactor": 5, "alphanumer": 5, "replac": 5, "strang": 5, "turn": 5, "previou": 5, "my_strange_titl": 5, "left": 5, "schema_titl": 5, "underscore_slug_": 5, "done": 5, "x3": 5, "x1": 5, "24": 5, "worth": 5, "lead": [5, 7], "perform": 5, "veri": 5, "100": 5, "slower": 5, "ensur": 5, "circumst": 5, "wrong": 5, "chosen": 5, "reduc": 5, "sacrif": 5, "int": 6, "unless": [6, 7], "help": 6, "definit": 6, "numer": 6, "timestamp": 6, "psql": 6, "someth": 6, "password": 6, "adapt": 6, "plain": 6, "advantag": 6, "games_dir_": 6, "sqlite3": 6, "my_databas": 6, "content": 6, "skip": 6, "profil": 6, "packag": 6, "resourc": 6, "primarykei": 6, "fetch": 7, "provid": 7, "aws_default_region": 7, "u": 7, "east": 7, "aws_access_key_id": 7, "access_key_id": 7, "aws_secret_access_kei": 7, "secret_access_kei": 7, "aws_endpoint": 7, "endpoint": 7, "aws_session_token": 7, "token": 7, "aws_container_credentials_relative_uri": 7, "aw": 7, "amazon": 7, "amazonec": 7, "developerguid": 7, "task": 7, "iam": 7, "role": 7, "html": 7, "aws_allow_http": 7, "permit": 7, "tl": 7, "commonli": 7, "alon": 7, "upload": 7, "someaccesskeyid": 7, "somesecretaccesskeyid": 7, "mybucket": 7, "mydata": 7, "flattenedoutput": 7, "store": 7, "constrain": 7, "serverless": 7, "read": 7, "twice": 7, "known": 7, "upfront": 7, "learn": 8, "visual": 8, "feedback": 8, "webserv": 8, "port": 8, "8080": 8, "localhost": 8, "127": 8, "max_siz": 8, "size": 8, "mb": 8, "maximum": 8, "500": 8, "no_brows": 8, "browser": 8}, "objects": {}, "objtypes": {}, "objnames": {}, "titleterms": {"chang": 0, "log": [0, 1], "0": 0, "19": 0, "14": 0, "2024": 0, "02": 0, "10": 0, "fix": 0, "13": 0, "01": 0, "26": 0, "12": 0, "2023": 0, "03": 0, "new": [0, 5], "8": 0, "06": 0, "21": 0, "6": 0, "5": 0, "07": 0, "4": 0, "05": 0, "3": 0, "11": 0, "1": 0, "18": 0, "17": 0, "2022": 0, "16": 0, "2": 0, "25": 0, "27": 0, "15": 0, "23": 0, "22": 0, "remov": 0, "24": 0, "04": 0, "28": 0, "09": 0, "9": 0, "7": 0, "2021": 0, "ad": 0, "20": 0, "develop": 1, "local": [1, 2, 3], "instal": [1, 2], "build": 1, "flatter": 2, "make": 2, "json": [2, 5, 6], "introduct": 2, "rational": 2, "exampl": [2, 7], "run": 2, "cli": [2, 5], "web": [2, 8], "interfac": [2, 8], "As": 2, "python": [2, 4, 5], "librari": [2, 4], "output": [2, 4, 5, 6], "file": [2, 3, 5, 6], "main": [2, 5], "tabl": [2, 5, 6], "One": [2, 5], "To": [2, 5], "mani": 2, "link": [2, 5], "field": [2, 5, 6], "csv": [2, 6], "input": 3, "sourc": 3, "stdin": 3, "http": 3, "": 3, "s3": [3, 7], "gzip": 3, "multipl": 3, "usag": [4, 5, 7], "creat": 4, "panda": 4, "datafram": 4, "option": 5, "refer": 5, "help": 5, "small": 5, "onli": 5, "format": 5, "name": 5, "pushdown": 5, "No": 5, "path": 5, "arrai": 5, "line": 5, "delemit": 5, "ndjson": 5, "stream": 5, "filter": 5, "forc": 5, "postgr": [5, 6], "schema": 5, "evolv": 5, "drop": 5, "inlin": 5, "prefix": 5, "separ": 5, "titl": 5, "stat": 5, "preview": 5, "thread": 5, "sql": [5, 6], "script": 5, "low": 5, "memori": 5, "api": 5, "xlsx": 6, "sqlite": 6, "parquet": 6, "postgresql": 6, "postgresql_schema": 6, "postgresql_load": 6, "sqlite_schema": 6, "sqlite_load": 6, "datapackag": 6, "url": 7, "region": 7, "authent": 7, "note": 7}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx": 57}, "alltitles": {"Change Log": [[0, "change-log"]], "[0.19.14] - 2024-02-10": [[0, "id1"]], "Fixed": [[0, "fixed"], [0, "id3"], [0, "id10"], [0, "id12"], [0, "id14"], [0, "id16"], [0, "id22"], [0, "id25"], [0, "id29"], [0, "id31"], [0, "id39"], [0, "id53"], [0, "id55"], [0, "id59"], [0, "id61"], [0, "id65"], [0, "id67"], [0, "id72"], [0, "id76"], [0, "id80"], [0, "id82"], [0, "id91"], [0, "id93"]], "[0.19.13] - 2024-01-26": [[0, "id2"]], "[0.19.12] - 2023-12-03": [[0, "id4"]], "New": [[0, "new"], [0, "id6"], [0, "id8"], [0, "id18"], [0, "id20"], [0, "id24"], [0, "id27"], [0, "id33"], [0, "id35"], [0, "id37"], [0, "id41"], [0, "id43"], [0, "id46"], [0, "id69"], [0, "id71"], [0, "id74"], [0, "id78"], [0, "id84"]], "[0.19.10] - 2023-10-02": [[0, "id5"]], "[0.19.8] - 2023-06-21": [[0, "id7"]], "[0.19.6] - 2023-06-21": [[0, "id9"]], "[0.19.5] - 2023-06-07": [[0, "id11"]], "[0.19.4] - 2023-05-26": [[0, "id13"]], "[0.19.3] - 2023-05-11": [[0, "id15"]], "[0.19.1] - 2023-05-10": [[0, "id17"]], "[0.18.0] - 2023-03-17": [[0, "id19"]], "[0.17.1] - 2023-01-07": [[0, "id21"]], "[0.17.0] - 2022-12-03": [[0, "id23"]], "[0.16.2] - 2022-10-25": [[0, "id26"]], "[0.16.2] - 2022-10-16": [[0, "id28"]], "[0.16.1] - 2022-10-16": [[0, "id30"]], "[0.16.0] - 2022-07-27": [[0, "id32"]], "[0.15.0] - 2022-07-27": [[0, "id34"]], "Changed": [[0, "changed"], [0, "id47"], [0, "id49"], [0, "id51"], [0, "id57"], [0, "id63"], [0, "id75"], [0, "id79"], [0, "id83"], [0, "id86"], [0, "id88"], [0, "id95"]], "[0.14.2] - 2022-07-27": [[0, "id36"]], "[0.14.1] - 2022-07-23": [[0, "id38"]], "[0.14.0] - 2022-07-22": [[0, "id40"]], "Removed": [[0, "removed"]], "[0.13.2] - 2022-06-24": [[0, "id42"]], "[0.13.1] - 2022-05-27": [[0, "id44"]], "[0.13.0] - 2022-04-28": [[0, "id45"]], "[0.12.12] - 2022-04-15": [[0, "id48"]], "[0.12.11] - 2022-04-09": [[0, "id50"]], "[0.12.10] - 2022-03-21": [[0, "id52"]], "[0.12.9] - 2022-03-07": [[0, "id54"]], "[0.12.8] - 2022-03-01": [[0, "id56"]], "[0.12.7] - 2022-02-27": [[0, "id58"]], "[0.12.6] - 2022-02-26": [[0, "id60"]], "[0.12.4] - 2022-02-18": [[0, "id62"]], "[0.12.3] - 2022-02-15": [[0, "id64"]], "[0.12.1] - 2022-02-15": [[0, "id66"]], "[0.12] - 2022-02-02": [[0, "id68"]], "[0.11] - 2022-01-25": [[0, "id70"]], "[0.10.1] - 2021-12-22": [[0, "id73"]], "[0.10] - 2021-12-22": [[0, "id77"]], "[0.9] - 2021-12-12": [[0, "id81"]], "[0.8] - 2021-12-01": [[0, "id85"]], "[0.7.1] - 2021-12-01": [[0, "id87"]], "[0.7] - 2021-11-24": [[0, "id89"]], "Added": [[0, "added"], [0, "id97"], [0, "id99"], [0, "id101"]], "[0.6.2] - 2021-11-24": [[0, "id90"]], "[0.6.1] - 2021-11-24": [[0, "id92"]], "[0.6] - 2021-11-23": [[0, "id94"]], "[0.5] - 2021-11-22": [[0, "id96"]], "[0.4] - 2021-11-20": [[0, "id98"]], "[0.3] - 2021-10-25": [[0, "id100"]], "Development": [[1, "development"]], "Local install": [[1, "local-install"]], "Builds": [[1, "builds"]], "Logging": [[1, "logging"]], "Flatterer. Making JSON flatterer": [[2, "flatterer-making-json-flatterer"]], "Introduction": [[2, "introduction"]], "Rationale": [[2, "rationale"]], "Install": [[2, "install"]], "Example JSON": [[2, "example-json"]], "Running Flatterer": [[2, "running-flatterer"]], "CLI": [[2, "cli"]], "Local web interface": [[2, "local-web-interface"]], "As python library": [[2, "as-python-library"]], "Output Files": [[2, "output-files"]], "Main Table": [[2, "main-table"]], "One To Many Table": [[2, "one-to-many-table"]], "Link Fields": [[2, "link-fields"]], "Fields CSV": [[2, "fields-csv"]], "Input Sources": [[3, "input-sources"]], "Local File": [[3, "local-file"]], "Stdin": [[3, "stdin"]], "HTTP(s)": [[3, "http-s"]], "S3": [[3, "s3"], [7, "s3"]], "GZIP": [[3, "gzip"]], "Multiple inputs": [[3, "multiple-inputs"]], "Python Library": [[4, "python-library"]], "Usage": [[4, "usage"]], "Output": [[4, "output"]], "Creating pandas DataFrames": [[4, "creating-pandas-dataframes"]], "Option Reference": [[5, "option-reference"]], "Help (CLI Only) ": [[5, "help-small-cli-only-small"]], "Output Formats": [[5, "output-formats"]], "CLI Usage": [[5, "cli-usage"], [5, "id1"], [5, "id3"], [5, "id5"], [5, "id7"], [5, "id9"], [5, "id11"], [5, "id13"], [5, "id15"], [5, "id17"], [5, "id19"], [5, "id21"], [5, "id23"], [5, "id25"], [5, "id27"], [5, "id29"], [5, "id31"], [5, "id33"], [5, "id35"], [5, "id37"], [5, "id39"], [5, "id41"], [5, "id43"], [5, "id45"], [5, "id47"]], "Python Usage": [[5, "python-usage"], [5, "id2"], [5, "id4"], [5, "id6"], [5, "id8"], [5, "id10"], [5, "id12"], [5, "id14"], [5, "id16"], [5, "id18"], [5, "id20"], [5, "id22"], [5, "id24"], [5, "id26"], [5, "id28"], [5, "id30"], [5, "id32"], [5, "id34"], [5, "id36"], [5, "id38"], [5, "id40"], [5, "id42"], [5, "id44"], [5, "id46"], [5, "id48"], [5, "id49"], [5, "id50"]], "Main Table Name": [[5, "main-table-name"]], "Pushdown Fields": [[5, "pushdown-fields"]], "No Link Fields": [[5, "no-link-fields"]], "Path to JSON Array": [[5, "path-to-json-array"]], "New Line Delemited JSON (NDJSON)": [[5, "new-line-delemited-json-ndjson"]], "JSON Stream": [[5, "json-stream"]], "JSON Path Filter": [[5, "json-path-filter"]], "Force": [[5, "force"]], "Postgres Schema": [[5, "postgres-schema"]], "Evolve Tables": [[5, "evolve-tables"]], "Drop Tables": [[5, "drop-tables"]], "Fields File": [[5, "fields-file"]], "Only Fields": [[5, "only-fields"]], "Tables File": [[5, "tables-file"]], "Only Tables": [[5, "only-tables"]], "Inline One To One": [[5, "inline-one-to-one"]], "Arrays as Table": [[5, "arrays-as-table"]], "Schema": [[5, "schema"]], "Table Prefix": [[5, "table-prefix"]], "Path Separator": [[5, "path-separator"]], "Schema Titles": [[5, "schema-titles"]], "Stats": [[5, "stats"]], "Preview": [[5, "preview"]], "Threads": [[5, "threads"]], "Sql Scripts": [[5, "sql-scripts"]], "Low Memory (api only)": [[5, "low-memory-api-only"]], "Outputs": [[6, "outputs"]], "CSV": [[6, "csv"]], "XLSX": [[6, "xlsx"]], "SQLITE": [[6, "sqlite"]], "POSTGRES": [[6, "postgres"]], "PARQUET": [[6, "parquet"]], "fields.csv": [[6, "fields-csv"]], "tables.csv": [[6, "tables-csv"]], "Postgresql Files": [[6, "postgresql-files"]], "postgresql_schema.sql": [[6, "postgresql-schema-sql"]], "postgresql_load.sql": [[6, "postgresql-load-sql"]], "Sqlite Files.": [[6, "sqlite-files"]], "sqlite_schema.sql": [[6, "sqlite-schema-sql"]], "sqlite_load.sql": [[6, "sqlite-load-sql"]], "datapackage.json": [[6, "datapackage-json"]], "URL": [[7, "url"]], "Region and authentication.": [[7, "region-and-authentication"]], "Example": [[7, "example"]], "Usage Note": [[7, "usage-note"]], "Web Interface": [[8, "web-interface"]]}, "indexentries": {}}) \ No newline at end of file