From d7709180ab5ffeb4c32f6ecb5215fd17d6b8befa Mon Sep 17 00:00:00 2001 From: Yee Hing Tong Date: Wed, 11 Dec 2024 12:31:08 -0800 Subject: [PATCH] remove comments about batch size (#6102) Signed-off-by: Yee Hing Tong --- .../data_types_and_io/pickle_type.md | 24 ------------------- 1 file changed, 24 deletions(-) diff --git a/docs/user_guide/data_types_and_io/pickle_type.md b/docs/user_guide/data_types_and_io/pickle_type.md index 6a1d84bd37..b0c497ef35 100644 --- a/docs/user_guide/data_types_and_io/pickle_type.md +++ b/docs/user_guide/data_types_and_io/pickle_type.md @@ -45,28 +45,4 @@ We have used a simple object here for demonstration purposes. :lines: 7-26 ``` -## Batch size - -By default, if the list subtype is unrecognized, a single pickle file is generated. -To optimize serialization and deserialization performance for scenarios involving a large number of items -or significant list elements, you can specify a batch size. -This feature allows for the processing of each batch as a separate pickle file. -The following example demonstrates how to set the batch size. - -```{literalinclude} /examples/data_types_and_io/data_types_and_io/pickle_type.py -:caption: data_types_and_io/pickle_type.py -:lines: 35-58 -``` - -:::{note} -The `welcome_superheroes` task will generate two pickle files: one containing two superheroes and the other containing one superhero. -::: - -You can run the workflows locally as follows: - -```{literalinclude} /examples/data_types_and_io/data_types_and_io/pickle_type.py -:caption: data_types_and_io/pickle_type.py -:lines: 62-64 -``` - [flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/master/examples/data_types_and_io/