Filesystem Loader/Writter #81
norberttech
started this conversation in
Ideas
Replies: 1 comment
-
After quick initial research, I found out that this approach makes sense mostly for small files that can be processed at once in memory. Most of the current adapters are memory safe, meaning that they can read huge (like gigabytes big) files without memory issues (files are read in chunks). |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Currently in order to read CSV file from external filesystem we would need to first download it locally and then use CSV Loader. This is not the most optimal and developer friendly way to work with CSVs.
This could be solved by creating Filesystem abstraction that could be implemented by Flysystem. Filesystem can come with 2 abstractions:
Reader(string $path, array $options = []): Rows
Writer(Rows $rows , array $options = []): void
This way our already existing adapters would only provide implementation for
Reader
andWriter
interfacesBeta Was this translation helpful? Give feedback.
All reactions