Is it possible to implement the batch data loading in TileLayer/MVTLayer?

[hifichevymane]

Hey, I want to implement the batch loading in TileLayer using loadInBatches function in loaders.gl. I am reading docs but I don't see the information that TileLayer supports batch loading and AsyncIterable objects. Is it possible to implement batch loading for TileLayer or should I create a custom layer for that?

[ibgreen]

This question could mean a few different things but perhaps this helps:

• In general, tiling and streaming (batched) loads are two ways to solve the same problem (incrementally loading big data sets), normally not combined.
• Normally your tiles are expected to be "small" enough so that you can load them atomically.

[hifichevymane]

Normally your tiles are expected to be "small" enough so that you can load them atomically.

I just have a problem that the tiles are so big to load, so I started to think about batched loading data.
So normally this is not expected to combine tiling and batched loading, but in theory that can be implemented, right?

[ibgreen]

Yes, deck.gl has a Tileset2D class that manages the loading of the tiles for the current viewport, and the TileLayer uses this class. I believe that Tileset2D can be extended and overridden by the application.

You'll face the issue that you now have a bunch of tiles that are not just "loaded or not loaded", they can be partially loaded and independently updating (topping off), so another layer of async tracking is needed.

[ibgreen]

But also you will need streaming loaders for the data in your tiles. If you just have enormous MVT tiles as a result of not applying aggressive options during traditional vector tiling, be aware that loaders.gl doesn't support batched loading of MVT tiles and it may not be super easy to implement due to the somewhat scattered binary layout of data in MVT tile protobufs.

You'd need to serve your tiles in a format that supports streaming loads. So it may grow into a bigger data architecture flavor effort rather than a focused point fix to improve performance.

[hifichevymane]

Got it, thank you so much for the answer!