Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changing zarr source does not reload zarr #101

Open
csteele2 opened this issue Sep 15, 2023 · 4 comments
Open

Changing zarr source does not reload zarr #101

csteele2 opened this issue Sep 15, 2023 · 4 comments

Comments

@csteele2
Copy link

csteele2 commented Sep 15, 2023

I have run into an issue. Much like the demo, I build the "source" of the zarr file using props. However, while I notice just about everything else reloads with new data, and the map will re-render, it does so with data from the previously loaded zarr. In other words, updating the zarr source in Raster doesn't seem to refresh the upstream tasks that actually load the data. If this is intended behavior, is there a way to force that reload?

@csteele2
Copy link
Author

Okay, after digging pretty deep, I was able to get raster to call createtiles by adding source as as dependency to the hook that runs createTiles. I would consider adding this in, because to remain memory efficient when building the zarr pyramids using my high resolution data, I did have to break them into multiple zarrs (by variable). The change I made was inside the repo (and had to retool the imports to read from mine), so I am not sure where to go from here, whether you would want to consider that change (since it still won't impact performance of those not swapping zarrs), or if you think its too special of case to consider.

@Jakidxav
Copy link

Jakidxav commented Jan 12, 2024

I have the same issue, though for a different reason. I want to show datasets that come from different data products on the same map, meaning that they often have different resolutions. I could regrid all of the data to have the same resolution, but am hoping to avoid that. I have not found a way to keep datasets with different resolutions in the same Zarr store, so I want to be able to re-render the <Raster /> component with a difference source prop for each dataset. However, I am finding it difficult to do.

The code for my current workaround can be found here: https://github.com/Jakidxav/carbonplan_examples . I based the example off of the demo code provided by CarbonPlan, but tweaked it to show that I am rendering data from separate Zarr stores. I am actually pretty happy with the way the code works, though it does involve a lot of code duplication every time I want to add a new dataset to the map. My main concern is that when I want to customize anything that needs to access the <Map /> component's state, I can't access it. For a concrete example, I wanted to customize the <RegionPicker /> component to look more like the one in the ncview-js package, but could not do so.

@csteele2 could you provide code for a small working example with the changes that you made? It seems like that may help my use case, as well. I tried getting the <Raster /> to call createTiles() in the hook with source as a dependency like you mentioned, but kept getting an error about my Mapbox content being null.

@katamartin
Copy link
Member

Thanks for opening this @csteele2 -- I think your suggestion is a good one!

In the meantime, another option is to render a different Raster per Zarr store. This could be done by mapping through an array of sources (similar to what is done in this example) or by simply setting the Raster key based on a unique identifier like source or variable.

@Jakidxav for your example, the key solution would look like:

        <Map zoom={2} center={[0, 0]}>
          <Fill
            color={theme.rawColors.background}
            source={'https://carbonplan-maps.s3.us-west-2.amazonaws.com/basemaps/ocean'}
            variable={'ocean'}
          />
          <Line
            color={theme.rawColors.primary}
            source={'https://carbonplan-maps.s3.us-west-2.amazonaws.com/basemaps/land'}
            variable={'land'}
          />
          <Raster
            key={ variable }
            display={ display }
            opacity={ opacity }
            source={ `https://storage.googleapis.com/carbonplan-maps/v2/demo/2d/${variable}` }
            variable={ variable }
            clim={ clim }
            colormap={ colormap }
            mode={ "texture" }
            regionOptions={{ setData: setRegionData }}
          />
          {...other children}
        </Map>

@Jakidxav
Copy link

Thank you @katamartin for taking a look at this! I tried using variable in a template literal like that before, but I hadn't been using key as a prop to <Raster />. This seems to be the piece I was missing and fixes the issues I was having.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants