Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuration overhaul #2

Open
favyen2 opened this issue Feb 9, 2024 · 2 comments
Open

Configuration overhaul #2

favyen2 opened this issue Feb 9, 2024 · 2 comments

Comments

@favyen2
Copy link
Collaborator

favyen2 commented Feb 9, 2024

Figure out how configuration files and loading config from them should work.

Right now some configuration loads arguments to a constructor directly (e.g. loading data source), while others have dedicated parsing functions.

Look into best practices but I think sticking with a function to parse the configuration into a class is good.

Dataset should store config as DatasetConfig or similar rather than keeping the decoded JSON dict.

favyen2 added a commit that referenced this issue Feb 9, 2024
Resolves #1.

Also changes item to have STGeometry instead of shp+time which resolves #5.

Partially deals with #2 by having data sources now provide config loading
function.
@favyen2
Copy link
Collaborator Author

favyen2 commented Feb 12, 2024

There is config object now but I think there are still some things to look into:

  • We should consider using YAML for dataset configuration so it's more readable. Since this config will mostly be manually edited.
  • Right now the user has to specify everything in the config file like bands and layer type even though some of that can come from the data source. We should consider compiling the YAML into a JSON like the current config format where missing values are automatically populated.

@favyen2
Copy link
Collaborator Author

favyen2 commented Nov 14, 2024

I think the bands / layer type is fine and JSON is okay too but we should use jsonargparse for parsing the configuration.

#104 uses jsonargparse to parse the tile store portion of the dataset config, maybe that can be a model to gradually switch to jsonargparse for other components.

We will need to decide whether we want to instantiate the data source objects though. It can be slow for some data sources. Maybe the client/bucket should be created when needed rather than during instantiation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant