Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prediction grid data size #336

Open
sigmafelix opened this issue Jun 10, 2024 · 0 comments
Open

Prediction grid data size #336

sigmafelix opened this issue Jun 10, 2024 · 0 comments

Comments

@sigmafelix
Copy link
Collaborator

The imputed dataset for the first study period is 525MB. The full space-time prediction grid will be 8,652,781 MB, which equals to 8.25 TB. Besides the feasibility of the prediction at such a large data (predicted outcomes will get a few thousandths of size, though), storage would be a problem. The total data size will increase linearly as additional study periods are accounted for. Perhaps we need to consider a DBMS to manage the data in case of the feature data getting too large to manage with qs, fst, or other file-based archives.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant