Exploratory data analysis

[Gunnar2908]

• Find ranges of all feature
• Find all features missing values
• Add temporal data for time
• Trend analysis,
• Seasonality analysis,
• Cyclicity Analysis,
• Autocorrelation,
• Outlier Detection #14
• Skewed
• Distribution Analysis
• Feature correlation
• Feature Importance
• Residual Analysis
• Anomaly detection #87
  • Check for non-legal values from https://www.meteomatics.com/en/api/available-parameters/alphabetic-list/

If the data is proving to be non-normality then we must find the distribution like log-normal, exponential, or even more domain-specific distributions like the Weibull distribution (often used for lifespan analyses). The professors have said that the data sets will not be the same but the public and private datasets will have the same distribution

[SverreNystad]

We need to fix some of the parts of EDA. We have much functionality added the last day, but we need to tidy it up and fix some bugs. They are not hard they just need some reading up on pandas docs. :)

[SverreNystad]

The bug is now fixed. We now have most of the analysis points

[SverreNystad]

Someone should read the docs for wavelets so we can have a greater understanding of wavelet analysis and experiment with different wavelets: https://pywavelets.readthedocs.io/en/latest/ref/wavelets.html

[SverreNystad]

Need to implement SHAP: https://medium.com/dataman-in-ai/explain-your-model-with-the-shap-values-bc36aac4de3d