YAIB wiki home
Welcome to the Yet Another ICU Benchmark wiki. We include information on how to use YAIB and how to extend it.
The authors of MIMIC-III and eICU have made a small demo dataset available to demonstrate their use. They can be found on Physionet: MIMIC-III Clinical Database Demo and eICU Collaborative Research Database Demo. These datasets are published under the Open Data Commons Open Database License v1.0 and can be used without credentialing procedure. We have created demo cohorts processed solely from these datasets for each of our currently supported task endpoints. To the best of our knowledge, this complies with the license and the respective dataset author's instructions. Usage of the task cohorts and the dataset is only permitted with the above license. We strongly recommend completing a human subject research training to ensure you properly handle human subject research data.
In the folder demo_data we provide processed publicly available demo datasets from eICU and MIMIC with the necessary labels
for Mortality at 24h,Sepsis, Akute Kidney Injury, Kidney Function, and Length of Stay.
If you do not yet have access to the ICU datasets, you can run the following command to train models for the included demo cohorts:
wandb sweep --verbose experiments/demo_benchmark_classification.yml
wandb sweep --verbose experiments/demo_benchmark_regression.yml
wandb agent <sweep_id>
Tip: You can choose to run each of the configurations on a SLURM cluster instance by
wandb agent --count 1 <sweep_id>
Note: You will need to have a wandb account and be logged in to run the above commands.
If you would like to use the full datasets to run your own benchmarks you can follow these three steps:
Here you can find the instructions to reproduce paper results
We document adding several components to our end-to-end pipeline:
- Datasets
- Clinical concepts
- Clinical tasks
- Evaluation metrics
- Imputation methods
- Preprocessing pipelines
- Models
The development guide for contributing new parts to YAIB can be found here.
The following repositories may be relevant as well:
- YAIB-cohorts: Cohort generation for YAIB.
- YAIB-models: Pretrained models for YAIB.
- ReciPys: Preprocessing package for YAIB pipelines.