Skip to content

DS3Lab/MLBench

Folders and files

NameName
Last commit message
Last commit date

Latest commit

908f59b · Sep 24, 2023

History

13 Commits
Sep 24, 2023
Sep 24, 2023
Jul 16, 2021
Jul 16, 2021
Sep 24, 2023
Sep 24, 2023

Repository files navigation

Benchmarking Machine Learning Clouds

We conduct a benchmark over several popular machine learning cloud platforms, including Google, Amazon AWS, Azure and Oracle. For reproducibility, we make all datasets and code public in this repo. Our benchmark contains two snapshots, mlbench and automlbanch.

mlbench

mlbench was conducted during 2016-2017 when Amazon and Azure provide ML training services.

  • Dataset
  • Configurations.
    • Amazon. Previous configurations have expired due to platform upgrades. Refer to the latest confugiration.
    • Azure. We share training workflows in this link. An Azure ML studio account is required.

automlbench

automlbench was conducted during 2020-2022 when many machine learning clouds provide automatic machine learning training. Our goal is to evaluate their AutoML functionalities.

  • Dataset
  • Code. We provide jupyter scripts for three platforms.
    • Google. Google does training and inference without any code.
    • AWS. We provide scripts for inference.
    • Azure. We provide scripts for inference.
    • Oracle. We provide scripts for training and inference.
  • Configurations. We show screenshots of how we configure each platform.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published