My Basic ML Project Environment Settings

• W&B(Experiment managing & Hyper Parameter Tuning)
• WIT(Model Analysis)
• Kubeflow(ML Pipeline)
  • How to use kubeflow pipeline on my project
    → There are 4 components in this pipeline (data extracting-preprocessing-training-predicting)
    → Every components should be uploaded by docker image
    • In Shell
      → cd kubeflow/{component}
      → docker build -t myname/{component_tag_name} .
      → docker login
      → docker push myname/{component_tag_name}
      → cd ..
      → python pipeline.py after all of components uploading
      → There is a service which sends email that has an error
• CodeClimate(Code Quality Management & Continuous Integration)
  • How to apply codeclimate's service on my project
    → CodeClimate
    → Login
    → Quality
    → Github connect
    → Create a new repository
    → Add a repo
    → click the repo
    • Activate Repo setting - Github - Summary comments (comments code reviews)
    • Activate Repo setting - Github - Pull request status updates + Webook on Github (auto-static analysis)
    • copy token in repo setting - test coverage
      → paste the token in .github/workflow/coverage.yml: CODECLIMATE_REPO_TOKEN: { }
      → Code commits
      → You can see where an error comes up in Github repository - pull requests
    • If you want to stop branch from merging code which has an error (you have to pay)
      • settings - Branches - Branch Protection rules
        → activate Require status checks to pass before merging / Require branches to be up to date before merging
        → add Coverage Report, Lint Code Base