Skip to content

Compute Studio 1.6.0
Compare
Choose a tag to compare
@hdoupe hdoupe released this 27 May 19:00
· 653 commits to master since this release
1.6.0
4ee4a42
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

  • Close-to-automatic updates and deploys. When a user creates or updates a model on the publish page, I get an email notification. Using the cs-publish package, I kick off a GH actions work flow:

    cs-publish -n [:owner]/[:title]

    I can then deploy the update/new model by merging the PR.

  • Simulations are run as Kuberenetes Jobs. This means that models no longer have to have at least one process running all of the time, and thus reducing C/S compute cost. One side-effect of doing this is that we no longer have Celery's constraint of only being allowed to use one process per worker.
    Screenshot from 2020-05-25 16-09-32

  • Secrets are now stored in Google Secret Manger or any other secret manager backend. This improves C/S security and makes it possible for modeler's to manage their own secrets from the webapp or rest api. This is similar to how GH lets you manage secrets for your Repository.

  • The workers have a new CLI tool to manage this:

export TAG=mytag CS_URL=https://compute.studio CS_API_TOKEN=secret PROJECT=gcp-project-name

# Update c/s api:
csw svc build
csw svc push
csw svc config -o - | kubectl apply -f -

# Update, get, delete c/s secrets
csw svc secrets set --secret-name CS_API_TOKEN --secret-value $CS_API_TOKEN
csw svc secrets get CS_API_TOKEN
csw svc secrets delete CS_API_TOKEN

# Update c/s model
csw models -n PSLmodels/Tax-Cruncher build
csw models -n PSLmodels/Tax-Cruncher test
csw models -n PSLmodels/Tax-Cruncher push

# Set staging tag
csw models -n PSLmodels/Tax-Cruncher stage $TAG

# Deploy and promote latest tag
csw models -n PSLmodels/Tax-Cruncher config -o - | kubectl apply -f - 
csw models -n PSLmodels/Tax-Cruncher promote

# Update, get, list, delete c/s model secrets
csw models -n PSLmodels/Tax-Cruncher secrets set --secret-name super --secret-value secret
csw models -n PSLmodels/Tax-Cruncher secrets get super
csw models -n PSLmodels/Tax-Cruncher secrets list
csw models -n PSLmodels/Tax-Cruncher secrets delete super

  • Also, the flask server has been re-written as a tornado app. This is advantageous because it's main job is to receive and submit requests. Async servers do very well with this type of work.

  • All model configuration data is now stored in the webapp. Modelers can update their repo url, repo tag, memory, and cpu requirements directly from the webapp. The build system will pull the values from the /publish/api/ endpoint.

Assets 2
Loading