Skip to content
This repository has been archived by the owner on Jul 10, 2024. It is now read-only.

Commit

Permalink
SUBMARINE-678. add pytorch template to mysql
Browse files Browse the repository at this point in the history
### What is this PR for?
The only predefined template currently in the database is tf-mnist
I add the pytorch template into the database

The following is the content
`
{
  "name": "pytorch-mnist",
  "author": "author",
  "description": "This is a template to run pytorch-mnist\n",
  "parameters": [
    {
      "name": "experiment_name",
      "required": "true",
      "description": "experiment name",
      "value": null
    }
  ],
  "experimentSpec": {
    "meta": {
      "name": "{{experiment_name}}",
      "namespace": "default",
      "framework": "PyTorch",
      "cmd": "python /var/mnist.py --backend gloo",
      "envVars": {
        "ENV_1": "ENV1"
      }
    },
    "environment": {
      "image": "apache/submarine:pytorch-dist-mnist-1.0"
    },
    "spec": {
      "Master": {
        "replicas": 1,
        "resources": "cpu=1,memory=1024M"
      },
      "Worker": {
        "replicas": 1,
        "resources": "cpu=1,memory=1024M"
      }
    }
  }
}
`

### What type of PR is it?
Improvement

### Todos
* [O] - add pytorch-mnist to database

### What is the Jira issue?
https://issues.apache.org/jira/projects/SUBMARINE/issues/SUBMARINE-678

### How should this be tested?

`
curl -X POST -H "Content-Type: application/json" -d '
{
    "params": {
        "experiment_name":"new-pytorch-mnist"
    }
}
' http://127.0.0.1:8080/api/v1/experiment/pytorch-mnist
`

### Screenshots (if appropriate)
![image](https://user-images.githubusercontent.com/19265751/100550083-6d0bf600-32b2-11eb-87f2-95d7aadd66a9.png)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: JohnTing <[email protected]>

Closes #466 from JohnTing/SUBMARINE-678 and squashes the following commits:

4c173b3 [JohnTing] add doc
65c0dd1 [JohnTing] Fix typo: pytprch->pytorch
9946e32 [JohnTing] add pytprch-mnist template to database
  • Loading branch information
JohnTing authored and pingsutw committed Nov 29, 2020
1 parent c360271 commit 514fa33
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/database/submarine-data.sql
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,5 @@ INSERT INTO `environment` VALUES
-- ----------------------------
INSERT INTO `experiment_template` (`id`, `experimentTemplate_name`, `experimentTemplate_spec`, `create_by`, `create_time`, `update_by`, `update_time`) VALUES
('experimentTemplate_1599498007985_0013', 'tf-mnist', '{\"name\": \"tf-mnist\", \"author\": \"author\", \"parameters\": [{\"name\": \"learning_rate\", \"value\": \"0.2\", \"required\": \"false\", \"description\": \"The parameter of train mnist.\"}, {\"name\": \"batch_size\", \"value\": \"150\", \"required\": \"false\", \"description\": \"The parameter of train mnist.\"}, {\"name\": \"experiment_name\", \"required\": \"true\", \"description\": \"experiment name, you should change it to avoid duplication with other experiment names.\"}, {\"name\": \"spec.Ps.replicas\", \"value\": \"1\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Ps.resourceMap.cpu\", \"value\": \"1\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Ps.resourceMap.memory\", \"value\": \"1000M\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Worker.replicas\", \"value\": \"1\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Worker.resourceMap.cpu\", \"value\": \"1\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Worker.resourceMap.memory\", \"value\": \"1000M\", \"required\": \"false\", \"description\": \"\"}], \"description\": \"This is a template to run tf-mnist.\", \"experimentSpec\": {\"meta\": {\"cmd\": \"python /var/tf_mnist/mnist_with_summaries.py --log_dir=/train/log --learning_rate={{learning_rate}} --batch_size={{batch_size}}\", \"name\": \"{{experiment_name}}\", \"envVars\": {\"ENV1\": \"ENV1\"}, \"framework\": \"TensorFlow\", \"namespace\": \"default\"}, \"spec\": {\"Ps\": {\"replicas\": 1, \"resources\": \"cpu=1,memory=1000M\", \"resourceMap\": {\"cpu\": \"1\", \"memory\": \"1000M\"}}, \"Worker\": {\"replicas\": 1, \"resources\": \"cpu=1,memory=1000M\", \"resourceMap\": {\"cpu\": \"1\", \"memory\": \"1000M\"}}}, \"environment\": {\"image\": \"apache/submarine:tf-mnist-with-summaries-1.0\"}}}', NULL, '2020-09-10 16:31:32', NULL, '2020-10-19 17:05:21');

INSERT INTO `experiment_template` (`id`, `experimentTemplate_name`, `experimentTemplate_spec`, `create_by`, `create_time`, `update_by`, `update_time`) VALUES('experimentTemplate_1606489231336_0014', 'pytorch-mnist', '{\"name\": \"pytorch-mnist\", \"author\": \"author\", \"parameters\": [{\"name\": \"experiment_name\", \"required\": \"true\", \"description\": \"experiment name\"}, {\"name\": \"spec.Master.replicas\", \"value\": \"1\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Master.resourceMap.cpu\", \"value\": \"1\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Master.resourceMap.memory\", \"value\": \"1024M\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Worker.replicas\", \"value\": \"1\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Worker.resourceMap.cpu\", \"value\": \"1\", \"required\": \"false\", \"description\": \"\"}, {\"name\": \"spec.Worker.resourceMap.memory\", \"value\": \"1024M\", \"required\": \"false\", \"description\": \"\"}], \"description\": \"This is a template to run pytorch-mnist\\n\", \"experimentSpec\": {\"meta\": {\"cmd\": \"python /var/mnist.py --backend gloo\", \"name\": \"{{experiment_name}}\", \"envVars\": {\"ENV_1\": \"ENV1\"}, \"framework\": \"PyTorch\", \"namespace\": \"default\"}, \"spec\": {\"Master\": {\"replicas\": 1, \"resources\": \"cpu=1,memory=1024M\", \"resourceMap\": {\"cpu\": \"1\", \"memory\": \"1024M\"}}, \"Worker\": {\"replicas\": 1, \"resources\": \"cpu=1,memory=1024M\", \"resourceMap\": {\"cpu\": \"1\", \"memory\": \"1024M\"}}}, \"environment\": {\"image\": \"apache/submarine:pytorch-dist-mnist-1.0\"}}}', NULL, '2020-11-29 17:56:10', NULL, '2020-11-29 17:56:10');
11 changes: 11 additions & 0 deletions docs/userdocs/k8s/run-experiment-template-rest.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,15 @@ curl -X POST -H "Content-Type: application/json" -d '
' http://127.0.0.1:8080/api/v1/experiment/my-tf-mnist-template
```

**Example Request:**
```sh
curl -X POST -H "Content-Type: application/json" -d '
{
"params": {
"experiment_name":"new-pytorch-mnist"
}
}
' http://127.0.0.1:8080/api/v1/experiment/pytorch-mnist
```

Register experiment template and more info see [Experiment Template API Reference](api/experiment-template.md).

0 comments on commit 514fa33

Please sign in to comment.