Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline versioning apply delete #2093

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
343ad64
updated swagger
Adir111 Feb 24, 2025
a5ae124
organised file + added version to pipeline
Adir111 Feb 24, 2025
26abde4
added explanation
Adir111 Feb 24, 2025
e1cd10a
updated swagger
Adir111 Feb 24, 2025
64b2654
updated swagger, now support deletion and changing version of pipeline
Adir111 Feb 24, 2025
169e6f7
added validator for pipeline version
Adir111 Feb 24, 2025
1eb2c18
added pipeline appyl and delete support for rest
Adir111 Feb 24, 2025
ab10f31
Merge branch 'master' into pipeline_versioning_apply_delete
Adir111 Feb 24, 2025
91dc191
formatted message
Adir111 Feb 24, 2025
e50d5c0
better readability
Adir111 Feb 24, 2025
95dfb01
.
Adir111 Feb 24, 2025
11d58bd
added _ (incorrect method name)
Adir111 Feb 24, 2025
faa469c
when applying new pipeline version, stopping the execution of the cur…
Adir111 Feb 24, 2025
dc1c4cb
updated error message
Adir111 Feb 24, 2025
502030b
updated error message
Adir111 Feb 24, 2025
c572b0f
now updating pipeline modified date when updating pipeline
Adir111 Feb 24, 2025
a0f162b
changed login to be only the token itself.
Adir111 Feb 24, 2025
b0b98cd
added dependency
Adir111 Feb 24, 2025
3ec5153
undo dependency addition
Adir111 Feb 24, 2025
74b0df3
sending back only the token itself
Adir111 Feb 24, 2025
85a8e24
undo login changes (not needed)
Adir111 Feb 24, 2025
8a2e834
updated test suite
Adir111 Feb 24, 2025
c1553ab
not needed
Adir111 Feb 25, 2025
817d22d
added applying tests
Adir111 Feb 25, 2025
fd43ec0
finished get and update tests
Adir111 Feb 25, 2025
679a0b2
added deletion unit tests
Adir111 Feb 25, 2025
6a867fa
removed 'force' from swagger for pipeline
Adir111 Feb 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 111 additions & 87 deletions core/api-server/api/graphql/schemas/pipeline-schema.js
Original file line number Diff line number Diff line change
@@ -1,94 +1,118 @@
const { gql } = require('apollo-server');

const pipelineTypeDefs = gql`
type Cron { enabled: Boolean pattern: String }

type Triggers { cron: Cron ,pipelines: [String]}

type ConcurrentPipelines { amount: Int rejectOnFailure: Boolean }

type Webhooks { progress:String, result:String }

type Streaming { flows:Object, defaultFlow:String }

type Options { batchTolerance: Int
ttl: Int
progressVerbosityLevel: String
concurrentPipelines: ConcurrentPipelines
activeTtl: Int
devMode: Boolean
devFolder: String
}

type Metrics { tensorboard: Boolean }
type Snapshot{
name: String
}

type Sampler{
search_space: Object
}

type HyperParams{
name: String
suggest: String
high: String
low: String
choices: [String]
sampler: Sampler
}
type Spec {
name: String
id: String
snapshot: Snapshot
description: String
mem: String
cpu: String
objectivePipeline: String
numberOfTrials: Int
hyperParams: [HyperParams]
sampler: Sampler
}
type Retry { policy: String limit: Int }

type PipelineNodes {
nodeName: String
algorithmName: String
ttl: Int
includeInResult: Boolean
batchOperation: String
metrics: Metrics
retry: Retry
spec: Spec
kind: String
stateType: String
input: [Object]
}



type Pipeline {
modified: Float
kind: String
name: String
description: String
priority: Int
experimentName: String
triggers: Triggers
options: Options
webhooks: Webhooks
streaming: Streaming
nodes: [PipelineNodes ]
flowInput: Object
}
type Metrics { tensorboard: Boolean }

type AutogeneratedPipelines { list: [Object],pipelinesCount:Int }

extend type Query {
pipelines:AutogeneratedPipelines
}
type Cron {
enabled: Boolean
pattern: String
}

type Triggers {
cron: Cron
pipelines: [String]
}

type ConcurrentPipelines {
amount: Int
rejectOnFailure: Boolean
}

type Webhooks {
progress: String
result: String
}

type Streaming {
flows: Object
defaultFlow: String
}

type Options {
batchTolerance: Int
ttl: Int
progressVerbosityLevel: String
concurrentPipelines: ConcurrentPipelines
activeTtl: Int
devMode: Boolean
devFolder: String
}

type Metrics {
tensorboard: Boolean
}

type Snapshot {
name: String
}

type Sampler {
search_space: Object
}

type HyperParams {
name: String
suggest: String
high: String
low: String
choices: [String]
sampler: Sampler
}

type Spec {
name: String
id: String
snapshot: Snapshot
description: String
mem: String
cpu: String
objectivePipeline: String
numberOfTrials: Int
hyperParams: [HyperParams]
sampler: Sampler
}

type Retry {
policy: String
limit: Int
}

type PipelineNodes {
nodeName: String
algorithmName: String
ttl: Int
includeInResult: Boolean
batchOperation: String
metrics: Metrics
retry: Retry
spec: Spec
kind: String
stateType: String
input: [Object]
}

type Pipeline {
modified: Float
kind: String
name: String
description: String
priority: Int
experimentName: String
triggers: Triggers
options: Options
webhooks: Webhooks
streaming: Streaming
nodes: [PipelineNodes]
flowInput: Object
version: String
}

type AutogeneratedPipelines {
list: [Object]
pipelinesCount: Int
}

extend type Query {
pipelines: AutogeneratedPipelines
}
`;

module.exports = pipelineTypeDefs;
2 changes: 1 addition & 1 deletion core/api-server/api/rest-api/routes/v1/exec.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ const routes = (options) => {
res.json({ jobId, gateways });
});
router.post('/getGraphByStreamingFlow', keycloak.getProtect(keycloakRoles.API_VIEW), async (req, res) => {
const { nodes, edges } = await Execution.getGraphByStreamingFlow(req.body);
const { nodes, edges } = await Execution._getGraphByStreamingFlow(req.body);
res.json({ nodes, edges });
});
router.post('/stop', keycloak.getProtect(keycloakRoles.API_VIEW), async (req, res) => {
Expand Down
8 changes: 8 additions & 0 deletions core/api-server/api/rest-api/routes/v1/versions.js
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,14 @@ const routes = (options) => {
const response = await pipelineVersionsService.getVersion(req.params);
res.json(response);
});
router.post('/pipelines/apply', keycloak.getProtect(keycloakRoles.API_EDIT), async (req, res) => {
const response = await pipelineVersionsService.applyVersion(req.body);
res.status(HttpStatus.StatusCodes.CREATED).json(response);
});
router.delete('/pipelines/:name/:version', keycloak.getProtect(keycloakRoles.API_DELETE), async (req, res) => {
const response = await pipelineVersionsService.deleteVersion(req.params);
res.json(response);
});

return router;
};
Expand Down
Loading
Loading