-
Notifications
You must be signed in to change notification settings - Fork 541
Add Jobs quickstart #1198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
siri-varma
wants to merge
56
commits into
dapr:master
Choose a base branch
from
siri-varma:users/svegiraju/jobs-quickstart
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Add Jobs quickstart #1198
Changes from all commits
Commits
Show all changes
56 commits
Select commit
Hold shift + click to select a range
8bd32a4
Add fundamentals example for Python
marcduiker cde572a
Add comments and types
marcduiker aad6f0e
Change import, add task chaining and fan-in fan-out
marcduiker ff1003c
Use lifespan for wf_runtime, add monitor
marcduiker 413627d
Remove logging
marcduiker 3c18d45
Add external events, use flush true for print statements
marcduiker c396ef6
Add child workflows
marcduiker e2592e4
Add workflow management
marcduiker 4f19e8f
Revert dapr.yaml changes for C#
marcduiker b175f0a
Add resiliency and compensation
marcduiker d12d612
Add Python README
marcduiker ece50b7
Add combined patterns (WIP)
marcduiker b2842f9
Update orderworkflow
marcduiker 4c71f90
Add challenges and tips code
marcduiker 19cb445
Try converting from SimpleNamespace to dataclass
marcduiker 88748b9
Change to pydantic models
marcduiker 32bf96c
Change to model_dump
marcduiker b106f26
Update tutorials/workflow/python/monitor-pattern/monitor/monitor_work…
marcduiker 3fb9bc3
Update tutorials/workflow/python/fan-out-fan-in/fan_out_fan_in/fanout…
marcduiker 75a0442
Update tutorials/workflow/python/monitor-pattern/monitor/monitor_work…
marcduiker 3985325
Update tutorials/workflow/python/monitor-pattern/monitor/monitor_work…
marcduiker 16c62e2
Update tutorials/workflow/python/monitor-pattern/monitor/monitor_work…
marcduiker bf232d4
Update tutorials/workflow/python/monitor-pattern/monitor/monitor_work…
marcduiker fc3cec0
Update tutorials/workflow/python/external-system-interaction/README.md
marcduiker a2616a8
Update tutorials/workflow/python/external-system-interaction/README.md
marcduiker ecf4103
Add univorn dependency
marcduiker 6f9c90b
Use order.id as instance_id
marcduiker aabfce4
Clean up
marcduiker 250ff4c
Update naming
marcduiker 508e9b0
Update log output
marcduiker f23b769
Fix workflow serialization
marcduiker aef9101
Update tutorials/workflow/python/fundamentals/basic/basic_workflow.py
marcduiker dfa70d9
Update tutorials/workflow/python/workflow-management/README.md
marcduiker decfde1
Update tutorials/workflow/python/workflow-management/README.md
marcduiker d70be34
Update tutorials/workflow/python/workflow-management/README.md
marcduiker 15b4dc5
Update tutorials/workflow/python/task-chaining/README.md
marcduiker 1dba48d
Update tutorials/workflow/python/workflow-management/README.md
marcduiker b86bacc
Update tutorials/workflow/python/combined-patterns/dapr.yaml
marcduiker 14702c2
Update tutorials/workflow/python/combined-patterns/dapr.yaml
marcduiker 785c595
Update tutorials/workflow/python/workflow-management/README.md
marcduiker fe1ba3c
Update tutorials/workflow/python/challenges-tips/payload_size_workflo…
marcduiker 2b2c899
Update tutorials/workflow/python/monitor-pattern/monitor/monitor_work…
marcduiker e92749b
fix(order-processor): provide default DAPR_HTTP_PORT fallback (3500) …
tommygood e5a77e1
Add Jobs quickstart
siri-varma c21ae11
REmove spaces (#1197)
siri-varma 2d004a4
Modify yaml
siri-varma 083db68
Amend
siri-varma 9b69dd1
Amend
siri-varma 4c808bf
Fix yaml
siri-varma bb86a28
Fix job
siri-varma ef4da09
Play
siri-varma c59ef9c
Fix things
siri-varma d242992
Fix readme
siri-varma 806f5ca
Merge branch 'master' into users/svegiraju/jobs-quickstart
siri-varma eea8c46
Merge branch 'master' into users/svegiraju/jobs-quickstart
alicejgibbons d2090f1
Merge branch 'master' into users/svegiraju/jobs-quickstart
siri-varma File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,167 @@ | ||
# Dapr Jobs API (SDK) | ||
|
||
In this quickstart, you'll schedule, get, and delete a job using Dapr's Job API. This API is responsible for scheduling and running jobs at a specific time or interval. | ||
|
||
Visit [this](https://docs.dapr.io/developing-applications/building-blocks/jobs/) link for more information about Dapr and the Jobs API. | ||
|
||
> **Note:** This example leverages the Java SDK. | ||
|
||
This quickstart includes two apps: | ||
|
||
- Job Scheduler, responsible for scheduling, retrieving and deleting jobs. | ||
- Job Service, receives the triggers for the scheduled jobs. | ||
|
||
## Run all apps with multi-app run template file | ||
|
||
This section shows how to run both applications at once using a [multi-app run template file](https://docs.dapr.io/developing-applications/local-development/multi-app-dapr-run/multi-app-overview/) with `dapr run -f .`. This enables to you test the interactions between multiple applications and will `schedule`, `run`, `get`, and `delete` jobs within a single process. | ||
|
||
1. Build the apps: | ||
|
||
<!-- STEP | ||
name: Build dependencies for job-service | ||
sleep: 1 | ||
--> | ||
|
||
```bash | ||
cd ./job-service | ||
mvn clean install | ||
cd .. | ||
``` | ||
|
||
<!-- END_STEP --> | ||
|
||
<!-- STEP | ||
name: Build dependencies for job-scheduler | ||
sleep: 1 | ||
--> | ||
|
||
```bash | ||
cd ./job-scheduler | ||
mvn clean install | ||
cd .. | ||
``` | ||
|
||
<!-- END_STEP --> | ||
|
||
2. Run the multi app run template: | ||
|
||
<!-- STEP | ||
name: Run multi app run template | ||
expected_stdout_lines: | ||
- '== APP - job-scheduler-sdk == Job Scheduled: {"name":"R2-D2","data":"T2lsIENoYW5nZQ==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null}' | ||
- '== APP - job-service-sdk == Starting Droid: R2-D2' | ||
- '== APP - job-scheduler-sdk == Scheduling a Job with name C-3PO' | ||
- '== APP - job-scheduler-sdk == Job Scheduled: {"name":"C-3PO","data":"TGltYiBDYWxpYnJhdGlvbg==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null}' | ||
- '== APP - job-service-sdk == Starting Droid: R2-D2' | ||
- '== APP - job-service-sdk == Executing Maintenance: Oil Change' | ||
- '== APP - job-service-sdk == Starting Droid: C-3PO' | ||
- '== APP - job-service-sdk == Executing Maintenance: Limb Calibration' | ||
- '== APP - job-scheduler-sdk == Getting Job: C-3PO' | ||
- '== APP - job-scheduler-sdk == Job Details: {"name":"C-3PO","data":"TGltYiBDYWxpYnJhdGlvbg==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null}' | ||
- '== APP - job-scheduler-sdk == Deleting Job: C-3PO' | ||
- '== APP - job-scheduler-sdk == Deleted Job: C-3PO' | ||
expected_stderr_lines: | ||
output_match_mode: substring | ||
match_order: none | ||
background: true | ||
sleep: 60 | ||
timeout_seconds: 120 | ||
--> | ||
|
||
```bash | ||
dapr run -f . | ||
``` | ||
|
||
The terminal console output should look similar to this, where: | ||
|
||
- The `R2-D2` job is being scheduled. | ||
- The `R2-D2` job is completed. | ||
- The `R2-D2` job is being retrieved. | ||
- The `C-3PO` job is being scheduled. | ||
- The `C-3PO` job is completed. | ||
- The `C-3PO` job is being retrieved. | ||
- The `C-3PO` job is being deleted. | ||
- The `C-3PO` job is deleted. | ||
|
||
```text | ||
== APP - job-scheduler-sdk == Job Scheduled: {"name":"R2-D2","data":"T2lsIENoYW5nZQ==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null} | ||
== APP - job-service-sdk == Starting Droid: R2-D2 | ||
== APP - job-scheduler-sdk == Scheduling a Job with name C-3PO | ||
== APP - job-scheduler-sdk == Job Scheduled: {"name":"C-3PO","data":"TGltYiBDYWxpYnJhdGlvbg==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null} | ||
== APP - job-service-sdk == Starting Droid: R2-D2 | ||
== APP - job-service-sdk == Executing Maintenance: Oil Change | ||
== APP - job-service-sdk == Starting Droid: C-3PO | ||
== APP - job-service-sdk == Executing Maintenance: Limb Calibration | ||
== APP - job-scheduler-sdk == Getting Job: C-3PO | ||
== APP - job-scheduler-sdk == Job Details: {"name":"C-3PO","data":"TGltYiBDYWxpYnJhdGlvbg==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null} | ||
== APP - job-scheduler-sdk == Deleting Job: C-3PO | ||
== APP - job-scheduler-sdk == Deleted Job: C-3PO | ||
== APP - job-service-sdk == Starting Droid: R2-D2 | ||
|
||
``` | ||
<!-- END_STEP --> | ||
|
||
3. Stop and clean up application processes. | ||
|
||
<!-- STEP | ||
name: Stop multi-app run | ||
--> | ||
|
||
```bash | ||
dapr stop -f . | ||
``` | ||
|
||
<!-- END_STEP --> | ||
|
||
## Run apps individually | ||
|
||
1. Open a terminal and run the `job-service` app. Build the dependencies if you haven't already. | ||
|
||
```bash | ||
cd ./job-service | ||
mvn clean install | ||
``` | ||
|
||
```bash | ||
dapr run --app-id job-scheduler-sdk --app-port 8080 --dapr-grpc-port 6200 --dapr-http-port 6390 --log-level debug -- java -jar target/JobService-0.0.1-SNAPSHOT.jar com.service.JobServiceStartup | ||
``` | ||
|
||
This makes sure we receive triggers as part of the scheduled jobs. | ||
|
||
2. Open another terminal and run the `job-scheduler` app. Build the dependencies if you haven't already. | ||
|
||
```bash | ||
cd ./job-scheduler | ||
mvn clean install | ||
``` | ||
|
||
```bash | ||
java -jar "target/JobsScheduler-0.0.1-SNAPSHOT.jar" | ||
``` | ||
|
||
This step is responsible for scheduling, getting and deleting jobs. | ||
|
||
3. Here is how the outputs will look like | ||
|
||
In the `job-scheduler` terminal window, the output should be: | ||
|
||
```text | ||
== APP - job-scheduler-sdk == Job Scheduled: {"name":"R2-D2","data":"T2lsIENoYW5nZQ==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null} | ||
== APP - job-scheduler-sdk == Scheduling a Job with name C-3PO | ||
== APP - job-scheduler-sdk == Job Scheduled: {"name":"C-3PO","data":"TGltYiBDYWxpYnJhdGlvbg==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null} | ||
== APP - job-scheduler-sdk == Getting Job: C-3PO | ||
== APP - job-scheduler-sdk == Job Details: {"name":"C-3PO","data":"TGltYiBDYWxpYnJhdGlvbg==","schedule":{"expression":"0/5 * * * * *"},"dueTime":null,"repeats":null,"ttl":null} | ||
== APP - job-scheduler-sdk == Deleting Job: C-3PO | ||
== APP - job-scheduler-sdk == Deleted Job: C-3PO | ||
``` | ||
|
||
In the `job-service` terminal window, the output should be: | ||
|
||
```text | ||
== APP - job-service-sdk == Starting Droid: R2-D2 | ||
== APP - job-service-sdk == Starting Droid: R2-D2 | ||
== APP - job-service-sdk == Executing Maintenance: Oil Change | ||
== APP - job-service-sdk == Starting Droid: C-3PO | ||
== APP - job-service-sdk == Executing Maintenance: Limb Calibration | ||
== APP - job-service-sdk == Starting Droid: R2-D2 | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
version: 1 | ||
apps: | ||
- appDirPath: ./job-service/ | ||
appID: job-service-sdk | ||
appPort: 8080 | ||
command: [ "java", "-jar", "target/JobService-0.0.1-SNAPSHOT.jar" ] | ||
appLogDestination: console | ||
daprdLogDestination: console | ||
- appDirPath: ./job-scheduler/ | ||
appID: job-scheduler-sdk | ||
daprGRPCPort: 6200 | ||
daprHTTPPort: 6390 | ||
appPort: 8080 | ||
command: ["java", "-jar", "target/JobScheduler-0.0.1-SNAPSHOT.jar"] | ||
appLogDestination: console | ||
daprdLogDestination: console | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
<?xml version="1.0" encoding="UTF-8"?> | ||
<project xmlns="http://maven.apache.org/POM/4.0.0" | ||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
<modelVersion>4.0.0</modelVersion> | ||
|
||
<groupId>com.service</groupId> | ||
<artifactId>JobScheduler</artifactId> | ||
<version>0.0.1-SNAPSHOT</version> | ||
<name>JobsService</name> | ||
<description>Demo for Dapr Jobs component</description> | ||
|
||
<dependencies> | ||
<dependency> | ||
<groupId>io.dapr</groupId> | ||
<artifactId>dapr-sdk</artifactId> | ||
<version>1.15.0-rc-5</version> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will wait to merge until 1.15.0 is out |
||
</dependency> | ||
</dependencies> | ||
|
||
<dependencyManagement> | ||
<dependencies> | ||
<dependency> | ||
<groupId>io.dapr</groupId> | ||
<artifactId>dapr-sdk</artifactId> | ||
<version>1.15.0-rc-5</version> | ||
</dependency> | ||
</dependencies> | ||
</dependencyManagement> | ||
|
||
<build> | ||
<plugins> | ||
<plugin> | ||
<groupId>org.springframework.boot</groupId> | ||
<artifactId>spring-boot-maven-plugin</artifactId> | ||
<version>3.3.1</version> | ||
<executions> | ||
<execution> | ||
<goals> | ||
<goal>repackage</goal> | ||
</goals> | ||
<configuration> | ||
<mainClass> | ||
com.service.JobScheduler | ||
</mainClass> | ||
</configuration> | ||
</execution> | ||
</executions> | ||
</plugin> | ||
<plugin> | ||
<groupId>org.apache.maven.plugins</groupId> | ||
<artifactId>maven-compiler-plugin</artifactId> | ||
<configuration> | ||
<source>9</source> | ||
<target>9</target> | ||
</configuration> | ||
</plugin> | ||
</plugins> | ||
</build> | ||
</project> |
79 changes: 79 additions & 0 deletions
79
jobs/java/sdk/job-scheduler/src/main/java/com/service/JobScheduler.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
package com.service; | ||
|
||
import com.fasterxml.jackson.core.JsonProcessingException; | ||
import com.fasterxml.jackson.databind.ObjectMapper; | ||
import io.dapr.client.DaprClientBuilder; | ||
import io.dapr.client.DaprPreviewClient; | ||
import io.dapr.client.domain.DeleteJobRequest; | ||
import io.dapr.client.domain.GetJobRequest; | ||
import io.dapr.client.domain.GetJobResponse; | ||
import io.dapr.client.domain.JobSchedule; | ||
import io.dapr.client.domain.ScheduleJobRequest; | ||
import io.dapr.config.Properties; | ||
import io.dapr.config.Property; | ||
import io.dapr.v1.DaprProtos; | ||
|
||
import java.util.Map; | ||
|
||
public class JobScheduler { | ||
|
||
/** | ||
* The main method of this app to register and fetch jobs. | ||
*/ | ||
public static void main(String[] args) throws Exception { | ||
Map<Property<?>, String> overrides = Map.of( | ||
Properties.HTTP_PORT, "6390", | ||
Properties.GRPC_PORT, "6200" | ||
); | ||
|
||
try (DaprPreviewClient client = new DaprClientBuilder().withPropertyOverrides(overrides).buildPreviewClient()) { | ||
|
||
// Schedule R2-D2 Job. | ||
String r2D2JobName = "R2-D2"; | ||
scheduleJob(client, r2D2JobName, "0/5 * * * * *", "Oil Change"); | ||
Thread.sleep(5000); | ||
|
||
// Retrieve the R2-D2 Job details. | ||
retrieveJob(client, r2D2JobName); | ||
|
||
// Schedule C3PO Job. | ||
String c3POJobName = "C-3PO"; | ||
scheduleJob(client, c3POJobName, "0/5 * * * * *", "Limb Calibration"); | ||
Thread.sleep(5000); | ||
|
||
// Retrieve C3PO Job details. | ||
retrieveJob(client, c3POJobName); | ||
|
||
// Wait for all the jobs to execute | ||
Thread.sleep(5000); | ||
|
||
// Delete the C-3PO Job | ||
deleteJob(client, c3POJobName); | ||
} | ||
} | ||
|
||
private static void scheduleJob(DaprPreviewClient client, String jobName, String cron, String data) | ||
throws JsonProcessingException { | ||
System.out.println("Scheduling a Job with name " + jobName ); | ||
ScheduleJobRequest request = new ScheduleJobRequest(jobName, | ||
JobSchedule.fromString(cron)).setData(data.getBytes()); | ||
client.scheduleJob(request).block(); | ||
|
||
ObjectMapper mapper = new ObjectMapper(); | ||
System.out.println("Job Scheduled: " + mapper.writeValueAsString(request)); | ||
} | ||
|
||
private static void retrieveJob(DaprPreviewClient client, String jobName) throws JsonProcessingException { | ||
System.out.println("Getting Job: " + jobName); | ||
GetJobResponse response = client.getJob(new GetJobRequest(jobName)).block(); | ||
|
||
ObjectMapper mapper = new ObjectMapper(); | ||
System.out.println("Job Details: " + mapper.writeValueAsString(response)); | ||
} | ||
|
||
private static void deleteJob(DaprPreviewClient client, String jobName) { | ||
System.out.println("Deleting Job: " + jobName); | ||
client.deleteJob(new DeleteJobRequest(jobName)).block(); | ||
System.out.println("Deleted Job: " + jobName); | ||
} | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.