Add hera tour to walkthrough (#1002)

* hera-tour part of walkthrough based on upcoming ArgoCon EU 2024 talk, link will be up after * Add `use_cases/testing_templates_and_workflows.py` to show how to test script templates and workflows * Tidy up other examples which will used in the talk --------- Signed-off-by: Elliot Gunton <[email protected]>
argoproj-labs · Mar 14, 2024 · 61bf73c · 61bf73c
1 parent 5c252ac
commit 61bf73c
Show file tree

Hide file tree

Showing 14 changed files with 622 additions and 30 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -45,7 +45,7 @@ respected. To solve this, add the following as a config in your `.vscode/launch.
 ```json
 {
    "name": "Debug Tests",
-   "type": "python",
+   "type": "debugpy", // "python" is now deprecated
    "request": "launch",
    "purpose": ["debug-test"],
    "console": "integratedTerminal",
@@ -69,7 +69,7 @@ Please keep in mind the following guidelines and practices when contributing to
 1. Add unit tests for any new code you write.
 1. Add an example, or extend an existing example, with any new features you may add. Use `make examples` to ensure that the documentation and examples are in sync.
 
-## Adding new Workflow tests
+## Adding new Workflow YAML generation tests
 
 Hera has an automated-test harness that is coupled with our documentation. In order to add new tests, please follow these steps -
 
@@ -88,7 +88,10 @@ In order to add a new workflow test to test Hera functionality, do the following
 
 ### Upstream Hera examples
 
-Tests that correspond to any [upstream Argo Workflow examples](https://github.com/argoproj/argo-workflows/tree/main/examples) should live in `examples/workflows/upstream/*.py`. These tests exist to ensure that Hera has complete parity with Argo Workflows and also to catch any regressions that might happen.
+Tests that correspond to any
+[upstream Argo Workflow examples](https://github.com/argoproj/argo-workflows/tree/main/examples) should live in
+`examples/workflows/upstream/*.py`. These tests exist to ensure that Hera has complete parity with Argo Workflows and
+also to catch any regressions that might happen.
 
 In order to add a new workflow test to test Hera functionality, do the following -
 
@@ -104,6 +107,52 @@ In order to add a new workflow test to test Hera functionality, do the following
 * If you would like to update the golden copy of the test files, you can run `make regenerate-test-data`
 * The golden copies must be checked in to ensure that regressions may be caught in the future
 
+## Adding new Workflow on-cluster tests
+
+Hera's CICD spins up Argo Workflows on a local Kubernetes cluster, which runs tests decorated with
+`@pytest.mark.on_cluster`. If you want to add more on-cluster tests, the easiest way is through a GitHub Codespace. You
+can then run the same `make` commands that run in CICD:
+
+```
+make install-k3d
+```
+
+This will install the k3d CLI.
+
+```
+make run-argo
+```
+
+This will create a cluster using k3d called `test-cluster`, then create a namespace called `argo` on it, applying the
+argo configuration, and patching the deployment to use `server` as the `auth-mode`, meaning the connection to submit the
+workflow doesn't require an authentication mechanism.
+
+You can then run existing on-cluster tests to ensure everything is set up correctly. This command also ports-forward the
+server's port.
+
+```
+make test-on-cluster
+```
+
+### Viewing the Argo UI from a Codespace
+
+> Before doing this, note that **anyone** will be able to connect using the Argo UI URL!
+
+Ensure Argo Workflows is running using the `make` command:
+
+```
+make run-argo
+```
+
+Forward the Server's port using kubectl:
+
+```
+kubectl -n argo port-forward deployment/argo-server 2746:2746
+```
+
+Then, go to the `PORTS` panel in VSCode, and add the `2746` port. You should see a green circle to the left of the port.
+Then right click on the `2746` row and set `Port Visibility` to `public`. You can then open the URL in your browser to view the Argo UI.
+
 ## Code of Conduct
 
 Please be mindful of, and adhere to, the CNCF's

diff --git a/Makefile b/Makefile
@@ -155,6 +155,7 @@ run-argo: ## Start the argo server
 	kubectl get namespace argo || kubectl create namespace argo
 	kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v$(ARGO_WORKFLOWS_VERSION)/install.yaml
 	kubectl patch deployment argo-server --namespace argo --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/args", "value": ["server", "--auth-mode=server"]}]'
+	kubectl create rolebinding default-admin --clusterrole=admin --serviceaccount=argo:default --namespace=argo
 	kubectl rollout status -n argo deployment/argo-server --timeout=120s --watch=true
 
 .PHONY: stop-argo

diff --git a/docs/examples/workflows/dags/dag_diamond_with_callable_decorators.md b/docs/examples/workflows/dags/dag_diamond_with_callable_decorators.md
@@ -8,14 +8,10 @@
 === "Hera"
 
     ```python linenums="1"
-    from hera.workflows import (
-        DAG,
-        Workflow,
-        script,
-    )
+    from hera.workflows import DAG, Workflow, script
 
 
-    @script(add_cwd_to_sys_path=False, image="python:alpine3.6")
+    @script(image="python:3.12")
     def echo(message):
         print(message)
 
@@ -76,8 +72,11 @@
         script:
           command:
           - python
-          image: python:alpine3.6
+          image: python:3.12
           source: |-
+            import os
+            import sys
+            sys.path.append(os.getcwd())
             import json
             try: message = json.loads(r'''{{inputs.parameters.message}}''')
             except: message = r'''{{inputs.parameters.message}}'''

diff --git a/docs/examples/workflows/upstream/dag_diamond.md b/docs/examples/workflows/upstream/dag_diamond.md
@@ -13,10 +13,7 @@ The upstream example can be [found here](https://github.com/argoproj/argo-workfl
     ```python linenums="1"
     from hera.workflows import DAG, Container, Parameter, Workflow
 
-    with Workflow(
-        generate_name="dag-diamond-",
-        entrypoint="diamond",
-    ) as w:
+    with Workflow(generate_name="dag-diamond-", entrypoint="diamond") as w:
         echo = Container(
             name="echo",
             image="alpine:3.7",

diff --git a/docs/examples/workflows/use-cases/testing_templates_and_workflows.md b/docs/examples/workflows/use-cases/testing_templates_and_workflows.md
@@ -0,0 +1,111 @@
+# Testing Templates And Workflows
+
+
+
+
+
+
+=== "Hera"
+
+    ```python linenums="1"
+    from hera.shared import global_config
+    from hera.workflows import DAG, RunnerScriptConstructor, Script, Workflow, WorkflowsService, script
+
+    try:
+        from pydantic.v1 import BaseModel
+    except ImportError:
+        from pydantic import BaseModel
+
+    global_config.set_class_defaults(Script, constructor=RunnerScriptConstructor())
+
+
+    class Rectangle(BaseModel):
+        length: float
+        width: float
+
+        def area(self) -> float:
+            return self.length * self.width
+
+
+    @script(constructor="runner", image="my-built-python-image")
+    def calculate_area_of_rectangle(rectangle: Rectangle) -> float:
+        return rectangle.area()
+
+
+    with Workflow(
+        generate_name="dag-",
+        entrypoint="dag",
+        namespace="argo",
+        workflows_service=WorkflowsService(host="https://localhost:2746"),
+    ) as w:
+        with DAG(name="dag"):
+            A = calculate_area_of_rectangle(
+                name="rectangle-1", arguments={"rectangle": Rectangle(length=1.2, width=3.4).json()}
+            )
+            B = calculate_area_of_rectangle(
+                name="rectangle-2", arguments={"rectangle": Rectangle(length=4.3, width=2.1).json()}
+            )
+            A >> B
+
+
+    def test_calculate_area_of_rectangle():
+        r = Rectangle(length=2.0, width=3.0)
+        assert calculate_area_of_rectangle(r) == 6.0
+
+
+    def test_create_workflow():
+        model_workflow = w.create(wait=True)
+        assert model_workflow.status and model_workflow.status.phase == "Succeeded"
+
+        echo_node = next(
+            filter(
+                lambda n: n.display_name == "echo",
+                model_workflow.status.nodes.values(),
+            )
+        )
+        assert echo_node.outputs.parameters[0].value == "my value"
+    ```
+
+=== "YAML"
+
+    ```yaml linenums="1"
+    apiVersion: argoproj.io/v1alpha1
+    kind: Workflow
+    metadata:
+      generateName: dag-
+      namespace: argo
+    spec:
+      entrypoint: dag
+      templates:
+      - dag:
+          tasks:
+          - arguments:
+              parameters:
+              - name: rectangle
+                value: '{"length": 1.2, "width": 3.4}'
+            name: rectangle-1
+            template: calculate-area-of-rectangle
+          - arguments:
+              parameters:
+              - name: rectangle
+                value: '{"length": 4.3, "width": 2.1}'
+            depends: rectangle-1
+            name: rectangle-2
+            template: calculate-area-of-rectangle
+        name: dag
+      - inputs:
+          parameters:
+          - name: rectangle
+        name: calculate-area-of-rectangle
+        script:
+          args:
+          - -m
+          - hera.workflows.runner
+          - -e
+          - examples.workflows.use_cases.testing_templates_and_workflows:calculate_area_of_rectangle
+          command:
+          - python
+          image: my-built-python-image
+          source: '{{inputs.parameters}}'
+    ```
+
diff --git a/docs/walk-through/about-hera.md b/docs/walk-through/about-hera.md
@@ -48,3 +48,5 @@ functionality in Hera. Read more in the [user guides](../user-guides/core-concep
 A natural extension of a Python DSL for Argo is tighter integration with Python scripts. This is where Hera improves the
 developer experience through its tailored classes and syntactic sugar to enable developers to easily orchestrate Python
 functions. Check out [Hello World](hello-world.md) to get started!
+
+If you want an introductory tour of Hera, check out the [Hera Tour](hera-tour.md)!