Skip to content

Commit

Permalink
docs: Best practices for handling flow outputs (#2199)
Browse files Browse the repository at this point in the history
  • Loading branch information
anna-geller authored Feb 10, 2025
1 parent 1b3fca0 commit d465eee
Showing 1 changed file with 60 additions and 0 deletions.
60 changes: 60 additions & 0 deletions content/docs/14.best-practices/7.outputs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
title: Managing and purging flow outputs
icon: /docs/icons/best-practices.svg
---

Best practices for handling flow outputs incl. purging large outputs and conditionally returning outputs.

## Conditionally returning outputs

When you have a flow that can return different outputs based on some condition, you can use a Pebble expression in the `outputs` section to conditionally return the output of task `A` if it wasn't skipped, and the output of task `B` otherwise.

```yaml
id: conditionallyReturnOutputs
namespace: company.team

inputs:
- id: runTask
type: BOOLEAN
defaults: true

tasks:
- id: taskA
runIf: "{{ inputs.runTask }}"
type: io.kestra.plugin.core.debug.Return
format: Hello World!

- id: taskB
type: io.kestra.plugin.core.debug.Return
format: Fallback output

outputs:
- id: flowOutput
type: STRING
value: "{{ tasks.taskA.state != 'SKIPPED' ? outputs.taskA.value : outputs.taskB.value }}"
```
## Purging large outputs
When you have a flow that generates large outputs that don't need to be kept after the execution is completed, you can use the `io.kestra.plugin.core.storage.PurgeExecutionFiles` task to purge the outputs of the current execution.

In the example below, the flow downloads a large file from an HTTP API and uploads it to an S3 bucket. After the file is uploaded to S3, it's no longer needed, so the `PurgeExecutionFiles` task is used to delete the file from the internal storage.

```yaml
id: extractLoadPurge
namespace: company.team
tasks:
- id: extract
type: io.kestra.plugin.core.http.Download
uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
- id: load
type: io.kestra.plugin.aws.s3.Upload
from: "{{ outputs.extractLargeFile.uri }}"
bucket: myBucket
key: largeFiles/orders.csv
- id: purge
type: io.kestra.plugin.core.storage.PurgeCurrentExecutionFiles
```

0 comments on commit d465eee

Please sign in to comment.