Skip to content

Commit

Permalink
docs: Add changelog page to the documentation (#776)
Browse files Browse the repository at this point in the history
  • Loading branch information
yohamta authored Jan 8, 2025
1 parent 37a4203 commit 235fe63
Show file tree
Hide file tree
Showing 23 changed files with 703 additions and 233 deletions.
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,6 +428,12 @@ Feel free to contribute in any way you want! Share ideas, questions, submit issu

We welcome any and all contributions!

## **Contributors**

<a href="https://github.com/dagu-org/dagu/graphs/contributors">
<img src="https://contrib.rocks/image?repo=dagu-org/dagu" />
</a>

## **License**

This project is licensed under the GNU GPLv3.
134 changes: 134 additions & 0 deletions docs/source/changelog.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
.. _ChangeLog:

Changelog
=========

Unreleased
----------

- **Docker Image**: Docker image now based on ``ubuntu:24.04`` and includes common tools and utilities (e.g., ``sudo``, ``git``, ``curl``, ``jq``, ``python3``, etc)
- **Regex support for precondition**: Added support for specifying regular expressions in the ``expected`` value with the ``re:`` prefix.

.. code-block:: yaml
steps:
- name: some_step
command: some_command
preconditions:
- condition: "`date '+%d'`"
expected: "re:0[1-9]" # Run only if the day is between 01 and 09
- **Command support for precondition**: Added support for using command for testing preconditions. If the command returns a non-zero exit code, the precondition is considered unmet and the step will be skipped.

.. code-block:: yaml
steps:
- name: some_step
command: some_command
preconditions:
- command: "test -f /tmp/some_file"
- **Support a list of key-value pairs for params**: Now you can specify a list of key-value pairs for ``params`` in the DAG file.

.. code-block:: yaml
params:
- PARAM1: value1
- PARAM2: value2
- **CLI**: Enable dagu start to support both named and positional parameters after the ``--`` separator. (:issue:`765` by :user:`kriyanshii`)

.. code-block:: bash
dagu start my_dag -- param1 param2 --param3 value3
# or
dagu start my_dag -- PARAM1=param1 PARAM2=param2 PARAM3=value3
- **Support for exitCode in continueOn**: Enhanced the ``continueOn`` attribute to support the ``exitCode`` field. The step will continue if the exit code matches the specified value when the step fails.

.. code-block:: yaml
steps:
- name: some_step
command: some_command
continueOn:
exitCode: 1 # Continue if the exit code is 0 or 1
- **Support for markSuccess in continueOn**: Added the ``markSuccess`` field to the ``continueOn`` attribute. If set to ``true``, the step will be marked as successful even if the command fails and the condition is met.

.. code-block:: yaml
steps:
- name: some_step
command: some_command
continueOn:
exitCode: 1
markSuccess: true # Mark the step as successful even if the command fails
You can specify multiple exit codes as a list.

.. code-block:: yaml
steps:
- name: some_step
command: some_command
continueOn:
exitCode: [1, 2] # Continue if the exit code is 1 or 2 when the step fails
- **Support for output in continueOn**: Added the ``output`` field to the ``continueOn`` attribute. The step will continue if the output (stdin or stdout) contains the specified value.

.. code-block:: yaml
steps:
- name: some_step
command: some_command
continueOn:
output: "some_output" # Continue if the output matches "some_output"
You can also use a regular expression for the ``output`` field with the ``re:`` prefix.

.. code-block:: yaml
steps:
- name: some_step
command: some_command
continueOn:
output: "re:^some_output" # Continue if the output starts with "some_output"
- **Support for piping in command**: Added support for piping in the command field.

.. code-block:: yaml
steps:
- name: some_step
command: "some_command | another_command"
- **Support for shell in command**: Added the ``shell`` field to the command attribute. By default, it uses ``$SHELL`` or ``/bin/sh`` if not set. If it cannot find the shell, it will run the program directly, so you can't use shell-specific features like ``&&``, ``||``, ``|``, etc.

.. code-block:: yaml
steps:
- name: some_step
command: "some_command"
shell: bash
- **Sub workflow execution output**: Now parent workflow will get the output of the subworkflow execution in the ``stdout``. It contains all output from the subworkflow execution. You can use the result in subsequent steps.

.. code-block:: json
{
"name": "some_subworkflow",
"params": "PARAM1=param1 PARAM2=param2",
"outputs": {
"RESULT1": "Some output",
"RESULT2": "Another output"
}
}
- **Support string in depends field**: Added support for specifying a string in the ``depends`` field.

.. code-block:: yaml
steps:
- name: some_step1
command: some_command
- name: some_step2
command: some_other_command
depends: some_step1
- **Support for environment variables in the most of the fields**: You can now use environment variables in most of the fields in the DAG configuration file.
- **Bugfix**: Fixed the issue where the DAG can't be edited when the DAG name contains ``.``.
- **UI**: Updated the visualization of the DAG in the Web UI for better readability.
- **Optimization**: Optimized the size of the saved state files by removing unnecessary information. This will reduce the disk space required for storing the history of the DAG runs.
5 changes: 5 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,16 @@

extensions = [
'sphinx_rtd_theme',
'sphinx.ext.extlinks',
]

templates_path = ['_templates']
exclude_patterns = []

extlinks = {
'issue': ('https://github.com/dagu-org/dagu/issues/%s', '#%s'),
'user': ('https://github.com/%s', '@%s'),
}

# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
Expand Down
11 changes: 10 additions & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ Quick Start
:ref:`schema-reference`
Schema reference.

:ref:`changelog`
History of changes.

.. toctree::
:caption: Installation
:hidden:
Expand Down Expand Up @@ -94,4 +97,10 @@ Quick Start
:hidden:

faq
contrib
contrib

.. toctree::
:caption: Changelog
:hidden:

changelog
2 changes: 1 addition & 1 deletion docs/source/schema.rst
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,7 @@ Each element in the top-level ``steps`` list has its own fields for customizatio
``depends``
~~~~~~~~~
Names of other steps that must complete before this step can run.
Names of other steps that must complete before this step can run. It can be a single step name or a list of step names.

``run``
~~~~~~
Expand Down
18 changes: 16 additions & 2 deletions docs/source/yaml_format.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,7 @@ Hello World
command: echo Hello $NAME
- name: Done
command: echo Done!
depends:
- Hello world
depends: Hello world
Using pipes (``|``) in commands:

Expand Down Expand Up @@ -58,6 +57,21 @@ Running a script:
echo hello world
echo goodbye world
Multiple dependencies:

.. code-block:: yaml
steps:
- name: step 1
command: echo hello
- name: step 2
command: echo world
- name: step 3
command: echo hello world
depends:
- step 1
- step 2
Schema Definition
~~~~~~~~~~~~~~~~
We provide a JSON schema to validate DAG files and enable IDE auto-completion:
Expand Down
13 changes: 6 additions & 7 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ module github.com/dagu-org/dagu
go 1.23

require (
github.com/Masterminds/sprig/v3 v3.2.3
github.com/adrg/xdg v0.5.0
github.com/docker/docker v27.4.1+incompatible
github.com/fsnotify/fsnotify v1.7.0
Expand All @@ -16,15 +17,15 @@ require (
github.com/go-openapi/validate v0.24.0
github.com/go-resty/resty/v2 v2.7.0
github.com/go-swagger/go-swagger v0.30.5
github.com/go-viper/mapstructure/v2 v2.2.1
github.com/golangci/golangci-lint v1.62.2
github.com/google/addlicense v1.1.1
github.com/google/uuid v1.6.0
github.com/imdario/mergo v0.3.16
github.com/itchyny/gojq v0.12.12
github.com/jedib0t/go-pretty/v6 v6.3.6
github.com/jessevdk/go-flags v1.5.0
github.com/mattn/go-shellwords v1.0.12
github.com/mitchellh/mapstructure v1.5.0
github.com/joho/godotenv v1.5.1
github.com/pkg/errors v0.9.1
github.com/robfig/cron/v3 v3.0.1
github.com/samber/slog-multi v1.2.0
Expand All @@ -38,6 +39,7 @@ require (
golang.org/x/text v0.21.0
gopkg.in/yaml.v2 v2.4.0
gotest.tools/gotestsum v1.12.0
mvdan.cc/sh/v3 v3.10.0
)

require (
Expand All @@ -54,7 +56,6 @@ require (
github.com/GaijinEntertainment/go-exhaustruct/v3 v3.3.0 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.3.0 // indirect
github.com/Masterminds/sprig/v3 v3.2.3 // indirect
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/OpenPeeDeeP/depguard/v2 v2.2.0 // indirect
github.com/alecthomas/go-check-sumtype v0.2.0 // indirect
Expand Down Expand Up @@ -113,7 +114,6 @@ require (
github.com/go-toolsmith/astp v1.1.0 // indirect
github.com/go-toolsmith/strparse v1.1.0 // indirect
github.com/go-toolsmith/typep v1.1.0 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/go-xmlfmt/xmlfmt v1.1.2 // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/gofrs/flock v0.12.1 // indirect
Expand Down Expand Up @@ -145,7 +145,6 @@ require (
github.com/jgautheron/goconst v1.7.1 // indirect
github.com/jingyugao/rowserrcheck v1.1.1 // indirect
github.com/jjti/go-spancheck v0.6.2 // indirect
github.com/joho/godotenv v1.5.1 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/julz/importas v0.1.0 // indirect
github.com/karamaru-alpha/copyloopvar v1.1.0 // indirect
Expand Down Expand Up @@ -173,6 +172,7 @@ require (
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/term v0.5.0 // indirect
Expand Down Expand Up @@ -266,12 +266,11 @@ require (
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
honnef.co/go/tools v0.5.1 // indirect
mvdan.cc/gofumpt v0.7.0 // indirect
mvdan.cc/sh/v3 v3.10.0 // indirect
mvdan.cc/unparam v0.0.0-20240528143540-8a5130ca722f // indirect
)

require (
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
Expand Down
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -484,8 +484,6 @@ github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=
github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mgechev/revive v1.5.1 h1:hE+QPeq0/wIzJwOphdVyUJ82njdd8Khp4fUIHGZHW3M=
Expand Down
6 changes: 0 additions & 6 deletions internal/cmdutil/cmdutil.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,6 @@ import (
"strconv"
"strings"
"unicode"

"github.com/mattn/go-shellwords"
)

var ErrCommandIsEmpty = fmt.Errorf("command is empty")
Expand Down Expand Up @@ -95,10 +93,6 @@ func SplitCommandWithEval(cmd string) (string, []string, error) {
return "", nil, err
}

parser := shellwords.NewParser()
parser.ParseBacktick = true
parser.ParseEnv = false

for _, command := range pipeline {
if len(command) < 2 {
continue
Expand Down
Loading

0 comments on commit 235fe63

Please sign in to comment.