Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add note to documentation about shell expansion #623

Merged
merged 1 commit into from
Sep 15, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/jobs/arrays.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,21 @@ contain each filepath on a separate line and then pass it to the submit command
$ hq submit --each-line entries.txt ...
```

!!! tip
To directly use an environment variable in the submitted command, you have to make sure that it will be expanded
when the command is executed, not when the command is submitted. You should also execute the command in a bash script
if you want to specify it directly and not via a script file.

For example, the following command is **incorrect**, as it will expand `HQ_ENTRY` during submission (probably to an
empty string) and submit a command `ls `:
```bash
$ hq submit --each-line files.txt ls $HQ_ENTRY
```
To actually submit the command `ls $HQ_ENTRY`, you can e.g. wrap the command in apostrophes and run it in a shell:
```bash
$ hq submit --each-line files.txt bash -c 'ls $HQ_ENTRY'
```

### JSON array
You can also specify the source using a JSON array stored inside a file. HyperQueue will then create a task for each
item in the array and pass the item as a JSON string to the corresponding task using the environment variable `HQ_ENTRY`.
Expand Down