Skip to content

Commit

Permalink
Initial commit for ycat
Browse files Browse the repository at this point in the history
  • Loading branch information
alxarch committed Feb 6, 2019
1 parent 1be7614 commit bcba269
Show file tree
Hide file tree
Showing 14 changed files with 808 additions and 386 deletions.
4 changes: 3 additions & 1 deletion .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
before:
hooks:
# # you may remove this if you don't use vgo
# - go mod download
- go mod download
# # you may remove this if you don't need go generate
# - go generate ./...
builds:
- env:
- CGO_ENABLED=0
main: ./cmd/ycat/main.go

archive:
replacements:
darwin: Darwin
Expand Down
132 changes: 83 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,90 +1,124 @@
# yjq
# ycat

YAML wrapper for [jq](https://stedolan.github.io/jq/) commandline JSON processor
Comand line processor for YAML/JSON files using [Jsonnet](https://jsonnet.org/)

```
Usage: yjq [JQ_ARG...]
yjq [options] [YAML_FILES...] -- [JQ_ARG...]
Usage: ycat [options|files...]
Options:
--cmd string Name of the jq command (default "jq")
-h, --help Show usage and exit
-i, --yaml-input Convert YAML input to JSON
-o, --yaml-output Convert JSON output to YAML
-h, --help Show help and exit
-y, --yaml [files...] Read YAML values from file(s)
-j, --json [files...] Read JSON values from file(s)
-o, --out {json|j|yaml|y} Set output format
--to-json Output JSON one value per line (same as -o json, -oj)
-a, --array Merge output values into an array
-e, --eval <snippet> Process values with Jsonnet
-m, --module <var>=<file> Load Jsonnet module into a local Jsonnet variable
--bind <var> Bind input value to a local Jsonnet variable (default _)
```

### Arguments

All non-option arguments are files to read from.
Format is guessed from extension and fallsback to YAML.

## Usage
If filename is `-` values are read from stdin using the last specified format or YAML

## Examples

Concatenate files to a single YAML stream (type is selected from extension)

```
yjq [JQ_ARG...]
$ ycat foo.yaml bar.yaml baz.json
```

By default all arguments are forwarded to `jq` and input/output is YAML. For example:
Concatenate files to single JSON stream (one item per-line)

```
$ echo 'name: foo' | yjq '.name+="_bar"'
name: foo_bar
$ ycat --to-json foo.yaml bar.yaml baz.json
$ ycat -o j foo.yaml bar.yaml baz.json
```

If there's a `--` argument, all arguments up to `--` are handled by `yjq`
and the rest are forwarded to `jq`.
Concatenate JSON values from `stdin` to a single YAML file

```
yjq [options] [YAML_FILES...] -- [JQ_ARG...]
$ ycat -j
```
For example:

Concatenate YAML from `a.txt`, `stdin`, `b.yaml` and JSON from `a.json`,

```
$ echo 'name: foo' | yjq -i -- '.name+="_bar"'
{
"name": "foo_bar"
}
$ ycat -y a.txt - b.txt -j a.json
```

### Options

- `-i`, `--yaml-input` Convert YAML input from `stdin` to JSON
- `-o`, `--yaml-output` Convert JSON output to YAML
- `--cmd <JQ>` Specify alternative jq command (default "jq")
Concatenate to YAML array

If neither `-i|--yaml-input` nor `-o|--yaml-output` is specified both input and output is YAML.
```
$ ycat -a a.json b.yaml
```

### Arguments

If `stdin` is not piped, `yjq` arguments are assumed to be YAML files to pipe to `jq` as JSON. If there are no arguments, interactive input from `stdin` is used (exit by sending `EOF` with `Ctrl-D`).
Concatenate YAML from `a.yaml` and `b.yaml` setting key `foo` to `bar` on each top level object

## YAML Input
```
$ ycat a.yaml b.yaml -e '_+{foo: "bar"}'
```

Multiple YAML values separated by `---` are passed to `jq` as separate values.
To combine them in an array use `jq`'s `-s|--slurp` option.
Add kubernetes namespace `foo` to all resources without namespace

```
$ ycat -e '_ + { metadata +: { namespace: if "namespace" in super then super.namespace else "foo" }}' *.yaml
```

## YAML Output
Process with [jq](http://stedolan.github.io/jq/) using a pipe

Results from `jq`'s output are separated by `---` in a single YAML value stream.
```
$ ycat -oj a.yaml b.json | jq ... | ycat
```

## Installation

Download an executable for your platform from github [releases]( https://github.com/alxarch/yjq/releases/latest).
Download an executable for your platform from github [releases]( https://github.com/alxarch/ycat/releases/latest).

Alternatively, assuming `$GOPATH/bin` is in `$PATH` just

```
go get github.com/alxarch/yjq
go get github.com/alxarch/ycat
```

## Caveats

When input is YAML `jq` argument `-R|--raw-input` is not supported and gets dropped.

When output is YAML the following `jq` arguments ara not supported and get dropped:
- `-r|--raw-output`
- `-a|--ascii-output`
- `-C|--color-output`
- `-j|--join-output`
- `--tab`
- `--indent`
- `-c|--compact-output`
## YAML Input

Multiple YAML values separated by `---\n` are processed separately.
Value reading stops at `...\n` or `EOF`.

## JSON Input

Multiple JSON values separated by whitespace are processed separately.
Value reading stops at `EOF`.

## YAML Output

Each result value is appended to the output with `---\n` separator.

## JSON Output

Each result value is appended into a new line of output.

## Jsonnet

[Jsonnet](https://jsonnet.org/) is a templating language from google that's really versatile in handling configuration files. Visit their site for more information.

Each value is bound to a local variable named `_` inside the snippet by default. Use `--bind` to change the name.

To use `Jsonnet` code from a file in the snippet use `-m <var>=<file>` and the exported value will be available as
a local variable in the snippet.

## TODO

- Add support for flow style YAML output, especially in long string values
- Intercept file arguments passed to `jq` and handle the ones with `.yaml,.yml` extension
- Add support for pretty printed output
- Add support for reading .txt files
- Add support for reading files as base64
- Add support for reading files as hex
- Add support for Jsonnet libraries
- Add support for sorting by JSONPath
Loading

0 comments on commit bcba269

Please sign in to comment.