Skip to content

Commit

Permalink
Update examples
Browse files Browse the repository at this point in the history
  • Loading branch information
nntrn committed Mar 4, 2024
1 parent b114d7d commit 283d85d
Show file tree
Hide file tree
Showing 8 changed files with 378 additions and 157 deletions.
20 changes: 0 additions & 20 deletions _snippets/describe-objects.md

This file was deleted.

80 changes: 0 additions & 80 deletions data-transform.md

This file was deleted.

1 change: 1 addition & 0 deletions data-transform.md
2 changes: 1 addition & 1 deletion functions/summary.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# summary
# Summary

```jq
def grouped_summary($item):
Expand Down
25 changes: 25 additions & 0 deletions functions/text.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Text functions

## Recursively split strings w/ new lines

```jq
def split_newlines($s):
if ((type == "string") and (($s|tostring|split("\n")|length) > 1)?)
then ($s|tostring|split("[\\r\\n]+([\\s]+)?";"x"))
elif (type == "object") then to_entries
else $s end;
def recuse_split_newlines: walk(split_newlines(.)|from_entries? // .);
```

## Quoting

```jq
def squo: [39]|implode;
def squote($text): [squo,$text,squo]|join("");
def dquote($text): "\"\($text)\"";
def unsmart($text): $text | gsub("[“”]";"\"") | gsub("[’‘]";"'");
def unsmart: . | unsmart;
```
64 changes: 31 additions & 33 deletions functions/unroll.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

```jq
[leaf_paths as $path | {
"key": $path | map(tostring) | join("_"),
"value": getpath($path)
"key": $path | map(tostring) | join("_"),
"value": getpath($path)
}] | from_entries
```

```console
$ cat data/nested.json|jq '[leaf_paths as $path | {
"key": $path | map(tostring) | join("_"),
"value": getpath($path)
$ cat data/nested.json|jq '[leaf_paths as $path | {
"key": $path | map(tostring) | join("_"),
"value": getpath($path)
}] | from_entries
'
{
Expand All @@ -23,54 +23,52 @@ $ cat data/nested.json|jq '[leaf_paths as $path | {

```jq
def categorize:
# Returns "object", "array" or "scalar" to indicate the category
# of the piped element.
if type == "object" then "object"
elif type == "array" then "array"
else "scalar"
end;
# Returns "object", "array" or "scalar" to indicate the category
# of the piped element.
if type == "object" then "object"
elif type == "array" then "array"
else "scalar"
end;
def pluck($category):
# Plucks the children of a particular category from piped element.
if categorize != "object"
then empty
else to_entries[]
| select(.value | categorize == $category)
| .value
end;
# Plucks the children of a particular category from piped element.
if categorize != "object"
then empty
else to_entries[]
| select(.value | categorize == $category)
| .value
end;
def split:
# Splits the piped element's children into arrays, scalars, and objects
# and returns a meta object containing the children seperated by these
# keys. If the piped element is a scalar or array, this does not look
# at the children, but just returns that element in the meta object.
if categorize == "scalar" then { objects: [], arrays: [], scalars: [.] }
elif categorize == "array" then { objects: [], arrays: [.], scalars: [] }
else { objects: [pluck("object")], arrays : [pluck("array")], scalars: [pluck("scalar")] }
end;
# Splits the piped element's children into arrays, scalars, and objects
# and returns a meta object containing the children seperated by these
# keys. If the piped element is a scalar or array, this does not look
# at the children, but just returns that element in the meta object.
if categorize == "scalar" then { objects: [], arrays: [], scalars: [.] }
elif categorize == "array" then { objects: [], arrays: [.], scalars: [] }
else { objects: [pluck("object")], arrays : [pluck("array")], scalars: [pluck("scalar")] }
end;
def unwrap:
# Unwraps an array recursively, until the elements of the returned array
# Unwraps an array recursively, until the elements of the returned array
# are either scalars or objects but not arrays. If piped element is not
# an array, returns the element as is.
if type != "array" then .
elif length == 0 then empty
else .[] | unwrap
end;
def extract($category):
# Extracts the elements of a particular category from the piped in array.
# If the piped in element is not an array, this fn acts as filter to
# Extracts the elements of a particular category from the piped in array.
# If the piped in element is not an array, this fn acts as filter to
# only return the element if it is of the desired category.
unwrap | select(.| categorize == $category);
def unroll:
def unroll:
# Unrolls the passed in object recursively until only scalars are left.
# Returns a row for each leaf node of tree structure of the object and
# elements of the row would be all the scalars encountered at all the
# ancestor levels of this left node.
. | .result += .state.scalars
| .state.objects += [.state.arrays | extract("object")]
| .state.objects += [.state.arrays | extract("scalar")]
Expand Down
80 changes: 80 additions & 0 deletions general/data-transform.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Data transformation

## json2ini

Adapted from <a>https://stackoverflow.com/a/76665197</a>

Data used: [nested.json](data/nested.json)

```
{
"server": {
"atx": {
"user": "annie",
"port": 22
}
},
"storage": {
"nyc": {
"user": "nntrn",
"port": 22
}
}
}
```

```sh
jq --stream -nr ' reduce (inputs | select(has(1))) as [$path, $val]
( {}; .[$path[:-1] | join(".")][$path[-1]] = $val )
| to_entries[]
| "[\(.key)]", (.value | to_entries[] | "\(.key) = \(.value)" )
, ""'
```

Output

```
[server.atx]
user = annie
port = 22
[storage.nyc]
user = nntrn
port = 22
```

## tsv2json

Adapted from <a>https://stackoverflow.com/a/55996042</a>

Data used: [tmp.tsv](data/tmp.tsv)

```
foo bar baz
1 2 3
4 5 6
```

```sh
jq -R 'split("[\\s\\t]+";"x") as $head
| [inputs | split("[\\s\\t]+";"x")]
| map( . as $row | reduce (keys|.[]) as $x
( {}; . + {"\($head[$x])":$row[$x]} ) )' data/tmp.tsv
```

Output

```json
[
{
"foo": "1",
"bar": "2",
"baz": "3"
},
{
"foo": "4",
"bar": "5",
"baz": "6"
}
]
```
25 changes: 2 additions & 23 deletions general/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ $ jq '.[] |= if .attr2 then (.attr2 = "bax") else . end'
[[Source]](https://github.com/stedolan/jq/issues/873#issuecomment-125385615)


## Slurpfiles
## Slurp-file

```console
$ jq --slurpfile cars data/cars.json '{titanic: .[0:2],cars:$cars[][0:2]}' data/titanic.json
$ jq --slurpfile cars data/cars.json '{titanic: .[0:1], cars: $cars[][0:1]}' data/titanic.json
{
"titanic": [
{
Expand All @@ -68,16 +68,6 @@ $ jq --slurpfile cars data/cars.json '{titanic: .[0:2],cars:$cars[][0:2]}' data/
"Siblings_Spouses_Aboard": 1,
"Parents_Children_Aboard": 0,
"Fare": 7.25
},
{
"Survived": 1,
"Pclass": 1,
"Name": "Mrs. John Bradley (Florence Briggs Thayer) Cumings",
"Sex": "female",
"Age": 38,
"Siblings_Spouses_Aboard": 1,
"Parents_Children_Aboard": 0,
"Fare": 71.2833
}
],
"cars": [
Expand All @@ -91,17 +81,6 @@ $ jq --slurpfile cars data/cars.json '{titanic: .[0:2],cars:$cars[][0:2]}' data/
"weight_lb_": 3821,
"0_60_mph_s_": 11,
"year": 1973
},
{
"name": "AMC Ambassador DPL",
"brand": "AMC",
"economy_mpg_": 15,
"cylinders": 8,
"displacement_cc_": 390,
"power_hp_": 190,
"weight_lb_": 3850,
"0_60_mph_s_": 8.5,
"year": 1970
}
]
}
Expand Down
Loading

0 comments on commit 283d85d

Please sign in to comment.