Design: CLI Output Policy

[mixmix]

I have an assumption that programmatic CLI output should be "easy to use in scripting" but that we are not aligned on what that means. The result is it keeps coming up in PR reviews.

Examples:

1. balance output:

1.2345 BITS

2. account create output:

Secret seed:        0x678d24823fe7513dd327151625e0718aed113082ffe879ebff6515b6d297fe3e
Public key (hex):   0x505e6b9d448edb98702304daa6eecec1111c6477983c8c0f16dd3976b238485d
AccountID:          0x505e6b9d448edb98702304daa6eecec1111c6477983c8c0f16dd3976b238485d
Public key (SS58):  5Dt5jgKvXymFVjrTAmcUpzr2wsHzBjsFvrF9JsLvYAWQc5Ch
SS58 Address:       5Dt5jgKvXymFVjrTAmcUpzr2wsHzBjsFvrF9JsLvYAWQc5Ch%   

Problem

If I was writing scripts using our CLI I want to be able to chain a bunch of commands together. To do this, I want outputs from CLI commands to be in some format that makes it easy for me to know how to parse, instead of having to write some custom parsing for each different output.

To me, the above examples are super annoying because sure they're nice to read as a human, but they are a pain to manipulate programmatically. We clearly have different expectations, and I really wanna align those so we have a clear idea of what we want to aim for and we don't keep having the same conversation

[mixmix]

Let's propose some patterns we could follow:

Proposal A

All programmatic CLI output should be parseable JSON.

Example outputs:

$ entropy balance mixmix
1.2314555

$ entropy balance mixmix --verbose
{ "amount": 1.2314555, "unit": "BITS" }

$ entropy account create mixmix
{ "seed": "0x678d24823fe7513dd327151625e0718aed113082ffe879ebff6515b6d297fe3e", "accountId": "0x505e6b9d448edb98702304daa6eecec1111c6477983c8c0f16dd3976b238485d", "SS58": "5Dt5jgKvXymFVjrTAmcUpzr2wsHzBjsFvrF9JsLvYAWQc5Ch" }

(maybe there could be a "simple" and "verbose" for this one too?)

[johnnymatthews]

Gonna copy your format here @mixmix:

Proposal B

All programmatic CLI output should be strings unless specifically told to output JSON using the verbose tag. JSON output should always be minified, and the user should rely on tools like JQ to prettify stuff if they want.

Example outputs:

Get balance:

$ entropy balance mixmix
1.2314555

$ entropy balance mixmix --verbose
{ "amount": 1.2314555, "unit": "BITS" }

Create account:

$ entropy account create mixmix
0x678d24823fe7513dd327151625e0718aed113082ffe879ebff6515b6d297fe3e

$ entropy account create mixmix --verbose
{ "seed": "0x678d24823fe7513dd327151625e0718aed113082ffe879ebff6515b6d297fe3e", "accountId": "0x505e6b9d448edb98702304daa6eecec1111c6477983c8c0f16dd3976b238485d", "SS58": "5Dt5jgKvXymFVjrTAmcUpzr2wsHzBjsFvrF9JsLvYAWQc5Ch" }

List accounts:

$ entropy account list
5GVv7vt82HKLQf2ErMtSm73H5ixK8apunEzWF74r7tzuNDr, 5Dt5jgKvXymFVjrTAmcUpzr2wsHzBjsFvrF9JsLvYAWQc5Ch

$ entropy account list --verbose
[{ "seed": "0x678d24823fe7513dd327151625e0718aed113082ffe879ebff6515b6d297fe3e", "accountId": "0x505e6b9d448edb98702304daa6eecec1111c6477983c8c0f16dd3976b238485d", "SS58": "5Dt5jgKvXymFVjrTAmcUpzr2wsHzBjsFvrF9JsLvYAWQc5Ch" }, {"seed": "9f38217c451a6d8e231f9432561a82752134567890abcdef0123456789abcdef0", "accountId": "1234567890abcdef01234567890abcdef01234567890abcdef0", "SS58": "9f38217c451a6d8e231f9432561a82752134567890abcdef0123456789abcdef0"}]

Notes

I'm not sure if --verbose is the correct term to use since it's always going to output JSON. Maybe we use --outputJSON or something like that. This way we can reserve the --verbose tag for something else.

[frankiebee]

I agree with @johnnymatthews although I would suggest these tags: -j/--json

[frankiebee]

Verbose should be reserved for logging or something else

[mixmix]

OOO, this is a super compelling and I think elegant solution to the tension @johnnymatthews
Can you speak to what you've seen other CLIs doing?

Bonus ideas

• i) could be a named output instead of a boolean flag
  • --output json
  • --output human
• ii) could set preference in config for users who always want to set output as json

  $ entropy set output json

[johnnymatthews]

Ya @frankiebee is right, -v / --verbose usually outputs bloody everything. Take rsync for example:

❯ rsync ./from/example_file_2 ./to/

Nothing was output.

❯ rsync from/example_file_2 ./to --verbose
example_file_2

sent 91 bytes  received 42 bytes  266.00 bytes/sec
total size is 0  speedup is 0.00

More information was output.

Based on this, we could use --verbose to output extra information; things like blocktime, cost for transation, transaction hash, all that good stuff.

---

i) could be a named output instead of a boolean flag -- @mixmix

What would human look like? If we're gonna support other formats, I think they should be recognized markup/down formats like YAML, XAML, TOML, CSV, etc.

---

could set preference in config -- @mixmix

Agreed! This would be a super useful option if/when we have multiple output options. We could likely have a bigger conversation about CLI personalization.

[rh0delta]

I'm all for standardizing the output for the CLI, but we should be a little pragmatic and not unify all outputs as the same. More specifically, the account output, I feel we should stick to the current output, as it follows the standards for what substrate users are already familiar with when generating addresses and keys outside of entropy.

Re: https://docs.substrate.io/reference/command-line-tools/subkey/

[mixmix]

Who are substrate users though @rh0delta ? I'd never heard of it, and I don't expect the average Entropy user to have heard of it. My problem with that output is I have zero idea what those outputs mean for me as an Entropy user. Is ... which one is my address? There seems to be repetition?

🙏 I'm trying to be pragmatic. Pragmatism is subjective! 😆