Adding secrets cat command and removing secrets get command

[aryanjassal]

Description

This PR tracks the status of implementing secrets cat command, which can be used to concatenate multiple secrets together.

Issues Fixed

• Relates to Implement secrets cat command #243
• Closes ENG-356

Tasks

• 1. Add secrets cat command
• 2. Remove secrets get command
• 3. Add support for concatenating multiple files from multiple vaults together
• 4. Redirect stdin to stdout if no arguments are provided
• 5. Add tests for new functionality

Final checklist

• Domain specific tests
• Full tests
• Updated inline-comment documentation
• Lint fixed
• Squash and rebased
• Sanity check the final build

[linear]

ENG-356 Implement `secrets cat` command

[tegefaulkes]

Some changes needed. I made some comments but also address what we discussed.

[tegefaulkes]

Weird, I replied to your question as part of a new review so it posted it twice.

[aryanjassal]

cat in UNIX, when being piped into another command, buffers the contents internally, accumulating stdout. Any errors written to stderr are printed immediately, and stdout is printed after all the files have been fully read. This results in the following output:

[aryanj@matrix-34xx:~]$ cat file1 nofile file2
content1
cat: nofile: No such file or directory
content2

[aryanj@matrix-34xx:~]$ cat file1 nofile file2 | cat
cat: nofile: No such file or directory
content1
content2

This implementation directly streams contents to stdout as we receive them. This results in the output being as follows:

[aryanj@matrix-34xx:~]$ polykey secrets cat vault:file1 vault:nofile vault:file2
content1
ErrorSecretsSecretUndefined: nofile: No such file or directory
content2

[aryanj@matrix-34xx:~]$ polykey secrets cat vault:file1 vault:nofile vault:file2 | cat
content1
ErrorSecretsSecretUndefined: nofile: No such file or directory
content2

It looks like the command is not doing anything, but if we redirect stderr to null in both cases, we get this:

[aryanj@matrix-34xx:~]$ cat file1 nofile file2 2> /dev/null
content1
content2

[aryanj@matrix-34xx:~]$ polykey secrets cat vault:file1 vault:nofile vault:file2 2> /dev/null
content1
content2

We currently don't detect if piping is being done or not, and we don't have a way of replicating this behaviour.

[CMCDragonkai]

cat in UNIX, when being piped into another command, buffers the contents internally, accumulating stdout. Any errors written to stderr are printed immediately, and stdout is printed after all the files have been fully read. This results in the following output:

[aryanj@matrix-34xx:~]$ cat file1 nofile file2
content1
cat: nofile: No such file or directory
content2

[aryanj@matrix-34xx:~]$ cat file1 nofile file2 | cat
cat: nofile: No such file or directory
content1
content2

This implementation directly streams contents to stdout as we receive them. This results in the output being as follows:

[aryanj@matrix-34xx:~]$ polykey secrets cat vault:file1 vault:nofile vault:file2
content1
ErrorSecretsSecretUndefined: nofile: No such file or directory
content2

[aryanj@matrix-34xx:~]$ polykey secrets cat vault:file1 vault:nofile vault:file2 | cat
content1
ErrorSecretsSecretUndefined: nofile: No such file or directory
content2

It looks like the command is not doing anything, but if we redirect stderr to null in both cases, we get this:

[aryanj@matrix-34xx:~]$ cat file1 nofile file2 2> /dev/null
content1
content2

[aryanj@matrix-34xx:~]$ polykey secrets cat vault:file1 vault:nofile vault:file2 2> /dev/null
content1
content2

We currently don't detect if piping is being done or not, and we don't have a way of replicating this behaviour.

1. The pipe buffering is still streamed, it's just dependent on chunk size.
2. You can detect if you are composed but that's a CLI concern not the PK agent concern.
3. Leave the pipe buffering to the CLI - it's done by default by the OS. You don't need to do it, you just switch modes.
4. Between agent to client you should still be streaming regardless of what the CLI is doing later.

[aryanjassal]

Between agent to client you should still be streaming regardless of what the CLI is doing later.

Yes, that is what I am currently doing. Brian mentioned writing the streamed contents we receive into a temporary file, and then stream the file's contents to stdout, but streaming the incoming data straight to stdout was much simpler and faster.

[tegefaulkes]

Try not to mix up what I said.

#282 (comment)

[aryanjassal]

All the checks have been passed, all the tasks have been finished, and the PR has also been approved. Merging.

[aryanjassal]

[CMCDragonkai]

Is it possible for you to disable your autosuggest before you record.

[aryanjassal]

Yes. I can actually edit the recording itself to remove the captured autosuggest lines. I will do this and upload the new recording when I next work.

[aryanjassal]

I have edited the GIF to not include autocompletion anymore. I have also sped it up a little bit. @CMCDragonkai

I will be attaching the raw .cast file for reference and archival purposes. The file will be attached as a text file as GitHub doesn't support attaching files with arbitrary extensions.

cat.cast