From 940a4028dfbfee6fe3f3ec5a0d251c92bfb77b15 Mon Sep 17 00:00:00 2001 From: irenarindos Date: Wed, 20 Mar 2024 16:45:14 +0000 Subject: [PATCH 1/4] backport of commit ca5c55e9be4ee36eac0567b17c0ba8e5f7e66105 --- website/content/docs/concepts/auditing.mdx | 159 ++++++++++----------- 1 file changed, 77 insertions(+), 82 deletions(-) diff --git a/website/content/docs/concepts/auditing.mdx b/website/content/docs/concepts/auditing.mdx index 138e7fafa4..71536b7b0b 100644 --- a/website/content/docs/concepts/auditing.mdx +++ b/website/content/docs/concepts/auditing.mdx @@ -101,19 +101,40 @@ Any session recording metadata that is attached to the storage bucket is deleted The BSR (Boundary Session Recording) defines a hierarchical directory structure of files and a binary file format. It contains all the data transmitted between a user and a target during a single session. -Boundary creates the top-level directory of the BSR as `.bsr`. This top level directory contains session summary information and subdirectories for connections. +Boundary creates the top-level directory of the BSR as `.bsr`. This top level directory contains session summary +information and subdirectories for connections. A BSR connections directory contains a summary of connections, as well as inbound and outbound requests. If you use a multiplexed protocol, there are subdirectories for the channels. +Every directory contains a SHA256SUMS and SHA256SUMS.sig file, to be used for cryptographically verifying the contents of +that directory. The SHA256SUMS file contains rows of file names paired with a checksum for file contents. The +SHA256SUMS.sign is a copy of the SHA256SUMS file, signed with the BSR’s private key. For more information on verifying a +session recording, refer to [Validating the integrity of session recordings](/boundary/docs/operations/session-recordings/validate-session-recordings). + +The example BSR below is for a multiplexed session recording with the ID `sr_iNCdGSREeX`. The session recording contains one connection, +`cr_3bB78W53Y9`. Connection `cr_3bB78W53Y9` contains two channels, `chr_VUnVuVnITu` and `chr_nITuVUnVuV`. +The files in each directory are explained in the following sections. + ``` └── sr_iNCdGSREeX.bsr ├── SHA256SUM ├── SHA256SUM.sig ├── bsrKey.pub + ├── pubKeyBsrSignature.sign + ├── pubKeySelfSignature.sign + ├── session-meta.json + ├── session-recording-summary.json + ├── session-recording.meta + ├── wrappedBsrKey + ├── wrappedPrivKey ├── cr_3bB78W53Y9.connection │ ├── SHA256SUM │ ├── SHA256SUM.sig + │ ├── connection-recording-summary.json + │ ├── connection-recording.meta + │ ├── requests-inbound.data + │ ├── requests-outbound.data │ ├── chr_VUnVuVnITu.channel │ │ ├── SHA256SUM │ │ ├── SHA256SUM.sig @@ -132,50 +153,29 @@ If you use a multiplexed protocol, there are subdirectories for the channels. │ │ ├── messages-outbound.data │ │ ├── requests-inbound.data │ │ └── requests-outbound.data - │ ├── connection-recording-summary.json - │ ├── connection-recording.meta - │ ├── requests-inbound.data - │ └── requests-outbound.data - ├── cr_W53Y93bB78.connection - │ ├── SHA256SUM - │ ├── SHA256SUM.sig - │ ├── chr_uVVuUITnVn.channel - │ │ ├── SHA256SUM - │ │ ├── SHA256SUM.sig - │ │ ├── channel-recording-summary.json - │ │ ├── channel-recording.meta - │ │ ├── messages-inbound.data - │ │ ├── messages-outbound.data - │ │ ├── requests-inbound.data - │ │ └── requests-outbound.data - │ ├── connection-recording-summary.json - │ ├── connection-recording.meta - │ ├── requests-inbound.data - │ └── requests-outbound.data - ├── pubKeyBsrSignature.sign - ├── pubKeySelfSignature.sign - ├── session-meta.json - ├── session-recording-summary.json - ├── session-recording.meta - ├── wrappedBsrKey - └── wrappedPrivKey + │ └── + └── + + ``` ### BSR Session folder -``` -└── sr_iNCdGSREeX.bsr - ├── SHA256SUM - ├── SHA256SUM.sig - ├── bsrKey.pub - ├── cr_3bB78W53Y9.connection - ├── pubKeyBsrSignature.sign - ├── pubKeySelfSignature.sign - ├── session-meta.json - ├── session-recording-summary.json - ├── session-recording.meta - ├── wrappedBsrKey - └── wrappedPrivKey -``` + A BSR session folder contains the following files: +- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents +- `SHA256SUM.sig` is a signature of the plaintext `SHA256SUM` file created with the private key. +- `bsrKey.pub` is the public ed25519 key. +- `pubKeySelfSignature.sign` is a self-signature of the plaintext public ed25519 key created with its private key. +- `pubKeyBsrSignature.sign` is a signature of the plaintext public ed25519 key created with the BSR key. +- `wrappedBsrKey` is the BSR key wrapped by the external KMS AES-GCM key that you configure. +- `wrappedPrivKey` is the private ed25519 key wrapped by the external KMS AES-GCM key that you configure. +- `session-meta.json` is a JSON file that contains metadata about the session, including the session id, endpoint, +user, target, host, worker, and credentials used to access the target. The intention of this file is to provide all information +relevant to the recorded session so that the BSR provides a complete snapshot of a session even in the absence of the Boundary +control plane. +- `session-recording.meta` is a plaintext file that contains metadata about the session, including the session id, protocol, +and connection id. +- `session-recording-summary.json` is a JSON file that contains a summary of the session recording, including the session id, connection count, +start time, end time, and any errors encountered during recording of the session. `session-recording.meta` file example: @@ -185,6 +185,19 @@ protocol: BSSH connection: cr_3bB78W53Y9.connection ``` +`session-recording-summary.json` file example: + +``` +{ + "Id": "sr_iNCdGSREeX", + "ConnectionCount": 1, + "StartTime": "2023-09-19T15:05:39.343307163Z", + "EndTime": "2023-09-19T15:08:02.953159598Z", + "Errors": "" +} + +``` + `session-meta.json` file example: ``` @@ -258,29 +271,16 @@ connection: cr_3bB78W53Y9.connection } ``` -`session-recording.json` file example: - -``` -id: sr_iNCdGSREeX -protocol: BSSH -connection: cr_3bB78W53Y9.connection -``` - -`SHA256SUM` and `SHA256SUM.sig` files are used for cryptographically verifying the contents of this directory. -For more information on `*.sign`, `bsrKey.pub`, `wrappedBsrKey`, and `wrappedPrivKey` files, refer to [Validating the integrity of session recordings](/boundary/docs/operations/session-recordings/validate-session-recordings). - ### BSR Connection folder - -``` -└── cr_W53Y93bB78.connection - ├── SHA256SUM - ├── SHA256SUM.sig - ├── chr_uVVuUITnVn.channel - ├── connection-recording-summary.json - ├── connection-recording.meta - ├── requests-inbound.data - └── requests-outbound.data -``` + A BSR connection folder contains the following files: +- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents +- `SHA256SUM.sig` is a signature of the plaintext `SHA256SUM` file created with the private key. +- `connection-recording.meta` is a plaintext file that contains metadata about the connection, including the connection id, +requests seen, channels opened under the connection, and any errors seen. +- `connection-recording-summary.json` is a JSON file that contains a summary of the connection, including the connection id, +start time, end time, bytes up, bytes down, and any errors encountered during recording the connection. +- `requests-inbound.data` is a binary file containing all inbound SSH request messages transmitted for the connection. +- `requests-outbound.data` is a binary file containing all outbound SSH request messages transmitted for the connection. `connection-recording.meta` file example: @@ -289,6 +289,7 @@ id: cr_W53Y93bB78 requests: outbound requests: inbound channel: chr_uVVuUITnVn.channel +error: error message would be appear here ``` `connection-recording-summary.json` file example: @@ -305,22 +306,19 @@ channel: chr_uVVuUITnVn.channel } ``` -`*.data` files are binary files containing all data transmitted during a session. -`SHA256SUM` and `SHA256SUM.sig` files are used for cryptographically verifying the contents of this directory. - ### BSR Channel folder - -``` -└── chr_uVVuUITnVn.channel - ├── SHA256SUM - ├── SHA256SUM.sig - ├── channel-recording-summary.json - ├── channel-recording.meta - ├── messages-inbound.data - ├── messages-outbound.data - ├── requests-inbound.data - └── requests-outbound.data -``` + A BSR connection folder contains the following files: +- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents +- `SHA256SUM.sig` is a signature of the plaintext `SHA256SUM` file created with the private key. +- `channel-recording.meta` is a plaintext file that contains metadata about the channel, including the channel id, +inbound and outbound requests seen, and inbound and outbound messages seen. +- `channel-recording-summary.json` is a JSON file that contains a summary of the channel, including the channel id, +start time, end time, bytes up, bytes down, channel type, session program, subsystem name (if applicable), exec program (if applicable), +and file transfer direction (if applicable). +- `requests-inbound.data` is a binary file containing all inbound SSH request messages transmitted for the channel. +- `requests-outbound.data` is a binary file containing all outbound SSH request messages transmitted for the channel. +- `messages-inbound.data` is a binary file containing all inbound SSH data transmitted for the channel. +- `messages-outbound.data` is a binary file containing all outbound SSH data transmitted for the channel. `channel-recording.meta` file example: @@ -353,7 +351,4 @@ requests: inbound } ``` -`*.data` files are binary files containing all data transmitted during a session. -`SHA256SUM` and `SHA256SUM.sig` files are used for cryptographically verifying the contents of this directory. - For more information, refer to the [overview of configuring session recording](/boundary/docs/configuration/session-recording). From bb37d04c21e0cb0f80ed2904ce9d95727ef8f18b Mon Sep 17 00:00:00 2001 From: irenarindos Date: Wed, 20 Mar 2024 18:19:15 +0000 Subject: [PATCH 2/4] backport of commit 7d6aec05d3d9cbdb5bedf3a2bf4ae22a38990e61 --- website/content/docs/concepts/auditing.mdx | 6 +++--- .../session-recordings/validate-session-recordings.mdx | 3 ++- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/website/content/docs/concepts/auditing.mdx b/website/content/docs/concepts/auditing.mdx index 71536b7b0b..1deafe8f5e 100644 --- a/website/content/docs/concepts/auditing.mdx +++ b/website/content/docs/concepts/auditing.mdx @@ -161,7 +161,7 @@ The files in each directory are explained in the following sections. ### BSR Session folder A BSR session folder contains the following files: -- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents +- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents. - `SHA256SUM.sig` is a signature of the plaintext `SHA256SUM` file created with the private key. - `bsrKey.pub` is the public ed25519 key. - `pubKeySelfSignature.sign` is a self-signature of the plaintext public ed25519 key created with its private key. @@ -273,7 +273,7 @@ connection: cr_3bB78W53Y9.connection ### BSR Connection folder A BSR connection folder contains the following files: -- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents +- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents. - `SHA256SUM.sig` is a signature of the plaintext `SHA256SUM` file created with the private key. - `connection-recording.meta` is a plaintext file that contains metadata about the connection, including the connection id, requests seen, channels opened under the connection, and any errors seen. @@ -308,7 +308,7 @@ error: error message would be appear here ### BSR Channel folder A BSR connection folder contains the following files: -- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents +- `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents. - `SHA256SUM.sig` is a signature of the plaintext `SHA256SUM` file created with the private key. - `channel-recording.meta` is a plaintext file that contains metadata about the channel, including the channel id, inbound and outbound requests seen, and inbound and outbound messages seen. diff --git a/website/content/docs/operations/session-recordings/validate-session-recordings.mdx b/website/content/docs/operations/session-recordings/validate-session-recordings.mdx index 727c014ee5..14801d1b0e 100644 --- a/website/content/docs/operations/session-recordings/validate-session-recordings.mdx +++ b/website/content/docs/operations/session-recordings/validate-session-recordings.mdx @@ -56,4 +56,5 @@ Follow these steps to validate a session recording: 1. Unwrap `wrappedBsrKey` using the external KMS you configured to retrieve the BSR key. 2. Unwrap `wrappedPrivKey` using the external KMS you configured to retrieve the private key. 3. Use the BSR key or the private key to verify the `bsrKey.pub` key using `go-kms-wrapping` HmacSha256(...). -4. When the key is verified, use the `bsrKey.pub` key to verify the BSR SHA256SUM file using `go-kms-wrapping` ed25519.Sign(...). \ No newline at end of file +4. When the key is verified, use the `bsrKey.pub` key to verify the BSR SHA256SUM.sig file using `go-kms-wrapping` ed25519.Sign(...). +5. After verifying the SHA256SUM.sig file, use the `sha256` commandline tool to verify BSR checksums using `sha256sum -c SHA256SUM` From 1075fd7e729ec61a3c1fb31b0d79ae6b0ad670fc Mon Sep 17 00:00:00 2001 From: irenarindos Date: Wed, 20 Mar 2024 18:46:16 +0000 Subject: [PATCH 3/4] backport of commit 892dbe7e3f71076c560463cf6b58c379bee00258 --- website/content/docs/concepts/auditing.mdx | 4 ++-- .../session-recordings/validate-session-recordings.mdx | 5 ++++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/website/content/docs/concepts/auditing.mdx b/website/content/docs/concepts/auditing.mdx index 1deafe8f5e..9f193edc6a 100644 --- a/website/content/docs/concepts/auditing.mdx +++ b/website/content/docs/concepts/auditing.mdx @@ -173,7 +173,7 @@ user, target, host, worker, and credentials used to access the target. The inten relevant to the recorded session so that the BSR provides a complete snapshot of a session even in the absence of the Boundary control plane. - `session-recording.meta` is a plaintext file that contains metadata about the session, including the session id, protocol, -and connection id. +and a connection ids. For each connection id listed, there should be a corresponding connection directory in the session directory. - `session-recording-summary.json` is a JSON file that contains a summary of the session recording, including the session id, connection count, start time, end time, and any errors encountered during recording of the session. @@ -276,7 +276,7 @@ connection: cr_3bB78W53Y9.connection - `SHA256SUM.sig` is a plaintext file that contains rows of file names paired with a checksum for file contents. - `SHA256SUM.sig` is a signature of the plaintext `SHA256SUM` file created with the private key. - `connection-recording.meta` is a plaintext file that contains metadata about the connection, including the connection id, -requests seen, channels opened under the connection, and any errors seen. +requests seen, channel ids, and any errors seen. For each channel id listed, there should be a corresponding channel directory in the connection directory. - `connection-recording-summary.json` is a JSON file that contains a summary of the connection, including the connection id, start time, end time, bytes up, bytes down, and any errors encountered during recording the connection. - `requests-inbound.data` is a binary file containing all inbound SSH request messages transmitted for the connection. diff --git a/website/content/docs/operations/session-recordings/validate-session-recordings.mdx b/website/content/docs/operations/session-recordings/validate-session-recordings.mdx index 14801d1b0e..7c5b5522e3 100644 --- a/website/content/docs/operations/session-recordings/validate-session-recordings.mdx +++ b/website/content/docs/operations/session-recordings/validate-session-recordings.mdx @@ -57,4 +57,7 @@ Follow these steps to validate a session recording: 2. Unwrap `wrappedPrivKey` using the external KMS you configured to retrieve the private key. 3. Use the BSR key or the private key to verify the `bsrKey.pub` key using `go-kms-wrapping` HmacSha256(...). 4. When the key is verified, use the `bsrKey.pub` key to verify the BSR SHA256SUM.sig file using `go-kms-wrapping` ed25519.Sign(...). -5. After verifying the SHA256SUM.sig file, use the `sha256` commandline tool to verify BSR checksums using `sha256sum -c SHA256SUM` +5. After verifying the SHA256SUM.sig file, use the `sha256` commandline tool to verify BSR checksums using `sha256sum -c SHA256SUM`. +6. Examine the *-meta.json files in the directory. For session-meta.json, every connection logged in the meta file should +correspond to a connection folder in the directory. For a connection-meta.json, every channel logged in the meta file should +correspond to a channel folder in the directory.. From 4fc0d9fd165595383fa6a6ba5b338c8d459830b3 Mon Sep 17 00:00:00 2001 From: irenarindos Date: Wed, 20 Mar 2024 19:08:39 +0000 Subject: [PATCH 4/4] backport of commit d463252d0efa95651073b6bf07dc31c48f937b1d --- .../session-recordings/validate-session-recordings.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/docs/operations/session-recordings/validate-session-recordings.mdx b/website/content/docs/operations/session-recordings/validate-session-recordings.mdx index 7c5b5522e3..b017fddd42 100644 --- a/website/content/docs/operations/session-recordings/validate-session-recordings.mdx +++ b/website/content/docs/operations/session-recordings/validate-session-recordings.mdx @@ -58,6 +58,6 @@ Follow these steps to validate a session recording: 3. Use the BSR key or the private key to verify the `bsrKey.pub` key using `go-kms-wrapping` HmacSha256(...). 4. When the key is verified, use the `bsrKey.pub` key to verify the BSR SHA256SUM.sig file using `go-kms-wrapping` ed25519.Sign(...). 5. After verifying the SHA256SUM.sig file, use the `sha256` commandline tool to verify BSR checksums using `sha256sum -c SHA256SUM`. -6. Examine the *-meta.json files in the directory. For session-meta.json, every connection logged in the meta file should -correspond to a connection folder in the directory. For a connection-meta.json, every channel logged in the meta file should -correspond to a channel folder in the directory.. +6. Examine the *.meta files in the directory. For session-recording.meta, every connection logged in the meta file should +correspond to a connection folder in the directory. For a connection-recording.meta, every channel logged in the meta file should +correspond to a channel folder in the directory.