ssh topology scanner initial implementation #557

FrimIdan · 2023-08-16T09:13:25Z

Description

The PR introduce a new family (infoFinder) and with it the new sshTopology scanner.
This is the first implementation to address #549.
I've run the cli (not connected to the server yet) on 2 test machines, client ssh and server ssh.

Client run output:

time="2023-08-16T07:57:44Z" level=info msg="Device is mounted. Device=/dev/xvdh1 MountPoint=/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42" app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="Scanning is in progress" app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="Running scanners..." app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="InfoFinder Run..." app=vmclarity family="info finder"
time="2023-08-16T07:57:44Z" level=info msg="Running with input=/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42 and source type=rootfs" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T07:57:44Z" level=info msg="Found home user dirs [/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user]" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T07:57:44Z" level=info msg="Running command: /usr/bin/ssh-keygen -E sha256 -l -f /mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user/.ssh/authorized_keys" family="info finder" scanner=sshTopology app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="Running command: /usr/bin/ssh-keygen -E sha256 -l -f /mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user/.ssh/known_hosts" family="info finder" scanner=sshTopology app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="Found ssh authorized keys fingerprints [{Type:sshAuthorizedKeys Path:/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user/.ssh/authorized_keys Data:2048 SHA256:YQuPOM8ld6FOA9HbKCgkCJWHuGt4aTRD7hstjJpRhxc idan-key-pair (RSA)}]" family="info finder" scanner=sshTopology app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="Found ssh known hosts fingerprints [{Type:sshKnownHosts Path:/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user/.ssh/known_hosts Data:256 SHA256:gv6snCwAl5+6fY2g5VkmETWb9Mv0zLRkMz8aQyQWAVc ec2-3-64-214-52.eu-central-1.compute.amazonaws.com (ED25519)} {Type:sshKnownHosts Path:/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user/.ssh/known_hosts Data:256 SHA256:cDmm4+e/BNwQpsk/Qhh39i2qiT6HcIs6qTLtIiMWzPg ec2-3-64-214-52.eu-central-1.compute.amazonaws.com (ECDSA)}]" family="info finder" scanner=sshTopology app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="Found ssh private keys paths [/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user/.ssh/id_rsa]" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T07:57:44Z" level=info msg="Running command: /usr/bin/ssh-keygen -E sha256 -l -f /mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user/.ssh/id_rsa" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T07:57:44Z" level=info msg="Found ssh private keys fingerprints [{Type:sshPrivateKeys Path:/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/home/ec2-user/.ssh/id_rsa Data:3072 SHA256:hl8YpVK8lTppCDSE7nTknvZHX/63kjwA77hqspESH/w [email protected] (RSA)}]" family="info finder" scanner=sshTopology app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="Found ssh daemon private keys paths [/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/etc/ssh/ssh_host_ecdsa_key /mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/etc/ssh/ssh_host_ed25519_key]" family="info finder" scanner=sshTopology app=vmclarity
time="2023-08-16T07:57:44Z" level=info msg="Running command: /usr/bin/ssh-keygen -E sha256 -l -f /mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/etc/ssh/ssh_host_ecdsa_key" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T07:57:44Z" level=info msg="Running command: /usr/bin/ssh-keygen -E sha256 -l -f /mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/etc/ssh/ssh_host_ed25519_key" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T07:57:44Z" level=info msg="Found ssh daemon private keys fingerprints [{Type:sshDaemonKeys Path:/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/etc/ssh/ssh_host_ecdsa_key Data:256 SHA256:fD5lHsrP3KuQI+x+UQEcbIjvUcW0yyNt+vll1X0rw+E [email protected] (ECDSA)} {Type:sshDaemonKeys Path:/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42/etc/ssh/ssh_host_ed25519_key Data:256 SHA256:LqznK3bfmgvLe+I9UWvDsgfp+h42KLKxUQhItQ2vahs [email protected] (ED25519)}]" scanner=sshTopology app=vmclarity family="info finder"
time="2023-08-16T07:57:44Z" level=info msg="Got result for job \"sshTopology\"" app=vmclarity family="info finder"
time="2023-08-16T07:57:46Z" level=info msg="Merging result from \"sshTopology\"" app=vmclarity family="info finder"
time="2023-08-16T07:57:46Z" level=info msg="InfoFinder Done..." app=vmclarity family="info finder"
time="2023-08-16T07:57:46Z" level=info msg="Writing results to /var/opt/vmclarity/infofinder.json..."
time="2023-08-16T07:57:46Z" level=info msg="Scan has been completed" app=vmclarity

{
  "Metadata": {
    "Timestamp": "2023-08-16T07:57:46.304241719Z",
    "Scanners": [
      "sshTopology"
    ],
    "InputScans": [
      {
        "InputType": "rootfs",
        "InputPath": "/mnt/snapshots/6d4864e4-7a11-4008-ba7f-4afbbafd8f42",
        "InputSize": 1635,
        "ScanStartTime": "2023-08-16T07:57:44.458682031Z",
        "ScanEndTime": "2023-08-16T07:57:44.514284648Z"
      }
    ]
  },
  "Infos": [
    {
      "ScannerName": "sshTopology",
      "type": "sshAuthorizedKeys",
      "path": "/home/ec2-user/.ssh/authorized_keys",
      "data": "2048 SHA256:YQuPOM8ld6FOA9HbKCgkCJWHuGt4aTRD7hstjJpRhxc idan-key-pair (RSA)"
    },
    {
      "ScannerName": "sshTopology",
      "type": "sshKnownHosts",
      "path": "/home/ec2-user/.ssh/known_hosts",
      "data": "256 SHA256:gv6snCwAl5+6fY2g5VkmETWb9Mv0zLRkMz8aQyQWAVc ec2-3-64-214-52.eu-central-1.compute.amazonaws.com (ED25519)"
    },
    {
      "ScannerName": "sshTopology",
      "type": "sshKnownHosts",
      "path": "/home/ec2-user/.ssh/known_hosts",
      "data": "256 SHA256:cDmm4+e/BNwQpsk/Qhh39i2qiT6HcIs6qTLtIiMWzPg ec2-3-64-214-52.eu-central-1.compute.amazonaws.com (ECDSA)"
    },
    {
      "ScannerName": "sshTopology",
      "type": "sshPrivateKeys",
      "path": "/home/ec2-user/.ssh/id_rsa",
      "data": "3072 SHA256:hl8YpVK8lTppCDSE7nTknvZHX/63kjwA77hqspESH/w [email protected] (RSA)"
    },
    {
      "ScannerName": "sshTopology",
      "type": "sshDaemonKeys",
      "path": "/etc/ssh/ssh_host_ecdsa_key",
      "data": "256 SHA256:fD5lHsrP3KuQI+x+UQEcbIjvUcW0yyNt+vll1X0rw+E [email protected] (ECDSA)"
    },
    {
      "ScannerName": "sshTopology",
      "type": "sshDaemonKeys",
      "path": "/etc/ssh/ssh_host_ed25519_key",
      "data": "256 SHA256:LqznK3bfmgvLe+I9UWvDsgfp+h42KLKxUQhItQ2vahs [email protected] (ED25519)"
    }
  ]
}

Server run output:

time="2023-08-16T08:06:42Z" level=info msg="Device is mounted. Device=/dev/xvdh1 MountPoint=/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c" app=vmclarity
time="2023-08-16T08:06:42Z" level=info msg="Scanning is in progress" app=vmclarity
time="2023-08-16T08:06:42Z" level=info msg="Running scanners..." app=vmclarity
time="2023-08-16T08:06:42Z" level=info msg="InfoFinder Run..." app=vmclarity family="info finder"
time="2023-08-16T08:06:42Z" level=info msg="Running with input=/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c and source type=rootfs" scanner=sshTopology app=vmclarity family="info finder"
time="2023-08-16T08:06:42Z" level=info msg="Found home user dirs [/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/home/ec2-user]" scanner=sshTopology app=vmclarity family="info finder"
time="2023-08-16T08:06:42Z" level=info msg="File (/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/home/ec2-user/.ssh/known_hosts) does not exist." app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T08:06:42Z" level=info msg="Found ssh known hosts fingerprints []" family="info finder" scanner=sshTopology app=vmclarity
time="2023-08-16T08:06:42Z" level=info msg="Running command: /usr/bin/ssh-keygen -E sha256 -l -f /mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/home/ec2-user/.ssh/authorized_keys" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T08:06:42Z" level=info msg="Found ssh private keys paths []" family="info finder" scanner=sshTopology app=vmclarity
time="2023-08-16T08:06:42Z" level=info msg="Found ssh private keys fingerprints []" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T08:06:42Z" level=info msg="Found ssh daemon private keys paths [/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/etc/ssh/ssh_host_ecdsa_key /mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/etc/ssh/ssh_host_ed25519_key]" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T08:06:42Z" level=info msg="Running command: /usr/bin/ssh-keygen -E sha256 -l -f /mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/etc/ssh/ssh_host_ecdsa_key" scanner=sshTopology app=vmclarity family="info finder"
time="2023-08-16T08:06:42Z" level=info msg="Found ssh authorized keys fingerprints [{Type:sshAuthorizedKeys Path:/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/home/ec2-user/.ssh/authorized_keys Data:3072 SHA256:hl8YpVK8lTppCDSE7nTknvZHX/63kjwA77hqspESH/w [email protected] (RSA)} {Type:sshAuthorizedKeys Path:/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/home/ec2-user/.ssh/authorized_keys Data:2048 SHA256:YQuPOM8ld6FOA9HbKCgkCJWHuGt4aTRD7hstjJpRhxc idan-key-pair (RSA)}]" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T08:06:42Z" level=info msg="Running command: /usr/bin/ssh-keygen -E sha256 -l -f /mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/etc/ssh/ssh_host_ed25519_key" app=vmclarity family="info finder" scanner=sshTopology
time="2023-08-16T08:06:42Z" level=info msg="Found ssh daemon private keys fingerprints [{Type:sshDaemonKeys Path:/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/etc/ssh/ssh_host_ecdsa_key Data:256 SHA256:cDmm4+e/BNwQpsk/Qhh39i2qiT6HcIs6qTLtIiMWzPg [email protected] (ECDSA)} {Type:sshDaemonKeys Path:/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c/etc/ssh/ssh_host_ed25519_key Data:256 SHA256:gv6snCwAl5+6fY2g5VkmETWb9Mv0zLRkMz8aQyQWAVc [email protected] (ED25519)}]" scanner=sshTopology app=vmclarity family="info finder"
time="2023-08-16T08:06:42Z" level=info msg="Got result for job \"sshTopology\"" family="info finder" app=vmclarity
time="2023-08-16T08:06:44Z" level=info msg="Merging result from \"sshTopology\"" app=vmclarity family="info finder"
time="2023-08-16T08:06:44Z" level=info msg="InfoFinder Done..." app=vmclarity family="info finder"
time="2023-08-16T08:06:44Z" level=info msg="Writing results to /var/opt/vmclarity/infofinder.json..."
time="2023-08-16T08:06:44Z" level=info msg="Scan has been completed" app=vmclarity

{
  "Metadata": {
    "Timestamp": "2023-08-16T08:06:44.260207052Z",
    "Scanners": [
      "sshTopology"
    ],
    "InputScans": [
      {
        "InputType": "rootfs",
        "InputPath": "/mnt/snapshots/923088ed-4080-49e5-8dee-f2fa5268df8c",
        "InputSize": 1653,
        "ScanStartTime": "2023-08-16T08:06:42.498016968Z",
        "ScanEndTime": "2023-08-16T08:06:42.516981101Z"
      }
    ]
  },
  "Infos": [
    {
      "ScannerName": "sshTopology",
      "type": "sshAuthorizedKeys",
      "path": "/home/ec2-user/.ssh/authorized_keys",
      "data": "3072 SHA256:hl8YpVK8lTppCDSE7nTknvZHX/63kjwA77hqspESH/w [email protected] (RSA)"
    },
    {
      "ScannerName": "sshTopology",
      "type": "sshAuthorizedKeys",
      "path": "/home/ec2-user/.ssh/authorized_keys",
      "data": "2048 SHA256:YQuPOM8ld6FOA9HbKCgkCJWHuGt4aTRD7hstjJpRhxc idan-key-pair (RSA)"
    },
    {
      "ScannerName": "sshTopology",
      "type": "sshDaemonKeys",
      "path": "/etc/ssh/ssh_host_ecdsa_key",
      "data": "256 SHA256:cDmm4+e/BNwQpsk/Qhh39i2qiT6HcIs6qTLtIiMWzPg [email protected] (ECDSA)"
    },
    {
      "ScannerName": "sshTopology",
      "type": "sshDaemonKeys",
      "path": "/etc/ssh/ssh_host_ed25519_key",
      "data": "256 SHA256:gv6snCwAl5+6fY2g5VkmETWb9Mv0zLRkMz8aQyQWAVc [email protected] (ED25519)"
    }
  ]
}

We can see the fingerprints relationship between the ssh daemon keys to the known_hosts and the private keys to authorized_keys.

Type of Change

[ ] Bug Fix
[X] New Feature
[ ] Breaking Change
[ ] Refactor
[ ] Documentation
[ ] Other (please describe)

Checklist

I have read the contributing guidelines
Existing issues have been referenced (where applicable)
I have verified this change is not present in other open pull requests
Functionality is documented
All code style checks pass
New code contribution is covered by automated tests
All new and existing tests pass

Dockerfile.cli

.families.yaml

pkg/shared/families/infofinder/sshtopology/scanner.go

pkg/shared/families/infofinder/types/scannerResult.go

pkg/shared/families/infofinder/testdata/rootfolder/.ssh/not_a_key

pkg/shared/families/infofinder/sshtopology/scanner.go

pkg/shared/families/infofinder/types/config.go

pkg/shared/families/infofinder/types/scannerResult.go

pkg/shared/families/infofinder/sshtopology/scanner.go

ssh topology scanner initial implementation

1c8a293

FrimIdan requested a review from a team as a code owner August 16, 2023 09:13

FrimIdan added 3 commits August 16, 2023 12:18

add missing licenses

6b21184

fix some lint issues

7c51187

remove bad unit test

12cc935

Tehsmash reviewed Aug 17, 2023

View reviewed changes

change infofinders to infofinder

87eeb03

fishkerez reviewed Aug 20, 2023

View reviewed changes

FrimIdan added 3 commits August 27, 2023 09:54

singular InfoType

db857d5

singular InfoType

de0444e

avoid using buffered channels

69cb8ad

FrimIdan requested review from Tehsmash and fishkerez August 30, 2023 06:33