-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kdump_Remote_SSH_Support #1714
Kdump_Remote_SSH_Support #1714
Changes from 3 commits
f7cc192
28467ff
cdba0c9
31ad8b1
7695f58
40c6dec
2f05dc9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,226 @@ | ||
# kdump_Remote_SSH | ||
|
||
## High Level Design Document | ||
**Rev 0.1** | ||
|
||
## Table of Contents | ||
|
||
<!-- TOC depthFrom:2 depthTo:4 withLinks:1 updateOnSave:1 orderedList:0 --> | ||
|
||
- [High Level Design Document](#high-level-design-document) | ||
- [Table of Contents](#table-of-contents) | ||
- [List of Tables](#list-of-tables) | ||
- [Revision](#revision) | ||
- [Overview](#about-this-manual) | ||
- [Scope](#scope) | ||
- [Definitions/Abbreviations](#definitionsabbreviations) | ||
- [Table 1: Abbreviations](#table-1-abbreviations) | ||
- [Introduction](#introduction) | ||
- [Requirements Overview <a name="requirements-overview"></a>](#requirements-overview-a-namerequirements-overviewa) | ||
- [Functional Requirements <a name="functional-requirements"></a>](#functional-requirements-a-namefunctional-requirementsa) | ||
- [Configuration and Management Requirements](#configuration-and-management-requirements) | ||
- [SSH Key Generation Requirements](#ssh-key-generation-requirement) | ||
- [kdump Remote Architechture](#kdump-remote-architechture) | ||
- [Functional Description](#functional-description) | ||
- [Design Changes](#design-changes) | ||
- [Configurations and Management](#configuration-and-management) | ||
- [CONFIG_DB Enhancements](#config_db-enhancements) | ||
- [SAI API](#sai-api) | ||
- [CLI/Yang MOdel Enhancements](#cliyang-model-enhancements) | ||
- [CLI Enhancements](#cli-enhancements) | ||
- [CLI Configuration Commands](#cli-configuration-commands) | ||
- [CLI Show Commands](#cli-show-commands) | ||
- [YANG Enhancement](#yang-enhancements) | ||
- [Warmboot Requirements](#warmboot-requirements) | ||
- [Test](#test) | ||
- [Unit Test cases](#unit-test-cases) | ||
- [Links](#links) | ||
|
||
<!-- /TOC --> | ||
|
||
## List of Tables | ||
|
||
[Table 1: Abbreviations](#table-1-abbreviations) | ||
|
||
## Revision | ||
|
||
Rev | Date | Author | Change Description | ||
:---: | :-----: | :------: | :--------- | ||
0.1 | 06/05/2024 | Ghulam Bahoo | Initial version | ||
## Overview | ||
This document outlines the configuration and usage of the kdump remote feature with ssh for the SONiC. | ||
|
||
## Scope | ||
|
||
This document describes how to configure remote kdump feature in SONiC infrastructure. | ||
|
||
## Definitions/Abbreviations | ||
|
||
### Table 1: Abbreviations | ||
|
||
| **Term** | **Meaning** | | ||
| ----------- | ---------------------- | | ||
| SSH | Secure Shell | | ||
| kdump | Kernel Dump | | ||
| NFS | Network File System | | ||
|
||
## Introduction | ||
Kdump, a built-in Linux kernel feature, generates and stores a crash dump file in the event of a kernel panic. Currently SONiC lacks the functionality of storing dump files on a remote server. It offloads the storage from switch by saving the crash reports to a separate designated server for offline analysis. This feature extends existing kdump feature by enabling remote dumps via ssh protocol, allowing you to transfer kernel crash data to a designated remote server. | ||
|
||
## Requirements Overview <a name="requirements-overview"></a> | ||
|
||
### Functional Requirements <a name="functional-requirements"></a> | ||
This section describes the SONiC requirements for kdump remote feature. | ||
|
||
At a high level the following should be supported: | ||
1. The kernel core dump files must be stored on the a remote ssh server. | ||
### Configuration and Management Requirements | ||
|
||
- CLI support for configuring remote kdump feature enable/disable via ssh. | ||
- CLI support for configuring username and hostname of ssh server (username@server_address). | ||
- CLI support for configuring SSH private key path for ssh server (SSH_private_Key_Path). | ||
- CLI support for displaying crededentials of ssh server. | ||
- CLI support for displaying state of kdump remote feature (enable/disable). | ||
### SSH Key Generation Requirement | ||
The system should authenticate with the remote server using SSH keys for secure access. | ||
|
||
``` | ||
admin@sonic: ssh-keygen | ||
``` | ||
|
||
User be prompted to choose a location to save the key pair. By default, it saves the private key to ~/.ssh/id_rsa and the public key to ~/.ssh/id_rsa.pub. | ||
|
||
``` | ||
admin@sonic: ssh-copy-id username@server_address | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this the manual setup apart from configuring kdump CLI commands? can this be abstracted within kdump command? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, it is a manual setup apart from kdump CLI commands. The manual setup ensures clarity and avoids the complexity and prompts that might arise from using kdump CLI commands. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you handle this as part of the Click command handler? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, we can implement this through the CLI, but it would be more suitable for the user to generate the key and save it to the desired path, which can then be configured in the "kdump-tools" file |
||
``` | ||
|
||
This helps automate passwordless SSH logins by copying public key to authorized servers. | ||
|
||
## kdump Remote Architechture | ||
 | ||
|
||
## Functional Description | ||
|
||
### Design Changes | ||
|
||
The SONiC kernel core dump remote functionality can be divided into two categories: | ||
|
||
1. Kernel core-dump generation service | ||
2. Storing Kernel core-dump files remotely | ||
|
||
Current SONiC lacks remote kernel dump functionality. To add this feature, consider enabling kdump for remote storage. | ||
|
||
We are suggesting modifying the exisitng SONiC configuration as following sonic-buildimage files. | ||
|
||
1. build_debian.sh (Addition) | ||
|
||
- Required for kdump_remote_ssh_dump: Initialize network interfaces and enable DHCP upon kernel crash. | ||
|
||
2. files/scripts/network_setup.sh (New Addition) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you add some details for this new change? is this only for eth0 interface? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, this script initializes the eth0 interface and enables dhcp on it. |
||
- A script to initialize the network interfaces and enable DHCP on them. | ||
3. files/script/network_setup.hook (New Addition) | ||
|
||
## Configuration and Management | ||
This section describes all types of configuration and management related design. Example sub-sections for "CLI" and "Config DB" are given below. | ||
|
||
|
||
|
||
### CONFIG_DB Enhancements | ||
New attributes will be introduced to "KDUMP" table in ConfigDB for maintaining remote kdump configurations. Below is the schema for this table. | ||
|
||
``` | ||
KDUMP_TABLE:{{config}} | ||
"enabled" :{{"false"|"true"}} | ||
"memory" :{{string}} | ||
"num_dumps" :{{number}} | ||
"remote" :{{"false"|"true"}} | ||
"ssh_connection_string" :{{string}} | ||
"ssh_private_key_path" :{{string}} | ||
``` | ||
|
||
### SAI API | ||
|
||
No SAI API change or addition is needed for this HLD. | ||
|
||
|
||
### CLI/YANG model Enhancements | ||
### CLI Enhancements | ||
|
||
#### CLI Configuration Commands | ||
New SONiC CLI commands are introduced to configure remote kdump feature. | ||
|
||
``` | ||
admin@sonic: sudo config kdump remote enable | ||
``` | ||
This commans will configure remote kdump feature in SONiC. | ||
|
||
``` | ||
admin@sonic: sudo config kdump remote ssh_connection_string | ||
``` | ||
|
||
``` | ||
admin@sonic: sudo config kdump remote ssh_private_key_path | ||
``` | ||
|
||
``` | ||
admin@sonic: sudo config kdump remote disable | ||
``` | ||
This command disables the remote kdump feature. | ||
#### CLI Show Commands | ||
An existing SONiC CLI command is used to display the current remote kdump feature configuraitons. | ||
|
||
``` | ||
admin@sonic: show kdump config | ||
``` | ||
|
||
Example output from the above command: | ||
``` | ||
admin@sonic:~$ show kdump config | ||
Kdump administrative mode: Enabled | ||
Kdump operational mode: Ready | ||
Kdump memory reservation: 512 | ||
Maximum number of Kdump files: 3 | ||
remote: true | ||
Kdump remote server user@ip/hostname: [email protected] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Have we tested the remote server reachability via mgmt VRF? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, but I will test it and keep posted. |
||
Kdump private key file path for remote ssh connection: /home/admin/.ssh/id_rsa | ||
``` | ||
|
||
### YANG Enhancements | ||
|
||
``` | ||
leaf remote { | ||
description "SSH Remote Config"; | ||
|
||
type string { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can the type be boolean to represent true/false instead of string? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, I will do this. |
||
pattern "true|false"; | ||
} | ||
} | ||
|
||
leaf ssh_connection_string { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it possible to add more than one remote IPs? e.g if one of the server is not reachable, is it possible to choose the alternate server to transfer the kdump data? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No this is not possible according to my understanding. There is variable #SSH in kdump-config file in /etc/default/kdump-tools directory, it allows you to add only one remote IP at a time. |
||
type string { | ||
length 1..128; | ||
} | ||
} | ||
|
||
leaf ssh_private_key_path { | ||
type string { | ||
length 1..128; | ||
} | ||
} | ||
|
||
``` | ||
|
||
### Warmboot Requirements | ||
|
||
Configuring kdump feature always requires a cold reboot of the switch. Warmboot is not supported while generating a core file in the event of a kernel crash. | ||
|
||
## Test | ||
|
||
### Unit Test Cases | ||
- Enable/Disable remote kdump feature. | ||
- Add/Remove SSH_Connection_String i.e. username@hostname. | ||
- Add/Remove SSH_Private_Key_Path. | ||
|
||
## Links | ||
- [White Paper: Red Hat Crash Utility](https://people.redhat.com/anderson/crash_whitepaper/) | ||
- [crash utility help pages](https://people.redhat.com/anderson/help.html) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo. crededentials, please do a spell-checker for this HLD.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated