Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] Add some initramfs debugging basics #796

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

[Draft] Add some initramfs debugging basics #796

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
df2799e
Add some initramfs debugging basics
fernflower Aug 30, 2022
3cb7a6a
[fixup] Add information about vagrant/libvirt pereference
fernflower Aug 31, 2022
35ea51c
[fixup] Add chroot howto
fernflower Sep 1, 2022
54f638a
[fixup] Apply suggestions from mreznik
fernflower Sep 5, 2022
c9b7be8
[squashme] remount /sysroot
fernflower Nov 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/source/debugging.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,31 @@ It's possible to go minimal and debug actor execution with remote debugger like
3. In a separate console connect to the debugger via network utility of your choice. The default port is 4444.

```nc localhost 4444```


### Initramfs

One of the biggest debugging challenges is exploring something in initramfs stage, as currently there is no network
connectivity (this might change soon though).

1. (can be skipped if you already ended up with an emergency console)
To get access to the emergency console right after leapp execution in initramfs stage has finished you should add an
`rd.break=leapp-upgrade` argument to the kernel commandline. One way to do this is by changing the code of the
fernflower marked this conversation as resolved.
Show resolved Hide resolved
[addupgradebootentry actor](https://github.com/oamg/leapp-repository/blob/master/repos/system_upgrade/common/actors/addupgradebootentry/libraries/addupgradebootentry.py#L23)

2. (can be skipped if you don't need any extra binaries) [TBD] Information on how to include additional binaries into
the initramfs

3. To get access to the common binaries change `PATH` accordingly. Setting
fernflower marked this conversation as resolved.
Show resolved Hide resolved
`PATH="$PATH:/sysroot/bin:/sysroot/sbin:/sysroot/usr/bin"` should do the trick.
If `/sysroot` is not mounted at the time you may need to mount it manually, e.g. with `systemctl start sysroot.mount`.

4. If the binaries are complaining about missing shared libraries, you could either set `LD_LIBRARY_PATH` variable
to `LD_LIBRARY_PATH=/sysroot/lib64:/lib64` or change root to /sysroot: `chroot /sysroot`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before chroot, kernel interfaces such as /dev /sys /proc should be mounted into the /sysroot. Otherwise some commands, e.g lsblk won't work.

I am not sure if you assume this knowledge or if it should be in the docs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should definitely be in the docs, assumed knowledge is always knowledge lost :)
Do you have a ready to run command to share?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can look at the leapp_dbg_chroot funtion in #997, but I am not 100% sure all of them are included here.

Maybe we can wait until #997 is merged and then just point to the function

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chroot is problematic from this POV (depends, what everything you would like to do inside and what are expectations). from the point of upgrades, we should always run it in the same way as leapp, creating the systemd-nspawn container with the all parameters.


5. If you need to make changes to files on /sysroot, make sure you remount it `mount -o remount,rw /sysroot`

6. [TBD] Put info how to collect the logs

> **_NOTE:_** When working in initramfs stage you will need a serial console. Though openstack machines can provide
you with a novnc console, unless you need a shared dev environment consider using vagrant/libvirt.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fernflower This is not good place to add this information. Keep in mind you are adding information about a initramfs debugging that is completely irrelevant for the framework. This is the framework documentation, not documentation for the content provided by leapp-repositories. If any debugging instrusctionts regarding that should be provided, it should not be here.

Copy link
Member Author

@fernflower fernflower Sep 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is related to leapp as a whole, not repositories only, and as leapp has been historically containing general debugging docs (like how to debug actors with rpdb information you see above) I decided to put initramfs debugging there as well.
I want this information to be public but I am not opposed to moving it, just tell where should I move it to?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can find el7toel8 directory inside docs. You need to come up where under docs specific for our upgrade repositories we should put it. We have already a task to separate leapp & leapp-repository docs, however there was no time to deal with it yet.