Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hack: Add remote lldb utilities to hack dir #596

Merged
merged 1 commit into from
Jun 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions HACKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,23 @@ with the target container host) is to just run a webserver on the host, e.g.
run `bootc usroverlay` once, and
`curl -L -o /usr/bin/bootc http://10.0.1.2:8080/target/release/bootc && restorecon /usr/bin/bootc`.

### Debugging via lldb

The `hack/lldb` directory contains an example of how to use lldb to debug bootc code.
`hack/lldb/deploy.sh` can be used to build and deploy a bootc VM in libvirt with an lldb-server
running as a systemd service. Depending on your editor, you can then connect to the lldb server
to use an interactive debugger, and set up the editor to build and push the new binary to the VM.
`hack/lldb/dap-example-vim.lua` is an example for neovim.

The VM can be connected to via `ssh test@bootc-lldb` if you have [nss](https://libvirt.org/nss.html)
enabled.

For some bootc install commands, it's simpler to run the lldb-server in a container, e.g.

```bash
sudo podman run --pid=host --network=host --privileged --security-opt label=type:unconfined_t -v /var/lib/containers:/var/lib/containers -v /dev:/dev -v .:/output localhost/bootc-lldb lldb-server platform --listen "*:1234" --server
```

## Running the tests

First, you can run many unit tests with `cargo test`.
Expand Down
18 changes: 18 additions & 0 deletions hack/lldb/Containerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
FROM quay.io/centos-bootc/centos-bootc-dev:stream9

COPY ./etc/sysctl.conf /etc/sysctl.conf
COPY ./etc/systemd/system/lldb-server.service /etc/systemd/system/lldb-server.service
COPY ./etc/sudoers.d/wheel-nopasswd /etc/sudoers.d
ARG sshpubkey

RUN dnf -y install lldb firewalld vim && \
firewall-offline-cmd --add-port 1025-65535/tcp && \
Comment on lines +8 to +9
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, interesting that you're adding firewalld here only to poke a lot of holes in the firewall...any reason for that?

(In the near future we may want to actually add firewalld to our base images though)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

firewalld is just really simple to use. I was surprised it's not in the base image. lldb uses the user defined port for the initial connection along with seemingly any random port in the 30000-* range.

systemctl enable lldb-server.service && \

# add test user
if test -z "$sshpubkey"; then echo "must provide sshpubkey"; exit 1; fi; \
useradd -G wheel test && \
mkdir -m 0700 -p /home/test/.ssh && \
echo $sshpubkey > /home/test/.ssh/authorized_keys && \
chmod 0600 /home/test/.ssh/authorized_keys && \
chown -R test: /home/test
122 changes: 122 additions & 0 deletions hack/lldb/dap-example-vim.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
-- This is an example of how to configure the DAP connection in an editor (neovim in this case)
-- It should be relatively straightforward to adapt to a different editor

local dap = require("dap")
local job = require("plenary.job")

-- This is a coroutine that runs the cargo build command and reports progress
local program = function()
return coroutine.create(function(dap_run_co)
local progress = require("fidget.progress")

local cargo_build_fidget = progress.handle.create({
title = "cargo build",
lsp_client = { name = "Debugger" },
percentage = 0,
})

local cargo_build_job = job:new({
command = "cargo",
args = { "build", "--color=never", "--profile=dev" },
cwd = vim.fn.getcwd(),
enable_handlers = true,
on_stderr = vim.schedule_wrap(function(_, output)
cargo_build_fidget:report({
message = output,
percentage = cargo_build_fidget.percentage + 0.3,
})
end),
on_exit = function(_, return_val)
vim.schedule(function()
if return_val ~= 0 then
cargo_build_fidget:report({
message = "Error during cargo build",
percentage = 100,
})
else
cargo_build_fidget:finish()
coroutine.resume(dap_run_co, vim.fn.getcwd() .. "/target/debug/bootc")
end
end)
end,
})

cargo_build_job:start()
end)
end

dap.adapters = {
lldb = {
executable = {
args = {
"--liblldb",
"~/.local/share/nvim/mason/packages/codelldb/extension/lldb/lib/liblldb.so",
"--port",
"${port}",
},
command = "~/.local/share/nvim/mason/packages/codelldb/extension/adapter/codelldb",
},
host = "127.0.0.1",
port = "${port}",
type = "server",
},
}

-- rust config that runs cargo build before opening dap ui and starting Debugger
-- shows cargo build status as fidget progress
-- the newly built bootc binary is copied to the VM and run in the lldb-server
dap.configurations.rust = {
{
args = { "status" },
cwd = "/",
name = "[remote] status",
program = program,
request = "launch",
console = "integratedTerminal",
stopOnEntry = false,
type = "lldb",
initCommands = {
"platform select remote-linux",
"platform connect connect://bootc-lldb:1234", -- connect to the lldb-server running in the VM
"file target/debug/bootc",
},
},
{
args = { "upgrade" },
cwd = "/",
name = "[remote] upgrade",
program = program,
request = "launch",
console = "integratedTerminal",
stopOnEntry = false,
type = "lldb",
initCommands = {
"platform select remote-linux",
"platform connect connect://bootc-lldb:1234",
"file target/debug/bootc",
},
},

-- The install command can connect to a container instead of a VM.
-- The following command is an example of how to run a container and start a lldb-server:
-- sudo podman run --pid=host --network=host --privileged --security-opt label=type:unconfined_t -v /var/lib/containers:/var/lib/containers -v /dev:/dev -v .:/output localhost/bootc-lldb lldb-server platform --listen "*:1234" --server
{
args = { "install", "to-disk", "--generic-image", "--via-loopback", "--skip-fetch-check", "~/.cache/bootc-dev/disks/test.raw" },
cwd = "/",
env = {
["RUST_LOG"] = "debug",
["BOOTC_DIRECT_IO"] = "on",
},
name = "[remote] install to-disk",
program = program,
request = "launch",
console = "integratedTerminal",
stopOnEntry = false,
type = "lldb",
initCommands = {
"platform select remote-linux",
"platform connect connect://127.0.0.1:1234", -- connect to the lldb-server running in the container
"file target/debug/bootc",
},
},
}
21 changes: 21 additions & 0 deletions hack/lldb/deploy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some overlap here with #593 a bit and of course - a lot of overlap with https://github.com/containers/podman-bootc ...I was really thinking about trying to make that a more generic tool, but it gets tricky.

There's so many virtualization wrappers on Linux...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, eventually podman-bootc should provide everything needed for this, but in the meantime I just wanted to use lldb. The primary (only?) thing preventing using podman-bootc is the networking stuff.


# connect to the VM using https://libvirt.org/nss.html

set -e

# build the container image
sudo podman build --build-arg "sshpubkey=$(cat ~/.ssh/id_rsa.pub)" -f Containerfile -t localhost/bootc-lldb .

# build the disk image
mkdir -p ~/.cache/bootc-dev/disks
rm -f ~/.cache/bootc-dev/disks/lldb.raw
truncate -s 10G ~/.cache/bootc-dev/disks/lldb.raw
sudo podman run --pid=host --network=host --privileged --security-opt label=type:unconfined_t -v /var/lib/containers:/var/lib/containers -v ~/.cache/bootc-dev/disks:/output -v /dev:/dev localhost/bootc-lldb bootc install to-disk --via-loopback --generic-image --skip-fetch-check /output/lldb.raw

# create a new VM in libvirt
set +e
virsh -c qemu:///system destroy bootc-lldb
virsh -c qemu:///system undefine --nvram bootc-lldb
set -e
sudo virt-install --name bootc-lldb --cpu host --vcpus 8 --memory 8192 --import --disk ~/.cache/bootc-dev/disks/lldb.raw --os-variant rhel9-unknown
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this one we could use --qemu-commandline to pass the ssh key via credentials too right? This is probably something to do via a patch to virt-install.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, I was playing around with the now defunct included-ssh-pubkey example and got it working so I just continued using it. Either way works for me.

2 changes: 2 additions & 0 deletions hack/lldb/etc/sudoers.d/wheel-nopasswd
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Enable passwordless sudo for the wheel group
%wheel ALL=(ALL) NOPASSWD: ALL
2 changes: 2 additions & 0 deletions hack/lldb/etc/sysctl.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
net.ipv6.conf.all.disable_ipv6 = 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why out of curiosity?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a weird issue where the lldb connection was trying to use the ipv6 address and failing, this fixed it. I have ipv6 turned off on my host.

net.ipv6.conf.default.disable_ipv6 = 1
13 changes: 13 additions & 0 deletions hack/lldb/etc/systemd/system/lldb-server.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[Unit]
Description=LLDB Server
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/root
ExecStart=lldb-server platform --listen "*:1234" --server --min-gdbserver-port 31200 --max-gdbserver-port 31202
Restart=on-failure

[Install]
WantedBy=multi-user.target
Loading