Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build]: fix libc-bin core dump issue #9444

Closed
wants to merge 2 commits into from

Conversation

xumia
Copy link
Collaborator

@xumia xumia commented Dec 4, 2021

Why I did it

Fix the docker-slave-bullseye build break issue caused by installing libc-bin core dumpd

See https://dev.azure.com/mssonic/build/_build/results?buildId=55683&view=logs&j=715ada69-f859-5f7e-8bcf-2a94c492da59&t=883c835f-394d-5b66-f09e-4bca8e91e07f

(Reading database ... 179213 files and directories currently installed.)
Preparing to unpack .../libc-bin_2.31-13+deb11u2_arm64.deb ...
Unpacking libc-bin (2.31-13+deb11u2) over (2.31-3) ...
Setting up libc-bin (2.31-13+deb11u2) ...
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)
The command '/bin/sh -c apt upgrade -y base-files' returned a non-zero code: 100

Similar issue: microsoft/WSL#4760

How I did it

A workaround to install libc-bin:

sudo apt install libc-bin || true
sudo mv /var/lib/dpkg/info/libc-bin.* /tmp/
sudo dpkg --remove --force-remove-reinstreq libc-bin || true
sudo dpkg --purge libc-bin || true
sudo apt install libc-bin
sudo mv /tmp/libc-bin.* /var/lib/dpkg/info/

Need to add a step to upgrade the libc-bin, before applying the workaround, installing libc-bin twice.

Another workaround is not to upgrade the libc-bin, using the old version.

apt upgrade -y base-files libc-bin=$(dpkg-query -W -f '${Version}' libc-bin)

How to verify it

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

@xumia
Copy link
Collaborator Author

xumia commented Dec 4, 2021

/azp run Azure.docker-slave-bullseye-arm64

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@xumia
Copy link
Collaborator Author

xumia commented Dec 4, 2021

/azp run Azure.docker-slave-bullseye

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@xumia
Copy link
Collaborator Author

xumia commented Dec 4, 2021

/azp run Azure.sonic-buildimage

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@xumia
Copy link
Collaborator Author

xumia commented Dec 4, 2021

/azp run Azure.docker-slave-buster-armhf

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@xumia
Copy link
Collaborator Author

xumia commented Dec 4, 2021

/azp run Azure.docker-slave-buster-arm64

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@xumia
Copy link
Collaborator Author

xumia commented Dec 4, 2021

/azp run Azure.docker-slave-bullseye-armhf

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@xumia
Copy link
Collaborator Author

xumia commented Dec 5, 2021

@xumia
Copy link
Collaborator Author

xumia commented Dec 5, 2021

The good one was running on respberry PI, may be a little different from the one running on the amd64 server.

@xumia
Copy link
Collaborator Author

xumia commented Dec 5, 2021

Trying to use another way, keep the libc-bin not upgraded.

@xumia
Copy link
Collaborator Author

xumia commented Dec 6, 2021

@lguohan , works fine now, could you please help review again?

@saiarcot895
Copy link
Contributor

I think the issue is that the wrong qemu version might be getting used for building the slave image. I have changes to fix that in #9274 to ensure that the qemu interpreters get registered. The current code has an issue where if the multiarch container is already downloaded, then the qemu interpreters aren't registered, which means whatever default qemu versions are being used will get used, which isn't good enough for arm64.

See this pipeline which builds the arm64 slave container.

@xumia
Copy link
Collaborator Author

xumia commented Dec 6, 2021

@saiarcot895 , thanks for fixing the issue in #9274 , my pr can be closed.
You mean the line below to fix the issue, right?

DOCKER_MULTIARCH_CHECK := docker run --rm --privileged multiarch/qemu-user-static:register --reset --credential yes

@saiarcot895
Copy link
Contributor

That line and the call being added here:

sonic-slave-base-build : sonic-build-hooks
ifeq ($(MULTIARCH_QEMU_ENVIRON), y)
	@$(DOCKER_MULTIARCH_CHECK)
endif

That change causes the qemu interpreters to get always registered, instead of tying it to whether or not the multiarch/qemu-user-static:register is present. Perhaps it could first check if it's registered or not first, and perhaps what interpreter is being used (is it just the path of the binary, or is the binary loaded into kernel memory?). But I went with just calling it always, since it's a low-cost operation that will succeed.

@xumia xumia closed this Dec 7, 2021
@judyjoseph judyjoseph removed the Request for 202111 Branch For PRs being requested for 202111 branch label Dec 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants