Skip to content
This repository has been archived by the owner on Jul 17, 2023. It is now read-only.

Solve arm64 kraft errors #64

Closed

Conversation

razvanvirtan
Copy link
Contributor

@razvanvirtan razvanvirtan commented Jul 27, 2021

This PR comes to solve issue #57. There are two problems with kraft on ARM64, solved by two different commits:

  • kraft build uses aarch64-linux-gnu-gcc, which in turn calls the x86_64 assembler, resulting in a build error. At first, I was thinking about replacing the current arm64 toolchain with the linaro one (as mentioned in a previous comment). Finally, I managed to solve this by adding a missing directory (aarch64-linux-gnu) to the final kraft container. This way, we don't need to add linaro anymore.
    In order for this to work fine, I think we should also update the unikraft/gcc:9.2.0-arm64-staging image from docker hub[1].

  • kraft run is always trying to enable hardware acceleration, even if the host architecture is different from the emulated one. This generates errors when trying to emulate ARM64 images on x86 hosts. The problem can be easily solved by checking the architecture in kvm.py and generating the right qemu command.

[1]https://hub.docker.com/layers/unikraft/gcc/9.2.0-arm64/images/sha256-3fd1ce6b6f7acb9e259212bf922b8b2090f10cf0e93621f94f04c51a92a5835e?context=explore

Signed-off-by: Răzvan Vîrtan [email protected]

Until now, aarch64 gcc was trying to use the x86 assembler, generating
a build error. This commit adds the missing `aarch64-linux-gnu`
directory to the kraft container, solving this issue.
`kraft run` is always trying to enable hardware acceleration, even
when the host has an achitecture different from the emulated one.
This commit solves the problem, taking into account the host architecture
when generating the qemu command.
@nderjung
Copy link
Member

Hey @razvanvirtan!

Thanks for taking the time to look into this, really appreciated!

Looks like the dockerfile can't be built on the gcc image:

#10 [gcc 5/5] COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/
#10 sha256:6b47cccdb8e29e627f2f5a463965f290687ca5dd1e07107b9b531f339326a5ec
#10 ERROR: "/out/aarch64-linux-gnu" not found: not found
------
 > [gcc 5/5] COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/:
------
Dockerfile.gcc:146
--------------------
 144 |     COPY --from=gcc-build /out/lib/ /lib/
 145 |     COPY --from=gcc-build /out/libexec/ /libexec/
 146 | >>> COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/
 147 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = failed to compute cache key: "/out/aarch64-linux-gnu" not found: not found
FATA[1561] failed to build: build: exit status 1        
FATA[1561] failed to run task: exit status 1            

@razvanvirtan
Copy link
Contributor Author

Hey @razvanvirtan!

Thanks for taking the time to look into this, really appreciated!

Looks like the dockerfile can't be built on the gcc image:

#10 [gcc 5/5] COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/
#10 sha256:6b47cccdb8e29e627f2f5a463965f290687ca5dd1e07107b9b531f339326a5ec
#10 ERROR: "/out/aarch64-linux-gnu" not found: not found
------
 > [gcc 5/5] COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/:
------
Dockerfile.gcc:146
--------------------
 144 |     COPY --from=gcc-build /out/lib/ /lib/
 145 |     COPY --from=gcc-build /out/libexec/ /libexec/
 146 | >>> COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/
 147 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = failed to compute cache key: "/out/aarch64-linux-gnu" not found: not found
FATA[1561] failed to build: build: exit status 1        
FATA[1561] failed to run task: exit status 1            

I think I know where the issue comes from, will come with a third commit after testing it.

Before this commit, the gcc container image couldn't be built on arm or x86_64,
generating a `file not found error`.
Replace `aarch64-linux-gnu` in Dockerfile.gcc with the GCC_PREFIX that is
architecture specific. This way, the problematic folder will always be found.
This should also allow `kraft build` to cross-compile images on any architecture.
@razvanvirtan
Copy link
Contributor Author

razvanvirtan commented Jul 29, 2021

@nderjung The gcc images can be built now. However, we still have problems with the kraft one. Without seeing the check output, my guess is that the system tries to build the kraft image using gcc images from dockerhub:
https://github.com/unikraft/kraft/blob/29c6fba81b885d5fb3b5d2dc76b976644828df30/package/docker/Dockerfile.kraft#L36

Since those images have been built using the initial Dockerfile.gcc, they don't contain the directories added in this PR, so the kraft container build fails when it tries to copy x86_64-linux-gnu. I think we will need to update the gcc images from dockerhub before this to work. (I can build the kraft container on my local, if I'm using gcc images compiled with the updated Dockerfile.gcc).

@nderjung
Copy link
Member

nderjung commented Jul 29, 2021

Hey, apologies for not being able to see the CI output yet, we're still migrating services...

Here's the relevant log:

#24 [kraft 14/31] COPY --from=gcc-arm64 /aarch64-linux-gnu/ /aarch64-linux-gnu
#24 sha256:4fddea67477d34a2ce98cb9f68cb4ad316dafd1e6196132997b1e2a28e08767f
#24 CANCELED
------
 > [kraft 10/31] COPY --from=gcc-arm /arm-linux-gnueabihf  /arm-linux-gnueabihf:
------
Dockerfile.kraft:56
--------------------
  54 |     COPY --from=gcc-arm /lib/gcc/             /lib/gcc
  55 |     COPY --from=gcc-arm /libexec/gcc/         /libexec/gcc
  56 | >>> COPY --from=gcc-arm /arm-linux-gnueabihf  /arm-linux-gnueabihf
  57 |     
  58 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = failed to compute cache key: "/arm-linux-gnueabihf" not found: not found

@razvanvirtan
Copy link
Contributor Author

razvanvirtan commented Jul 29, 2021

Hey, apologies for not being able to see the CI output yet, we're still migrating services...

Here's the relevant log:

#24 [kraft 14/31] COPY --from=gcc-arm64 /aarch64-linux-gnu/ /aarch64-linux-gnu
#24 sha256:4fddea67477d34a2ce98cb9f68cb4ad316dafd1e6196132997b1e2a28e08767f
#24 CANCELED
------
 > [kraft 10/31] COPY --from=gcc-arm /arm-linux-gnueabihf  /arm-linux-gnueabihf:
------
Dockerfile.kraft:56
--------------------
  54 |     COPY --from=gcc-arm /lib/gcc/             /lib/gcc
  55 |     COPY --from=gcc-arm /libexec/gcc/         /libexec/gcc
  56 | >>> COPY --from=gcc-arm /arm-linux-gnueabihf  /arm-linux-gnueabihf
  57 |     
  58 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = failed to compute cache key: "/arm-linux-gnueabihf" not found: not found

Thanks!
Seems to confirm my initial thoughts. We could update these images:

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants