Missing Yarn Packages When Building Image Offline

[spyr0-sec]

Description:

During the ui-builder process, step [ui-builder 2/8] RUN yarn install fails as the two following packages are missing:

@rollup/rollup-linux-arm64-musl@npm:4.13.2
@rollup/rollup-linux-x64-musl@npm:4.13.2

I assume this is because they are not in the yarn cache?

We have not encountered this issue when building images of previous releases.

Component(s) Affected:

• UI
• API
• Neo4j
• PostgreSQL
• Data Collector (SharpHound, AzureHound)
• Other (tooling, documentation, etc.)

Steps to Reproduce:

1. docker build -f dockerfiles/bloodhound.Dockerfile

Expected Behavior:

Docker image builds with no error

Actual Behavior:

Docker build fails with error as seen above.

Screenshots/Code Snippets/Sample Files:

If applicable, add screenshots, relevant code snippets, or sample files that help illustrate the issue.

Environment Information:

BloodHound: v5.8.1

Collector: N/A

OS: Ubuntu 22.04

Browser (if UI related): N/A

Node.js (if UI related: N/A

Go (if API related): N/A

Database (if persistence related): N/A

Docker (if using Docker): v25.0.1

Additional Information:

Package info - https://socket.dev/npm/package/@rollup/rollup-linux-x64-musl

Potential Solution (Optional):

I have tried to download the zip and add it to the cache manually but that does not seem to work. Sorry I do not understand Yarn enough to investigate further. Each of the zips in the .yarn/cache folder also seem to be suffixed with GUIDs so assume they are locked somehow?

Related Issues:

N/A

Contributor Checklist:

• I have searched the issue tracker to ensure this bug hasn't been reported before or is not already being addressed.
• I have provided clear steps to reproduce the issue.
• I have included relevant environment information details.
• I have attached necessary supporting documents.
• I have checked that any JSON files I am attempting to upload to BloodHound are valid.

[StephenHinck]

@spyr0-sec - I tested this locally on main with the command docker build -f dockerfiles/bloodhound.Dockerfile .

Can you confirm you're still seeing this? If not, can you give some additional information about the steps you took so I can try to replicate?

[spyr0-sec]

Thanks for getting back to me @StephenHinck

I can confirm I am seeing this when building from the main and v5.8.1 branches:

Reverting back to v5.8.0 I can build the image with no error.

[spyr0-sec]

Hey @StephenHinck

Just as a heads up we are still experiencing this issue on the v5.9.0 pre-release branches. Apologies I assume its a fairly easy fix but I just don't understand enough about npm / yarn.

I've tried downloading the packages manually (https://www.npmjs.com/package/@rollup/rollup-linux-x64-musl/v/4.13.2) but can't seem to work out how to add them to the cache as the files within the repo are all prefixed with hashes etc so I assume there is a pipeline / process that I'm not aware of.

Thanks!

[StephenHinck]

@spyr0-sec, I understand. This must have something to do with our CI automation, so it's unlikely to "just fix itself," as we're likely missing some packages to pull for the yarn cache. My familiarity with NPM/yarn is similarly limited - we'll get this on the team's plate to look at, but I don't have a timeline presently.

[elikmiller]

@spyr0-sec these packages are indeed missing from the cache. Long-term we can make sure these are included!

Short-term, you should be able to fetch the missing packages over the internet when running docker build .... The error message you shared indicates that a networking issue is preventing you from doing so.

While we can't fully troubleshoot your issue I suggest peeking at https://stackoverflow.com/questions/31763418/docker-build-has-no-network-but-docker-run-has for suggestions or leads.

---

Edit: I totally missed that you are wanting to do offline builds (per the issue title) - apologies!

Locally, you can modify .yarnrc.yml and append the following configuration to the supportedArchitectures section.

supportedArchitectures:
  os:
    - current
    - darwin
    - linux
    - win32
  cpu:
    - current
    - x64
    - arm64

  libc: // <- you need to append this libc section and the three below options
    - current
    - "glibc"
    - "musl"

Then, running yarn locally should download the missing packages and place them into the cache for you - at which point they would be available for use when running docker build offline. This assumes you are okay running yarn online once. If not, you'll need to wait for a PR.

[spyr0-sec]

@elikmiller

Appreciate your assistance on this, I was able to manually import the dependencies which satisfied that build step.

Unfortunately I'm now seeing a new error:

> [ui-builder 3/8] RUN yarn build:
1.181 /bloodhound/packages/javascript/bh-shared-ui/node_modules/rollup/dist/shared/rollup.js:65
1.181             textEncoder ??= new TextEncoder();

Having zero knowledge of JS, I asked CoPilot which returned the following:

The error you're seeing is due to the use of the nullish coalescing assignment (??=) operator, which is a feature in more recent versions of JavaScript (ES2020). This operator assigns a value to a variable only if that variable is null or undefined.

I assume when I performed a yarn build on my online machine, it was running a different version of JS than what the docker image has? If this is the case do you know what version it has and I will rerun with the same.

[elikmiller]

@spyr0-sec You don't need to run yarn build locally - you just needed to have those missing dependencies in the .yarn/cache folder.

When running docker build offline, assuming those new cached packages are available, you should be able to complete the build. Hope this helps!

[spyr0-sec]

@elikmiller

Apologies this was an environmental issue again from our side (joys of an air-gapped environment!).

We are now able to build images again, I'll keep this ticket open until the rollup packages hopefully get merged into the release.

Thanks again for all your help on this!