Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update riscv64 ubuntu to 22.04, the next lts version #1

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: build
on:
workflow_dispatch:
inputs:
version:
type: string
description: Node.js version to build
required: true
arch:
type: string
description: Architecture to build
default: riscv64
jobs:
build:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- name: prepare
run: bin/prepare-images.sh "fetch-source ${{ github.event.inputs.arch }}"
- name: create build directories
run: mkdir -p build/download build/logs build/staging build/.ccache
- name: build
run: bin/build.sh ${{ github.event.inputs.version }} ${{ github.event.inputs.arch }}
env:
workdir: build
- name: save logs
uses: actions/upload-artifact@v3
if: always()
with:
name: logs
path: build/logs
- name: save download
uses: actions/upload-artifact@v3
with:
name: download
path: build/download
- name: Release
uses: softprops/action-gh-release@v1
with:
tag_name: ${{ github.event.inputs.version }}
files: build/download/release/${{ github.event.inputs.version }}/*
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/workdir
18 changes: 16 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,26 @@ This list of officially supported platforms is available in the Node.js [BUILDIN

**unofficial-builds** attempts to provide basic Node.js binaries for some platforms that either not supported or only partially supported by Node.js. This project **does not provide any guarantees** and its results are not rigorously tested. Builds made available at nodejs.org have very high quality standards for code quality, support on the relevant platforms and for timing and methods of delivery. Builds made available by unofficial-builds have minimal or no testing; the platforms may have no inclusion in the official Node.js test infrastructure. These builds are made available for the convenience of their user community but those communities are expected to assist in their maintenance.

## Run

Execute the following commands to cross compile riscv64 node binaries. Select the node version on the `bin/build.sh` call.

```sh
./bin/prepare-images.sh "fetch-source riscv64"
mkdir -p build/download
mkdir -p build/logs
mkdir -p build/staging
mkdir -p build/.ccache
workdir=$PWD/build/ bin/build.sh v18.16.0 riscv64
```

## Builds

* **linux-x64-musl**: Linux x64 binaries compiled against [musl libc](https://www.musl-libc.org/) version 1.1.20. Primarily useful for users of Alpine Linux 3.9 and later. Linux x64 with musl is considered "Experimental" by Node.js but the Node.js test infrastructure includes some Alpine test servers so support is generally good. These Node.js builds require the `libstdc++` package to be installed on Alpine Linux, which is not installed by default. You can add this by running `apk add libstdc++`.
* **linux-x64-glibc-217**: Linux x64, compiled with glibc 2.17 to support [older Linux distros](https://en.wikipedia.org/wiki/Glibc#Version_history), QNAP QTS 4.x and 5.x, and Synology DSM 7, and other environments where a newer glibc is unavailable.
* **linux-x86**: Linux x86 (32-bit) binaries compiled against libc 2.17, similar to the way the official [linux-x64 binaries are produced](https://github.com/nodejs/node/blob/master/BUILDING.md#official-binary-platforms-and-toolchains). 32-bit Linux binaries were dropped for Node.js 10 and 32-bit support is now considered "Experimental".
* **linux-armv6l**: Linux ARMv6 binaries, cross-compiled on Ubuntu 16.04 with a [custom GCC 6 toolchain](https://github.com/rvagg/rpi-newer-crosstools) (for Node.js versions earlier than 16) or Ubuntu 18.04 with a [custom GCC 8 toolchain](https://github.com/rvagg/rpi-newer-crosstools) (for Node.js 16 and later) in a similar manner to the official linux-armv7l binaries. Binaries are optimized for `armv6zk` which is suitable for Raspberry Pi devices (1, 1+ and Zero in particular). ARMv6 binaries were dropped from Node.js 12 and ARMv6 support is now considered "Experimental".
* **riscv64**: Linux riscv64 (RISC-V), cross compiled on Ubuntu 20.04 with the toolchain which the Adoptium project uses (for now...). Built with --openssl-no-asm (Should be with --with-intl=none but that gets overriden)
* **riscv64**: Linux riscv64 (RISC-V), cross compiled on Ubuntu 22.04 with the ubuntu toolchain. Built with --openssl-no-asm (Should be with --with-intl=none but that gets overriden)

"Experimental" status for Node.js is defined as:
> Experimental: May not compile or test suite may not pass. The core team does not create releases for these platforms. Test failures on experimental platforms do not block releases. Contributions to improve support for these platforms are welcome.
Expand All @@ -41,7 +55,7 @@ The build process can be described as:

1. This repository is cloned onto the unofficial-builds server whenever it is updated (triggered via [github-webhook](https://github.com/rvagg/github-webhook)) and Docker images contained within the [`/recipes`](/recipes) directory are built by means of the [`/bin/deploy.sh`](/bin/deploy.sh) script which in turn calls the [`/bin/prepare-images.sh`](/bin/prepare-images.sh) script.
2. A periodic service runs every 5 minutes via systemd on the server which calls [`/bin/periodic.sh`](/bin/periodic.sh) script.
3. `periodic.sh` calls [`/bin/check-releases.sh`](/bin/check-releases.sh) for each release line being checked ("release", "rc", etc.). Any new versions that check-releases.sh finds are added to the build queue via [`/bin/queue-push.sh`](/bin/queue-push.sh) (the build queue uses a locking mechanism to prevent concurrent changes).
3. `periodic.sh` calls [`/bin/check-releases.sh`](/bin/check-releases.sh) for each release line being checked ("release", "rc", etc.). Any new versions that check-releases.sh finds are added to the build queue via [`/bin/queue-push.sh`](/bin/queue-push.sh) (the build queue uses a locking mechanism to prevent concurrent changes).
4. `periodic.sh` calls [`/bin/build-if-queued.sh`](/bin/build-if-queued.sh) which will execute a build if there is at least one build in the queue and no builds are currently running. [`/bin/queue-pop.sh`](/bin/queue-pop.sh) is used to atomically remove the next build from the queue. Note that only zero or one build per periodic run is executed. If the queue has more than one build, these will be deferred until later periodic runs.
5. When `build-if-queued.sh` encounters a build in the queue that it can execute, it calls [`/bin/build.sh`](/bin/build.sh) to perform the build. This script iterates through the images that have been pre-built from the [`/recipes`](/recipes) directory, starting with the [`/recipes/fetch-source`](/recipes/fetch-source) recipe that fetches the source file for the given version and validates official releases using GPG keys. Optionally, a recipe might have a `should-build` file which is used to determine if the recipe should run for a specific Node.js version. Each recipe is passed this source and is given a staging directory to place its binaries in. After all recipes are finished, builds are promoted to the <https://unofficial-builds.nodejs.org/download/> directory along with a SHASUMS256.txt file and the index.tab and index.json files for that release type are updated.

Expand Down
12 changes: 10 additions & 2 deletions bin/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ recipes=" \
musl \
armv6l \
armv6l-pre16 \
x64-glibc-217 \
x64-pointer-compression \
x64-usdt \
riscv64 \
Expand All @@ -29,6 +30,7 @@ if [[ "X${1}" = "X" ]]; then
fi

fullversion="$1"
shift
. ${__dirname}/_decode_version.sh
decode "$fullversion"
# see _decode_version for all of the magic variables now set and available for use
Expand Down Expand Up @@ -57,6 +59,9 @@ distdir_promote="${distdir}/${disttype_promote}"
distoutdir="${distdir_promote}/${fullversion}"
mkdir -p $distoutdir

# give write permission to all, to avoid docker permission issues
chmod 777 -R ${workdir}

# Build fetch-source, needs to be the first and must succeed
docker run --rm \
-v ${sourcedir}:/out \
Expand All @@ -65,16 +70,19 @@ docker run --rm \
> ${thislogdir}/fetch-source.log 2>&1

# Build all other recipes
for recipe in $recipes; do
for recipe in ${1:-$recipes}; do
# each recipe has 3 variable components:
# - individiaul ~/.ccache directory
# - individual ~/.ccache directory
# - a ~/node.tar.xz file that fetch-source has downloaded
# - an output /out directory that puts generated assets into a staging directory
ccachemount="-v ${ccachedir}/${recipe}/:/home/node/.ccache/"
mkdir -p "${ccachedir}/${recipe}"
sourcemount="-v ${sourcefile}:/home/node/node.tar.xz"
stagingmount="-v ${stagingoutdir}:/out"

# give write permission to all (again), to avoid docker permission issues
chmod 777 -R ${ccachedir}/${recipe}

shouldbuild="${__dirname}/../recipes/$recipe/should-build.sh"

if [ -f "$shouldbuild" ]; then
Expand Down
3 changes: 2 additions & 1 deletion bin/prepare-images.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
__dirname="$(CDPATH= cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
image_tag_pfx=unofficial-build-recipe-

for recipe in $(ls ${__dirname}/../recipes/); do
recipes=${1:-$(ls ${__dirname}/../recipes/)}
for recipe in $recipes; do
docker build ${__dirname}/../recipes/${recipe}/ -t ${image_tag_pfx}${recipe} --build-arg UID=1000 --build-arg GID=1000
done

Expand Down
11 changes: 5 additions & 6 deletions recipes/riscv64/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM ubuntu:20.04
FROM ubuntu:22.04

ARG GID=1000
ARG UID=1000
Expand All @@ -13,15 +13,14 @@ RUN apt-get update \
&& apt-get update \
&& apt-get install -y \
git \
g++-9 \
g++-9 \
curl \
make \
python3 \
python3-distutils \
python3-distutils \
ccache \
xz-utils

RUN curl https://ci.adoptopenjdk.net/userContent/riscv/riscv_toolchain_linux64.tar.xz | tar xJf - -C /opt
g++-riscv64-linux-gnu \
gcc-riscv64-linux-gnu

COPY --chown=node:node run.sh /home/node/run.sh

Expand Down
4 changes: 2 additions & 2 deletions recipes/riscv64/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ cd "node-${fullversion}"

export CC_host="ccache gcc-9"
export CXX_host="ccache g++-9"
export CC="ccache /opt/riscv_toolchain_linux/bin/riscv64-unknown-linux-gnu-gcc"
export CXX="ccache /opt/riscv_toolchain_linux/bin/riscv64-unknown-linux-gnu-g++"
export CC="ccache riscv64-linux-gnu-gcc"
export CXX="ccache riscv64-linux-gnu-g++"

make -j$(getconf _NPROCESSORS_ONLN) binary V= \
DESTCPU="riscv64" \
Expand Down
32 changes: 32 additions & 0 deletions recipes/x64-glibc-217/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
FROM centos:7

ARG GID=1000
ARG UID=1000

RUN groupadd --gid $GID node \
&& adduser --gid $GID --uid $UID node

COPY cloudlinux.repo /etc/yum.repos.d/cloudlinux.repo

RUN yum install -y epel-release \
&& yum upgrade -y \
&& yum install -y \
git \
curl \
make \
python2 \
python3 \
ccache \
xz-utils \
devtoolset-9 \
glibc-devel

COPY --chown=node:node run.sh /home/node/run.sh

VOLUME /home/node/.ccache
VOLUME /out
VOLUME /home/node/node.tar.xz

USER node

ENTRYPOINT [ "/home/node/run.sh" ]
9 changes: 9 additions & 0 deletions recipes/x64-glibc-217/cloudlinux.repo
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[cloudlinux-sclo-devtoolset-8]
name=Cloudlinux devtoolset-8
baseurl=https://repo.cloudlinux.com/cloudlinux/7/sclo/devtoolset-8/x86_64/
gpgcheck=0

[cloudlinux-sclo-devtoolset-9]
name=Cloudlinux devtoolset-9
baseurl=https://repo.cloudlinux.com/cloudlinux/7/sclo/devtoolset-9/x86_64/
gpgcheck=0
37 changes: 37 additions & 0 deletions recipes/x64-glibc-217/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/usr/bin/env bash

set -e
set -x

release_urlbase="$1"
disttype="$2"
customtag="$3"
datestring="$4"
commit="$5"
fullversion="$6"
source_url="$7"
config_flags=""

cd /home/node

tar -xf node.tar.xz
cd "node-${fullversion}"

export CC="ccache gcc"
export CXX="ccache g++"
export MAJOR_VERSION=$(echo ${fullversion} | cut -d . -f 1 | tr --delete v)

. /opt/rh/devtoolset-9/enable

make -j$(getconf _NPROCESSORS_ONLN) binary V= \
DESTCPU="x64" \
ARCH="x64" \
VARIATION="glibc-217" \
DISTTYPE="$disttype" \
CUSTOMTAG="$customtag" \
DATESTRING="$datestring" \
COMMIT="$commit" \
RELEASE_URLBASE="$release_urlbase" \
CONFIG_FLAGS="$config_flags"

mv node-*.tar.?z /out/
10 changes: 10 additions & 0 deletions recipes/x64-glibc-217/should-build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash -xe

__dirname=$1
fullversion=$2

. ${__dirname}/_decode_version.sh

decode "$fullversion"

test "$major" -ge "18"
Loading