Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: prebuilt librocksdb in docker image #2318

Merged
merged 29 commits into from
Nov 13, 2024
Merged
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
8fba694
ci: prebuild librocksdb in docker image
lklimek Nov 8, 2024
e056b9f
chore: fix dockerfile build logic
lklimek Nov 8, 2024
714d999
chore: move test image build to gh runners
lklimek Nov 8, 2024
35de1fa
chore: don't run build-rust-deps when not needed
lklimek Nov 8, 2024
c8871b5
chore: update rust-decimal to trigger workflow
lklimek Nov 8, 2024
efe95f0
chore: update rocsdb in drive-abci
lklimek Nov 8, 2024
cb28ec2
chore(dpp): fix some linter warn to trigger ci
lklimek Nov 8, 2024
242836c
chore: fix Dockerfile according to coderabbitai feedback
lklimek Nov 12, 2024
a8119ad
refactor(Dockerfile): put env vars into /root/env file that is copied…
lklimek Nov 12, 2024
d7c3342
build: update sccache in gha
lklimek Nov 12, 2024
ccd8426
chore: fix sccache
lklimek Nov 12, 2024
886a93d
chore: sccache not working correctly
lklimek Nov 12, 2024
6c98a9e
fix: build fails for some reason
lklimek Nov 12, 2024
5afd8fb
chore: minor cleanup
lklimek Nov 12, 2024
38f94fe
chore: sime fixes + install snappy
lklimek Nov 12, 2024
ecabc9e
chore: fix
lklimek Nov 12, 2024
07a0c46
chore: clang
lklimek Nov 12, 2024
bf2161c
chore: minor fixes
lklimek Nov 12, 2024
6740cb5
build: install snappy lib
lklimek Nov 13, 2024
939befa
Merge remote-tracking branch 'origin/v1.6-dev' into build/prebuild-ro…
lklimek Nov 13, 2024
19dc0f1
chore: self-review and some docs
lklimek Nov 13, 2024
54792fd
chore: replace /root/env with /etc/profile
lklimek Nov 13, 2024
c1abdb0
chore: revert some cargo.toml changes that are out of scope
lklimek Nov 13, 2024
f491026
chore: fix snappy in ubuntu x86-64
lklimek Nov 13, 2024
650103f
Revert "chore: replace /root/env with /etc/profile"
lklimek Nov 13, 2024
58ac034
build: move to github-hosted runners
lklimek Nov 13, 2024
cd55eaa
feat(Dockerfile): on amd64,. build rocksdb optimized for haswell cpu
lklimek Nov 13, 2024
9ec5437
Revert "build: move to github-hosted runners"
lklimek Nov 13, 2024
a1156e0
Merge remote-tracking branch 'origin/v1.6-dev' into build/prebuild-ro…
lklimek Nov 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: fix
lklimek committed Nov 12, 2024

Verified

This commit was signed with the committer’s verified signature.
emma-sg Emma Segal-Grossman
commit ecabc9e309cec1029a7d51691b88d7d96ffd77eb
27 changes: 6 additions & 21 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -185,15 +185,19 @@ RUN <<EOS
fi

if [ -n "${RUSTC_WRAPPER}" ]; then
echo "export CXX='${RUSTC_WRAPPER} clang++'" >> /root/env
echo "export CC='${RUSTC_WRAPPER} clang'" >> /root/env
echo "export CXX='${RUSTC_WRAPPER} cpp'" >> /root/env
echo "export CC='${RUSTC_WRAPPER} cc'" >> /root/env
echo "export RUSTC_WRAPPER='${RUSTC_WRAPPER}'" >> /root/env
echo "export SCCACHE_SERVER_PORT=$((RANDOM+1025))" >> /root/env
fi

cat /root/env
EOS

# Some security settings
ONBUILD ARG ACTIONS_RUNTIME_TOKEN
ONBUILD ARG AWS_SECRET_ACCESS_KEY

# Image containing compolation dependencies; used to overcome lack of interpolation in COPY --from
FROM deps-${RUSTC_WRAPPER:-base} AS deps-compilation
# Stage intentionally left empty
@@ -207,14 +211,10 @@ FROM deps-compilation AS deps-rocksdb
RUN mkdir -p /tmp/rocksdb
WORKDIR /tmp/rocksdb

ARG ACTIONS_RUNTIME_TOKEN
ARG AWS_SECRET_ACCESS_KEY

RUN git clone https://github.com/facebook/rocksdb.git -b v8.10.2 --depth 1 . && \
source /root/env && \
PORTABLE=1 make -j$(nproc) static_lib && \
mkdir -p /opt/rocksdb/usr/local/lib && \
strip librocksdb.a && \
cp librocksdb.a /opt/rocksdb/usr/local/lib/ && \
cp -r include /opt/rocksdb/usr/local/ && \
cd / && \
@@ -244,10 +244,6 @@ FROM deps-rocksdb AS deps

WORKDIR /platform

# Configure credentials requied by sccache
ARG ACTIONS_RUNTIME_TOKEN
ARG AWS_SECRET_ACCESS_KEY

# Install wasm-bindgen-cli in the same profile as other components, to sacrifice some performance & disk space to gain
# better build caching
RUN --mount=type=cache,sharing=shared,id=cargo_registry_index,target=${CARGO_HOME}/registry/index \
@@ -273,9 +269,6 @@ FROM deps AS build-planner
WORKDIR /platform
COPY . .

ARG ACTIONS_RUNTIME_TOKEN
ARG AWS_SECRET_ACCESS_KEY

RUN source $HOME/.cargo/env && \
source /root/env && \
cargo chef prepare --recipe-path recipe.json
@@ -296,10 +289,6 @@ WORKDIR /platform

COPY --from=build-planner /platform/recipe.json recipe.json

# Configure credentials requied by sccache
ARG ACTIONS_RUNTIME_TOKEN
ARG AWS_SECRET_ACCESS_KEY

# Build dependencies - this is the caching Docker layer!
RUN --mount=type=cache,sharing=shared,id=cargo_registry_index,target=${CARGO_HOME}/registry/index \
--mount=type=cache,sharing=shared,id=cargo_registry_cache,target=${CARGO_HOME}/registry/cache \
@@ -350,10 +339,6 @@ WORKDIR /platform

COPY --from=build-planner /platform/recipe.json recipe.json

# Configure credentials requied by sccache
ARG ACTIONS_RUNTIME_TOKEN
ARG AWS_SECRET_ACCESS_KEY

# Build dependencies - this is the caching Docker layer!
RUN --mount=type=cache,sharing=shared,id=cargo_registry_index,target=${CARGO_HOME}/registry/index \
--mount=type=cache,sharing=shared,id=cargo_registry_cache,target=${CARGO_HOME}/registry/cache \