Skip to content

Commit

Permalink
Dockerfile for 1brc
Browse files Browse the repository at this point in the history
  • Loading branch information
radughiorma committed Jan 14, 2024
1 parent 7235968 commit f0cc867
Show file tree
Hide file tree
Showing 5 changed files with 113 additions and 2 deletions.
13 changes: 13 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
.github
.git
.idea
*.log
*.zip
measurements.txt
measurements_1B.txt
out_expected.txt
out_expected_1B.txt
target/

Dockerfile
.env
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,4 @@ out/
# 1BRC
/measurements*.txt
/*.out
out_expected.txt
out_expected*.txt
78 changes: 78 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#
# Copyright 2023 The original authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# syntax=docker/dockerfile:1
# Use an official Fedora Linux as a base image
FROM fedora:latest as build-sdkman
ARG JAVA_VERSIONS
# Maintain a variable providing a list of required packages (for easier maintenance)
ENV _PACKAGES="curl zip unzip"
# Maintain a variable providing a list of java versions to always install (for easier maintenance)
ENV _JAVA_VERSIONS="21.0.1-open"
# Install necessary packages
RUN dnf -y update && dnf -y install ${_PACKAGES}
# Clear DNF caches to reduce image size
RUN dnf clean all
# Install SDKMAN!
RUN curl -s "https://get.sdkman.io" | bash
# this SHELL command is needed to allow using source
SHELL ["/bin/bash", "-c"]
# Look for SDKMAN and load it into the shell session
RUN echo "source /root/.sdkman/bin/sdkman-init.sh" >> /root/.bashrc
# Install default java versions
RUN for JAVA_VERSION in $_JAVA_VERSIONS; do source "/root/.sdkman/bin/sdkman-init.sh" && sdk install java $JAVA_VERSION; done
# Install jbang
RUN source "/root/.sdkman/bin/sdkman-init.sh" && sdk install jbang
# Install SDK versions
RUN for JAVA_VERSION in $JAVA_VERSIONS; do source "/root/.sdkman/bin/sdkman-init.sh" && sdk install java $JAVA_VERSION; done

FROM fedora:latest as build-1brc
ARG PACKAGES
# Maintain a variable providing a list of required packages (for easier maintenance)
ENV _PACKAGES="diffutils perl-Digest-SHA hyperfine jq numactl bc gcc zlib-devel git"

COPY --from=build-sdkman /root/.sdkman /root/.sdkman
# Source sdkman in all shells (alternative to adding to PATH)
# this SHELL command is needed to allow using source
SHELL ["/bin/bash", "-c"]
# Look for SDKMAN and load it into the shell session
RUN echo "source /root/.sdkman/bin/sdkman-init.sh" >> /root/.bashrc
# Manually add sdkman to PATH
ENV PATH="/root/.sdkman/bin:$PATH"
# Install necessary packages
RUN dnf -y update && dnf -y install ${_PACKAGES}
# Install requested extra build packages
RUN if [ -n "$PACKAGES" ] ; then dnf -y install $PACKAGES ; else echo "PACKAGES is empty, run docker build --build_arg PACKAGES=<extra packages to install in the image> ." ; fi
# Clear DNF caches to reduce image size
RUN dnf clean all

ARG REPOSITORY="https://github.com/gunnarmorling/1brc.git"
# Checkeout the repository
RUN git clone $REPOSITORY

WORKDIR 1brc
# Initialize SDKMAN and download project dependencies to work offline
RUN source "/root/.sdkman/bin/sdkman-init.sh" \
# Cache local repo
&& ./mvnw dependency:go-offline \
# Build the project
&& ./mvnw package

FROM build-1brc
# Set the working directory in the container to /app
WORKDIR /1brc
# Copy the files from the workspace
COPY / .
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,26 @@ or directly on the .java file:
When you run this, it will generate a flamegraph in profile.html. You can then open this in a browser and see where your program is spending its time.
## Building the Docker image
Build the image by default it will install java 21.0.1-open.
❗To keep the image size reasonable it will not generate the measurements_1B.txt and out_expected.txt files
```shell
docker build . -t 1brc
```
To configure the build image you can pass the following build_arg
* --build-arg JAVA_VERSION=\<additional java distribuitions to install>
* --build-arg PACKAGES=\<additional packages to install with dnf>
```shell
docker build -build-arg JAVA_VERSIONS="21.0.1-graal 21.0.1-graalce" . -t 1brc
```
When running the container you need to specify the measurements_1B.txt and out_expected.txt files.
```shell
docker run -it -v ${pwd}/measurements_1B.txt:/1brc/measurements_1B.txt -v ${pwd}/out_expected_1B.txt:/1brc/out_expected.txt 1brc
```

## Rules and limits

* Any of these Java distributions may be used:
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@
<exclude>**/.dontdelete</exclude>
<exclude>**/measurements*.txt</exclude>
<exclude>**/measurements*.out</exclude>
<exclude>out_expected.txt</exclude>
<exclude>out_expected*.txt</exclude>
<exclude>github_users.txt</exclude>
<!-- Cliff asked to be named as the copyright holder for his entry; -->
<exclude>src/main/java/dev/morling/onebrc/CalculateAverage_cliffclick.java</exclude>
Expand Down

0 comments on commit f0cc867

Please sign in to comment.