Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade JVM environment #382

Merged
merged 6 commits into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions .github/workflows/environment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ name: Environments build and test
on:
pull_request:
branches: [master]
env:
FISSION_CLI_VERSION: 1.20.3

jobs:
# Job to run change detection
check:
Expand Down Expand Up @@ -118,14 +121,14 @@ jobs:
with:
filters: .github/workflows/filters/filters.yaml
- name: Helm
uses: Azure/setup-helm@v1
uses: Azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # v3.5
with:
version: 'v3.0.0'
version: v3.13.0
- name: Kind Clutser
uses: engineerd/[email protected]
with:
image: kindest/node:v1.21.1
version: v0.11.1
image: kindest/node:v1.25.16
version: v0.23.0
config: kind.yaml
- name: Configuring and testing the Installation
run: |
Expand All @@ -134,12 +137,12 @@ jobs:
kubectl get nodes
- name: Install Fission CLI
run: |
curl -Lo fission https://github.com/fission/fission/releases/download/1.12.0/fission-cli-linux \
curl -Lo fission https://github.com/fission/fission/releases/download/${{ env.FISSION_CLI_VERSION }}/fission-${{ env.FISSION_CLI_VERSION }}-linux-amd64 \
&& chmod +x fission && sudo mv fission /usr/local/bin/
- name: jvm
uses: hiberbee/[email protected]
with:
skaffold-version: '1.39.2'
skaffold-version: '2.13.1'
command: run
profile: jvm
tag: latest
Expand Down
6 changes: 3 additions & 3 deletions jvm/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM maven:3.5-jdk-8 as BUILD
FROM maven:3.9.9-eclipse-temurin-22-alpine AS build
WORKDIR /usr/src/myapp/

# To reuse the build cache, here we split maven dependency
Expand All @@ -9,8 +9,8 @@ RUN mvn dependency:go-offline
COPY src /usr/src/myapp/src/
RUN mvn package

FROM openjdk:8-jdk-alpine
FROM eclipse-temurin:22-jdk-alpine
VOLUME /tmp
COPY --from=BUILD /usr/src/myapp/target/env-java-0.0.1-SNAPSHOT.jar /app.jar
COPY --from=build /usr/src/myapp/target/env-java-0.0.1-SNAPSHOT.jar /app.jar
ENTRYPOINT java ${JVM_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /app.jar --server.port=8888
EXPOSE 8888
2 changes: 1 addition & 1 deletion jvm/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
-include ../rules.mk

.PHONY: all
all: jvm-builder jvm-env-img
all: jvm-env-img

jvm-env-img: Dockerfile
4 changes: 2 additions & 2 deletions jvm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This is the JVM environment for Fission.

It's a Docker image containing a OpenJDK8 runtime, along with a
It's a Docker image containing a OpenJDK22 runtime by [Eclipse Temurin](https://hub.docker.com/_/eclipse-temurin), along with a
dynamic loader. A few dependencies are included in the
pom.xml file.

Expand Down Expand Up @@ -67,4 +67,4 @@ JVM environment uses Tomcat HTTP server by default as it is included in spring w

## Java and JVM builder

JVM environment builder is based on OpenJDK8 and Maven 3.5.4 version. The default build command runs `mvn clean package` and uses the target/*with-dependencies.jar file for function. The default build command can be overridden as long as the uber jar file is copied to ${DEPLOY_PKG}.
JVM environment builder is based on OpenJDK22 built by Eclipse Temurin and Maven 3.9.9 version. The default build command runs `mvn clean package` and uses the target/*with-dependencies.jar file for function. The default build command can be overridden as long as the uber jar file is copied to ${DEPLOY_PKG}.
117 changes: 84 additions & 33 deletions jvm/builder/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,50 +1,101 @@
## Fission builder base image
ARG BUILDER_IMAGE=fission/builder:latest
FROM ${BUILDER_IMAGE}
FROM ${BUILDER_IMAGE} AS builder

## Section copied from the openjdk:8-jdk-alpine Dockerfile - (https://github.com/docker-library/openjdk/blob/47a6539cd18023dafb45db9013455136cc0bca07/8/jdk/alpine/Dockerfile)
## Section copied from the eclipse-temurin:22-jdk-alpine Dockerfile - (https://github.com/adoptium/containers/blob/07677395574f5d3462c3b6fdf5f6c4a0a350b683/22/jdk/alpine/Dockerfile)

FROM alpine:3.8
FROM alpine:3.20

ENV LANG C.UTF-8
RUN { \
echo '#!/bin/sh'; \
echo 'set -e'; \
echo; \
echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"'; \
} > /usr/local/bin/docker-java-home \
&& chmod +x /usr/local/bin/docker-java-home
ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk
ENV PATH $PATH:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin
ENV JAVA_HOME=/opt/java/openjdk
ENV PATH=$JAVA_HOME/bin:$PATH

ENV JAVA_VERSION 8u181
## Use "fuzzy" version matching to pin the version to a major/minor release
ENV JAVA_ALPINE_VERSION "~8"
# Default to UTF-8 file.encoding
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'

RUN set -x \
&& apk add --no-cache \
openjdk8="$JAVA_ALPINE_VERSION" \
&& [ "$JAVA_HOME" = "$(docker-java-home)" ]
RUN set -eux; \
apk add --no-cache \
# java.lang.UnsatisfiedLinkError: libfontmanager.so: libfreetype.so.6: cannot open shared object file: No such file or directory
# java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11FontManager
# https://github.com/docker-library/openjdk/pull/235#issuecomment-424466077
fontconfig ttf-dejavu \
# utilities for keeping Alpine and OpenJDK CA certificates in sync
# https://github.com/adoptium/containers/issues/293
ca-certificates p11-kit-trust \
# locales ensures proper character encoding and locale-specific behaviors using en_US.UTF-8
musl-locales musl-locales-lang \
# jlink --strip-debug on 13+ needs objcopy: https://github.com/docker-library/openjdk/issues/351
# Error: java.io.IOException: Cannot run program "objcopy": error=2, No such file or directory
binutils \
tzdata \
# Contains `csplit` used for splitting multiple certificates in one file to multiple files, since keytool can
# only import one at a time.
coreutils \
# Needed to extract CN and generate aliases for certificates
openssl \
; \
rm -rf /var/cache/apk/*

ENV JAVA_VERSION=jdk-22.0.2+9

RUN set -eux; \
ARCH="$(apk --print-arch)"; \
case "${ARCH}" in \
aarch64) \
ESUM='8ac93a2d5a55aabbc0f7156c2f9032026e87c185689d628ef8a4184b6e9ab006'; \
BINARY_URL='https://github.com/adoptium/temurin22-binaries/releases/download/jdk-22.0.2%2B9/OpenJDK22U-jdk_aarch64_alpine-linux_hotspot_22.0.2_9.tar.gz'; \
;; \
x86_64) \
ESUM='49f73414824b1a7c268a611225fa4d7ce5e25600201e0f1cd59f94d1040b5264'; \
BINARY_URL='https://github.com/adoptium/temurin22-binaries/releases/download/jdk-22.0.2%2B9/OpenJDK22U-jdk_x64_alpine-linux_hotspot_22.0.2_9.tar.gz'; \
;; \
*) \
echo "Unsupported arch: ${ARCH}"; \
exit 1; \
;; \
esac; \
wget -O /tmp/openjdk.tar.gz ${BINARY_URL}; \
echo "${ESUM} */tmp/openjdk.tar.gz" | sha256sum -c -; \
mkdir -p "$JAVA_HOME"; \
tar --extract \
--file /tmp/openjdk.tar.gz \
--directory "$JAVA_HOME" \
--strip-components 1 \
--no-same-owner \
; \
rm -f /tmp/openjdk.tar.gz ${JAVA_HOME}/lib/src.zip;

RUN set -eux; \
echo "Verifying install ..."; \
fileEncoding="$(echo 'System.out.println(System.getProperty("file.encoding"))' | jshell -s -)"; [ "$fileEncoding" = 'UTF-8' ]; rm -rf ~/.java; \
echo "javac --version"; javac --version; \
echo "java --version"; java --version; \
echo "Complete."
COPY --chmod=755 entrypoint.sh /__cacert_entrypoint.sh
ENTRYPOINT ["/__cacert_entrypoint.sh"]

CMD ["jshell"]
## Section copied from the Maven Dockerfile

RUN apk add --no-cache curl tar bash procps
RUN apk add --no-cache bash procps curl tar openssh-client

ARG MAVEN_VERSION=3.5.4
ARG USER_HOME_DIR="/root"
ARG SHA=ce50b1c91364cb77efe3776f756a6d92b76d9038b0a0782f7d53acf1e997a14d
ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
LABEL org.opencontainers.image.title="Apache Maven"
LABEL org.opencontainers.image.source=https://github.com/carlossg/docker-maven
LABEL org.opencontainers.image.url=https://github.com/carlossg/docker-maven
LABEL org.opencontainers.image.description="Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information."

RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
&& echo "${SHA} /tmp/apache-maven.tar.gz" | sha256sum -c - \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
ENV MAVEN_HOME=/usr/share/maven

ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
COPY --from=maven:3.9.9-eclipse-temurin-11 ${MAVEN_HOME} ${MAVEN_HOME}
COPY --from=maven:3.9.9-eclipse-temurin-11 /usr/local/bin/mvn-entrypoint.sh /usr/local/bin/mvn-entrypoint.sh
COPY --from=maven:3.9.9-eclipse-temurin-11 /usr/share/maven/ref/settings-docker.xml /usr/share/maven/ref/settings-docker.xml

RUN ln -s ${MAVEN_HOME}/bin/mvn /usr/bin/mvn

ARG MAVEN_VERSION=3.9.9
ARG USER_HOME_DIR="/root"
ENV MAVEN_CONFIG="$USER_HOME_DIR/.m2"

## Fission builder specific section
COPY --from=builder /builder /builder
ADD build.sh /usr/local/bin/build
EXPOSE 8001
2 changes: 1 addition & 1 deletion jvm/builder/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
-include ../../rules.mk

.PHONY: all
all: jvm-env-builder-img
all: jvm-builder-img

jvm-builder-img : Dockerfile
104 changes: 104 additions & 0 deletions jvm/builder/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
#!/usr/bin/env sh
# ------------------------------------------------------------------------------
# NOTE: THIS FILE IS GENERATED VIA "generate_dockerfiles.py"
#
# PLEASE DO NOT EDIT IT DIRECTLY.
# ------------------------------------------------------------------------------
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This script defines `sh` as the interpreter, which is available in all POSIX environments. However, it might get
# started with `bash` as the shell to support dotted.environment.variable.names which are not supported by POSIX, but
# are supported by `sh` in some Linux flavours.

set -e

TMPDIR=${TMPDIR:-/tmp}

# JDK truststore location
JRE_CACERTS_PATH=$JAVA_HOME/lib/security/cacerts

# Opt-in is only activated if the environment variable is set
if [ -n "$USE_SYSTEM_CA_CERTS" ]; then

if [ ! -w "$TMPDIR" ]; then
echo "Using additional CA certificates requires write permissions to $TMPDIR. Cannot create truststore."
exit 1
fi

# Figure out whether we can write to the JVM truststore. If we can, we'll add the certificates there. If not,
# we'll use a temporary truststore.
if [ ! -w "$JRE_CACERTS_PATH" ]; then
# We cannot write to the JVM truststore, so we create a temporary one
JRE_CACERTS_PATH_NEW=$(mktemp)
echo "Using a temporary truststore at $JRE_CACERTS_PATH_NEW"
cp "$JRE_CACERTS_PATH" "$JRE_CACERTS_PATH_NEW"
JRE_CACERTS_PATH=$JRE_CACERTS_PATH_NEW
# If we use a custom truststore, we need to make sure that the JVM uses it
export JAVA_TOOL_OPTIONS="${JAVA_TOOL_OPTIONS} -Djavax.net.ssl.trustStore=${JRE_CACERTS_PATH} -Djavax.net.ssl.trustStorePassword=changeit"
fi

tmp_store=$(mktemp)

# Copy full system CA store to a temporary location
trust extract --overwrite --format=java-cacerts --filter=ca-anchors --purpose=server-auth "$tmp_store" > /dev/null

# Add the system CA certificates to the JVM truststore.
keytool -importkeystore -destkeystore "$JRE_CACERTS_PATH" -srckeystore "$tmp_store" -srcstorepass changeit -deststorepass changeit -noprompt > /dev/null

# Clean up the temporary truststore
rm -f "$tmp_store"

# Import the additional certificate into JVM truststore
for i in /certificates/*crt; do
if [ ! -f "$i" ]; then
continue
fi
tmp_dir=$(mktemp -d)
BASENAME=$(basename "$i" .crt)

# We might have multiple certificates in the file. Split this file into single files. The reason is that
# `keytool` does not accept multi-certificate files
csplit -s -z -b %02d.crt -f "$tmp_dir/$BASENAME-" "$i" '/-----BEGIN CERTIFICATE-----/' '{*}'

for crt in "$tmp_dir/$BASENAME"-*; do
# Create an alias for the certificate
ALIAS=$(openssl x509 -in "$crt" -noout -subject -nameopt -space_eq | sed -n 's/^.*CN=\([^,]*\).*$/\1/p')

# Add the certificate to the JVM truststore
keytool -import -noprompt -alias "$ALIAS" -file "$crt" -keystore "$JRE_CACERTS_PATH" -storepass changeit >/dev/null
done
done

# Add additional certificates to the system CA store. This requires write permissions to several system
# locations, which is not possible in a container with read-only filesystem and/or non-root container.
if [ "$(id -u)" -eq 0 ]; then

# Copy certificates from /certificates to the system truststore, but only if the directory exists and is not empty.
# The reason why this is not part of the opt-in is because it leaves open the option to mount certificates at the
# system location, for whatever reason.
if [ -d /certificates ] && [ "$(ls -A /certificates 2>/dev/null)" ]; then
cp -La /certificates/* /usr/local/share/ca-certificates/
fi
update-ca-certificates
else
# If we are not root, we cannot update the system truststore. That's bad news for tools like `curl` and `wget`,
# but since the JVM is the primary focus here, we can live with that.
true
fi
fi

# Let's provide a variable with the correct path for tools that want or need to use it
export JRE_CACERTS_PATH

exec "$@"
4 changes: 2 additions & 2 deletions jvm/envconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@
],
"name": "JVM Environment",
"readme": "https://github.com/fission/environments/tree/master/jvm",
"runtimeVersion": "8",
"runtimeVersion": "22",
"shortDescription": "JVM environment based on Spring Boot server",
"status": "Stable",
"version": "1.31.1"
"version": "1.31.2"
}
]
2 changes: 1 addition & 1 deletion jvm/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.1.RELEASE</version>
<version>3.3.2</version>
</parent>

<dependencies>
Expand Down
Loading