Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prdp 163 feat quarkus java engine #83

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
5e02bd7
quarkus_app_test: introducing quarkus based app for pdf-engine-java
alexyey Oct 10, 2023
f402a2a
temp commit
alessio-cialini Oct 11, 2023
f0da050
quarkus_app_test: introducing reactive management
alessio-cialini Oct 11, 2023
50913d0
quarkus_app_test: introducing reactive management
alessio-cialini Oct 11, 2023
c70e2fb
quarkus_app_test: introducing quarkus based app for pdf-engine-java
alexyey Oct 10, 2023
d892526
temp commit
alessio-cialini Oct 11, 2023
cfc6ea0
quarkus_app_test: introducing reactive management
alessio-cialini Oct 11, 2023
023d90c
quarkus_app_test: introducing reactive management
alessio-cialini Oct 11, 2023
da31bcd
quarkus_app_test: introducing reactive management
alessio-cialini Oct 17, 2023
a949aa8
Merge remote-tracking branch 'origin/quarkus_app_test' into quarkus_a…
alessio-cialini Oct 17, 2023
e910449
[PRDP-163] feat: Introducing changes to define Quarkus Java Engine an…
alessio-cialini Oct 18, 2023
165d239
[PRDP-163] feat: Updated error management on reactive streams
alessio-cialini Oct 18, 2023
55d22f2
[PRDP-163] feat: Updated info endpoint and client service
alessio-cialini Oct 18, 2023
4aa5c9c
[PRDP-163] feat: Updated info service tests
alessio-cialini Oct 18, 2023
9ab2d3a
[PRDP-163] feat: Updated tests, introducing configurations for applic…
alessio-cialini Oct 19, 2023
c425b90
Merge branch 'main' into PRDP-163-feat-quarkus-java-engine
alessio-cialini Oct 19, 2023
2332612
[PRDP-163] feat: Updated run_docker.sh
alessio-cialini Oct 19, 2023
2df45cc
Merge remote-tracking branch 'origin/PRDP-163-feat-quarkus-java-engin…
alessio-cialini Oct 19, 2023
0054755
[PRDP-163] feat: Updated GeneratePdfResource
alessio-cialini Oct 19, 2023
7673fa0
[PRDP-163] feat: Updated file management
alessio-cialini Oct 19, 2023
01f92e6
[PRDP-163] feat: Updated GeneratePDFServiceImplTest
alessio-cialini Oct 20, 2023
2d85c6b
[PRDP-163] feat: Updated GeneratePDFServiceImpl management of Vertx o…
alessio-cialini Oct 20, 2023
8d21f10
[PRDP-163] feat: Updated GeneratePDFServiceImpl management of Vertx o…
alessio-cialini Oct 20, 2023
183dc6f
[PRDP-163] feat: Updated GeneratePDFServiceImpl management of Vertx o…
alessio-cialini Oct 20, 2023
3c2689c
[PRDP-163] feat: Updated GeneratePDFServiceImpl management of Vertx o…
alessio-cialini Oct 20, 2023
7ef2c8c
[PRDP-163] chore: Removing code smells
alessio-cialini Oct 20, 2023
5b36c0f
[PRDP-163] feat: updated GeneratePdfResource and GeneratePdfResourceTest
alessio-cialini Oct 23, 2023
762a959
[PRDP-163] feat: GeneratePDFServiceImpl and PdfEngineClientImplTest
alessio-cialini Oct 25, 2023
e6ec840
Merge branch 'main' into PRDP-163-feat-quarkus-java-engine
alessio-cialini Oct 25, 2023
c2c509e
[PRDP-163] feat: Updated code_review.yml, GeneratePdfResource
alessio-cialini Oct 26, 2023
00dbba4
Merge remote-tracking branch 'origin/PRDP-163-feat-quarkus-java-engin…
alessio-cialini Oct 26, 2023
a716be3
[PRDP-163] feat: Updated tests
alessio-cialini Oct 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions .github/maven_code_review/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
name: Maven Code Review
description: "Code Review for Pull Request"

inputs:
github_token:
required: true
type: string
description: Github Token
sonar_token:
required: true
type: string
description: Sonar Token for the login
project_key:
required: true
type: string
description: Key of the project on SonarCloud
coverage_exclusions:
required: false
type: string
description: Files to exclude from coverage
default: '**/config/*,**/*Mock*,**/model/**,**/entity/*'
cpd_exclusions:
required: false
type: string
description: Files to exclude from code duplication
default: '**/model/**,**/entity/*'
jdk_version:
required: true
type: string
description: JDK version
default: 11
maven_version:
required: true
type: string
description: Maven version
default: 3.8.2

runs:
using: "composite"
steps:
- uses: actions/checkout@v2

- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: ${{ inputs.jdk_version }}

- name: Set up Maven
uses: stCarolas/[email protected]
with:
maven-version: ${{ inputs.maven_version }}

- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2

- name: Cache SonarCloud packages
uses: actions/cache@v1
with:
path: ~/.sonar-project.properties/cache
key: ${{ runner.os }}-sonar-project.properties
restore-keys: ${{ runner.os }}-sonar-project.properties

- name: Build and analyze on Pull Requests
if: ${{ github.event_name == 'pull_request' }}
shell: bash
run: mvn -B verify org.sonarsource.scanner.maven:sonar-maven-plugin:sonar
-Dsonar.organization=pagopa
-Dsonar.projectKey=${{ env.PROJECT_KEY }}
-Dsonar.coverage.jacoco.xmlReportPaths=./target/jacoco-report/jacoco.xml
-Dsonar.coverage.exclusions=${{inputs.coverage_exclusions}}
-Dsonar.cpd.exclusions=${{inputs.cpd_exclusions}}
-Dsonar.host.url=https://sonarcloud.io
-Dsonar.login=${{ inputs.sonar_token }}
-Dsonar.pullrequest.key=${{ github.event.pull_request.number }}
-Dsonar.pullrequest.branch=${{ github.head_ref }}
-Dsonar.pullrequest.base=${{ github.base_ref }}
env:
# Needed to get some information about the pull request, if any
GITHUB_TOKEN: ${{ inputs.github_token }}
# SonarCloud access token should be generated from https://sonarcloud.io/account/security/
SONAR_TOKEN: ${{ inputs.sonar_token }}

- name: Build and analyze on Push main
if: ${{ github.event_name != 'pull_request' }}
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.github_token }}
SONAR_TOKEN: ${{ inputs.sonar_token }}
run: mvn -B verify org.sonarsource.scanner.maven:sonar-maven-plugin:sonar
-Dsonar.organization=pagopa
-Dsonar.projectKey=${{ env.PROJECT_KEY }}
-Dsonar.coverage.jacoco.xmlReportPaths=./target/jacoco-report/jacoco.xml
-Dsonar.coverage.exclusions=${{inputs.coverage_exclusions}}
-Dsonar.cpd.exclusions=${{inputs.cpd_exclusions}}
-Dsonar.branch.name=${{ github.head_ref }}
-Dsonar.host.url=https://sonarcloud.io
-Dsonar.login=${{ inputs.sonar_token }}
7 changes: 5 additions & 2 deletions .github/workflows/code_review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,16 @@ jobs:

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- uses: actions/[email protected]
- name: Code Review
uses: pagopa/github-actions-template/[email protected]
uses: ./.github/maven_code_review
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
sonar_token: ${{ secrets.SONAR_TOKEN }}
project_key: ${{env.PROJECT_KEY}}
coverage_exclusions: "**/config/*,**/*Mock*,**/model/**,**/entity/*"
jdk_version: 17
maven_version: 3.9.3
coverage_exclusions: "**/config/*,**/*Mock*,**/exception/**,**/model/**,**/entity/*,**/producer/**,**/enumeration/**"
cpd_exclusions: "**/model/**,**/entity/*"

code-review-node:
Expand Down
1 change: 1 addition & 0 deletions .mvn/wrapper/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
maven-wrapper.jar
18 changes: 18 additions & 0 deletions .mvn/wrapper/maven-wrapper.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.3/apache-maven-3.9.3-bin.zip
wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar
38 changes: 38 additions & 0 deletions .mvn/wrapper/settings.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
https://maven.apache.org/xsd/settings-1.0.0.xsd">
<profiles>
<profile>
<id>securecentral</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<!--Override the repository (and pluginRepository) "central" from the
Maven Super POM -->
<repositories>
<repository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
</repository>
<repository>
<id>com.e-iceblue</id>
<name>e-iceblue</name>
<url>https://repo.e-iceblue.com/nexus/content/groups/public/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
</settings>
89 changes: 57 additions & 32 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,43 +1,68 @@
ARG JAVA_VERSION=11
# This image additionally contains function core tools – useful when using custom extensions
FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION-build AS installer-env

COPY . /src/java-function-app
RUN echo $(ls -1 /src/java-function-app)
RUN chmod 777 /src/java-function-app/agent/config.yaml
RUN cd /src/java-function-app && \
## Stage 1 : build with maven builder image with native capabilities
FROM quay.io/quarkus/ubi-quarkus-graalvmce-builder-image:22.3-java17 AS build
COPY --chown=quarkus:quarkus mvnw /code/mvnw
COPY --chown=quarkus:quarkus .mvn /code/.mvn
COPY --chown=quarkus:quarkus pom.xml /code/
USER quarkus
WORKDIR /code
RUN chmod 777 mvnw
RUN chmod 777 .mvn
RUN chmod 777 mvnw
RUN chmod 777 .mvn
RUN ./mvnw -B org.apache.maven.plugins:maven-dependency-plugin:3.1.2:resolve -s .mvn/wrapper/settings.xml
RUN ./mvnw -B org.apache.maven.plugins:maven-dependency-plugin:3.1.2:resolve-plugins -s .mvn/wrapper/settings.xml
COPY src /code/src
COPY agent /code/agent
ARG QUARKUS_PROFILE
ARG APP_NAME

USER root
RUN echo $(ls -1 /code/src)
RUN chmod 777 /code/agent/config.yaml
# install wget
RUN microdnf install -y wget
# install jmx agent
RUN cd /code && \
wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.19.0/jmx_prometheus_javaagent-0.19.0.jar && \
curl -o 'opentelemetry-javaagent.jar' -L 'https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.25.1/opentelemetry-javaagent.jar' && \
mkdir -p /home/site/wwwroot && \
mvn clean package -Dmaven.test.skip=true && \
cd ./target/azure-functions/ && \
cd $(ls -d */|head -n 1) && \
cp -a . /home/site/wwwroot && \
cp /src/java-function-app/agent/config.yaml /home/site/wwwroot/config.yaml && \
cp -r /src/java-function-app/node /home/site/wwwroot/node
curl -o 'applicationinsights-agent.jar' -L 'https://github.com/microsoft/ApplicationInsights-Java/releases/download/3.4.17/applicationinsights-agent-3.4.17.jar'

RUN chmod 777 /src/java-function-app/jmx_prometheus_javaagent-0.19.0.jar && \
cp /src/java-function-app/jmx_prometheus_javaagent-0.19.0.jar /home/site/wwwroot/jmx_prometheus_javaagent-0.19.0.jar
# build the application
RUN ./mvnw package -DskipTests=true -Dquarkus.application.name=$APP_NAME -Dquarkus.profile=$QUARKUS_PROFILE

RUN chmod 777 /src/java-function-app/opentelemetry-javaagent.jar && \
cp /src/java-function-app/opentelemetry-javaagent.jar /home/site/wwwroot/opentelemetry-javaagent.jar
RUN mkdir -p /code/target/otel && \
chmod 777 /code/opentelemetry-javaagent.jar && \
cp /code/opentelemetry-javaagent.jar /code/target/otel/opentelemetry-javaagent.jar

RUN cp /src/java-function-app/start.sh /home/site/wwwroot/start.sh
RUN mkdir -p /code/target/appins && \
chmod 777 /code/applicationinsights-agent.jar && \
cp /code/applicationinsights-agent.jar /code/target/appins/applicationinsights-agent.jar

# This image is ssh enabled
#FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION-appservice
# This image isn't ssh enabled
FROM mcr.microsoft.com/azure-functions/java:3.0-java$JAVA_VERSION
RUN mkdir -p /code/target/jmx && \
cp /code/agent/config.yaml /code/target/jmx/config.yaml

ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
AzureFunctionsJobHost__Logging__Console__IsEnabled=true
RUN chmod 777 /code/jmx_prometheus_javaagent-0.19.0.jar && \
cp /code/jmx_prometheus_javaagent-0.19.0.jar /code/target/jmx/jmx_prometheus_javaagent-0.19.0.jar

EXPOSE 80
EXPOSE 12345
COPY --from=installer-env ["/home/site/wwwroot", "/home/site/wwwroot"]
FROM registry.access.redhat.com/ubi8/openjdk-17:1.14

USER root
ENV LANGUAGE='en_US:en'

# We make four distinct layers so if there are application changes the library layers can be re-used
COPY --from=build /code/target/quarkus-app/lib/ /deployments/lib/
COPY --from=build /code/target/quarkus-app/*.jar /deployments/
COPY --from=build /code/target/quarkus-app/app/ /deployments/app/
COPY --from=build /code/target/quarkus-app/quarkus/ /deployments/quarkus/
COPY --from=build /code/target/jmx/ /deployments/
COPY --from=build /code/target/otel/ /deployments/
COPY --from=build /code/target/appins/ /deployments/

EXPOSE 8080
EXPOSE 12345
USER 185

RUN chmod 777 /home/site/wwwroot/start.sh
ARG QUARKUS_PROFILE
ARG APP_NAME

CMD ["/home/site/wwwroot/start.sh"]
ENV JAVA_OPTS="-Dquarkus.http.host=0.0.0.0 -Dquarkus.application.name=$APP_NAME -Dquarkus.profile=$QUARKUS_PROFILE -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
ENV JAVA_APP_JAR="/deployments/quarkus-run.jar"
59 changes: 59 additions & 0 deletions Dockerfile_integration_test
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
## Stage 1 : build with maven builder image with native capabilities
FROM quay.io/quarkus/ubi-quarkus-graalvmce-builder-image:22.3-java17 AS build
COPY --chown=quarkus:quarkus mvnw /code/mvnw
COPY --chown=quarkus:quarkus .mvn /code/.mvn
COPY --chown=quarkus:quarkus pom.xml /code/
USER quarkus
WORKDIR /code
RUN chmod 777 mvnw
RUN chmod 777 .mvn
RUN ./mvnw -B org.apache.maven.plugins:maven-dependency-plugin:3.1.2:resolve -s .mvn/wrapper/settings.xml
RUN ./mvnw -B org.apache.maven.plugins:maven-dependency-plugin:3.1.2:resolve-plugins -s .mvn/wrapper/settings.xml
COPY src /code/src
COPY agent /code/agent
ARG QUARKUS_PROFILE
ARG APP_NAME

USER root
RUN echo $(ls -1 /code/src)
RUN chmod 777 /code/agent/config.yaml
# install wget
RUN microdnf install -y wget
# install jmx agent
RUN cd /code && \
wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.19.0/jmx_prometheus_javaagent-0.19.0.jar

# build the application
RUN ./mvnw package -DskipTests=true -Dquarkus.application.name=$APP_NAME -Dquarkus.profile=$QUARKUS_PROFILE

RUN mkdir -p /code/target/jmx && \
cp /code/agent/config.yaml /code/target/jmx/config.yaml

RUN chmod 777 /code/jmx_prometheus_javaagent-0.19.0.jar && \
cp /code/jmx_prometheus_javaagent-0.19.0.jar /code/target/jmx/jmx_prometheus_javaagent-0.19.0.jar

FROM registry.access.redhat.com/ubi8/openjdk-17:1.14

ENV LANGUAGE='en_US:en'

# We make four distinct layers so if there are application changes the library layers can be re-used
COPY --from=build /code/target/quarkus-app/lib/ /deployments/lib/
COPY --from=build /code/target/quarkus-app/*.jar /deployments/
COPY --from=build /code/target/quarkus-app/app/ /deployments/app/
COPY --from=build /code/target/quarkus-app/quarkus/ /deployments/quarkus/
COPY --from=build /code/target/jmx/ /deployments/

EXPOSE 8080
EXPOSE 12345
USER 185

ARG QUARKUS_PROFILE
ARG APP_NAME

ENV JAVA_OPTS="-Dquarkus.http.host=0.0.0.0 -Dquarkus.application.name=$APP_NAME -Dquarkus.profile=$QUARKUS_PROFILE -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
ENV JAVA_APP_JAR="/deployments/quarkus-run.jar"

ARG SLEEP_FOR_INTEGRATION

USER root
RUN sleep $SLEEP_FOR_INTEGRATION &
6 changes: 3 additions & 3 deletions docker/run_docker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,15 @@ done

stack_name=$(cd .. && basename "$PWD")
docker build -t pagopa-pdf-engine-node ../node
docker build -t pagopa-pdf-engine ../
docker build -t pagopa-pdf-engine -f ../Dockerfile_integration_test ../
docker run -d -p 3000:3000 --name="${stack_name}-node" --network=host pagopa-pdf-engine-node
docker run -d -p 60486:80 --name="${stack_name}" --network=host pagopa-pdf-engine
docker run -d -p 8080:8080 --name="${stack_name}" --network=host pagopa-pdf-engine

# waiting the containers
printf 'Waiting for the service'
attempt_counter=0
max_attempts=50
until [ $(curl -s -o /dev/null -w "%{http_code}" http://localhost/info) -eq 200 ]; do
until [ $(curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/info) -eq 200 ]; do
if [ ${attempt_counter} -eq ${max_attempts} ];then
echo "Max attempts reached"
exit 1
Expand Down
13 changes: 7 additions & 6 deletions helm/values-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,14 @@ microservice-chart:
type: Utilization # Allowed types are 'Utilization' or 'AverageValue'
value: "75"
envConfig:
ENV: "dev"
WEBSITE_SITE_NAME: "pagopapdfengine" # required to show cloud role name in application insights
FUNCTIONS_WORKER_RUNTIME: "java"
APP_NAME: "pdf-engine"
APP_ENVIRONMENT: "dev"
WEBSITE_SITE_NAME: 'pagopapdfengine' # required to show cloud role name in application insights
APP_LOGGING_LEVEL: 'DEBUG'
DEFAULT_LOGGING_LEVEL: 'INFO'
HTML_TEMPLATE_FILE_NAME: "template"
WORKING_DIRECTORY_PATH: "/temp"
PDF_ENGINE_NODE_INFO_ENDPOINT: "https://api.dev.platform.pagopa.it/shared/pdf-engine-node/v1/info"
PDF_ENGINE_NODE_GENERATE_ENDPOINT: "https://api.dev.platform.pagopa.it/shared/pdf-engine-node/v1/generate-pdf"
PDF_ENGINE_NODE_GENERATE_ENDPOINT: "https://api.dev.platform.pagopa.it/shared/pdf-engine-node/v1"
ENABLE_ECS_CONSOLE: "true"
CONSOLE_LOG_THRESHOLD: "DEBUG"
CONSOLE_LOG_PATTERN: "%d{HH:mm:ss.SSS}[%thread]%-5level%logger{36}-%msg%n"
Expand All @@ -87,7 +88,7 @@ microservice-chart:
OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector.elastic-system.svc:4317"
OTEL_LOGS_EXPORTER: none
OTEL_TRACES_SAMPLER: "always_on"
JAVA_TOOL_OPTIONS: "-javaagent:/home/site/wwwroot/jmx_prometheus_javaagent-0.19.0.jar=12345:/home/site/wwwroot/config.yaml -javaagent:/home/site/wwwroot/opentelemetry-javaagent.jar"
JAVA_TOOL_OPTIONS: "-javaagent:/deployments/jmx_prometheus_javaagent-0.19.0.jar=12345:/deployments/config.yaml -javaagent:/deployments/opentelemetry-javaagent.jar -javaagent:/deployments/applicationinsights-agent.jar"
envSecret:
APPLICATIONINSIGHTS_CONNECTION_STRING: 'ai-d-connection-string'
OTEL_EXPORTER_OTLP_HEADERS: 'elastic-otel-token-header'
Expand Down
Loading
Loading