-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Configurable Kafka Properties, Dev Container Java Update & GCS BLOB Support #59
Changes from 22 commits
8ffd14f
a4bebce
c733e46
371f2a3
f8708f1
ab2a4e2
3a66222
1b9a660
8bf4e57
81f997c
2639dc0
02adfa7
52590fa
45c3c68
1cc54d0
011ae1f
c8a04ab
f7159cf
67baf0e
5409374
b664cf8
65d9185
3ca277a
b2083fb
ef8c8c1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,28 +1,30 @@ | ||
# See here for image contents: https://github.com/microsoft/vscode-dev-containers/tree/v0.217.4/containers/java/.devcontainer/base.Dockerfile | ||
# Install openJDK version 21 (includes maven, gradle, and node) | ||
FROM cimg/openjdk:21.0.2-node | ||
|
||
# [Choice] Java version (use -bullseye variants on local arm64/Apple Silicon): 11, 17, 11-bullseye, 17-bullseye, 11-buster, 17-buster | ||
ARG VARIANT="17" | ||
FROM mcr.microsoft.com/vscode/devcontainers/java:0-${VARIANT} | ||
# set user to root to allow apt-get to run | ||
USER root | ||
|
||
# [Option] Install Maven | ||
ARG INSTALL_MAVEN="false" | ||
ARG MAVEN_VERSION="" | ||
# [Option] Install Gradle | ||
ARG INSTALL_GRADLE="false" | ||
ARG GRADLE_VERSION="" | ||
RUN if [ "${INSTALL_MAVEN}" = "true" ]; then su vscode -c "umask 0002 && . /usr/local/sdkman/bin/sdkman-init.sh && sdk install maven \"${MAVEN_VERSION}\""; fi \ | ||
&& if [ "${INSTALL_GRADLE}" = "true" ]; then su vscode -c "umask 0002 && . /usr/local/sdkman/bin/sdkman-init.sh && sdk install gradle \"${GRADLE_VERSION}\""; fi | ||
ARG USERNAME=vscode | ||
ARG USER_UID=1000 | ||
ARG USER_GID=$USER_UID | ||
|
||
# [Choice] Node.js version: none, lts/*, 16, 14, 12, 10 | ||
ARG NODE_VERSION="none" | ||
RUN if [ "${NODE_VERSION}" != "none" ]; then su vscode -c "umask 0002 && . /usr/local/share/nvm/nvm.sh && nvm install ${NODE_VERSION} 2>&1"; fi | ||
|
||
# [Optional] Uncomment this section to install additional OS packages. | ||
# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \ | ||
# && apt-get -y install --no-install-recommends <your-package-list-here> | ||
# Create non-root user vscode with sudo support | ||
ENV DEBIAN_FRONTEND=noninteractive | ||
RUN apt-get update \ | ||
# | ||
# Create a non-root user to use if preferred - see https://aka.ms/vscode-remote/containers/non-root-user. | ||
&& groupadd --gid $USER_GID $USERNAME \ | ||
&& useradd -s /bin/bash --uid $USER_UID --gid $USER_GID -m $USERNAME \ | ||
&& apt-get install -y sudo \ | ||
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME\ | ||
&& chmod 0440 /etc/sudoers.d/$USERNAME | ||
|
||
# [Optional] Uncomment this line to install global node packages. | ||
# RUN su vscode -c "source /usr/local/share/nvm/nvm.sh && npm install -g <your-package-here>" 2>&1 | ||
# RUN npm install -g <your-package-list-here> | ||
|
||
# install kafkacat for testing purposes | ||
RUN apt-get update && apt-get install -y kafkacat | ||
RUN apt-get update && apt-get install -y kafkacat | ||
|
||
# [Optional] Uncomment this section to install additional OS packages. | ||
# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \ | ||
# && apt-get -y install --no-install-recommends <your-package-list-here> |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -22,4 +22,10 @@ services: | |
KAFKA_TYPE: ${KAFKA_TYPE} | ||
CONFLUENT_KEY: ${CONFLUENT_KEY} | ||
CONFLUENT_SECRET: ${CONFLUENT_SECRET} | ||
KAFKA_ENABLE_AUTO_COMMIT: ${KAFKA_ENABLE_AUTO_COMMIT} | ||
KAFKA_AUTO_COMMIT_INTERVAL_MS: ${KAFKA_AUTO_COMMIT_INTERVAL_MS} | ||
KAFKA_SESSION_TIMEOUT_MS: ${KAFKA_SESSION_TIMEOUT_MS} | ||
GOOGLE_APPLICATION_CREDENTIALS: '/google/gcp_service_account.json' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this google credential value: ${GOOGLE_APPLICATION_CREDENTIALS} There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes. The value of the environment variable is mounted to |
||
volumes: | ||
- ${GOOGLE_APPLICATION_CREDENTIALS}:/google/gcp_service_account.json | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -35,6 +35,12 @@ services: | |
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY} | ||
HEADER_ACCEPT: ${HEADER_ACCEPT} | ||
HEADER_X_API_KEY: ${HEADER_X_API_KEY} | ||
KAFKA_ENABLE_AUTO_COMMIT: ${KAFKA_ENABLE_AUTO_COMMIT} | ||
KAFKA_AUTO_COMMIT_INTERVAL_MS: ${KAFKA_AUTO_COMMIT_INTERVAL_MS} | ||
KAFKA_SESSION_TIMEOUT_MS: ${KAFKA_SESSION_TIMEOUT_MS} | ||
GOOGLE_APPLICATION_CREDENTIALS: '/google/gcp_service_account.json' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The same as above There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The value of the environment variable is mounted to /google/gcp_service_account.json and then that file is referenced in the application as GOOGLE_APPLICATION_CREDENTIALS. |
||
volumes: | ||
- ${GOOGLE_APPLICATION_CREDENTIALS}:/google/gcp_service_account.json | ||
depends_on: | ||
- kafka | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
*.json | ||
!sample_gcp_service_account.json |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# Google Resources | ||
|
||
This folder is used as a location to put the GCP service account json file that will be used for the S3 Depositor. Please name the file gcp_service_account.json, a sample file has been provided in this folder. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
{ | ||
"type": "service_account", | ||
"project_id": "", | ||
"private_key_id": "", | ||
"private_key": "", | ||
"client_email": "", | ||
"client_id": "", | ||
"auth_uri": "https://accounts.google.com/o/oauth2/auth", | ||
"token_uri": "https://oauth2.googleapis.com/token", | ||
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", | ||
"client_x509_cert_url": "" | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a .env file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A
sample.env
file is provided. In the README the user is instructed to copy this file and rename the copy to.env