Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create github action for docker container building and push #54

Merged
merged 80 commits into from
Dec 14, 2023
Merged
Show file tree
Hide file tree
Changes from 79 commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
db0f8de
first test
ywkim312 Aug 10, 2023
697be11
added name
ywkim312 Aug 10, 2023
657eb25
remove dependency
ywkim312 Aug 10, 2023
b281e70
second try
ywkim312 Aug 10, 2023
ee77bf5
change variable name
ywkim312 Aug 10, 2023
a35621a
added work dir
ywkim312 Aug 10, 2023
d29fd27
rearranged variable location
ywkim312 Aug 10, 2023
7db18a6
changed file path
ywkim312 Aug 10, 2023
c2a2e3a
saving purpose
ywkim312 Aug 10, 2023
d11e4a8
intial test
ywkim312 Nov 9, 2023
5caee62
Update docker.yml
ywkim312 Nov 9, 2023
277f9ee
Update docker.yml
ywkim312 Nov 9, 2023
e29c2b8
Second test
ywkim312 Nov 9, 2023
96d5fcb
third test
ywkim312 Nov 9, 2023
4908c9f
4th test
ywkim312 Nov 9, 2023
286d3ac
Update docker.yml
ywkim312 Nov 9, 2023
37b646f
Update docker.yml
ywkim312 Nov 9, 2023
5cf8cb7
Update docker.yml
ywkim312 Nov 9, 2023
baba45c
Update docker.yml
ywkim312 Nov 9, 2023
e81484f
Update docker.yml
ywkim312 Nov 9, 2023
c05f667
Update docker.yml
ywkim312 Nov 9, 2023
76a2493
Update docker.yml
ywkim312 Nov 9, 2023
cbc22b2
Update docker.yml
ywkim312 Nov 9, 2023
9408dd4
Update docker.yml
ywkim312 Nov 9, 2023
7dc655e
Update docker.yml
ywkim312 Nov 9, 2023
9bff365
Update docker.yml
ywkim312 Nov 9, 2023
41654c7
Update docker.yml
ywkim312 Nov 9, 2023
ba379f4
Update docker.yml
ywkim312 Nov 9, 2023
d24df60
Update docker.yml
ywkim312 Nov 9, 2023
2350cea
Update docker.yml
ywkim312 Nov 9, 2023
4fa3b2e
Update docker.yml
ywkim312 Nov 9, 2023
d58647c
Update docker.yml
ywkim312 Nov 9, 2023
3ede53e
Update docker.yml
ywkim312 Nov 9, 2023
28701aa
Update docker.yml
ywkim312 Nov 9, 2023
abdfde7
Update docker.yml
ywkim312 Nov 9, 2023
2d60a34
moved dockerfile to root directory
ywkim312 Nov 9, 2023
9a0b6a9
fixed indentation
ywkim312 Nov 9, 2023
b2bee7e
fixed name
ywkim312 Nov 9, 2023
bb69b58
redo from beginning
ywkim312 Nov 9, 2023
d272d52
Update docker.yml
ywkim312 Nov 9, 2023
f256642
changed to matrix
ywkim312 Nov 9, 2023
71cf444
Merge branch '26-github-action-build-and-push' of github.com:ncsa/sta…
ywkim312 Nov 9, 2023
ffc4e94
fixed indentation
ywkim312 Nov 9, 2023
4f2f9b5
Update docker.yml
ywkim312 Nov 9, 2023
012428c
added publish routine
ywkim312 Nov 9, 2023
4b9cfd2
Merge branch '26-github-action-build-and-push' of github.com:ncsa/sta…
ywkim312 Nov 9, 2023
1136c22
updated secret
ywkim312 Nov 9, 2023
f710350
updated docker hub registry
ywkim312 Nov 9, 2023
eb04404
added histogram to matrix
ywkim312 Nov 9, 2023
381493e
added version tag
ywkim312 Nov 9, 2023
81f1a99
Update docker.yml
ywkim312 Nov 9, 2023
75530f8
added version print
ywkim312 Nov 9, 2023
c962824
fixed merge conflict
ywkim312 Nov 9, 2023
996ef38
made only to build and publish when it gets merged to main
ywkim312 Nov 9, 2023
23028dc
Merge branch 'develop' into 26-github-action-build-and-push
ywkim312 Nov 15, 2023
810868f
testing post to github
ywkim312 Nov 15, 2023
7b51b4e
testing github loging
ywkim312 Nov 15, 2023
e0d2ce9
Merge branch 'develop' into 26-github-action-build-and-push
ywkim312 Dec 4, 2023
3dbaeae
modified github image name
ywkim312 Dec 4, 2023
e562515
added other containers
ywkim312 Dec 4, 2023
c9b4883
fixed container names
ywkim312 Dec 4, 2023
d0962db
made to make only merged to develop
ywkim312 Dec 4, 2023
a0d5ea8
rename the variables
ywkim312 Dec 5, 2023
f42b434
rename comments
ywkim312 Dec 5, 2023
fce2de4
added clowder containers
ywkim312 Dec 5, 2023
a55545a
added test
ywkim312 Dec 5, 2023
cb17e9c
fixed repo name
ywkim312 Dec 5, 2023
f3a5061
removed pull request launch
ywkim312 Dec 5, 2023
55b357b
added manual trigger
ywkim312 Dec 5, 2023
a25517e
1st test
ywkim312 Dec 5, 2023
40f35d9
2nd test
ywkim312 Dec 5, 2023
5162f0d
3rd test
ywkim312 Dec 5, 2023
098e08e
4th test
ywkim312 Dec 5, 2023
26d0a7e
5th test
ywkim312 Dec 5, 2023
c2582b8
rolled back
ywkim312 Dec 5, 2023
5ea16ef
create new file
ywkim312 Dec 5, 2023
c1f6f40
added job to manual trigger
ywkim312 Dec 5, 2023
ff05e2f
modified docker.yml
ywkim312 Dec 7, 2023
967e50b
modified to build only for develop and main
ywkim312 Dec 7, 2023
bb08da0
modified push destination
ywkim312 Dec 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
235 changes: 235 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,235 @@
name: CI

on:
push:
branches:
- main
- develop
- 'release/*'

tags:
- '*'

longshuicy marked this conversation as resolved.
Show resolved Hide resolved
pull_request:

env:
MAIN_REPO: ncsa/standalone-smm-analytics

jobs:

# ----------------------------------------------------------------------
# DOCKER BUILD
# ----------------------------------------------------------------------
docker:
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
name:
- autophrase
- histogram
- check_screen_name
- classification_predict
- classification_train
- classification_split
- collect_reddit_comment
- crimson_hexagon_monitors
- image_crawler
- name_entity_recognition
- network_analysis
- preprocessing
- screen_name_prompt
- sentiment_analysis
- topic_modeling
ywkim312 marked this conversation as resolved.
Show resolved Hide resolved
- clowder_create_collection
- clowder_create_dataset
- clowder_create_space
- clowder_list
- clowder_upload_file
include:
- name: autophrase
dockerfile: Dockerfile
workdir: containerized_analytics/smile/autophrase
docker_repo_tag: socialmediamacroscope/autophrase
github_repo_tag: ncsa/autophrase
- name: histogram
dockerfile: Dockerfile
workdir: containerized_analytics/smile/histogram
docker_repo_tag: socialmediamacroscope/histogram
github_repo_tag: ncsa/histogram
- name: check_screen_name
dockerfile: Dockerfile
workdir: containerized_analytics/smile/check_screen_name
docker_repo_tag: socialmediamacroscope/check_screen_name
github_repo_tag: ncsa/check_screen_name
- name: classification_predict
dockerfile: Dockerfile
workdir: containerized_analytics/smile/classification_predict
docker_repo_tag: socialmediamacroscope/classification_predict
github_repo_tag: ncsa/classification_predict
- name: classification_train
dockerfile: Dockerfile
workdir: containerized_analytics/smile/classification_train
docker_repo_tag: socialmediamacroscope/classification_train
github_repo_tag: ncsa/classification_train
- name: classification_split
dockerfile: Dockerfile
workdir: containerized_analytics/smile/classification_split
docker_repo_tag: socialmediamacroscope/classification_split
github_repo_tag: ncsa/classification_split
- name: collect_reddit_comment
dockerfile: Dockerfile
workdir: containerized_analytics/smile/collect_reddit_comment
docker_repo_tag: socialmediamacroscope/collect_reddit_comment
github_repo_tag: ncsa/collect_reddit_comment
- name: crimson_hexagon_monitors
dockerfile: Dockerfile
workdir: containerized_analytics/smile/crimson_hexagon_monitors
docker_repo_tag: socialmediamacroscope/crimson_hexagon_monitors
github_repo_tag: ncsa/crimson_hexagon_monitors
- name: image_crawler
dockerfile: Dockerfile
workdir: containerized_analytics/smile/image_crawler
docker_repo_tag: socialmediamacroscope/image_crawler
github_repo_tag: ncsa/image_crawler
- name: name_entity_recognition
dockerfile: Dockerfile
workdir: containerized_analytics/smile/name_entity_recognition
docker_repo_tag: socialmediamacroscope/name_entity_recognition
github_repo_tag: ncsa/name_entity_recognition
- name: network_analysis
dockerfile: Dockerfile
workdir: containerized_analytics/smile/network_analysis
docker_repo_tag: socialmediamacroscope/network_analysis
github_repo_tag: ncsa/network_analysis
- name: preprocessing
dockerfile: Dockerfile
workdir: containerized_analytics/smile/preprocessing
docker_repo_tag: socialmediamacroscope/preprocessing
github_repo_tag: ncsa/preprocessing
- name: screen_name_prompt
dockerfile: Dockerfile
workdir: containerized_analytics/smile/screen_name_prompt
docker_repo_tag: socialmediamacroscope/screen_name_prompt
github_repo_tag: ncsa/screen_name_prompt
- name: sentiment_analysis
dockerfile: Dockerfile
workdir: containerized_analytics/smile/sentiment_analysis
docker_repo_tag: socialmediamacroscope/sentiment_analysis
github_repo_tag: ncsa/sentiment_analysis
- name: topic_modeling
dockerfile: Dockerfile
workdir: containerized_analytics/smile/topic_modeling
docker_repo_tag: socialmediamacroscope/topic_modeling
github_repo_tag: ncsa/topic_modeling
- name: clowder_create_collection
dockerfile: Dockerfile
workdir: containerized_analytics/clowder/clowder_create_collection
docker_repo_tag: socialmediamacroscope/clowder_create_collection
github_repo_tag: ncsa/clowder_create_collection
- name: clowder_create_dataset
dockerfile: Dockerfile
workdir: containerized_analytics/clowder/clowder_create_dataset
docker_repo_tag: socialmediamacroscope/clowder_create_dataset
github_repo_tag: ncsa/clowder_create_dataset
- name: clowder_create_space
dockerfile: Dockerfile
workdir: containerized_analytics/clowder/clowder_create_space
docker_repo_tag: socialmediamacroscope/clowder_create_space
github_repo_tag: ncsa/clowder_create_space
- name: clowder_list
dockerfile: Dockerfile
workdir: containerized_analytics/clowder/clowder_list
docker_repo_tag: socialmediamacroscope/clowder_list
github_repo_tag: ncsa/clowder_list
- name: clowder_upload_file
dockerfile: Dockerfile
workdir: containerized_analytics/clowder/clowder_upload_file
docker_repo_tag: socialmediamacroscope/clowder_upload_file
github_repo_tag: ncsa/clowder_upload_file
steps:
# checkout source code
- uses: actions/checkout@v2

# calculate version information
- name: version information
run: |
if [ "${{ github.event.release.target_commitish }}" != "" ]; then
BRANCH="${{ github.event.release.target_commitish }}"
elif [[ "${{github.event_name}}" == "pull_request" ]]; then
BRANCH="PR-${{github.event.pull_request.number}}"
else
BRANCH=${GITHUB_REF##*/}
fi
echo "GITHUB_BRANCH=${BRANCH}" >> $GITHUB_ENV

if [ "$BRANCH" == "main" ]; then
VERSION=$(cat manual_jb/content/_config.yml | grep "version" | head -1 | awk -F= "{print $2 }" | sed "s/[[:alpha:]|,|-|\"|:|#|[:space:]]//g" | sed "s/-//g")
VERSIONS="latest"
OLDVERSION=""
TMPVERSION=$VERSION
while [ "$OLDVERSION" != "$TMPVERSION" ]; do
VERSIONS="${VERSIONS} ${TMPVERSION}"
OLDVERSION="${TMPVERSION}"
TMPVERSION=$(echo ${OLDVERSION} | sed 's/\.[0-9]*$//')
done
echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "TAGS=${VERSIONS}" >> $GITHUB_ENV
elif [ "$BRANCH" == "develop" ]; then
echo "VERSION=develop" >> $GITHUB_ENV
echo "TAGS=develop" >> $GITHUB_ENV
else
echo "VERSION=testing" >> $GITHUB_ENV
echo "TAGS=${BRANCH}" >> $GITHUB_ENV
fi

- name: Print Version tag
run: echo "${{ env.TAGS }}"

# login to registries
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build docker image
if: |
github.repository == env.MAIN_REPO &&
github.event_name == 'push' &&
(env.GITHUB_BRANCH == 'main' || env.GITHUB_BRANCH == 'develop')
uses: elgohr/[email protected]
with:
dockerfile: ${{ matrix.dockerfile }}
name: ${{ matrix.docker_repo_tag }}
no_push: true
workdir: ${{ matrix.workdir }}

- name: Publish doc image to Docker Hub
if: |
github.repository == env.MAIN_REPO &&
github.event_name == 'push' &&
env.GITHUB_BRANCH == 'main'
uses: elgohr/[email protected]
with:
registry: docker.io
name: ${{ matrix.docker_repo_tag }}
username: ${{ secrets.HUB_USERNAME }}
password: ${{ secrets.HUB_PASSWORD }}
tags: "${{ env.TAGS }}"
workdir: ${{ matrix.workdir }}

- name: Publish doc image to Github
if: |
github.repository == env.MAIN_REPO &&
github.event_name == 'push' &&
env.GITHUB_BRANCH == 'main'
ywkim312 marked this conversation as resolved.
Show resolved Hide resolved
uses: elgohr/[email protected]
with:
registry: ghcr.io
name: ${{ matrix.github_repo_tag }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
tags: "${{ env.TAGS }}"
workdir: ${{ matrix.workdir }}
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- Github action for building containers [#26](https://github.com/ncsa/standalone-smm-analytics/issues/26)

### Changed
- Reorganized repository structure [#10](https://github.com/ncsa/standalone-smm-analytics/issues/10)

Expand Down
Loading