Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding GitHub actions #13

Open
wants to merge 32 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
7d99b4e
add client.RemoveContainer
HansikaW Jul 24, 2020
64d328f
update README.md
HansikaW Jul 24, 2020
ba98712
remove util
HansikaW Jul 25, 2020
982b65a
add push manager validation
HansikaW Jul 25, 2020
7c359ef
add containerrunner
HansikaW Jul 25, 2020
ae07fd9
add webhook scheduler
HansikaW Jul 25, 2020
8bc0ef7
remove webhooks
HansikaW Jul 25, 2020
660c431
add containerremover
HansikaW Jul 26, 2020
2ca6952
fixing unit testing and path issue
HansikaW Jul 26, 2020
369f1e9
add caching module
HansikaW Aug 14, 2020
657b6ae
update containerscheduler with cache
HansikaW Aug 14, 2020
c776024
add event scheduler
HansikaW Aug 15, 2020
3fa7d1f
change event scheduler
HansikaW Aug 16, 2020
4517f89
remove containerhistory
HansikaW Aug 18, 2020
1747f57
add port change func
HansikaW Aug 18, 2020
c70d9bd
remove deployment detail
HansikaW Aug 18, 2020
571274c
add cache updater
HansikaW Aug 19, 2020
62fda05
recreate process
HansikaW Aug 19, 2020
dae8b50
recreate dockerpuller with cacheupdater
HansikaW Aug 21, 2020
ca21684
add cacheupadte
HansikaW Aug 21, 2020
e96e8cf
issue fixed[remove static array]
HansikaW Aug 22, 2020
f572571
info log location changes and history table view
HansikaW Aug 22, 2020
ac3f6a1
add flag -d
HansikaW Aug 23, 2020
033f81a
add curl commands
HansikaW Aug 23, 2020
503c948
add ukiyo-agent
HansikaW Aug 23, 2020
c8fdf3c
fixing unit testing
HansikaW Aug 23, 2020
8a9e560
Merge branch 'master' into scheduer
HansikaW Aug 23, 2020
c1e0bd8
add scheduled feature
HansikaW Aug 26, 2020
fec7051
minor fixes
HansikaW Aug 29, 2020
ebb6630
add caching_test
HansikaW Aug 29, 2020
febc198
Merge branch 'scheduer' of https://github.com/HansikaW/ukiyo into sch…
HansikaW Aug 29, 2020
014615f
add GitHub actions
HansikaW Aug 29, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: CI

on:
push:
branches: [ master ]
pull_request:
branches: [ master ]

jobs:

build:
name: Build
runs-on: ubuntu-latest
steps:

- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.13
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v2

- name: Get dependencies
run: |
go get -v -t -d ./...
if [ -f Gopkg.toml ]; then
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
dep ensure
fi

- name: Build
run: go build -v .

- name: Test
run: go test ./test/registryapilayer_test.go ./test/schedulerapilayer_test.go ./test/caching_test.go
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM alpine

RUN apk add --no-cache tzdata
COPY ukiyo .
COPY dbs .
RUN chmod 777 ./ukiyo
Expand Down
148 changes: 124 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,17 @@

## Idea incubation

- ukiyo will act as a watcher for docker containers. It will run alongside with the other running containers and will be responsible for automatic updates. Updates will be based on push based model compared to existing solutions such as [watchtower](https://github.com/containrrr/watchtower) and [ouroboros](https://github.com/pyouroboros/ouroboros)
- ukiyo will act as a watcher for docker containers. It will run alongside with the other running containers and will be responsible for automatic updates and schedule the container deployment time. Updates will be based on push based model compared to existing solutions such as [watchtower](https://github.com/containrrr/watchtower) and [ouroboros](https://github.com/pyouroboros/ouroboros)

- Push events will be recived from ukiyo via webhooks. Docker registries provide webhooks to subscribe and listen to image changes. Locally running images will change only after such an event is received by ukiyo.

- Pull based model can be implemented as an optional way of updating the running containers.
- ukiyo itself runs as a Docker container or else runs as a separate background process a daemon.

## Components

- Container manager
- Push manager (webhooks configuration)
- Scheduler
- OPTIONAL - Pull based update implementation

## Execution modes

Expand All @@ -22,14 +21,6 @@ Two modes of execution
- As a container running alongside other containers (Should mount docker.sock to run docker commands inside the ukiyo docker container)
- As a standalone executable

## Language

- Go lang

### Go style guide

- https://github.com/rajikaimal/go-styleguide

## Dev Setup Guide

Setup docker
Expand All @@ -47,33 +38,142 @@ $ go build -ldflags="-s -w" -o ukiyo main.go

Docker build command
```sh
$ docker build -f Dockerfile -t agentukiyo/ukiyo .
$ docker push agentukiyo/ukiyo
$ docker run -p 8080:8080 \
$ docker build -f Dockerfile -t agentukiyo/ukiyo:tag .
$ docker push agentukiyo/ukiyo:tag
```

The ukiyo-agent is available at https://hub.docker.com/r/agentukiyo/ukiyo

Run ukiyo-agent
```sh
$ docker run -d \
-p 8080:8080 \
-- name ukiyo \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /home/reinventor/dbs:/dbs \
agentukiyo/ukiyo
-v /home/ukiyo/dbs:/dbs \
agentukiyo/ukiyo:tag
```

Add webhook to your dockerhub repository
```
http:{serverIP}:8080/ukiyo-web-hook
```
For pull the containers, require the save Docker Hub authentications inside the ukiyo agent. Therefore you need to use curl or postman request to save the Docker keys.

Add your own docker registy details
```
http:{serverIP}:8080/add-container
http:{serverIP}:8080/save-container-access-keys
{
"username":"agentukiyo",
"repoName":"Ukiyo Docker registry",
"accessToken":"f44e334e-1440-4166-a16f-d8fc9d0eb188",
"email":"[email protected]",
"username":"docker registry name"
"desc":"docker registry description"
"accessToken":"docker registry accesstoken"
"email":"your email"
"serverAddress":"http://docker.io/v1"
}
```

```sh
curl -X POST \
-H "Accept: Application/json" \
-H "Content-Type: application/json" http://{serverIP}:8080/save-container-access-keys \
-d '{"username":"docker registry name","desc":"docker registry description",
"accessToken":"docker registry accesstoken","email":"your email",
"serverAddress":"http://docker.io/v1"}'
```

Update your own docker registy details
```
http:{serverIP}:8080/edit-container-access-keys
{
"username":"docker registry name"
"desc":"edited description"
"accessToken":"new docker registry accesstoken"
"email":"new email"
"serverAddress":"http://docker.io/v1"
}
```

```sh
curl -X POST \
-H "Accept: Application/json" \
-H "Content-Type: application/json" http://{serverIP}:8080/edit-container-access-keys \
-d '{"username":"docker registry name","desc":"docker registry description",
"accessToken":"docker registry accesstoken","email":"your email",
"serverAddress":"http://docker.io/v1"}'
```

Delete your docker registy details
```
http://{serverIP}:8080/delete-container-access-keys/{registryname}
```

```sh
curl -X DELETE \
-H "Accept: Application/json" \
-H "Content-Type: application/json" http://{serverIP}:8080/delete-container-access-keys/{registryname}
```

Ukiyo will do the deployment in order to configured deployment details. Using save-repository-scheduled-time API you can apply the deployment on the fly or scheduled time

Add your deployment details and set schedule deployment time
```
http://{serverIP}:8080/save-repository-scheduled-time
{
"name": "repository name",
"bindingPort":
[{
"exportPort": "8180",
"internalPort": "80"
},
{
"exportPort": "443",
"internalPort": "443"
}],
"scheduledTime": "Aug 17 2020 00:40:50 AM",
"scheduledDowntime": false
}
```

http:{serverIP}:8080/edit-container-token
```sh
curl -X POST \
-H "Accept: Application/json" \
-H "Content-Type: application/json" http://{serverIP}:8080/save-repository-scheduled-time \
-d '{"name": "repository name","bindingPort": [{ "exportPort": "8180", "internalPort": "80" },
{ "exportPort": "443", "internalPort": "443" }],
"scheduledTime": "Aug 17 2020 00:40:50 AM", "scheduledDowntime": false}'
```

Change the deployment schedule
```
http://{serverIP}:8080/edit-repository-scheduled-time
{
"username":"agentukiyo",
"accessToken":"f44e334e-1440-4166-a16f-d8fc9d0eb188"
"name": "repository name",
"bindingPort":
[{
"exportPort": "8180",
"internalPort": "80"
}],
"scheduledTime": "Aug 17 2020 00:40:50 AM",
"scheduledDowntime": false
}
```

```sh
curl -X POST \
-H "Accept: Application/json" \
-H "Content-Type: application/json" http://{serverIP}:8080/edit-repository-scheduled-time \
-d '{"name": "repository name","bindingPort": [{ "exportPort": "8180", "internalPort": "80" },
{ "exportPort": "443", "internalPort": "443" }],
"scheduledTime": "Aug 17 2020 00:40:50 AM", "scheduledDowntime": true}'
```

Delete your deployment details
```
http://{serverIP}:8080/remove-repository-scheduled-time/{repositoryname}
```

```sh
curl -X DELETE \
-H "Accept: Application/json" \
-H "Content-Type: application/json" http://{serverIP}:8080/remove-repository-scheduled-time/{repositoryname}
```
101 changes: 101 additions & 0 deletions api/registryapilayer/registryapilayer.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
package registryapilayer

import (
"encoding/json"
"github.com/gin-gonic/gin"
"github.com/patrickmn/go-cache"
"log"
"net/http"
"time"
"ukiyo/internal/containeraccess"
"ukiyo/pkg/jencoder"
)

type History struct {
Time string
Action string
Status int
Comment string
}

const (
_TimeZone = "Asia/Kolkata"
)

func SaveContainerAccessKeys(r *gin.Engine, cache *cache.Cache) {
r.POST("/save-container-access-keys", func(c *gin.Context) {
var containerKey containeraccess.ContainerKeys
var responseObj containeraccess.ResponseObj
c.ShouldBindJSON(&containerKey)
log.Println("save-container-access-keys | request : " + jencoder.PrintJson(containerKey))

if len(containerKey.UserName) > 0 && len(containerKey.AccessToken) > 0 && len(containerKey.ServerAddress) > 0 {
res := containeraccess.InsertDockerRegData(containerKey)
responseObj = res
} else {
responseObj.ResponseCode = 1
responseObj.ResponseDesc = "Invalid Parameter : save-container key"
}
log.Println("save-container-access-keys | response : " + jencoder.PrintJson(responseObj))
SetHistory(responseObj, cache)
c.JSON(http.StatusOK, responseObj)
})
}

func EditContainerAccessKeys(r *gin.Engine, cache *cache.Cache) {
r.POST("/edit-container-access-keys", func(c *gin.Context) {
var containerKey containeraccess.ContainerKeys
var responseObj containeraccess.ResponseObj
c.ShouldBindJSON(&containerKey)
log.Println("edit-container-access-keys | request : " + jencoder.PrintJson(containerKey))

if len(containerKey.UserName) > 0 && len(containerKey.AccessToken) > 0 {
res := containeraccess.UpdateDockerRegData(containerKey)
responseObj = res
} else {
responseObj.ResponseCode = 1
responseObj.ResponseDesc = "Invalid Parameter edit-container key"
}
log.Println("edit-container-access-keys | response :" + jencoder.PrintJson(responseObj))
SetHistory(responseObj, cache)
c.JSON(http.StatusOK, responseObj)
})
}

func DeleteContainerAccessKeys(r *gin.Engine, cache *cache.Cache) {
r.DELETE("/delete-container-access-keys/:userName", func(c *gin.Context) {
var responseObj containeraccess.ResponseObj
name := c.Param("userName")
log.Println("delete-container-access-keys | request : userName=" + name)
if len(name) > 0 {
res := containeraccess.DeleteDockerRegData(name)
responseObj = res
} else {
responseObj.ResponseCode = 1
responseObj.ResponseDesc = "Invalid Parameter delete-container key"
}
log.Println("delete-container-access-keys | response : " + jencoder.PrintJson(responseObj))
SetHistory(responseObj, cache)
c.JSON(http.StatusOK, responseObj)
})
}

func SetHistory(obj containeraccess.ResponseObj, cash *cache.Cache) {
var historyArray []History
var history History
loc, _ := time.LoadLocation(_TimeZone)
history.Time = time.Now().In(loc).Format("2006-01-02 15:04:05")
history.Status = obj.ResponseCode
history.Action = obj.ResponseDesc
history.Comment = obj.ResponseDesc

if x, _, found := cash.GetWithExpiration("history"); found {
err := json.Unmarshal(jencoder.PassJson(x), &historyArray)
if err != nil {
log.Println(err)
}
}

historyArray = append(historyArray, history)
cash.Set("history", historyArray, 5*time.Minute)
}
Loading