Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintained agenda-rest #207

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
2d2dc19
cli: fix commander import and options usage
the-vampiire Mar 3, 2022
1e1c7f9
package: update name, repo, remove snyk
the-vampiire Mar 3, 2022
2a474e0
job: fix DELETE error
the-vampiire Mar 3, 2022
d24edd2
linting fixes
the-vampiire Mar 3, 2022
69fd435
target node 16
the-vampiire Mar 3, 2022
36e9e46
1.0.1
the-vampiire Mar 3, 2022
7183fb0
api: add healthcheck endpoint and test
the-vampiire Mar 3, 2022
278f936
1.0.2
the-vampiire Mar 3, 2022
5c9d5cc
remove dist before build
the-vampiire Mar 3, 2022
1dea578
docker: lean base and mongo-atlas whitelist script
the-vampiire Mar 3, 2022
a1b71ab
1.0.3
the-vampiire Mar 3, 2022
606ce7b
1.0.4
the-vampiire Mar 3, 2022
ec94370
add scripts
the-vampiire Mar 3, 2022
62ec162
1.0.3
the-vampiire Mar 3, 2022
ea1d19e
version script
the-vampiire Mar 3, 2022
12822ac
eslint: ignore dist WAS CAUSING FUCKING TEMPLATE STRING ISSUE!!!
the-vampiire Mar 3, 2022
992bef1
remove version script?
the-vampiire Mar 3, 2022
58a38b0
1.0.3
the-vampiire Mar 3, 2022
ea43129
fix publish script
the-vampiire Mar 3, 2022
7ceb4e3
ignore docker.env
the-vampiire Mar 3, 2022
f406a56
docker: use version arg
the-vampiire Mar 3, 2022
303e3a9
docker/scripts: mongo atlas fix API res parsing
the-vampiire Mar 3, 2022
dc32a57
scripts: push-registry for GCP and DO
the-vampiire Mar 3, 2022
0a8644c
scripts: add missing context in push-registry
the-vampiire Mar 3, 2022
26607bb
docker: add CMD and sleep in entrypoint
the-vampiire Mar 4, 2022
d7c5cd4
wait 30s for whitelist
the-vampiire Mar 4, 2022
f570243
docker: improve mongo whitelist script behavior
the-vampiire Mar 4, 2022
1a7f9f1
update mongo whitelist script
the-vampiire Mar 4, 2022
0f63937
fix whitelist entrypoint ( `` -> $() )
the-vampiire Mar 4, 2022
e961117
docker/scripts: fix whitelist sleep
the-vampiire Mar 6, 2022
717404d
scripts: push-registry and env
the-vampiire Mar 6, 2022
d5344c3
docker: fix healthcheck, set default VERSION arg
the-vampiire Mar 10, 2022
a236274
docker: fix default VERSION
the-vampiire Mar 10, 2022
606e71d
remove docker/scripts from ignore
the-vampiire Mar 10, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions .babelrc
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
{
"plugins": [
],
"presets": [
["@babel/preset-env", {
"plugins": [],
"presets": [
[
"@babel/preset-env",
{
"targets": {
"node": "12.16"
"node": "16.14"
}
}]
}
]
]
}
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
**/*
!./docker/scripts/*
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
**/dist/*
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
push-registry.env

dist
package-lock.json

Expand Down
8 changes: 0 additions & 8 deletions .snyk

This file was deleted.

40 changes: 35 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,38 @@
FROM node:latest
FROM node:16.14-alpine AS base

RUN npm install -g agenda-rest
ARG VERSION=1.0.3
ENV VERSION=${VERSION}

#expose
EXPOSE 4040
ENV API_PORT=8008
ENV MONGO_DB_URL=mongodb+srv://user:password@host/db-name

CMD ['agenda-rest']
# bash: /bin/bash
# curl: /usr/bin/curl
RUN apk update && apk add --no-cache curl bash

RUN npm install -g [email protected]
RUN npm install -g @nftoolkit/agenda-rest@${VERSION}

EXPOSE ${API_PORT}

HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
CMD curl -s -f localhost:${API_PORT}/health || exit 1

CMD agenda-rest --port ${API_PORT} --dburi ${MONGO_DB_URL}

FROM base AS mongo-atlas-whitelist

ENV SERVICE_NAME='scheduler'

ENV MONGO_ATLAS_API_PK=''
ENV MONGO_ATLAS_API_SK=''
ENV MONGO_ATLAS_API_PROJECT_ID=''

# used for whitelisting script
RUN apk update && apk add --no-cache jq

COPY ./docker/scripts/mongo-atlas-whitelist-entrypoint.sh /tmp/entrypoint.sh

ENTRYPOINT [ "/bin/bash", "/tmp/entrypoint.sh" ]
# NOTE: must re-define the CMD because entrypoint is "overridden" (relative to default docker-entrypoint.sh)
CMD agenda-rest --port ${API_PORT} --dburi ${MONGO_DB_URL}
22 changes: 12 additions & 10 deletions cli.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env node

const program = require("commander");
const { program } = require("commander");

program
.option("-u, --dburi <dburi>", "[optional] Full Mongo connection string")
Expand All @@ -25,21 +25,23 @@ program
)
.parse(process.argv);

const options = program.opts();

const settings = require("./settings");

settings.dburi = program.dburi || settings.dburi;
settings.dbname = program.dbname || settings.dbname;
settings.dbhost = program.dbhost || settings.dbhost;
settings.appId = program.key || settings.appId;
settings.timeout = program.timeout || settings.timeout;
if (program.agenda_settings) {
settings.agenda = JSON.parse(program.agenda_settings);
settings.dburi = options.dburi || settings.dburi;
settings.dbname = options.dbname || settings.dbname;
settings.dbhost = options.dbhost || settings.dbhost;
settings.appId = options.key || settings.appId;
settings.timeout = options.timeout || settings.timeout;
if (options.agenda_settings) {
settings.agenda = JSON.parse(options.agenda_settings);
}

const { app, agenda } = require("./dist");

const server = app.listen(program.port, () => {
console.log(`App listening on port ${program.port}.`);
const server = app.listen(options.port, () => {
console.log(`App listening on port ${options.port}.`);
});

async function graceful() {
Expand Down
132 changes: 132 additions & 0 deletions docker/scripts/mongo-atlas-whitelist-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
#!/usr/bin/env bash

# -- ENV -- #
# SERVICE_NAME
# MONGO_ATLAS_API_PK
# MONGO_ATLAS_API_SK
# MONGO_ATLAS_API_PROJECT_ID
# -- ENV -- #

set -e

mongo_api_base_url='https://cloud.mongodb.com/api/atlas/v1.0'

check_for_deps() {
deps=(
bash
curl
jq
)

for dep in "${deps[@]}"; do
if [ ! "$(command -v $dep)" ]
then
echo "dependency [$dep] not found. exiting"
exit 1
fi
done
}

make_mongo_api_request() {
local request_method="$1"
local request_url="$2"
local data="$3"

curl \
--silent \
--user "$MONGO_ATLAS_API_PK:$MONGO_ATLAS_API_SK" --digest \
--header "Accept: application/json" \
--header "Content-Type: application/json" \
--request "$request_method" "$request_url" \
--data "$data"
}

get_access_list_endpoint() {
echo -n "$mongo_api_base_url/groups/$MONGO_ATLAS_API_PROJECT_ID/accessList"
}

get_service_ip() {
echo -n "$(curl https://ipinfo.io/ip -s)"
}

get_previous_service_ip() {
local access_list_endpoint=$(get_access_list_endpoint)

local previous_ip=$(make_mongo_api_request 'GET' "$access_list_endpoint" \
| jq --arg SERVICE_NAME "$SERVICE_NAME" -r \
'.results[]? as $results | $results.comment | if test("\\[\($SERVICE_NAME)\\]") then $results.ipAddress else empty end'
)

echo "$previous_ip"
}

whitelist_service_ip() {
local current_service_ip="$1"
local comment="Hosted IP of [$SERVICE_NAME] [set@$(date +%s)]"

if (( "${#comment}" > 80 )); then
echo "comment field value will be above 80 char limit: \"$comment\""
echo "comment would be too long due to length of service name [$SERVICE_NAME] [${#SERVICE_NAME}]"
echo "change comment format or service name then retry. exiting to avoid mongo API failure"
exit 1
fi

echo "whitelisting service IP [$current_service_ip] with comment value: \"$comment\""

response=$(make_mongo_api_request \
'POST' \
"$(get_access_list_endpoint)?pretty=true" \
"[
{
\"comment\" : \"$comment\",
\"ipAddress\": \"$current_service_ip\"
}
]" \
| jq -r 'if .error then . else empty end'
)

if [[ -n "$response" ]];
then
echo 'API error whitelisting service'
echo "$response"
exit 1
else
echo "whitelist request successful"
echo "waiting 60s for whitelist to propagate to cluster"
sleep 60
fi
}

delete_previous_service_ip() {
local previous_service_ip="$1"

echo "deleting previous service IP address of [$SERVICE_NAME]"

make_mongo_api_request \
'DELETE' \
"$(get_access_list_endpoint)/$previous_service_ip"
}

set_mongo_whitelist_for_service_ip() {
local current_service_ip=$(get_service_ip)
local previous_service_ip=$(get_previous_service_ip)

if [[ -z "$previous_service_ip" ]]; then
echo "service [$SERVICE_NAME] has not yet been whitelisted"

whitelist_service_ip "$current_service_ip"
elif [[ "$current_service_ip" == "$previous_service_ip" ]]; then
echo "service [$SERVICE_NAME] IP has not changed"
else
echo "service [$SERVICE_NAME] IP has changed from [$previous_service_ip] to [$current_service_ip]"

delete_previous_service_ip "$previous_service_ip"
whitelist_service_ip "$current_service_ip"
fi
}

check_for_deps
set_mongo_whitelist_for_service_ip

# run CMD
exec "$@"
27 changes: 14 additions & 13 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,21 +1,24 @@
{
"name": "agenda-rest",
"version": "1.3.1",
"name": "@nftoolkit/agenda-rest",
"version": "1.0.3",
"description": "Scheduling as a Service",
"main": "./dist/index.js",
"jsnext:main": "./src/index.js",
"scripts": {
"snyk-protect": "snyk protect",
"format": "prettier-eslint --eslint-config-path ./.eslintrc.js --write $PWD'/**/*.js'",
"dev": "webpack --mode development",
"build": "webpack --mode production",
"build": "rm -rf dist && webpack --mode production",
"test": "ava ./dist/test.js && npm run format",
"start": "npm run dev && node cli.js",
"prepublish": "npm run snyk-protect && npm run build"
"preversion": "npm test",
"prepublish": "npm run build",
"publish": "npm publish",
"pushregistry": "bash push-registry.sh",
"postpublish": "npm run pushregistry gcp && npm run pushregistry do"
},
"repository": {
"type": "git",
"url": "git+ssh://[email protected]/agenda/agenda-rest.git"
"url": "git+ssh://[email protected]/nftoolkit/agenda-rest.git"
},
"files": [
"cli.js",
Expand All @@ -38,9 +41,9 @@
"agenda-rest": "cli.js"
},
"bugs": {
"url": "https://github.com/agenda/agenda-rest/issues"
"url": "https://github.com/nftoolkit/agenda-rest/issues"
},
"homepage": "https://github.com/agenda/agenda-rest#README",
"homepage": "https://github.com/nftoolkit/agenda-rest#README",
"devDependencies": {
"@babel/cli": "7.14.8",
"@babel/core": "7.15.0",
Expand All @@ -57,7 +60,7 @@
"webpack-cli": "4.7.2"
},
"dependencies": {
"agenda": "^4.0.0",
"agenda": "^4.2.1",
"async-counter": "^1.1.0",
"babel-runtime": "^6.26.0",
"commander": "^8.0.0",
Expand All @@ -68,8 +71,7 @@
"pythonic": "^2.0.3",
"regenerator-runtime": "^0.13.3",
"request": "^2.88.0",
"request-promise": "^4.2.4",
"snyk": "^1.424.2"
"request-promise": "^4.2.4"
},
"engines": {
"node": ">=8"
Expand All @@ -90,6 +92,5 @@
5
]
}
},
"snyk": true
}
}
Loading