-
-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Feat/dockerize #14
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# Include any files or directories that you don't want to be copied to your | ||
# container here (e.g., local build artifacts, temporary files, etc.). | ||
# | ||
# For more help, visit the .dockerignore file reference guide at | ||
# https://docs.docker.com/go/build-context-dockerignore/ | ||
|
||
**/.DS_Store | ||
**/.classpath | ||
**/.dockerignore | ||
**/.env | ||
**/.git | ||
**/.gitignore | ||
**/.project | ||
**/.settings | ||
**/.toolstarget | ||
**/.vs | ||
**/.vscode | ||
**/*.*proj.user | ||
**/*.dbmdl | ||
**/*.jfm | ||
**/bin | ||
**/charts | ||
**/docker-compose* | ||
**/compose.y*ml | ||
**/Dockerfile* | ||
**/node_modules | ||
**/npm-debug.log | ||
**/obj | ||
**/secrets.dev.yaml | ||
**/values.dev.yaml | ||
LICENSE | ||
README.md | ||
|
||
/shove |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
*~ | ||
/cmd/shove/shove | ||
/shove | ||
/redis |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
ARG GO_VERSION=1.23.1 | ||
FROM --platform=$BUILDPLATFORM golang:${GO_VERSION} AS build | ||
WORKDIR /src | ||
|
||
RUN --mount=type=cache,target=/go/pkg/mod/ \ | ||
--mount=type=bind,source=go.sum,target=go.sum \ | ||
--mount=type=bind,source=go.mod,target=go.mod \ | ||
go mod download -x | ||
|
||
ARG TARGETARCH | ||
|
||
RUN --mount=type=cache,target=/go/pkg/mod/ \ | ||
--mount=type=bind,target=. \ | ||
CGO_ENABLED=0 GOARCH=$TARGETARCH go build -o /bin/shove ./cmd/shove/main.go | ||
|
||
FROM alpine:latest AS final | ||
|
||
RUN --mount=type=cache,target=/var/cache/apk \ | ||
apk --update add \ | ||
ca-certificates \ | ||
tzdata \ | ||
&& \ | ||
update-ca-certificates | ||
|
||
ARG UID=10001 | ||
RUN adduser \ | ||
--disabled-password \ | ||
--gecos "" \ | ||
--home "/nonexistent" \ | ||
--shell "/sbin/nologin" \ | ||
--no-create-home \ | ||
--uid "${UID}" \ | ||
appuser | ||
USER appuser | ||
|
||
COPY --from=build /bin/shove /bin/shove | ||
|
||
EXPOSE 8322 | ||
|
||
CMD ["/bin/shove"] |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,11 +9,13 @@ This is the replacement for [Pulsus](https://github.com/pennersr/pulsus) which h | |
## Overview | ||
|
||
Design: | ||
|
||
- Asynchronous: a push client can just fire & forget. | ||
- Multiple workers per push service. | ||
- Less moving parts: when using Redis, you can push directly to the queue, bypassing the need for the Shove server to be up and running. | ||
|
||
Supported push services: | ||
|
||
- APNS | ||
- Email: supports automatic creation of email digests in case the rate limit | ||
is exceeded | ||
|
@@ -24,20 +26,19 @@ Supported push services: | |
- Web Push | ||
|
||
Features: | ||
|
||
- Feedback: asynchronously receive information on invalid device tokens. | ||
- Queueing: both in-memory and persistent via Redis. | ||
- Exponential back-off in case of failure. | ||
- Prometheus support. | ||
- Squashing of messages in case rate limits are exceeded. | ||
|
||
|
||
## Why? | ||
|
||
- https://github.com/appleboy/gorush/issues/386#issuecomment-479191179 | ||
|
||
- https://github.com/mercari/gaurun/issues/115 | ||
|
||
|
||
## Usage | ||
|
||
### Running | ||
|
@@ -86,28 +87,27 @@ Usage: | |
VAPID public key | ||
-webpush-vapid-public-key string | ||
VAPID public key | ||
-webpush-vapid-keys-file string | ||
VAPID keys file path | ||
-webpush-workers int | ||
The number of workers pushing Web messages (default 8) | ||
|
||
|
||
Start the server: | ||
|
||
$ shove \ | ||
-api-addr localhost:8322 \ | ||
-queue-redis redis://redis:6379 \ | ||
-fcm-credentials-file /etc/shove/fcm/credentials.json \ | ||
-apns-certificate-path /etc/shove/apns/production/bundle.pem -apns-sandbox-certificate-path /etc/shove/apns/sandbox/bundle.pem \ | ||
-webpush-vapid-public-key=$VAPID_PUBLIC_KEY -webpush-vapid-private-key=$VAPID_PRIVATE_KEY \ | ||
-webpush-vapid-key-file=/etc/shove/webpush/vapid-keys.json \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This uses |
||
-telegram-bot-token $TELEGRAM_BOT_TOKEN | ||
|
||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For readability, I would prefer to keep 2 newlines before starting a new section. |
||
### APNS | ||
|
||
Push an APNS notification: | ||
|
||
$ curl -i --data '{"service": "apns", "headers": {"apns-priority": 10, "apns-topic": "com.shove.app"}, "payload": {"aps": { "alert": "hi"}}, "token": "81b8ecff8cb6d22154404d43b9aeaaf6219dfbef2abb2fe313f3725f4505cb47"}' http://localhost:8322/api/push/apns | ||
|
||
|
||
A successful push results in: | ||
|
||
HTTP/1.1 202 Accepted | ||
|
@@ -117,7 +117,6 @@ A successful push results in: | |
|
||
OK | ||
|
||
|
||
### FCM | ||
|
||
Push an FCM notification: | ||
|
@@ -134,7 +133,6 @@ Or, post JSON: | |
|
||
$ curl -i --data '{"url": "http://localhost:8000/api/webhook", "headers": {"foo": "bar"}, "data": {"hello": "world!"}}' http://localhost:8322/api/push/webhook | ||
|
||
|
||
### WebPush | ||
|
||
Push a WebPush notification: | ||
|
@@ -145,7 +143,6 @@ The subscription (serialized as a JSON string) is used for receiving | |
feedback. Alternatively, you can specify an optional `token` parameter as done | ||
in the example above. | ||
|
||
|
||
### Telegram | ||
|
||
Push a Telegram notification: | ||
|
@@ -157,12 +154,10 @@ Shove requires strings to be passed. For users that disconnected from your bot | |
the chat ID will be communicated back through the feedback mechanism. Here, the | ||
token will equal the unreachable chat ID. | ||
|
||
|
||
### Receive Feedback | ||
|
||
Outdated/invalid tokens are communicated back. To receive those, you can periodically query the feedback channel to receive token feedback, and remove those from your database: | ||
|
||
|
||
$ curl -X POST 'http://localhost:8322/api/feedback' | ||
|
||
{ | ||
|
@@ -173,7 +168,6 @@ Outdated/invalid tokens are communicated back. To receive those, you can periodi | |
] | ||
} | ||
|
||
|
||
|
||
In order to keep your SMTP server safe from being blacklisted, the email service | ||
|
@@ -190,7 +184,7 @@ automatically digested. | |
|
||
Push an email: | ||
|
||
$ curl -i -X POST --data @./scripts/email.json http://localhost:8322/api/push/email | ||
$ curl -i -X POST --data @./scripts/email.json http://localhost:8322/api/push/email | ||
|
||
If you send too many emails, you'll notice that they are digested, and at a | ||
later time, one digest mail is being sent: | ||
|
@@ -208,13 +202,12 @@ later time, one digest mail is being sent: | |
2021/03/23 21:16:12 email: Rate to [email protected] exceeded, email digested | ||
2021/03/23 21:16:18 email: Sending digest email | ||
|
||
|
||
### Redis Queues | ||
|
||
Shove is being used to push a high volume of notifications in a production | ||
environment, consisting of various microservices interacting together. In such a | ||
scenario, it is important that the various services are not too tightly coupled | ||
to one another. For that purpose, Shove offers the ability to post | ||
to one another. For that purpose, Shove offers the ability to post | ||
notifications directly to a Redis queue. | ||
|
||
Posting directly to the Redis queue, instead of using the HTTP service | ||
|
@@ -226,7 +219,6 @@ payloads being pushed, as they are mostly handed over as is to the upstream | |
services. So, when using Shove this way, the client is responsible for handing | ||
over a raw payload. Here's an example: | ||
|
||
|
||
package main | ||
|
||
import ( | ||
|
@@ -263,7 +255,6 @@ over a raw payload. Here's an example: | |
} | ||
} | ||
|
||
|
||
## Status | ||
|
||
Used in production, over at: | ||
|
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -2,6 +2,7 @@ package main | |||||
|
||||||
import ( | ||||||
"context" | ||||||
"encoding/json" | ||||||
"flag" | ||||||
"os" | ||||||
"os/signal" | ||||||
|
@@ -37,7 +38,8 @@ var redisURL = flag.String("queue-redis", "", "Use Redis queue (Redis URL)") | |||||
var webhookWorkers = flag.Int("webhook-workers", 0, "The number of workers pushing Webhook messages") | ||||||
|
||||||
var webPushVAPIDPublicKey = flag.String("webpush-vapid-public-key", "", "VAPID public key") | ||||||
var webPushVAPIDPrivateKey = flag.String("webpush-vapid-private-key", "", "VAPID public key") | ||||||
var webPushVAPIDPrivateKey = flag.String("webpush-vapid-private-key", "", "VAPID private key") | ||||||
var webPushVAPIDKeysJSON = flag.String("webpush-vapid-keys-json", "", "JSON file containing VAPID keys") | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
(that's consistent with e.g. |
||||||
var webPushWorkers = flag.Int("webpush-workers", 8, "The number of workers pushing Web messages") | ||||||
|
||||||
var telegramBotToken = flag.String("telegram-bot-token", "", "Telegram bot token") | ||||||
|
@@ -150,9 +152,31 @@ func main() { | |||||
slog.Error("Failed to add WebPush service", "error", err) | ||||||
os.Exit(1) | ||||||
} | ||||||
} | ||||||
|
||||||
if *telegramBotToken != "" { | ||||||
} else if *webPushVAPIDKeysJSON != "" { | ||||||
content, err := os.ReadFile(*webPushVAPIDKeysJSON) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would be good to add a check here that both |
||||||
if err != nil { | ||||||
slog.Error("Failed to read WebPush VAPID keys file", "error", err) | ||||||
os.Exit(1) | ||||||
} | ||||||
var vapidKeys struct { | ||||||
PublicKey string `json:"publicKey"` | ||||||
PrivateKey string `json:"privateKey"` | ||||||
} | ||||||
err = json.Unmarshal(content, &vapidKeys) | ||||||
if err != nil { | ||||||
slog.Error("Failed to unmarshal WebPush VAPID keys file, please check your JSON file containt 'publicKey' and 'privatekey' keys", "error", err) | ||||||
os.Exit(1) | ||||||
} | ||||||
web, err := webpush.NewWebPush(vapidKeys.PublicKey, vapidKeys.PrivateKey, newServiceLogger("webpush")) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't we need to check here that after the unmarshal public key and private key are really set? |
||||||
if err != nil { | ||||||
slog.Error("Failed to setup WebPush service", "error", err) | ||||||
os.Exit(1) | ||||||
} | ||||||
if err := s.AddService(web, *webPushWorkers, services.SquashConfig{}); err != nil { | ||||||
slog.Error("Failed to add WebPush service", "error", err) | ||||||
os.Exit(1) | ||||||
} | ||||||
} else if *telegramBotToken != "" { | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Issue: This should be an |
||||||
tg, err := telegram.NewTelegramService(*telegramBotToken, newServiceLogger("telegram")) | ||||||
if err != nil { | ||||||
slog.Error("Failed to setup Telegram service", "error", err) | ||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
services: | ||
shove: | ||
build: | ||
context: . | ||
target: final | ||
container_name: shove | ||
volumes: | ||
- ./shove/webpush:/etc/shove/webpush | ||
command: | ||
[ | ||
"shove", | ||
"-api-addr", | ||
"0.0.0.0:8322", | ||
"-queue-redis", | ||
"redis://redis:6379", | ||
"-webpush-vapid-keys-json", | ||
"/etc/shove/webpush/vapid-keys.json", | ||
] | ||
|
||
depends_on: | ||
- redis | ||
ports: | ||
- 8322:8322 | ||
redis: | ||
image: redis:7-alpine | ||
container_name: redis | ||
volumes: | ||
- ./shove/redis/data:/data | ||
ports: | ||
- 6379:6379 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you split this into two separate MRs? One for docker related functionality, the other for the vapid keys file.