Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Feat/dockerize #14

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Include any files or directories that you don't want to be copied to your
# container here (e.g., local build artifacts, temporary files, etc.).
#
# For more help, visit the .dockerignore file reference guide at
# https://docs.docker.com/go/build-context-dockerignore/

**/.DS_Store
**/.classpath
**/.dockerignore
**/.env
**/.git
**/.gitignore
**/.project
**/.settings
**/.toolstarget
**/.vs
**/.vscode
**/*.*proj.user
**/*.dbmdl
**/*.jfm
**/bin
**/charts
**/docker-compose*
**/compose.y*ml
**/Dockerfile*
**/node_modules
**/npm-debug.log
**/obj
**/secrets.dev.yaml
**/values.dev.yaml
LICENSE
README.md

/shove
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
*~
/cmd/shove/shove
/shove
/redis
40 changes: 40 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
ARG GO_VERSION=1.23.1
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you split this into two separate MRs? One for docker related functionality, the other for the vapid keys file.

FROM --platform=$BUILDPLATFORM golang:${GO_VERSION} AS build
WORKDIR /src

RUN --mount=type=cache,target=/go/pkg/mod/ \
--mount=type=bind,source=go.sum,target=go.sum \
--mount=type=bind,source=go.mod,target=go.mod \
go mod download -x

ARG TARGETARCH

RUN --mount=type=cache,target=/go/pkg/mod/ \
--mount=type=bind,target=. \
CGO_ENABLED=0 GOARCH=$TARGETARCH go build -o /bin/shove ./cmd/shove/main.go

FROM alpine:latest AS final

RUN --mount=type=cache,target=/var/cache/apk \
apk --update add \
ca-certificates \
tzdata \
&& \
update-ca-certificates

ARG UID=10001
RUN adduser \
--disabled-password \
--gecos "" \
--home "/nonexistent" \
--shell "/sbin/nologin" \
--no-create-home \
--uid "${UID}" \
appuser
USER appuser

COPY --from=build /bin/shove /bin/shove

EXPOSE 8322

CMD ["/bin/shove"]
25 changes: 8 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@ This is the replacement for [Pulsus](https://github.com/pennersr/pulsus) which h
## Overview

Design:

- Asynchronous: a push client can just fire & forget.
- Multiple workers per push service.
- Less moving parts: when using Redis, you can push directly to the queue, bypassing the need for the Shove server to be up and running.

Supported push services:

- APNS
- Email: supports automatic creation of email digests in case the rate limit
is exceeded
Expand All @@ -24,20 +26,19 @@ Supported push services:
- Web Push

Features:

- Feedback: asynchronously receive information on invalid device tokens.
- Queueing: both in-memory and persistent via Redis.
- Exponential back-off in case of failure.
- Prometheus support.
- Squashing of messages in case rate limits are exceeded.


## Why?

- https://github.com/appleboy/gorush/issues/386#issuecomment-479191179

- https://github.com/mercari/gaurun/issues/115


## Usage

### Running
Expand Down Expand Up @@ -86,28 +87,27 @@ Usage:
VAPID public key
-webpush-vapid-public-key string
VAPID public key
-webpush-vapid-keys-file string
VAPID keys file path
-webpush-workers int
The number of workers pushing Web messages (default 8)


Start the server:

$ shove \
-api-addr localhost:8322 \
-queue-redis redis://redis:6379 \
-fcm-credentials-file /etc/shove/fcm/credentials.json \
-apns-certificate-path /etc/shove/apns/production/bundle.pem -apns-sandbox-certificate-path /etc/shove/apns/sandbox/bundle.pem \
-webpush-vapid-public-key=$VAPID_PUBLIC_KEY -webpush-vapid-private-key=$VAPID_PRIVATE_KEY \
-webpush-vapid-key-file=/etc/shove/webpush/vapid-keys.json \
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This uses -key- whereas above it says -keys-.

-telegram-bot-token $TELEGRAM_BOT_TOKEN


Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For readability, I would prefer to keep 2 newlines before starting a new section.

### APNS

Push an APNS notification:

$ curl -i --data '{"service": "apns", "headers": {"apns-priority": 10, "apns-topic": "com.shove.app"}, "payload": {"aps": { "alert": "hi"}}, "token": "81b8ecff8cb6d22154404d43b9aeaaf6219dfbef2abb2fe313f3725f4505cb47"}' http://localhost:8322/api/push/apns


A successful push results in:

HTTP/1.1 202 Accepted
Expand All @@ -117,7 +117,6 @@ A successful push results in:

OK


### FCM

Push an FCM notification:
Expand All @@ -134,7 +133,6 @@ Or, post JSON:

$ curl -i --data '{"url": "http://localhost:8000/api/webhook", "headers": {"foo": "bar"}, "data": {"hello": "world!"}}' http://localhost:8322/api/push/webhook


### WebPush

Push a WebPush notification:
Expand All @@ -145,7 +143,6 @@ The subscription (serialized as a JSON string) is used for receiving
feedback. Alternatively, you can specify an optional `token` parameter as done
in the example above.


### Telegram

Push a Telegram notification:
Expand All @@ -157,12 +154,10 @@ Shove requires strings to be passed. For users that disconnected from your bot
the chat ID will be communicated back through the feedback mechanism. Here, the
token will equal the unreachable chat ID.


### Receive Feedback

Outdated/invalid tokens are communicated back. To receive those, you can periodically query the feedback channel to receive token feedback, and remove those from your database:


$ curl -X POST 'http://localhost:8322/api/feedback'

{
Expand All @@ -173,7 +168,6 @@ Outdated/invalid tokens are communicated back. To receive those, you can periodi
]
}


### Email

In order to keep your SMTP server safe from being blacklisted, the email service
Expand All @@ -190,7 +184,7 @@ automatically digested.

Push an email:

$ curl -i -X POST --data @./scripts/email.json http://localhost:8322/api/push/email
$ curl -i -X POST --data @./scripts/email.json http://localhost:8322/api/push/email

If you send too many emails, you'll notice that they are digested, and at a
later time, one digest mail is being sent:
Expand All @@ -208,13 +202,12 @@ later time, one digest mail is being sent:
2021/03/23 21:16:12 email: Rate to [email protected] exceeded, email digested
2021/03/23 21:16:18 email: Sending digest email


### Redis Queues

Shove is being used to push a high volume of notifications in a production
environment, consisting of various microservices interacting together. In such a
scenario, it is important that the various services are not too tightly coupled
to one another. For that purpose, Shove offers the ability to post
to one another. For that purpose, Shove offers the ability to post
notifications directly to a Redis queue.

Posting directly to the Redis queue, instead of using the HTTP service
Expand All @@ -226,7 +219,6 @@ payloads being pushed, as they are mostly handed over as is to the upstream
services. So, when using Shove this way, the client is responsible for handing
over a raw payload. Here's an example:


package main

import (
Expand Down Expand Up @@ -263,7 +255,6 @@ over a raw payload. Here's an example:
}
}


## Status

Used in production, over at:
Expand Down
32 changes: 28 additions & 4 deletions cmd/shove/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package main

import (
"context"
"encoding/json"
"flag"
"os"
"os/signal"
Expand Down Expand Up @@ -37,7 +38,8 @@ var redisURL = flag.String("queue-redis", "", "Use Redis queue (Redis URL)")
var webhookWorkers = flag.Int("webhook-workers", 0, "The number of workers pushing Webhook messages")

var webPushVAPIDPublicKey = flag.String("webpush-vapid-public-key", "", "VAPID public key")
var webPushVAPIDPrivateKey = flag.String("webpush-vapid-private-key", "", "VAPID public key")
var webPushVAPIDPrivateKey = flag.String("webpush-vapid-private-key", "", "VAPID private key")
var webPushVAPIDKeysJSON = flag.String("webpush-vapid-keys-json", "", "JSON file containing VAPID keys")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var webPushVAPIDKeysJSON = flag.String("webpush-vapid-keys-json", "", "JSON file containing VAPID keys")
var webPushVAPIDKeysFile = flag.String("webpush-vapid-keys-file", "", "JSON file containing VAPID keys")

(that's consistent with e.g. -fcm-credentials-file)

var webPushWorkers = flag.Int("webpush-workers", 8, "The number of workers pushing Web messages")

var telegramBotToken = flag.String("telegram-bot-token", "", "Telegram bot token")
Expand Down Expand Up @@ -150,9 +152,31 @@ func main() {
slog.Error("Failed to add WebPush service", "error", err)
os.Exit(1)
}
}

if *telegramBotToken != "" {
} else if *webPushVAPIDKeysJSON != "" {
content, err := os.ReadFile(*webPushVAPIDKeysJSON)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to add a check here that both webPushVAPIDPublicKey and webPushVAPIDPrivateKey are empty.

if err != nil {
slog.Error("Failed to read WebPush VAPID keys file", "error", err)
os.Exit(1)
}
var vapidKeys struct {
PublicKey string `json:"publicKey"`
PrivateKey string `json:"privateKey"`
}
err = json.Unmarshal(content, &vapidKeys)
if err != nil {
slog.Error("Failed to unmarshal WebPush VAPID keys file, please check your JSON file containt 'publicKey' and 'privatekey' keys", "error", err)
os.Exit(1)
}
web, err := webpush.NewWebPush(vapidKeys.PublicKey, vapidKeys.PrivateKey, newServiceLogger("webpush"))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need to check here that after the unmarshal public key and private key are really set?

if err != nil {
slog.Error("Failed to setup WebPush service", "error", err)
os.Exit(1)
}
if err := s.AddService(web, *webPushWorkers, services.SquashConfig{}); err != nil {
slog.Error("Failed to add WebPush service", "error", err)
os.Exit(1)
}
} else if *telegramBotToken != "" {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue: This should be an else if -- Telegram is independent from the webpush config.

tg, err := telegram.NewTelegramService(*telegramBotToken, newServiceLogger("telegram"))
if err != nil {
slog.Error("Failed to setup Telegram service", "error", err)
Expand Down
30 changes: 30 additions & 0 deletions compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
services:
shove:
build:
context: .
target: final
container_name: shove
volumes:
- ./shove/webpush:/etc/shove/webpush
command:
[
"shove",
"-api-addr",
"0.0.0.0:8322",
"-queue-redis",
"redis://redis:6379",
"-webpush-vapid-keys-json",
"/etc/shove/webpush/vapid-keys.json",
]

depends_on:
- redis
ports:
- 8322:8322
redis:
image: redis:7-alpine
container_name: redis
volumes:
- ./shove/redis/data:/data
ports:
- 6379:6379