diff --git a/README.md b/README.md index 77f7f8f..55050a4 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ This is a reliable distributed key-value store based on the raft algorithm, and go get github.com/RealFax/RedQueen@latest ``` -[Code example](https://github.com/RealFax/RedQueen/tree/master/client/example) +[Code example](https://github.com/RealFax/RedQueen/tree/master/pkg/client/example) ## Write & Read _RedQueen based on raft algorithm has the characteristics of single node write (Leader node) and multiple node read (Follower node)._ @@ -53,8 +53,12 @@ Read order: `Environment Variables | Program Arguments -> Configuration File` - `RQ_DATA_DIR ` Node data storage directory - `RQ_LISTEN_PEER_ADDR ` Node-to-node communication (Raft RPC) listening address, cannot be `0.0.0.0` - `RQ_LISTEN_CLIENT_ADDR ` Node service listening (gRPC API) address +- `RQ_LISTEN_HTTP_ADDR ` Node service listening (http API) address - `RQ_MAX_SNAPSHOTS ` Maximum number of snapshots - `RQ_REQUESTS_MERGED ` Whether to enable request merging +- `RQ_AUTO_TLS ` Whether to enable auto tls +- `RQ_TLS_CERT_FILE ` TLS certificate path +- `RQ_TLS_KEY_FILE ` TLS key path - `RQ_STORE_BACKEND ` Storage backend (default: nuts) - `RQ_NUTS_NODE_NUM ` - `RQ_NUTS_SYNC ` Whether to enable synchronous disk writes @@ -62,6 +66,8 @@ Read order: `Environment Variables | Program Arguments -> Configuration File` - `RQ_NUTS_RW_MODE ` Write mode - `RQ_CLUSTER_BOOTSTRAP ` Cluster information (e.g., node-1@127.0.0.1:5290, node-2@127.0.0.1:4290) - `RQ_DEBUG_PPROF ` Enable pprof debugging +- `RQ_BASIC_AUTH ` Basic auth list (e.g., admin:123456,root:toor) + ### Program Arguments - `-config-file ` Configuration file path. Note: If this parameter is set, the following parameters will be ignored, and the configuration file will be used. @@ -69,8 +75,12 @@ Read order: `Environment Variables | Program Arguments -> Configuration File` - `-data-dir ` Node data storage directory - `-listen-peer-addr ` Node-to-node communication (Raft RPC) listening address, cannot be `0.0.0.0` - `-listen-client-addr ` Node service listening (gRPC API) address +- `-listen-http-addr ` Node service listening (http API) address - `-max-snapshots ` Maximum number of snapshots - `-requests-merged ` Whether to enable request merging +- `-auto-tls ` Whether to enable auto tls +- `-tls-cert-file ` TLS certificate path +- `-tls-key-file ` TLS key path - `-store-backend ` Storage backend (default: nuts) - `-nuts-node-num ` - `-nuts-sync ` Whether to enable synchronous disk writes @@ -78,6 +88,7 @@ Read order: `Environment Variables | Program Arguments -> Configuration File` - `-nuts-rw-mode ` Write mode - `-cluster-bootstrap ` Cluster information (e.g., node-1@127.0.0.1:5290, node-2@127.0.0.1:4290) - `-d-pprof ` Enable pprof debugging +- `-basic-auth ` Basic auth list (e.g., admin:123456,root:toor) ### Configuration File ```toml @@ -89,6 +100,11 @@ listen-client-addr = "127.0.0.1:5230" max-snapshots = 5 requests-merged = false + [node.tls] + auto = true + cert-file = "" + key-file = "" + [store] # backend options # nuts @@ -110,6 +126,10 @@ backend = "nuts" [misc] pprof = false + +[basic-auth] +root = "toor" +admin = "123456" ``` ### _About More Usage (e.g., Docker Single/Multi-node Deployment), Please Refer to [**Wiki**](https://github.com/RealFax/RedQueen/wiki)_ 🤩 diff --git a/README_zh.md b/README_zh.md index b84fdab..c324f44 100644 --- a/README_zh.md +++ b/README_zh.md @@ -20,7 +20,7 @@ _灵感来源于《生化危机》中的超级计算机(Red Queen), 分布式key go get github.com/RealFax/RedQueen@latest ``` -[代码示例](https://github.com/RealFax/RedQueen/tree/master/client/example) +[代码示例](https://github.com/RealFax/RedQueen/tree/master/pkg/client/example) ## 写入 & 读取 _基于raft算法实现的RedQueen具备单节点写入(Leader node)多节点读取(Follower node)的特性_ @@ -53,8 +53,12 @@ RedQueen在内部实现了一个互斥锁, 并提供grpc接口调用 - `RQ_DATA_DIR ` 节点数据存储目录 - `RQ_LISTEN_PEER_ADDR ` 节点间通信监听(raft rpc)地址, 不可为 `0.0.0.0` - `RQ_LISTEN_CLIENT_ADDR ` 节点服务监听(grpc api)地址 +- `RQ_LISTEN_HTTP_ADDR ` 节点服务监听(http api)地址 - `RQ_MAX_SNAPSHOTS ` 最大快照数量 - `RQ_REQUESTS_MERGED ` 是否开启合并请求 +- `RQ_AUTO_TLS ` 是否启用auto tls +- `RQ_TLS_CERT_FILE ` tls certificate文件路径 +- `RQ_TLS_KEY_FILE ` tls key文件路径 - `RQ_STORE_BACKEND ` 存储后端(默认nuts) - `RQ_NUTS_NODE_NUM ` - `RQ_NUTS_SYNC ` 是否启用同步写入磁盘 @@ -62,6 +66,7 @@ RedQueen在内部实现了一个互斥锁, 并提供grpc接口调用 - `RQ_NUTS_RW_MODE ` 写入模式 - `RQ_CLUSTER_BOOTSTRAP ` 集群信息 (例如 node-1@127.0.0.1:5290, node-2@127.0.0.1:4290) - `RQ_DEBUG_PPROF ` 启用pprof调试 +- `RQ_BASIC_AUTH ` basic auth的信息 (例如 admin:123456,root:toor) ### 程序参数 - `-config-file ` 配置文件路径. note: 设置该参数后, 将会忽略以下参数, 使用配置文件 @@ -69,8 +74,12 @@ RedQueen在内部实现了一个互斥锁, 并提供grpc接口调用 - `-data-dir ` 节点数据存储目录 - `-listen-peer-addr ` 节点间通信监听(raft rpc)地址, 不可为 `0.0.0.0` - `-listen-client-addr ` 节点服务监听(grpc api)地址 +- `-listen-http-addr ` 节点服务监听(http api)地址 - `-max-snapshots ` 最大快照数量 - `-requests-merged ` 是否开启合并请求 +- `-auto-tls ` 是否启用auto tls +- `-tls-cert-file ` tls certificate文件路径 +- `-tls-key-file ` tls key文件路径 - `-store-backend ` 存储后端(默认nuts) - `-nuts-node-num ` - `-nuts-sync ` 是否启用同步写入磁盘 @@ -78,6 +87,7 @@ RedQueen在内部实现了一个互斥锁, 并提供grpc接口调用 - `-nuts-rw-mode ` 写入模式 - `-cluster-bootstrap ` 集群信息 (例如 node-1@127.0.0.1:5290, node-2@127.0.0.1:4290) - `-d-pprof ` 启用pprof调试 +- `-basic-auth ` basic auth信息 (例如 admin:123456,root:toor) ### 配置文件 ```toml @@ -86,9 +96,15 @@ id = "node-1" data-dir = "/tmp/red_queen" listen-peer-addr = "127.0.0.1:5290" listen-client-addr = "127.0.0.1:5230" +listen-http-addr = "127.0.0.1:5231" max-snapshots = 5 requests-merged = false + [node.tls] + auto = true + cert-file = "" + key-file = "" + [store] # backend options # nuts @@ -110,6 +126,10 @@ backend = "nuts" [misc] pprof = false + +[basic-auth] +root = "toor" +admin = "123456" ``` ### _关于更多用法(例如docker单/多节点部署), 请参考 [**Wiki**](https://github.com/RealFax/RedQueen/wiki)_ 🤩 diff --git a/config.toml.exam b/config.toml.exam index dff93dd..ab3f55c 100644 --- a/config.toml.exam +++ b/config.toml.exam @@ -35,12 +35,6 @@ backend = "nuts" name = "node-1" peer-addr = "127.0.0.1:3290" -[log] -# logger options -# zap, internal -logger = "" -debug = false - [misc] pprof = false diff --git a/internal/rqd/config/config.go b/internal/rqd/config/config.go index 620d219..19a870c 100644 --- a/internal/rqd/config/config.go +++ b/internal/rqd/config/config.go @@ -65,7 +65,6 @@ type ClusterBootstrap struct { } type Cluster struct { - Token string `toml:"token"` Bootstrap []ClusterBootstrap `toml:"bootstrap"` } @@ -147,9 +146,6 @@ func bindServerFromArgs(cfg *Config, args ...string) error { f.BoolVar(&cfg.Store.Nuts.StrictMode, "nuts-strict-mode", DefaultStoreNutsStrictMode, "enable strict mode") f.Var(newValidatorStringValue[EnumNutsRWMode](DefaultStoreNutsRWMode, &cfg.Store.Nuts.RWMode), "nuts-rw-mode", "select read & write mode, options: fileio, mmap") - // main config::cluster - f.StringVar(&cfg.Cluster.Token, "cluster-token", "", "") - // main config::cluster::bootstrap(s) // in cli: node-1@peer_addr,node-2@peer_addr f.Var(newClusterBootstrapsValue("", &cfg.Cluster.Bootstrap), "cluster-bootstrap", "bootstrap at cluster startup, e.g. : node-1@peer_addr,node-2@peer_addr") @@ -189,9 +185,6 @@ func bindServerFromEnv(cfg *Config) { EnvBoolVar(&cfg.Store.Nuts.StrictMode, "RQ_NUTS_STRICT_MODE", DefaultStoreNutsStrictMode) BindEnvVar(newValidatorStringValue[EnumNutsRWMode](DefaultStoreNutsRWMode, &cfg.Store.Nuts.RWMode), "RQ_NUTS_RW_MODE") - // main config::cluster - EnvStringVar(&cfg.Cluster.Token, "RQ_CLUSTER_TOKEN", "") - // main config::cluster::bootstrap(s) BindEnvVar(newClusterBootstrapsValue("", &cfg.Cluster.Bootstrap), "RQ_CLUSTER_BOOTSTRAP") diff --git a/internal/rqd/server.go b/internal/rqd/server.go index 01f3178..6be8449 100644 --- a/internal/rqd/server.go +++ b/internal/rqd/server.go @@ -9,6 +9,7 @@ import ( "github.com/RealFax/RedQueen/internal/rqd/store" "github.com/RealFax/RedQueen/pkg/dlocker" "github.com/RealFax/RedQueen/pkg/expr" + "github.com/RealFax/RedQueen/pkg/grpcutil" "github.com/RealFax/RedQueen/pkg/httputil" "github.com/RealFax/RedQueen/pkg/tlsutil" "github.com/hashicorp/go-hclog" @@ -141,8 +142,13 @@ func (s *Server) registerRPCServer() { opts := make([]grpc.ServerOption, 0, 8) if s.tlsConfig != nil { opts = append(opts, grpc.Creds(credentials.NewTLS(s.tlsConfig))) + } + if s.cfg.BasicAuth != nil && len(s.cfg.BasicAuth) != 0 { + auth := grpcutil.NewBasicAuth(grpcutil.NewMemoryBasicAuthFunc(s.cfg.BasicAuth)) + opts = append(opts, grpc.UnaryInterceptor(auth.Unary), grpc.StreamInterceptor(auth.Stream)) } + if s.grpcServer == nil { s.grpcServer = grpc.NewServer(opts...) } diff --git a/internal/version/version.go b/internal/version/version.go index 23087c3..8cd42aa 100644 --- a/internal/version/version.go +++ b/internal/version/version.go @@ -4,7 +4,7 @@ import "strings" const ( Major = "0" - Minor = "7" + Minor = "8" Patch = "1" ) diff --git a/pkg/balancer/lb_rand.go b/pkg/balancer/lb_rand.go index 98b2c20..36cbf1d 100644 --- a/pkg/balancer/lb_rand.go +++ b/pkg/balancer/lb_rand.go @@ -2,6 +2,7 @@ package balancer import ( "crypto/rand" + "github.com/RealFax/RedQueen/pkg/expr" "github.com/pkg/errors" "math/big" ) @@ -13,8 +14,7 @@ type randomBalance[K comparable, V any] struct { func (b *randomBalance[K, V]) Next() (V, error) { size := b.Size() if size == 0 { - var empty V - return empty, errors.New("empty load balance list") + return expr.Zero[V](), errors.New("empty load balance list") } idx, _ := rand.Int(rand.Reader, big.NewInt(int64(size))) diff --git a/pkg/balancer/lb_round_robin.go b/pkg/balancer/lb_round_robin.go index baad990..f627666 100644 --- a/pkg/balancer/lb_round_robin.go +++ b/pkg/balancer/lb_round_robin.go @@ -1,6 +1,7 @@ package balancer import ( + "github.com/RealFax/RedQueen/pkg/expr" "github.com/pkg/errors" "sync/atomic" ) @@ -13,8 +14,7 @@ type roundRobinBalance[K comparable, V any] struct { func (b *roundRobinBalance[K, V]) Next() (V, error) { size := b.Size() if size == 0 { - var empty V - return empty, errors.New("empty load balance list") + return expr.Zero[V](), errors.New("empty load balance list") } next := b.current.Add(1) % size b.current.CompareAndSwap(b.current.Load(), next) diff --git a/pkg/client/example/basic-key-value/case.go b/pkg/client/example/basic-key-value/case.go index 4424c9d..3a520a3 100644 --- a/pkg/client/example/basic-key-value/case.go +++ b/pkg/client/example/basic-key-value/case.go @@ -2,7 +2,7 @@ package main import ( "context" - client2 "github.com/RealFax/RedQueen/pkg/client" + "github.com/RealFax/RedQueen/pkg/client" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" "log" @@ -12,9 +12,9 @@ import ( ) func main() { - c, err := client2.New(context.Background(), []string{ - //"127.0.0.1:3230", - //"127.0.0.1:4230", + c, err := client.New(context.Background(), []string{ + "127.0.0.1:3230", + "127.0.0.1:4230", "127.0.0.1:5230", }, grpc.WithTransportCredentials(insecure.NewCredentials())) if err != nil { @@ -23,7 +23,7 @@ func main() { defer c.Close() // watch case - watcher := client2.NewWatcher([]byte("Key1")) + watcher := client.NewWatcher([]byte("Key1")) go func() { if wErr := c.Watch(context.Background(), watcher); err != nil { log.Fatal("client watch error:", wErr) diff --git a/pkg/client/example/distributed-lock/case.go b/pkg/client/example/distributed-lock/case.go index 1577b13..8bdb83a 100644 --- a/pkg/client/example/distributed-lock/case.go +++ b/pkg/client/example/distributed-lock/case.go @@ -2,7 +2,7 @@ package main import ( "context" - client2 "github.com/RealFax/RedQueen/pkg/client" + "github.com/RealFax/RedQueen/pkg/client" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" "log" @@ -13,7 +13,7 @@ func main() { ctx, cancel := context.WithCancel(context.Background()) defer cancel() - c, err := client2.New(ctx, []string{ + c, err := client.New(ctx, []string{ "127.0.0.1:3230", "127.0.0.1:4230", "127.0.0.1:5230", @@ -23,7 +23,7 @@ func main() { } defer c.Close() - mu := client2.NewMutexLock(ctx, c, 120, "lock_object") + mu := client.NewMutexLock(ctx, c, 120, "lock_object") if err = mu.Lock(); err != nil { log.Fatal("client lock error:", err) diff --git a/pkg/client/example/with-auth/case.go b/pkg/client/example/with-auth/case.go new file mode 100644 index 0000000..ce3f7ad --- /dev/null +++ b/pkg/client/example/with-auth/case.go @@ -0,0 +1,34 @@ +package main + +import ( + "context" + "github.com/RealFax/RedQueen/pkg/client" + "github.com/RealFax/RedQueen/pkg/grpcutil" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials/insecure" + "log" +) + +func main() { + // create basic-auth client + authBroker := grpcutil.NewBasicAuthClient("admin", "123456") + + c, err := client.New(context.Background(), []string{ + "127.0.0.1:3230", + "127.0.0.1:4230", + "127.0.0.1:5230", + }, + grpc.WithTransportCredentials(insecure.NewCredentials()), + // setup auth client interceptor + grpc.WithUnaryInterceptor(authBroker.Unary), + grpc.WithStreamInterceptor(authBroker.Stream), + ) + if err != nil { + log.Fatal(err) + } + defer c.Close() + + if _, err = c.Get(context.Background(), []byte("Key1"), nil); err != nil { + log.Fatal("client get error:", err) + } +} diff --git a/pkg/grpcutil/client_handler.go b/pkg/grpcutil/client_handler.go new file mode 100644 index 0000000..ddfeac6 --- /dev/null +++ b/pkg/grpcutil/client_handler.go @@ -0,0 +1,42 @@ +package grpcutil + +import ( + "context" + "google.golang.org/grpc" + "google.golang.org/grpc/metadata" +) + +type BasicAuthClient struct { + authKey string +} + +func (c BasicAuthClient) ctxWrap(ctx context.Context) context.Context { + return metadata.NewOutgoingContext(ctx, metadata.MD{ + MetadataAuthorization: []string{c.authKey}, + }) +} + +func (c BasicAuthClient) Unary( + ctx context.Context, + method string, req, reply any, + cc *grpc.ClientConn, + invoker grpc.UnaryInvoker, + opts ...grpc.CallOption, +) error { + return invoker(c.ctxWrap(ctx), method, req, reply, cc, opts...) +} + +func (c BasicAuthClient) Stream( + ctx context.Context, + desc *grpc.StreamDesc, + cc *grpc.ClientConn, + method string, + streamer grpc.Streamer, + opts ...grpc.CallOption, +) (grpc.ClientStream, error) { + return streamer(c.ctxWrap(ctx), desc, cc, method, opts...) +} + +func NewBasicAuthClient(username, password string) *BasicAuthClient { + return &BasicAuthClient{authKey: BuildAuthorization(username, password)} +} diff --git a/pkg/grpcutil/server_handler.go b/pkg/grpcutil/server_handler.go new file mode 100644 index 0000000..c0b9af0 --- /dev/null +++ b/pkg/grpcutil/server_handler.go @@ -0,0 +1,71 @@ +package grpcutil + +import ( + "context" + "crypto/subtle" + "google.golang.org/grpc" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/metadata" + "google.golang.org/grpc/status" +) + +const ( + MetadataAuthorization string = "Authorization" +) + +type BasicAuthFunc func(username, password string) bool +type BasicAuth struct { + authFC BasicAuthFunc +} + +func (a BasicAuth) auth(ctx context.Context) error { + md, ok := metadata.FromIncomingContext(ctx) + if !ok { + return status.Error(codes.InvalidArgument, "failed get metadata") + } + + authorization := md.Get(MetadataAuthorization) + if len(authorization) != 1 { + return status.Error(codes.InvalidArgument, "invalid metadata 'Authorization'") + } + + if !ParseAuthorization(authorization[0], a.authFC) { + return status.Error(codes.Unauthenticated, "unauthenticated") + } + + return nil +} + +func (a BasicAuth) Unary( + ctx context.Context, + req any, + _ *grpc.UnaryServerInfo, + handler grpc.UnaryHandler, +) (any, error) { + if err := a.auth(ctx); err != nil { + return nil, err + } + return handler(ctx, req) +} + +func (a BasicAuth) Stream( + srv any, + ss grpc.ServerStream, + _ *grpc.StreamServerInfo, + handler grpc.StreamHandler, +) error { + if err := a.auth(ss.Context()); err != nil { + return err + } + return handler(srv, ss) +} + +func NewBasicAuth(fc BasicAuthFunc) *BasicAuth { + return &BasicAuth{authFC: fc} +} + +func NewMemoryBasicAuthFunc(users map[string]string) BasicAuthFunc { + return func(username, password string) bool { + return subtle.ConstantTimeCompare([]byte(users[username]), []byte(password)) == 1 + } +} diff --git a/pkg/grpcutil/util.go b/pkg/grpcutil/util.go new file mode 100644 index 0000000..93425f1 --- /dev/null +++ b/pkg/grpcutil/util.go @@ -0,0 +1,33 @@ +package grpcutil + +import ( + "bytes" + "encoding/base64" + "github.com/RealFax/RedQueen/pkg/hack" + "strings" +) + +func ParseAuthorization(auth string, fc BasicAuthFunc) bool { + p, err := base64.StdEncoding.DecodeString(auth) + if err != nil { + return false + } + + xp := strings.Split(hack.Bytes2String(p), ":") + if len(xp) != 2 { + return false + } + + return fc(xp[0], xp[1]) +} + +func BuildAuthorization(username, password string) string { + b := bytes.Buffer{} + b.Grow(len(username) + len(password) + 1) + + b.WriteString(username) + b.WriteRune(':') + b.WriteString(password) + + return base64.StdEncoding.EncodeToString(b.Bytes()) +}