Skip to content

Commit

Permalink
Update Documentation to Reflect GatewayD’s Support for Multiple Proxi…
Browse files Browse the repository at this point in the history 
…es and Configuration Blocks (#61)

* Added Configuration block into glossary
* made changes related to loadBalancer in server
* made changes related to configuration block for related component clinets, pools,proxies
* fixed pool link in Clients md
* docs: update environment variables format
- Update configuration example to reflect new nested block structure for clients and servers
- Adjust environment variable section to include the new five-part structure
- Update Mermaid diagram to illustrate the five-part environment variable breakdown
* Update using-gatewayd/configuration.md

Co-authored-by: Mostafa Moradian <[email protected]>
Signed-off-by: sina <[email protected]>

* Update using-gatewayd/configuration.md

Co-authored-by: Mostafa Moradian <[email protected]>
Signed-off-by: sina <[email protected]>

* docs: Update README with enhanced server configuration details
- Added detailed descriptions for new load balancer configuration options:
  - `consistentHash` settings with `useSourceIp` parameter.
  - Support for `loadBalancingRules` and `distribution` parameters for strategies like `WEIGHTED_ROUND_ROBIN`.
- Expanded possible values for `loadBalancer.strategy` to include `RANDOM` and `WEIGHTED_ROUND_ROBIN`.
- Updated example configuration to reflect the new settings and provide clarity on usage.
* Replaced all activeWrites to writes

---------

Signed-off-by: sina <[email protected]>
Co-authored-by: Mostafa Moradian <[email protected]>
  • Loading branch information
@sinadarbouy @mostafa
sinadarbouy and mostafa authored Sep 10, 2024
1 parent 7e6dac2 commit 2dc2557
Show file tree
Hide file tree
Showing 6 changed files with 101 additions and 60 deletions.
7 changes: 6 additions & 1 deletion miscellaneous/glossary.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-05-31 20:16:38
last_modified_date: 2024-07-30 16:55:16
layout: default
title: Glossary
description: Glossary of GatewayD terms
Expand Down Expand Up @@ -28,6 +28,11 @@ Each configuration file contains multiple configuration objects that correspond

To enable multi-tenancy, GatewayD supports configuring multiple instances of each (configuration) object using configuration groups. All the default configuration objects have a single configuration group called `default`, except the `API`.

## Configuration block
Within each configuration group, GatewayD allows for further segmentation of settings through the use of configuration blocks. A configuration block represents a specific set of parameters within a configuration group, tailored to a particular instance.

Configuration blocks are supported for the following configuration objects: `clients`, `pools` and `proxies`.

## Configuration parameter

A configuration object has one or many configuration parameters to set up the corresponding object. For example, the `output` parameter on the `logger` object is used to set the outputs by the `default` (configuration group) logger.
Expand Down
61 changes: 36 additions & 25 deletions using-gatewayd/configuration.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-05-31 20:16:38
last_modified_date: 2024-07-30 16:36:53
layout: default
title: Configuration
description: GatewayD is fully configurable via various sources, including default values, YAML config files, environment variables, CLI flags and plugins.
Expand Down Expand Up @@ -74,28 +74,31 @@ metrics:

clients:
default:
network: tcp
address: localhost:5432
tcpKeepAlive: False
tcpKeepAlivePeriod: 30s # duration
receiveChunkSize: 8192
receiveDeadline: 0s # duration, 0ms/0s means no deadline
receiveTimeout: 0s # duration, 0ms/0s means no timeout
sendDeadline: 0s # duration, 0ms/0s means no deadline
dialTimeout: 60s # duration, 0ms/0s means no timeout
# Retry configuration
retries: 3 # 0 means no retry
backoff: 1s # duration
backoffMultiplier: 2.0 # 0 means no backoff
disableBackoffCaps: false
writes: # ⬅️ Configuration block
network: tcp
address: localhost:5432
tcpKeepAlive: False
tcpKeepAlivePeriod: 30s # duration
receiveChunkSize: 8192
receiveDeadline: 0s # duration, 0ms/0s means no deadline
receiveTimeout: 0s # duration, 0ms/0s means no timeout
sendDeadline: 0s # duration, 0ms/0s means no deadline
dialTimeout: 60s # duration, 0ms/0s means no timeout
# Retry configuration
retries: 3 # 0 means no retry
backoff: 1s # duration
backoffMultiplier: 2.0 # 0 means no backoff
disableBackoffCaps: false

pools:
default:
size: 10
writes:
size: 10

proxies:
default:
healthCheckPeriod: 60s # duration
writes:
healthCheckPeriod: 60s # duration

servers:
default:
Expand All @@ -107,6 +110,11 @@ servers:
certFile: ""
keyFile: ""
handshakeTimeout: 5s # duration
loadBalancer:
# Load balancer strategies can be found in config/constants.go
strategy: ROUND_ROBIN
consistentHash:
useSourceIp: true

api:
enabled: True
Expand Down Expand Up @@ -158,24 +166,27 @@ plugins:

## Environment variables

All configuration parameters have a corresponding environment variables, except in certain cases. All environment variables are prefixed with `GATEWAYD` and are in snake case. For example, the `GATEWAYD_LOGGERS_DEFAULT_OUTPUT` environment variable can be set to the outputs required by the default logger and consists of four parts:
All configuration parameters have a corresponding environment variables, except in certain cases. All environment variables are prefixed with `GATEWAYD` and are in snake case. For example, the `GATEWAYD_CLIENTS_DEFAULT_WRITES_NETWORK` environment variable can be set to the network type for the writes in the default client configuration and consists of five parts:

1. Prefix: all environment variables are prefixed with `GATEWAYD`.
2. Object: the configuration object, in this case `LOGGERS`.
2. Object: the configuration object, in this case `CLIENTS`.
3. Group: the configuration group, in this case `DEFAULT`.
4. Parameter: the configuration parameter, in this case `OUTPUT`.
4. Block: the configuration block, in this case `WRITES` (if applicable).
5. Parameter: the configuration parameter, in this case `NETWORK`.

```mermaid
flowchart TD
A(GATEWAYD_LOGGERS_DEFAULT_OUTPUT)
A(GATEWAYD_CLIENTS_DEFAULT_WRITES_NETWORK)
A --> GATEWAYD
A --> LOGGERS
A --> CLIENTS
A --> DEFAULT
A --> OUTPUT
A -.-> WRITES
A --> NETWORK
GATEWAYD --> Prefix
LOGGERS --> Object
CLIENTS --> Object
DEFAULT --> Group
OUTPUT --> Parameter
WRITES -.-> Block
NETWORK --> Parameter
```

## Runtime configuration
Expand Down
33 changes: 17 additions & 16 deletions using-gatewayd/global-configuration/clients.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-05-31 20:16:38
last_modified_date: 2024-07-29 17:35:18
layout: default
title: Clients
description: GatewayD client configuration
Expand All @@ -10,7 +10,7 @@ grand_parent: Using GatewayD

# Clients

GatewayD supports multiple client configurations. Each client in each configuration group will connect to the same database server specified in the configuration parameters and will be added to its corresponding [pool](pools) based on their configuration group, i.e. `default`.
GatewayD supports multiple client configurations. Each client within a configuration group will connect to a database server specified in the configuration parameters and will be added to its corresponding [pool](pools) based on their configuration group (e.g., `default`) and configuration block (e.g., `writes`).

## Configuration parameters

Expand All @@ -33,18 +33,19 @@ GatewayD supports multiple client configurations. Each client in each configurat
```yaml
clients:
default:
network: tcp
address: localhost:5432
tcpKeepAlive: False
tcpKeepAlivePeriod: 30s # duration
receiveChunkSize: 8192
receiveDeadline: 0s # duration, 0ms/0s means no deadline
receiveTimeout: 0s # duration, 0ms/0s means no timeout
sendDeadline: 0s # duration, 0ms/0s means no deadline
dialTimeout: 60s # duration, 0ms/0s means no timeout
# Retry configuration
retries: 3 # 0 means no retry
backoff: 1s # duration
backoffMultiplier: 2.0 # 0 means no backoff
disableBackoffCaps: false
writes:
network: tcp
address: localhost:5432
tcpKeepAlive: False
tcpKeepAlivePeriod: 30s # duration
receiveChunkSize: 8192
receiveDeadline: 0s # duration, 0ms/0s means no deadline
receiveTimeout: 0s # duration, 0ms/0s means no timeout
sendDeadline: 0s # duration, 0ms/0s means no deadline
dialTimeout: 60s # duration, 0ms/0s means no timeout
# Retry configuration
retries: 3 # 0 means no retry
backoff: 1s # duration
backoffMultiplier: 2.0 # 0 means no backoff
disableBackoffCaps: false
```
5 changes: 3 additions & 2 deletions using-gatewayd/global-configuration/pools.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-05-31 20:16:38
last_modified_date: 2024-07-29 17:37:00
layout: default
title: Pools
description: GatewayD pool configuration
Expand All @@ -23,5 +23,6 @@ The size of the pool defines the maximum capacity of the pool. Upon start, Gatew
```yaml
pools:
default:
size: 10
writes:
size: 10
```
9 changes: 5 additions & 4 deletions using-gatewayd/global-configuration/proxies.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-05-31 20:16:38
last_modified_date: 2024-07-30 16:34:34
layout: default
title: Proxies
description: GatewayD proxy configuration
Expand All @@ -12,7 +12,7 @@ grand_parent: Using GatewayD

The proxy object is used to proxy connections between database clients and servers.

GatewayD supports a fixed proxy that creates a pool with a fixed number of connection to the database server. It honors the pool capacity, and if the number of connections from the clients is more than the capacity, new connections will be rejected.
GatewayD supports multiple proxies, each creating its own pool of connections to the database server. Each proxy pool honors its capacity, and if the number of connections from the clients exceeds the capacity, new connections will be rejected.

The PostgreSQL database expects new connections to authenticate before keeping them connected forever, thus the TCP connections from GatewayD will be timed out and dropped. A health check scheduler is started when creating connections to the database. If there are connections available in the available connections pool after the `healthCheckPeriod` is reached, it will remove and recreate new TCP connections to the database and put them in the pool.

Expand All @@ -32,5 +32,6 @@ Each proxy has two pools:
```yaml
proxies:
default:
healthCheckPeriod: 60s # duration
```
writes:
healthCheckPeriod: 60s # duration
```
46 changes: 34 additions & 12 deletions using-gatewayd/global-configuration/servers.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-05-31 20:16:38
last_modified_date: 2024-07-30 16:45:29
layout: default
title: Servers
description: GatewayD server configuration
Expand All @@ -14,24 +14,46 @@ The server object runs to listen for incoming connections from database clients.

## Configuration parameters

| Name | Type | Default value | Possible values | Description |
| ---------------- | ------- | ------------- | --------------- | ----------------------------------- |
| network | string | tcp | tcp, unix | The network protocol to use |
| address | string | 0.0.0.0:15432 | Valid host:port | The address to listen on |
| enableTicker | boolean | False | True, False | Whether to enable the ticker or not |
| tickInterval | string | 5s | Valid duration | The interval of the ticker |
| enableTLS | boolean | False | True, False | Whether to enable TLS or not |
| certFile | string | | Valid path | The path to the TLS certificate |
| keyFile | string | | Valid path | The path to the TLS key |
| handshakeTimeout | string | 5s | Valid duration | The timeout for TLS handshake |
| Name | Type | Default value | Possible values | Description |
| -------------------------------------- | ------- | ------------- | -------------------------------------------- | --------------------------------------------------------------------------- |
| network | string | tcp | tcp, unix | The network protocol to use |
| address | string | 0.0.0.0:15432 | Valid host:port | The address to listen on |
| enableTicker | boolean | False | True, False | Whether to enable the ticker or not |
| tickInterval | string | 5s | Valid duration | The interval of the ticker |
| enableTLS | boolean | False | True, False | Whether to enable TLS or not |
| certFile | string | | Valid path | The path to the TLS certificate |
| keyFile | string | | Valid path | The path to the TLS key |
| handshakeTimeout | string | 5s | Valid duration | The timeout for TLS handshake |
| loadBalancer | object | | | Configuration for the load balancer |
| loadBalancer.strategy | string | ROUND_ROBIN | ROUND_ROBIN, RANDOM, WEIGHTED_ROUND_ROBIN | The strategy used to distribute connections |
| loadBalancer.consistentHash | object | | | Configuration for consistent hash-based load balancing |
| loadBalancer.consistentHash.useSourceIp | boolean | False | True, False | Whether to use the source IP for consistent hashing |
| loadBalancer.loadBalancingRules | array | | List of rules | Optional configuration for strategies that support rules (e.g., WEIGHTED_ROUND_ROBIN) |
| loadBalancingRules.condition | string | DEFAULT | DEFAULT | Condition for the load balancing rule (currently, only "DEFAULT" is supported) |
| loadBalancingRules.distribution | array | | List of proxyName and weight pairs | Defines the weight distribution for proxies in the load balancing strategy |
| distribution.proxyName | string | | Valid proxy name | Name of the proxy server (e.g., "writes", "reads") |
| distribution.weight | integer | | Positive integer | Weight assigned to a proxy in the load balancing distribution |

## Example configuration

### Example Configuration

```yaml
servers:
default:
network: tcp
address: 0.0.0.0:15432
loadBalancer:
strategy: ROUND_ROBIN # ROUND_ROBIN, RANDOM, WEIGHTED_ROUND_ROBIN
consistentHash:
useSourceIp: true
# Optional configuration for strategies that support rules (e.g., WEIGHTED_ROUND_ROBIN)
# loadBalancingRules:
# - condition: "DEFAULT" # Currently, only the "DEFAULT" condition is supported
# distribution:
# - proxyName: "writes"
# weight: 70
# - proxyName: "reads"
# weight: 30
enableTicker: False
tickInterval: 5s # duration
enableTLS: False
Expand Down

0 comments on commit 2dc2557

Please sign in to comment.