Skip to content

Commit 3334c04

Browse files
vfoucaultvfoucault
committed
doc: added README.md
* udpated makefile
1 parent e10eaad commit 3334c04

File tree

2 files changed

+253
-12
lines changed

2 files changed

+253
-12
lines changed

‎Makefile‎

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -9,18 +9,9 @@ default: dev
99
dev:
1010
CGO_ENABLED=0 go build -o bin/$(PLUGIN_NAME) cmd/$(PLUGIN_NAME)/main.go
1111

12-
.PHONY: bootstrap
13-
bootstrap:
14-
@echo "Downloading tools ..."
15-
@go generate -tags tools tools/tools.go
16-
# This should only ever be performed once, so we lean on the cmd/ directory
17-
# to indicate whether this has already been done.
18-
@if [ "$(PLUGIN_NAME)" != "$(REPO_DIR)" ]; then \
19-
echo "Renaming cmd/$(PLUGIN_NAME) to cmd/$(REPO_DIR) ..."; \
20-
mv cmd/$(PLUGIN_NAME) to cmd/$(REPO_DIR); \
21-
echo "Renaming Go module to github.com/hashicorp/$(REPO_DIR) ..."; \
22-
go mod edit -module github.com/hashicorp/$(REPO_DIR); \
23-
fi
12+
.PHONY: build
13+
build:
14+
go build -o clickhouse-database-plugin -ldflags="-s -w" ./cmd/vault-plugin-database-clickhouse/main.go
2415

2516
.PHONY: test
2617
test: fmtcheck

‎README.md‎

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
# Clickhouse Databaase Secret Engine
2+
3+
This plugin provides clickhouse connectivity for clickhouse database using SQL user management
4+
5+
# Build & Tests
6+
7+
make test will run the whole test suite
8+
9+
```bash
10+
~# make test
11+
```
12+
13+
make build will run build the corresponding plugin for the current os/arch
14+
15+
```bash
16+
~# make build
17+
```
18+
19+
## Installation
20+
21+
The Vault plugin system is documented on the [Vault documentation site](https://www.vaultproject.io/docs/internals/plugins.html).
22+
23+
You will need to define a plugin directory using the `plugin_directory` configuration directive, then place the
24+
`vault-plugin-database-clickhouse` executable generated above, into the directory.
25+
26+
**Please note:** Versions v0.2.0 onwards of this plugin are incompatible with Vault versions before 1.6.0 due to an update of the database plugin interface.
27+
28+
Sample commands for registering and starting to use the plugin:
29+
30+
```bash
31+
~# SHA256=$(shasum -a 256 plugins/clickhouse-database-plugin | cut -d' ' -f1)
32+
33+
~# vault secrets enable database
34+
35+
~# vault plugin enable -sha256=$SHA256 database clickhouse-database-plugin
36+
```
37+
38+
At this stage you are now ready to initialize the plugin to connect to clickhouse cluster using unencrypted or encrypted communications.
39+
40+
Prior to initializing the plugin, ensure that you have created an administration account. Vault will use the user specified here to create/update/revoke database credentials. That user must have the appropriate permissions to perform actions upon other database users.
41+
42+
43+
## Pre requisites
44+
45+
Clickhouse user management must be switched to SQL as follows
46+
47+
eg. file /etc/clickhouse-server/users.d/vaultadminuser.xml
48+
49+
```xml
50+
51+
<clickhouse>
52+
<users>
53+
<vault_user>
54+
<profile>default</profile>
55+
<quota>default</quota>
56+
<password_sha256_hex>65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5</password_sha256_hex>
57+
<access_management>1</access_management>
58+
</vault_user>
59+
</users>
60+
</clickhouse>
61+
```
62+
63+
Also, roles must be defined in advance as per the vault roles.
64+
65+
```sql
66+
CREATE ROLE readonly ON CLUSTER '{cluster_name}' SETTINGS max_execution_time=30, max_concurrent_queries_for_user=30, max_threads=8, max_query_size=50485760, max_memory_usage=32819380224, max_memory_usage_for_user=33356251136, max_ast_elements=50000000, distributed_product_mode='local', log_queries=1, distributed_group_by_no_merge=1, optimize_move_to_prewhere=0, readonly=2, optimize_min_equality_disjunction_chain_length=100;
67+
```
68+
69+
## Cluster creation statements
70+
71+
When dealing with a clickhouse cluster, (multiple replicas/shards) we may use the `ON CLUSTER` statement
72+
73+
Creation statement
74+
75+
```bash
76+
creation_statements="CREATE USER '{{name}}' IDENTIFIED BY '{{password}}' ON CLUSTER 'my_cluster'; GRANT readonly TO '{{name}}' ON CLUSTER 'my_cluster';SET DEFAULT ROLE readonly TO '{{name}}'"
77+
```
78+
79+
## Default statements
80+
81+
Default revocation statements:
82+
83+
```sql
84+
DROP USER IF EXISTS '{{name}}';
85+
```
86+
87+
Default Rotate credential statement
88+
89+
```sql
90+
ALTER USER IF EXISTS '{{name}}' IDENTIFIED BY '{{password}}';
91+
```
92+
93+
Default username template
94+
95+
```bash
96+
`{{ printf "v-%s-%s-%s-%s" (.DisplayName | truncate 10) (.RoleName | truncate 10) (random 20) (unix_time) | truncate 32 }}`
97+
```
98+
99+
## Plugin configuration
100+
101+
| Setting | Description | Type | default value |
102+
|-----------------|:----------------------------------------------------|-----:|---------------|
103+
| tls | TLS secure connection to clickhouse | bool | false |
104+
| tls_skip_verify | Whether to check certificate CA upon TLS connection | bool | true |
105+
106+
## Running a dev vault
107+
108+
```bash
109+
~# docker run -p 8200:8200 -it contentsquareplatform/vault-plugin-database-clickhouse:1.13.1-latest server -dev -dev-plugin-dir=/vault/plugins -dev-root-token-id=bladibla
110+
...
111+
2023-04-14T15:32:04.888Z [INFO] identity: entities restored
112+
2023-04-14T15:32:04.888Z [INFO] identity: groups restored
113+
2023-04-14T15:32:04.888Z [INFO] core: post-unseal setup complete
114+
2023-04-14T15:32:04.888Z [INFO] core: vault is unsealed
115+
2023-04-14T15:32:04.890Z [INFO] expiration: revoked lease: lease_id=auth/token/root/ha010bb14be140f3cdf07b143e150b6cdc75f822e4ac925f65b106553363ccdb2
116+
2023-04-14T15:32:04.891Z [INFO] core: successful mount: namespace="" path=secret/ type=kv version=""
117+
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
118+
and starts unsealed with a single unseal key. The root token is already
119+
authenticated to the CLI, so you can immediately begin using Vault.
120+
121+
You may need to set the following environment variables:
122+
123+
$ export VAULT_ADDR='http://0.0.0.0:8200'
124+
125+
The unseal key and root token are displayed below in case you want to
126+
seal/unseal the Vault or re-authenticate.
127+
128+
Unseal Key: iOhLjm7fixKtDJbMicErCveh0A7GB/6i+4UstXG0Udo=
129+
Root Token: bladibla
130+
131+
The following dev plugins are registered in the catalog:
132+
- clickhouse-database-plugin << The clickhouse plugin has been loaded.
133+
134+
Development mode should NOT be used in production installations!
135+
```
136+
137+
```bash
138+
~# export VAULT_ADDR=http://localhost:8200
139+
~# vault login
140+
Token (will be hidden): ********
141+
Success! You are now authenticated. The token information displayed below
142+
is already stored in the token helper. You do NOT need to run "vault login"
143+
again. Future Vault requests will automatically use this token.
144+
145+
Key Value
146+
--- -----
147+
token bladibla
148+
token_accessor EgHmgtCbYhCEIxmTUV2RiQYn
149+
token_duration ∞
150+
token_renewable false
151+
token_policies ["root"]
152+
identity_policies []
153+
policies ["root"]
154+
```
155+
156+
Check the plugin & version
157+
158+
```bash
159+
~# vault plugin list database
160+
Name Version
161+
---- -------
162+
cassandra-database-plugin v1.13.1+builtin.vault
163+
clickhouse-database-plugin v0.1.0
164+
couchbase-database-plugin v0.9.0+builtin
165+
elasticsearch-database-plugin v0.13.1+builtin
166+
hana-database-plugin v1.13.1+builtin.vault
167+
influxdb-database-plugin v1.13.1+builtin.vault
168+
mongodb-database-plugin v1.13.1+builtin.vault
169+
mongodbatlas-database-plugin v0.9.0+builtin
170+
mssql-database-plugin v1.13.1+builtin.vault
171+
mysql-aurora-database-plugin v1.13.1+builtin.vault
172+
mysql-database-plugin v1.13.1+builtin.vault
173+
mysql-legacy-database-plugin v1.13.1+builtin.vault
174+
mysql-rds-database-plugin v1.13.1+builtin.vault
175+
postgresql-database-plugin v1.13.1+builtin.vault
176+
redis-database-plugin v0.2.0+builtin
177+
redis-elasticache-database-plugin v0.2.0+builtin
178+
redshift-database-plugin v1.13.1+builtin.vault
179+
snowflake-database-plugin v0.7.0+builtin
180+
```
181+
182+
## Configure the clickhouse plugin
183+
184+
Enable vault database mount path
185+
186+
```bash
187+
~# vault secrets enable database
188+
Success! Enabled the database secrets engine at: database/
189+
190+
```
191+
192+
Create a connection to your clickhouse deployment
193+
194+
```bash
195+
~# vault write database/config/my-clickhouse \
196+
plugin_name=clickhouse-database-plugin \
197+
plugin_version=v0.1.0 \
198+
allowed_roles="readonly" \
199+
connection_url="clickhouse://my-clickhouse-server:9000?username={{username}}&password={{password}}" \
200+
username="vault_user" \
201+
password="mySEcreTP@assw0Rd" \
202+
username_template="{{.DisplayName}}-{{.RoleName}}-{{unix_time}}-{{random 8}}"
203+
Success! Data written to: database/config/my-clickhouse
204+
```
205+
206+
Create a vault role to use the db connection
207+
208+
```bash
209+
~# vault write database/roles/readonly \
210+
creation_statements="CREATE USER '{{name}}' IDENTIFIED BY '{{password}}'; GRANT readonly TO '{{name}}';SET DEFAULT ROLE readonly TO '{{name}}'" \
211+
revocation_statements="REVOKE readonly FROM '{{name}}'; DROP USER '{{name}}';" \
212+
default_ttl="1h" \
213+
db_name="my-clickhouse"
214+
Success! Data written to: database/roles/readonly
215+
```
216+
217+
Request a database token for role readonly
218+
219+
```bash
220+
vault read database/creds/readonly
221+
222+
Key Value
223+
--- -----
224+
lease_id database/creds/readonly/sYXjvvB50ZjXtygRQepYZtTr
225+
lease_duration 1h
226+
lease_renewable true
227+
password dTbla2n-0talksjf4uK-rWJ
228+
username token-readonly-1681487166-1hAOaK6V
229+
```
230+
231+
Check on clickhouse side
232+
233+
```bash
234+
clickhouse-server:9000|default :) SHOW USERS
235+
236+
SHOW USERS
237+
238+
Query id: 15553c58-9b60-41ce-a19a-f0ff38b35cd8
239+
240+
┌─name───────────────────────────────┐
241+
│ default │
242+
│ token-readonly-1681487166-1hAOaK6V │
243+
│ vault_users │
244+
└────────────────────────────────────┘
245+
246+
3 rows in set. Elapsed: 0.001 sec.
247+
248+
clickhouse-server:9000|default :)
249+
```
250+

0 commit comments

Comments
 (0)