-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add pages about storage, configuration (#107)
* Add pages about storage, configuration * sort alphabetically (#110) --------- Co-authored-by: Raúl Barroso <[email protected]>
- Loading branch information
Showing
8 changed files
with
66 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
--- | ||
title: 'Configuration' | ||
sidebar_position: 1 | ||
--- | ||
|
||
|
||
Conduit accepts CLI flags, environment variables and a configuration file to | ||
configure its behavior. Each CLI flag has a corresponding environment variable | ||
and a corresponding field in the configuration file. Conduit uses the value for | ||
each configuration option based on the following priorities: | ||
|
||
- **CLI flags** (highest priority) - if a CLI flag is provided it will always be | ||
respected, regardless of the environment variable or configuration file. To | ||
see a full list of available flags run `conduit --help`. | ||
- **Environment variables** (lower priority) - an environment variable is only | ||
used if no CLI flag is provided for the same option. Environment variables | ||
have the prefix `CONDUIT` and contain underscores instead of dots and | ||
hyphens (e.g. the flag `-db.postgres.connection-string` corresponds | ||
to `CONDUIT_DB_POSTGRES_CONNECTION_STRING`). | ||
- **Configuration file** (lowest priority) - Conduit by default loads the | ||
file `conduit.yaml` placed in the same folder as Conduit. The path to the file | ||
can be customized using the CLI flag `-config`. It is not required to provide | ||
a configuration file and any value in the configuration file can be overridden | ||
by an environment variable or a flag. The file content should be a YAML | ||
document where keys can be hierarchically split on `.`. For example: | ||
|
||
```yaml | ||
db: | ||
type: postgres # corresponds to flag -db.type and env variable CONDUIT_DB_TYPE | ||
postgres: | ||
connection-string: postgres://localhost:5432/conduitdb # -db.postgres.connection-string or CONDUIT_DB_POSTGRES_CONNECTION_STRING | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
--- | ||
title: 'Storage' | ||
sidebar_position: 7 | ||
--- | ||
|
||
Conduit's own data (information about pipelines, connectors, etc.) can be stored | ||
in the following databases: | ||
* BadgerDB (default) | ||
* PostgreSQL | ||
* SQLite | ||
|
||
It's also possible to store all the data in memory, which is sometimes useful | ||
for development purposes. | ||
|
||
The database type used can be configured with the `db.type` parameter (through | ||
any of the [configuration](/docs/features/configuration) options in Conduit). | ||
For example, the CLI flag to use a PostgresSQL database with Conduit is as | ||
follows: `-db.type=postgres`. | ||
|
||
Changing database parameters (e.g. the PostgreSQL connection string) is done | ||
through parameters of the following form: `db.<db type>.<parameter name>`. For | ||
example, the CLI flag to use a PostgreSQL instance listening on `localhost:5432` | ||
would be: `-db.postgres.connection-string=postgres://localhost:5432/conduitdb`. | ||
|
||
The full example in our case would be: | ||
```shell | ||
./conduit -db.type=postgres -db.postgres.connection-string="postgresql://localhost:5432/conduitdb" | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters