Skip to content

Commit

Permalink
feat(graphql clickhouse): add ClickHouse DB adapter PE-6488
Browse files Browse the repository at this point in the history
This change adds support for a new ClickHouse based DB backend. It can
be used in combination with the SQLite DB backend to enable batch
loading of historical data from Parquet. It also opens up the
possibility of higher DB performance and scalability. In its current
state it should be considered a technology preview. It won't be useful
to most users until we either provide Parquet files to load into it or
automate flushing of the SQLite DB to it (both are planned in future
release). It also is not intended to be standalone solution. It supports
bulk loading and efficient GraphQL querying of transactions and data
items, but it relies on SQLite (or potentially another OLTP in the
future) to index recent data. These limitations allow greatly simplified
schema and query construction. Querying the new ClickHouse DB for
transaction and data items via GraphQL is enabled by setting the
'CLICKHOUSE_URL' environment variable.
  • Loading branch information
djwhitt committed Sep 6, 2024
1 parent 4ee04c1 commit 267ea0e
Show file tree
Hide file tree
Showing 10 changed files with 492 additions and 6 deletions.
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
"@aws-lite/client": "^0.21.7",
"@aws-lite/s3": "^0.1.21",
"@permaweb/aoconnect": "^0.0.56",
"@clickhouse/client": "^1.3.0",
"apollo-server-express": "^3.13.0",
"arbundles": "^0.11.1",
"arweave": "^1.14.4",
Expand Down
2 changes: 1 addition & 1 deletion src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ app.use(dataRouter);
app.use(arweaveRouter);

// GraphQL
const apolloServerInstanceGql = apolloServer(system.db, {
const apolloServerInstanceGql = apolloServer(system.gqlQueryable, {
introspection: true,
persistedQueries: false,
});
Expand Down
3 changes: 3 additions & 0 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,9 @@ export const MAX_DATA_ITEM_QUEUE_SIZE = +env.varOrDefault(
'100000',
);

// ClickHouse URL
export const CLICKHOUSE_URL = env.varOrUndefined('CLICKHOUSE_URL');

//
// ArNS and sandboxing
//
Expand Down
Loading

0 comments on commit 267ea0e

Please sign in to comment.