Skip to content

Commit

Permalink
docs: add missing sections
Browse files Browse the repository at this point in the history
  • Loading branch information
burmecia committed Dec 16, 2024
1 parent 84b0fc2 commit b33096f
Show file tree
Hide file tree
Showing 16 changed files with 600 additions and 353 deletions.
51 changes: 34 additions & 17 deletions docs/catalog/airtable.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,37 @@ values (
returning key_id;
```

### Connecting to Airtable

We need to provide Postgres with the credentials to connect to Airtable, and any additional options. We can do this using the `create server` command:


=== "With Vault"

```sql
create server airtable_server
foreign data wrapper airtable_wrapper
options (
api_key_id '<key_ID>' -- The Key ID from above.
);
```

=== "Without Vault"

```sql
create server airtable_server
foreign data wrapper airtable_wrapper
options (
api_key '<your_api_key>'
);
```

### Create a schema

We recommend creating a schema to hold all the foreign tables:

```sql
create schema airtable;
create schema if not exists airtable;
```

## Entities
Expand All @@ -78,7 +103,7 @@ The Airtable Wrapper supports data reads from Airtable's [Records](https://airta
#### Usage

```sql
create foreign table my_foreign_table (
create foreign table airtable.my_foreign_table (
name text
-- other fields
)
Expand All @@ -104,16 +129,8 @@ This FDW doesn't support query pushdown.

This will create a "foreign table" inside your Postgres database called `airtable_table`:

#### Operations

| Object | Select | Insert | Update | Delete | Truncate |
| ------ | :----: | :----: | :----: | :----: | :------: |
| Tables ||||||

#### Usage

```sql
create foreign table airtable_table (
create foreign table airtable.airtable_table (
name text,
notes text,
content text,
Expand All @@ -127,18 +144,18 @@ options (
);
```

#### Notes
You can now fetch your Airtable data from within your Postgres database:

- The `base_id` option specifies the Airtable Base ID the table belongs to
- The `table_id` option specifies the Airtable table ID
- You can fetch data using standard SQL: `select * from airtable_table;`
```sql
select * from airtable.airtable_table;
```

### Query an Airtable view

We can also create a foreign table from an Airtable View called `airtable_view`:

```sql
create foreign table airtable_view (
create foreign table airtable.airtable_view (
name text,
notes text,
content text,
Expand All @@ -156,5 +173,5 @@ options (
You can now fetch your Airtable data from within your Postgres database:

```sql
select * from airtable_view;
select * from airtable.airtable_view;
```
50 changes: 34 additions & 16 deletions docs/catalog/auth0.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,17 +53,43 @@ values (
returning key_id;
```

### Connecting to Auth0

We need to provide Postgres with the credentials to connect to Airtable, and any additional options. We can do this using the `create server` command:

=== "With Vault"

```sql
create server auth0_server
foreign data wrapper auth0_wrapper
options (
api_key_id '<key_ID>' -- The Key ID from above.
);
```

=== "Without Vault"

```sql
-- create server and specify custom options
create server auth0_server
foreign data wrapper auth0_wrapper
options (
url 'https://dev-<tenant-id>.us.auth0.com/api/v2/users',
api_key '<your_api_key>'
);
```

### Create a schema

We recommend creating a schema to hold all the foreign tables:

```sql
create schema auth0;
create schema if not exists auth0;
```

## Entities

The Auth0 Wrapper supports data reads from Auth0's [Management API List users endpoint](https://auth0.com/docs/api/management/v2/users/get-users) endpoint (_read only_).
The Auth0 Wrapper supports data reads from Auth0 API.

### Users

Expand All @@ -78,7 +104,7 @@ The Auth0 Wrapper supports data reads from Auth0's [Management API List users en
#### Usage

```sql
create foreign table my_foreign_table (
create foreign table auth0.my_foreign_table (
name text
-- other fields
)
Expand All @@ -102,16 +128,8 @@ This FDW doesn't support query pushdown.

This example demonstrates querying Auth0 users data.

#### Operations

| Object | Select | Insert | Update | Delete | Truncate |
| ------ | :----: | :----: | :----: | :----: | :------: |
| Users ||||||

#### Usage

```sql
create foreign table auth0_table (
create foreign table auth0.auth0_table (
created_at text,
email text,
email_verified bool,
Expand All @@ -123,8 +141,8 @@ create foreign table auth0_table (
);
```

#### Notes
You can now fetch your Auth0 data from within your Postgres database:

- The `object` option must be set to 'users'
- Query the table using standard SQL: `select * from auth0_table;`
- The table provides read-only access to Auth0 user data
```sql
select * from auth0.auth0_table;
```
52 changes: 43 additions & 9 deletions docs/catalog/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,12 +76,46 @@ values (
returning key_id;
```

### Connecting to BigQuery

We need to provide Postgres with the credentials to connect to BigQuery, and any additional options. We can do this using the `create server` command:

=== "With Vault"

```sql
create server bigquery_server
foreign data wrapper bigquery_wrapper
options (
sa_key_id '<key_ID>', -- The Key ID from above.
project_id 'your_gcp_project_id',
dataset_id 'your_gcp_dataset_id'
);
```

=== "Without Vault"

```sql
create server bigquery_server
foreign data wrapper bigquery_wrapper
options (
sa_key '
{
"type": "service_account",
"project_id": "your_gcp_project_id",
...
}
',
project_id 'your_gcp_project_id',
dataset_id 'your_gcp_dataset_id'
);
```

### Create a schema

We recommend creating a schema to hold all the foreign tables:

```sql
create schema bigquery;
create schema if not exists bigquery;
```

## Options
Expand Down Expand Up @@ -116,7 +150,7 @@ The BigQuery Wrapper supports data reads and writes from BigQuery tables and vie
#### Usage

```sql
create foreign table my_bigquery_table (
create foreign table bigquery.my_bigquery_table (
id bigint,
name text,
ts timestamp
Expand All @@ -140,7 +174,7 @@ This FDW supports `where`, `order by` and `limit` clause pushdown.

## Inserting Rows & the Streaming Buffer

This foreign data wrapper uses BigQuery’s `insertAll` API method to create a `streamingBuffer` with an associated partition time. **Within that partition time, the data cannot be updated, deleted, or fully exported**. Only after the time has elapsed (up to 90 minutes according to [BigQuery’s documentation](https://cloud.google.com/bigquery/docs/streaming-data-into-bigquery)); can you perform operations.
This foreign data wrapper uses BigQuery’s `insertAll` API method to create a `streamingBuffer` with an associated partition time. **Within that partition time, the data cannot be updated, deleted, or fully exported**. Only after the time has elapsed (up to 90 minutes according to [BigQuery’s documentation](https://cloud.google.com/bigquery/docs/streaming-data-into-bigquery)), can you perform operations.

If you attempt an `UPDATE` or `DELETE` statement on rows while in the streamingBuffer, you will get an error of `UPDATE` or `DELETE` statement over table datasetName - note that tableName would affect rows in the streaming buffer, which is not supported.

Expand Down Expand Up @@ -170,7 +204,7 @@ insert into your_project_id.your_dataset_id.people values
This example will create a "foreign table" inside your Postgres database called `people` and query its data:

```sql
create foreign table people (
create foreign table bigquery.people (
id bigint,
name text,
ts timestamp
Expand All @@ -181,15 +215,15 @@ create foreign table people (
location 'EU'
);

select * from people;
select * from bigquery.people;
```

### Data modify example

This example will modify data in a "foreign table" inside your Postgres database called `people`, note that `rowid_column` option is mandatory:

```sql
create foreign table people (
create foreign table bigquery.people (
id bigint,
name text,
ts timestamp
Expand All @@ -202,15 +236,15 @@ create foreign table people (
);

-- insert new data
insert into people(id, name, ts)
insert into bigquery.people(id, name, ts)
values (4, 'Yoda', '2023-01-01 12:34:56');

-- update existing data
update people
update bigquery.people
set name = 'Anakin Skywalker'
where id = 1;

-- delete data
delete from people
delete from bigquery.people
where id = 2;
```
34 changes: 14 additions & 20 deletions docs/catalog/cal.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ By default, Postgres stores FDW credentials inside `pg_catalog.pg_foreign_server
insert into vault.secrets (name, secret)
values (
'cal',
'<Cal.com API Key>' -- Cal.com API key
'<Cal.com API key>' -- Cal.com API key
)
returning key_id;
```
Expand Down Expand Up @@ -138,6 +138,8 @@ Supported objects are listed below:

## Entities

The Cal.com Wrapper supports data reads and booking writes from the Cal.com API.

### Profile

This is an object representing your Cal.com user profile.
Expand Down Expand Up @@ -168,7 +170,6 @@ create foreign table cal.my_profile (
#### Notes

- The `attrs` column contains all profile attributes in JSON format
- Query using standard SQL: `select * from cal.my_profile`

### Event Types

Expand Down Expand Up @@ -197,8 +198,7 @@ create foreign table cal.event_types (
#### Notes

- The `attrs` column contains all event type attributes in JSON format
- Extract specific fields using JSON operators
- Example:
- Extract specific fields using JSON operators, for example:
```sql
select
etg->'profile'->>'name' as profile,
Expand Down Expand Up @@ -353,13 +353,7 @@ Below are some examples on how to use Cal.com foreign tables.

### Basic example

This example will create a "foreign table" inside your Postgres database and query its data. First, we can create a schema to hold all the Cal.com foreign tables.

```sql
create schema if not exists cal;
```

Then create the foreign table and query it, for example:
This example will create a "foreign table" inside your Postgres database and query its data.

```sql
create foreign table cal.my_profile (
Expand Down Expand Up @@ -424,15 +418,15 @@ Once we know an event type ID (we can get it from above example, here we suppose
```
insert into cal.bookings(attrs)
values (
'{
"start": "2024-12-12T10:30:00.000Z",
"eventTypeId": 123456,
"attendee": {
"name": "Test Name",
"email": "[email protected]",
"timeZone": "America/New_York"
}
}'::jsonb
'{
"start": "2024-12-12T10:30:00.000Z",
"eventTypeId": 123456,
"attendee": {
"name": "Test Name",
"email": "[email protected]",
"timeZone": "America/New_York"
}
}'::jsonb
);
```

Expand Down
Loading

0 comments on commit b33096f

Please sign in to comment.