Add airtable-cache prototype #16
Conversation
| @@ -0,0 +1,35 @@ | |||
| # airtable-cache | |||
|
|
|||
| A caching proxy to Airtable. Consumers can generally ignore the details, and just pretend this is Airtable itself for standard [Airtable API](https://airtable.com/developers/web/api/) routes. For example, with [`airtable-ts`](https://github.com/domdomegg/airtable-ts): | |||
There was a problem hiding this comment.
Need to explain why this exists, i.e.:
Airtable has low rate limits
Airtable is slow (sometimes)
|
|
||
| To deploy a new version, simply commit to the master branch. GitHub Actions automatically handles CD. | ||
|
|
||
| ## TODO |
There was a problem hiding this comment.
Delete this TODO - has now been implemented
| i.get('/v0/:baseId/:tableId', cachedGetHandler); | ||
| i.get('/v0/:baseId/:tableId/:recordId', cachedGetHandler); | ||
|
|
||
| // TODO: clear out base schema |
| @@ -0,0 +1,74 @@ | |||
| import z from 'zod'; | |||
| import { sql } from 'kysely'; | |||
| import { db } from '../db/client'; | |||
There was a problem hiding this comment.
I reckon we should probably scrap the entire database, and just use an in-memory cache.
Postgres uses up a bunch of memory in practice (~200MB) and we probably don't need all its functionality.
Maybe we want to keep Postgres if we want to set up webhooks, so we don't set up webhooks then forget about them.
| @@ -0,0 +1,226 @@ | |||
| import { fastify, FastifyReply, FastifyRequest } from 'fastify'; | |||
There was a problem hiding this comment.
File is a little messy and probably too long: tidy up and split into multiple files
| @@ -3,4 +3,5 @@ import env from '../env'; | |||
|
|
|||
| export default new AirtableTs({ | |||
| apiKey: env.AIRTABLE_PERSONAL_ACCESS_TOKEN, | |||
| endpointUrl: 'http://localhost:8001', | |||
There was a problem hiding this comment.
This should be dynamic, and also needs to work in prod
| return instance; | ||
| }; | ||
|
|
||
| const cachedGetHandler = async (req: FastifyRequest, reply: FastifyReply) => { |
There was a problem hiding this comment.
Maybe consider setting up webhooks on Airtable so that if the records change through other routes e.g. different API changes, or manual edits in Airtable, the cache isn't stale.
Or at least make it very easy to clear the cache - maybe a web UI or something.
| await instance.register(errorHandlerPlugin); | ||
| await instance.register(fastifyCors); | ||
|
|
||
| await instance.register(async (i) => { |
There was a problem hiding this comment.
At the moment, this is a free proxy for any requests to Airtable, and will cache anybody's requests there.
We should instead validate the token is from some allowlisted set.
No description provided.