feat(world-postgres-redis): add PostgreSQL + Redis world implementation #155
Conversation
Implement workflow world using PostgreSQL for durable storage and Redis for queue processing and streaming. Uses Redis Lists with Pub/Sub for job queues and Redis Streams for real-time data streaming, providing 3-6x faster performance than pg-boss while maintaining the same guarantees.
|
|
@kschmelter13 is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
Fix the queue worker to use BRPOP for blocking item retrieval, enhancing performance and reliability. Each worker now utilizes a separate Redis client to prevent connection blocking.
Renamed the PostgresWorldConfig interface to PostgresRedisWorldConfig to better reflect its purpose. Added redisUrl property to the configuration, ensuring the Redis URL is sourced from the config in the createWorld and createQueue functions.
…e error handling in queue processing Updated the TTL for idempotency keys from 10 minutes to 24 hours to provide a longer safety net against memory leaks. Enhanced error handling in the queue worker by adding a retry mechanism for BRPOP errors, ensuring graceful recovery from network issues.
|
@kschmelter13 Thanks for contributing! We love supporting community-created worlds as much as we can, while also making sure the community isn't bottlenecked by us reviewing/maintaining all the worlds. Since a world is just an npm package, do you want to take a stab at publishing your world as an npm package yourself? If you provide us with a link, we'll add your package to the list of community worlds on useworkflow.dev If you think this should be part of the main workflow repository instead, please let us know your reasoning. Thanks! |
2 participants
Implement workflow world using PostgreSQL for durable storage and Redis for queue processing and streaming. Uses Redis Lists with Pub/Sub for job queues and Redis Streams for real-time data streaming, providing 3-6x faster performance than pg-boss while maintaining the same guarantees. #106