Reference implementation for AWS world and corresponding documentation update #167
Conversation
|
|
@karthikscale3 is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub. |
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
| workflow: process.env.WORKFLOW_AWS_WORKFLOW_QUEUE_URL!, | ||
| step: process.env.WORKFLOW_AWS_STEP_QUEUE_URL!, | ||
| }, | ||
| streamBucket: process.env.WORKFLOW_AWS_STREAM_BUCKET!, |
There was a problem hiding this comment.
Critical runtime error: Non-null assertions on environment variables that may not be defined will cause the application to crash when AWS resources aren't properly configured.
View Details
📝 Patch Details
diff --git a/packages/world-aws/world/config.ts b/packages/world-aws/world/config.ts
index 8c6278b..dcc8e47 100644
--- a/packages/world-aws/world/config.ts
+++ b/packages/world-aws/world/config.ts
@@ -27,6 +27,25 @@ export function getDefaultConfig(): AWSWorldConfig {
region: process.env.AWS_REGION || process.env.WORKFLOW_AWS_REGION,
});
+ // Validate required environment variables
+ const requiredEnvVars: Record<string, string | undefined> = {
+ WORKFLOW_AWS_WORKFLOW_QUEUE_URL: process.env.WORKFLOW_AWS_WORKFLOW_QUEUE_URL,
+ WORKFLOW_AWS_STEP_QUEUE_URL: process.env.WORKFLOW_AWS_STEP_QUEUE_URL,
+ WORKFLOW_AWS_STREAM_BUCKET: process.env.WORKFLOW_AWS_STREAM_BUCKET,
+ };
+
+ const missingVars = Object.entries(requiredEnvVars)
+ .filter(([, value]) => !value)
+ .map(([key]) => key);
+
+ if (missingVars.length > 0) {
+ throw new Error(
+ `Missing required AWS World environment variables: ${missingVars.join(', ')}. ` +
+ `Please ensure these are set before using AWS World. ` +
+ `See https://github.com/vercel/workflow-devkit for setup instructions.`
+ );
+ }
+
const config = {
region:
process.env.AWS_REGION || process.env.WORKFLOW_AWS_REGION || 'us-east-1',
@@ -39,10 +58,10 @@ export function getDefaultConfig(): AWSWorldConfig {
process.env.WORKFLOW_AWS_STREAMS_TABLE || 'workflow_stream_chunks',
},
queues: {
- workflow: process.env.WORKFLOW_AWS_WORKFLOW_QUEUE_URL!,
- step: process.env.WORKFLOW_AWS_STEP_QUEUE_URL!,
+ workflow: requiredEnvVars.WORKFLOW_AWS_WORKFLOW_QUEUE_URL as string,
+ step: requiredEnvVars.WORKFLOW_AWS_STEP_QUEUE_URL as string,
},
- streamBucket: process.env.WORKFLOW_AWS_STREAM_BUCKET!,
+ streamBucket: requiredEnvVars.WORKFLOW_AWS_STREAM_BUCKET as string,
// CRITICAL: In Lambda, AWS credentials are auto-provided BUT we should NOT pass them explicitly
// The AWS SDK will automatically use the Lambda execution role if we don't provide credentials
credentials: undefined, // Always use Lambda's IAM role, never explicit credentials
Analysis
Missing environment variable validation in getDefaultConfig()
What fails: getDefaultConfig() in packages/world-aws/world/config.ts uses TypeScript non-null assertions (!) on three required environment variables (WORKFLOW_AWS_WORKFLOW_QUEUE_URL, WORKFLOW_AWS_STEP_QUEUE_URL, WORKFLOW_AWS_STREAM_BUCKET) without runtime validation. When these variables are missing, the function returns a config object with undefined values that later cause failures when passed to AWS SDK methods.
How to reproduce:
# In an environment where the required env vars are not set
unset WORKFLOW_AWS_WORKFLOW_QUEUE_URL
unset WORKFLOW_AWS_STEP_QUEUE_URL
unset WORKFLOW_AWS_STREAM_BUCKET
# Import and call getDefaultConfig
node -e "const {getDefaultConfig} = require('./packages/world-aws/dist/world/config.js'); getDefaultConfig();"Result before fix: Returns config object with undefined values for queue URLs and bucket. When these are passed to SQS operations (e.g., in packages/world-aws/world/queue.ts line 27-28), the AWS SDK throws "Missing required parameter QueueUrl".
Expected: Should immediately throw a clear error listing which required environment variables are missing, preventing silent failures.
Fix implemented: Added validation in getDefaultConfig() that:
- Checks for presence of required environment variables at function start
- Throws a clear error with list of missing variables if any are not set
- Replaced non-null assertions with type-safe variable access after validation
This ensures misconfiguration is caught immediately with a helpful error message instead of failing later in AWS SDK calls.
| const streams = new dynamodb.Table(this, 'WorkflowStreamChunks', { | ||
| tableName: 'workflow_stream_chunks', | ||
| partitionKey: { name: 'streamId', type: dynamodb.AttributeType.STRING }, | ||
| sortKey: { name: 'chunkIndex', type: dynamodb.AttributeType.NUMBER }, |
There was a problem hiding this comment.
| sortKey: { name: 'chunkIndex', type: dynamodb.AttributeType.NUMBER }, | |
| sortKey: { name: 'chunkId', type: dynamodb.AttributeType.STRING }, |
DynamoDB schema mismatch: The stream chunks table is defined with a NUMBER sort key (chunkIndex), but the code writes STRING values (chunkId) to it, causing DynamoDB operations to fail.
View Details
Analysis
DynamoDB Stream Chunks Table Schema Mismatch
What fails: WorkflowInfrastructure.createTables() defines the stream chunks DynamoDB table with a sort key named chunkIndex (NUMBER type), but createStreamer() writes items with a chunkId (STRING type) sort key, causing DynamoDB PutItem/UpdateItem operations to fail at runtime.
How to reproduce:
// In packages/world-aws/src/cdk/index.ts line 144
const streams = new dynamodb.Table(this, 'WorkflowStreamChunks', {
tableName: 'workflow_stream_chunks',
partitionKey: { name: 'streamId', type: dynamodb.AttributeType.STRING },
sortKey: { name: 'chunkIndex', type: dynamodb.AttributeType.NUMBER }, // ❌ Wrong
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
removalPolicy: cdk.RemovalPolicy.DESTROY,
});
// In packages/world-aws/world/streamer.ts line 38-67
const streamChunk: StreamChunk = {
streamId: name,
chunkId, // ❌ STRING value like "chnk_01ARYZ5ADG..."
s3Key,
eof: false,
createdAt: new Date().toISOString(),
};
await dynamoClient.send(
new PutCommand({
TableName: tableName,
Item: streamChunk, // Writes to table expecting NUMBER sort key
})
);Result: DynamoDB rejects the PutItem operation with validation error because the table schema expects a NUMBER attribute chunkIndex but receives a STRING attribute chunkId.
Expected: The sort key should be named chunkId (STRING type) to match what createStreamer() writes. This is correctly defined in packages/world-aws/lib/schema-utils.ts line 147 and packages/world-aws/lib/workflow-stack.ts line 192-194, where both use chunkId (STRING). Since WorkflowInfrastructure is exported in package.json exports.cdk, users deploying this infrastructure will encounter failures when attempting to write stream data.
1 participant
AWS World: Reference implementation for serverless workflow execution
Summary
This PR introduces
@workflow/world-awsas a reference implementation for running Workflow DevKit on AWS infrastructure. It demonstrates how to build a serverless world implementation using DynamoDB, SQS, S3, and Lambda, providing a blueprint for custom AWS-based workflow backends.Features
AWS Infrastructure Integration
Developer Experience
aws-workflowprovides commands for bootstrapping infrastructure, deploying workflows, and managing resourcesnpx aws-workflow bootstrap -ycreates all required AWS resourcesImplementation Details
Package Structure
@workflow/world-postgresstructure for consistency across reference implementations@workflow/tsconfigDocumentation
/docs/deploying/world/aws-worldStatus
Use Cases
This reference implementation can be used as:
Cost Estimate
Free tier eligible. Typical cost: $5-20/month for moderate usage.