mangum does not cache database connection pools

[geospatial-jeff]

It appears that mangum runs ASGI lifecycle events (startup, shutdown etc.) on each lambda function invocation instead of run once and cached. This adds an extra ~200ms of runtime to each invocation and can exhaust available database connections unless lambda is calling through a proxy (ex. RDS Proxy) configured to aggressively drop idle connections.

This seems to fix the problem for pgstac:

# app.py

async def connect_to_database(settings: Settings):
    db = DB()
    logger.info("Connecting to database.")
    readpool = await db.create_pool(settings.reader_connection_string, settings)
    writepool = await db.create_pool(settings.writer_connection_string, settings)
    logger.info("Succesfully connected to database.")
    return readpool, writepool

# Define read/write pool in global scope, lambda will cache across invocations.
loop = asyncio.get_event_loop()
READPOOL, WRITEPOOL = loop.run_until_complete(connect_to_database(settings))

# Update startup event.
# Also need to delete the shutdown event.
@app.on_event("startup")
async def startup_event():
    """Connect to database on startup."""
    app.state.readpool = READPOOL
    app.state.writepool = WRITEPOOL

[vincentsarago]

I think the easiest way to fix this in the lambda handler is to do

handler = Mangum(app, lifespan="off")


if "AWS_EXECUTION_ENV" in os.environ:
    loop = asyncio.get_event_loop()
    loop.run_until_complete(app.router.startup())

ref Kludex/mangum#211

The issue then is that the shutdown is never reached

[alukach]

I think a section in the documentation regarding runtime-dependent tips would be helpful. In addition to this topic, disabling pgSTAC's default DB pooling parameters (ie setting both to 1) can be very impactful for limiting pressure on DB connections.

stac-fastapi/stac_fastapi/pgstac/stac_fastapi/pgstac/config.py

Lines 32 to 33 in 5e1ea82

	db_min_conn_size: int = 10
	db_max_conn_size: int = 10

[alukach]

@vincentsarago I tested this fix but it appears that the startup and shutdown events still trigger:

[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:01 AM Running in AWS mode
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:01 AM starting app
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:01 AM START RequestId: 9cc5e438-cac0-4f3c-9267-e539c7f15a99 Version: $LATEST
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:01 AM starting app
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM shutting down app
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM END RequestId: 9cc5e438-cac0-4f3c-9267-e539c7f15a99
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM REPORT RequestId: 9cc5e438-cac0-4f3c-9267-e539c7f15a99    Duration: 388.32 ms      Billed Duration: 389 ms Memory Size: 128 MB     Max Memory Used: 70 MB  Init Duration: 559.05 ms
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM START RequestId: d4c3c231-dfcf-464e-8e20-672d3373da14 Version: $LATEST
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM starting app
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM shutting down app
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM END RequestId: d4c3c231-dfcf-464e-8e20-672d3373da14
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM REPORT RequestId: d4c3c231-dfcf-464e-8e20-672d3373da14    Duration: 20.45 ms       Billed Duration: 21 ms  Memory Size: 128 MB     Max Memory Used: 70 MB
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM START RequestId: 9d699e80-9ae2-4d1a-8aab-4f4ec4c7d6c3 Version: $LATEST
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM starting app
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM shutting down app
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM END RequestId: 9d699e80-9ae2-4d1a-8aab-4f4ec4c7d6c3
[/aws/lambda/FastApiTest-MyFunction3BAA72D1-6NPaUVZDVlQV] 7:23:02 AM REPORT RequestId: 9d699e80-9ae2-4d1a-8aab-4f4ec4c7d6c3    Duration: 5.34 ms        Billed Duration: 6 ms   Memory Size: 128 MB     Max Memory Used: 70 MB

[vincentsarago]

@alukach you need to set lifespan="off" Kludex/mangum#211 (comment)