Data Api plus Aurora Serverless v2

[max-webster]

This pull request updates the Aurora lending library example for Python, the Aurora item tracker example for Python, and the resources/cdk/aurora_serverless_app CloudFormation and CDK setup code. All with changes
to get everything to work with Aurora Serverless v2 and PostgreSQL instead of MySQL.

The Postgres engine switch is needed because currently, Data API for Serverless v2 only supports Postgres.

The PHP and Java examples that use this same resources/cdk/aurora_serverless_app resource setup should continue to work because I already added PostgreSQL-specific setup info into the READMEs. The actual SQL in those examples is generic and should work equally in PostgreSQL or MySQL.

Addresses #5722.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[max-webster]

This branch has conflicts that must be resolved

Hmm I did pull down main on March 15 and did all the conflict resolution with my files in the submitted branch. That was e9aa5e2. I expect some strangeness with python/cross_service/aurora_rest_lending_library/rds_tools/aurora_tools.py because it was deleted in a recent commit but I put it back. The other files that are listed are the ones that I conflict-resolved as part of my git rebase on the branch.

[max-webster]

I had to do a lot more in aurora_serverless_app/setup.ts than I wanted - retrieve the default VPC, get the AWS account ID and AWS Region out of environment variables, assert that it was OK to have public subnets in the VPC. And still there was a warning about the egress rules in the security group. It's possible that the CDK part could be simplified a bit and made more consistent with the setup.yaml file that I modified independently.

[max-webster]

The commit that caused the change in all the conflicting files is:

dac6a42 Python - SecretsManager : Remove snippets from Code Library & delete unused files (#5797)

which also touches a number of files outside the directories I'm concerned with.

[ford-at-aws]

Needs to not change 5,000+ files. Just the files you modified.

[DavidSouther]

Hi, Max - not sure how this got in this state, but this should untangle you:

git remote add aws-doc-sdk-examples [email protected]:awsdocs/aws-doc-sdk-examples.git
git switch master
git fetch aws-doc-sdk-examples main
git reset --hard aws-doc-sdk-examples/main
git switch -c tmp
git cherry-pick e9aa5e2
git cherry-pick 28bcd16
git cherry-pick 1456bb1
git switch data_api_plus_serverless_v2_final_final
git reset --hard tmp
git push --force

That should end up with a branch with just the three commits from Friday. If there were more that I missed in the log, of course add their cherry-picks in order. Here's the copy I made: https://github.com/awsdocs/aws-doc-sdk-examples/compare/main...DavidSouther:aws-doc-sdk-examples:tmp_max_webster?expand=1

[max-webster]

I don't think those 3 cherry-picks have everything. This bottom one should include about 10 files:

commit 2c122793bcd7a47e41535afea5570195fb946e90
Author: John Russell <[email protected]>
Date:   Fri Mar 15 13:35:08 2024 -0700

    Bring back bits of code needed for the Serverless v2 mods to work
    
    Some setup code and helper functions were deleted in an interim commit
    while these changes were in progress.

:100644 100644 dc3465636 ba411ffd6 M    python/cross_service/aurora_rest_lending_library/library_demo.py

Maybe those changes got diverted elsewhere in the history when I did the conflict resolution with

dac6a4224... Python - SecretsManager : Remove snippets from Code Library & delete unused files (#5797)

[DavidSouther]

Gotcha. In that case, you'll need to find them in your local clone's history? They're likely still findable in your reflog, though they may be hiding. Let me know if you'd like to schedule some pair programming time to see if we can find them?

[ford-at-aws]

Question - what happened with this PR that you said would resolve the need for using the SDK to create resources that can be created by the CDK?

I don't want to add scripts for creating resources unless we absolutely have to in order to showcase the feature you want to show.

[max-webster]

I used the extra DatabaseCluster attribute in resources/cdk/aurora_serverless_app/setup.ts to illustrate how to set up an Aurora Serverless v2 cluster + instance. In python/cross_service/aurora_rest_lending_library, I left the original Python setup code so that that example had the same constituent pieces as before. With Serverless v2, there is very little that's different on the application code side - the big difference is in the setup and teardown logic. So between the 2 examples (Lending Library with API calls and Item Tracker with CFN and CDK), I wanted to keep demonstrations of all the places you would take out the old keywords, function calls, and assumptions from Serverless v1 and what you would change them to in Serverless v2. Otherwise the crucial facts that people need to learn are abstracted and glossed over.

[max-webster]

I don't mind adding in an alternative setup mechanism as its own discrete change. I didn't delete the config.yml file from the Lending Library example. But let's work out the CDK best practices first with the Item Tracker updates, because that project was already using CDK. I had to make more changes in resources/cdk/aurora_serverless_app/setup.ts than I really wanted to related to VPCs and subnets. Maybe the result could be improved to handle all edge cases. (What if the user doesn't have environment variables declared for default account and region, what if the user doesn't have any VPCs or security groups set up already, what if the user is already at their account/region limit of 5 VPCs.)