Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade memory for pieriandx glue lambdas #634

Merged
merged 7 commits into from
Nov 2, 2024

Conversation

@alexiswl alexiswl requested a review from victorskl October 31, 2024 23:47
@alexiswl alexiswl self-assigned this Oct 31, 2024
@victorskl victorskl added the fix label Oct 31, 2024
@victorskl victorskl linked an issue Oct 31, 2024 that may be closed by this pull request
@alexiswl
Copy link
Member Author

alexiswl commented Nov 1, 2024

Figured out the lambda error.

Thank you, you were right! Logging.info not writing to console. Lambdas was getting stuck waking up the redcap lambda because I used

props.redcapLambdaObj.latestVersion.grantInvoke(getDataFromRedCapPyLambdaObj.currentVersion);

But props.redcapLambdaObj.latestVersion ARN resolves to just arn:aws:lambda:ap-southeast-2:843407916570:function:redcap-apis-dev-lambda-function:$LATEST

Rather than the actual version number.

With print statements in the error section for the wake_up_lambdas step I get

Error warming up lambda: An error occurred (AccessDeniedException) when calling the Invoke operation: User: arn:aws:sts::843407916570:assumed-role/OrcaBusBeta-StackyMcStack-cttsov2topieriandxgetData-9VmCQBR11lGe/OrcaBusBeta-StackyMcStack-cttsov2topieriandxgetDat-jd4Ts7DTHeYC is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:ap-southeast-2:843407916570:function:redcap-apis-dev-lambda-function because no identity-based policy allows the lambda:InvokeFunction action

So instead of
props.redcapLambdaObj.latestVersion.grantInvoke(getDataFromRedCapPyLambdaObj.currentVersion);
Instead need to switch to

    getDataFromRedCapPyLambdaObj.addToRolePolicy(
      new iam.PolicyStatement(
        {
          actions: ['lambda:Invoke'],
          resources: [
            `${props.redcapLambdaObj.functionArn}`,
            `${props.redcapLambdaObj.functionArn}:${props.redcapLambdaObj.latestVersion.version}`
          ],
        }
      )
    )

The fact that logger.info isn't working is concerning, I used the logger command across almost all python functions, will try debug this for a future PR

@alexiswl
Copy link
Member Author

alexiswl commented Nov 1, 2024

    getDataFromRedCapPyLambdaObj.addToRolePolicy(
      new iam.PolicyStatement(
        {
          actions: ['lambda:Invoke'],
          resources: [
            `${props.redcapLambdaObj.functionArn}`,
            `${props.redcapLambdaObj.functionArn}:${props.redcapLambdaObj.latestVersion.version}`
          ],
        }
      )
    )

This didn't work, latestVersion.version also returns just $LATEST rather than the number, using a NagSuppression instead

@alexiswl alexiswl force-pushed the bugfix/upgrade-lambda-mem-sizes branch 2 times, most recently from 1eb597a to 19f3ce5 Compare November 1, 2024 07:57
@victorskl
Copy link
Member

Suppress the nag globally if need be. For now.

Would be nice if we can merge this fix in. Before Run 236 hits to O3. No pressure! :D

@alexiswl
Copy link
Member Author

alexiswl commented Nov 2, 2024

Ditto, adding the cdk nag suppression after the infringing line / construct method resolved the issue

@alexiswl alexiswl merged commit 6da7013 into main Nov 2, 2024
6 checks passed
@alexiswl alexiswl deleted the bugfix/upgrade-lambda-mem-sizes branch November 2, 2024 02:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add PierianDx Pipeline Manager as a service
2 participants