Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

concept to fix #2339 #2340

Closed
wants to merge 1 commit into from
Closed

concept to fix #2339 #2340

wants to merge 1 commit into from

Conversation

taylordowns2000
Copy link
Member

Description

This PR drops dataclips when they're too big to fix #2339

Validation steps

  1. Set up a workflow that creates an 11MB dataclip at some point in the workflow
  2. Run it and verify that the dataclips are NOT saved for each step that's over 10MB

Pre-submission checklist

  • I have performed a self-review of my code.
  • I have implemented and tested all related authorization policies. (e.g., :owner, :admin, :editor, :viewer)
  • I have updated the changelog.

@taylordowns2000 taylordowns2000 requested a review from stuartc July 29, 2024 15:35
@taylordowns2000
Copy link
Member Author

closing as this is likely too late to check byte_size. OpenFn/kit#740 will prevent us from sending really big payloads back to lightning, but we should still apply some more defensive stuff on this side

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

40mb dataclips can crash the web app
1 participant