Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: fail fast on forked process #2765

Merged
merged 3 commits into from
Aug 12, 2024
Merged

feat: fail fast on forked process #2765

merged 3 commits into from
Aug 12, 2024

Conversation

Tom-Newton
Copy link
Contributor

@Tom-Newton Tom-Newton commented Aug 12, 2024

Description

The tokio runtime does not support forked processes, so fail fast if we detect this scenario

  • Avoids difficult to debug deadlocks
  • Using the deltalake python bindings with multiprocessing can easily run into this problem because the default multiprocessing start method is fork on linux systems

Related Issue(s)

Documentation

@github-actions github-actions bot added the binding/python Issues for the Python package label Aug 12, 2024
Copy link

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

@Tom-Newton Tom-Newton changed the title Fail fast on forked process feat: Fail fast on forked process Aug 12, 2024
@Tom-Newton Tom-Newton changed the title feat: Fail fast on forked process feat: fail fast on forked process Aug 12, 2024
@ion-elgreco ion-elgreco added this pull request to the merge queue Aug 12, 2024
Copy link

@westonpace westonpace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea!

@westonpace
Copy link

westonpace commented Aug 12, 2024

(ignore my review...I thought I was looking at a different repo 🤦 )

Merged via the queue into delta-io:main with commit 9bdb8ac Aug 12, 2024
26 of 29 checks passed
@ion-elgreco
Copy link
Collaborator

@westonpace :P

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/python Issues for the Python package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Regression in Python multiprocessing support
4 participants