Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport #40412 to 14.0.2 #1369

Merged
merged 4 commits into from
Apr 23, 2024
Merged

Conversation

galipremsagar
Copy link
Contributor

@galipremsagar galipremsagar commented Apr 18, 2024

This PR backports a python memory leak patch to 14.0.2: apache/arrow#40412

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

@galipremsagar
Copy link
Contributor Author

@conda-forge-admin, please rerender

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@galipremsagar galipremsagar changed the title Backport #40412 to 14.0.2 Backport #40412 to 14.0.2 Apr 18, 2024
Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@h-vetinari h-vetinari added the automerge Merge the PR when CI passes label Apr 18, 2024
@galipremsagar
Copy link
Contributor Author

@h-vetinari This might need a CI restart too

@galipremsagar galipremsagar mentioned this pull request Apr 19, 2024
5 tasks
@galipremsagar
Copy link
Contributor Author

@conda-forge-admin, please rerender

@github-actions github-actions bot removed the automerge Merge the PR when CI passes label Apr 20, 2024
Copy link
Contributor

Hi! This is the friendly conda-forge automerge bot!

Commits were made to this PR after the automerge label was added. For security reasons, I have disabled automerge by removing the automerge label. Please add the automerge label again (or ask a maintainer to do so) if you'd like to enable automerge again!

@jakirkham
Copy link
Member

Had tried restarting the failing job yesterday evening, but it still fails. Unfortunately the log is sparse on details except for the job stopping at some point. Typically this is a sign of resource limitations

Guessing this is either too many threads or too little space on the VM. Option to try would be...

  1. Reduce # of threads
  2. Freeing more space
  3. Using a swap file

More details on 2 & 3 in this doc, which would involve updating this conda-forge.yml section

azure:
free_disk_space: true

@h-vetinari
Copy link
Member

The aarch builds have gotten incredibly flaky recently, but were fine before. Most of the issues happens in the tests, where we're getting errors for fetching something from the web, presumably because the QEMU emulation is too slow. The jobs just straight up dying happens much more rarely, when we get a particularly slow agent.

Freeing more space

You're literally pointing out how we're doing that already...? There's no "more" either - it's on or off. I doubt that reducing the threads helps (will just make the build even slower and more prone to timeouts), and I doubt that the memory is the issue, but happy to be proven wrong. 🙃

@jakirkham
Copy link
Member

Gotcha thanks for the context

Well there are finer grained settings that do more. They are not turned on by default as they can be a bit slower and in a couple cases builds needed the components removed

That said, no need to spend time doing that if we don't think it will help. Just think up options

@h-vetinari
Copy link
Member

This has become a reproducible failure at

[134/139] Building CXX object CMakeFiles/gandiva.dir/gandiva.cpp.o
[135/139] Linking CXX shared module release/gandiva.cpython-311-aarch64-linux-gnu.so

for both v14 & v15 (though not v12 & v13).

@h-vetinari
Copy link
Member

It's a GCC 12 problem. The CUDA build passes because it uses GCC 10; Downgrading the non-CUDA job to GCC 10 would be possible, but I've also tested that GCC 13 works and prefer that. I'll finish this off with the libgoogle migration, and will come back to this afterwards.

@h-vetinari h-vetinari merged commit 5779fc8 into conda-forge:14.x Apr 23, 2024
1 of 12 checks passed
@galipremsagar
Copy link
Contributor Author

Thanks @h-vetinari !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants