Skip to content

[testing] Add validation for concurrent cache in Conan #18098

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: develop2
Choose a base branch
from

Conversation

uilianries
Copy link
Member

Hello!

This new validation introduces a test for concurrent. This initial validation started in this gist: https://gist.github.com/uilianries/e0438f6ac92f8444a1cb89c6faa3a4a7

So far, we found some conflicts:

  • The test package uses the same folder when building, so parallel build will overwrite it
  • Export source and Export will overwrite the same artifacts
  • The result in Mac is a busy/locked file and folder

/cc @AbrilRBS @memsharded

Changelog: Omit
Docs: Omit

  • Refer to the issue that supports this Pull Request.
  • If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
  • I've read the Contributing guide.
  • I've followed the PEP8 style guides for Python code.
  • I've opened another PR in the Conan docs repo to the develop branch, documenting this one.

from conan.test.assets.genconanfile import GenConanfile
from conan.test.utils.tools import TestClient

_parallel_options = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to start first with concurrency for conan config install/install-pkg which hopefully would be a bit easier problem

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the test test_parallel_config to validate conan config install

"""
num_builds = len(_parallel_options)
test_client = TestClient()
test_client.run("new cmake_lib -d name=parallel -d version=0.1.0")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The concurrency over test_package would be a different problem. Lets focus first on the cache concurrency, the test_package can have other mechanisms, for example using build_folder_vars using the option will create independent folders.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified the test to execute without test package. Plus, added a new one only to validate conan export. Please, see 86e341d

It's not possible to guarantee the concurrent failure as it may depends
on the operate system as well. For instance, in MacOS the failure ocurrs
less than in Linux (based on tests executed in the CI)

Signed-off-by: Uilian Ries <[email protected]>
Signed-off-by: Uilian Ries <[email protected]>
@uilianries uilianries marked this pull request as ready for review April 9, 2025 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants