"Failed to build the project and import the benchmark suite" with no apparent errors in output

[sflis]

I was trying out asv with my project but failed to get it event to run getting this output

$asv run
· Creating environments
· Discovering benchmarks
·· Uninstalling from virtualenv-py3.12
·· Building 698da480 <CAV-101376-expose-encoding-field> for virtualenv-py3.12...
·· Installing 698da480 <CAV-101376-expose-encoding-field> into virtualenv-py3.12
·· Failed to build the project and import the benchmark suite. 

when running in verbose mode I'm getting this:

   Building wheels for collected packages: dcap
     Building wheel for dcap (pyproject.toml): started
     Building wheel for dcap (pyproject.toml): finished with status 'done'
     Created wheel for dcap: filename=dcap-0.0.4.post17+git.698da480-py3-none-any.whl size=22315 sha256=a208536bddb65150a963462139a4156ab570ce73abc7129e43a41357908e501e
     Stored in directory: /tmp/pip-ephem-wheel-cache-3qlrv7ij/wheels/d6/c4/aa/2fddef2f981ba26e6e5d96b79c993cb2203ead973b3aa818db
   Successfully built dcap
·· Running '/usr/bin/git name-rev --name-only --exclude=remotes/* --no-undefined 698da480f911c26d790dc1062f19da2d6ea96690'
   OUTPUT -------->
   CAV-101376-expose-encoding-field
·· Installing 698da480 <CAV-101376-expose-encoding-field> into virtualenv-py3.12
·· Failed to build the project and import the benchmark suite.

with still no apparent errors, this includes the previous steps omitted from the output shown above. What would be the next step to track down the issue?

[HaoZeke]

Can you provide more details for reproducibility? Crucially, the repo (if it is open source) and also if the command used to install the project normally works outside ASV.

[bdice]

I'm seeing the same problem, and another user reported the same. glotzerlab/signac#1017 (comment)

$ asv run in https://github.com/glotzerlab/signac/ is enough to reproduce the failure.

Here is a more extensive reproducer with an older Python version (3.10 instead of 3.12):

git clone https://github.com/glotzerlab/signac
cd signac
conda create -n py310 python=3.10
conda activate py310
pip install -e .
pip install asv
asv run

Output:

$ asv run
Couldn't load asv.plugins._mamba_helpers because
No module named 'libmambapy'
· Creating environments...
· Discovering benchmarks
·· Uninstalling from virtualenv-py3.10
·· Building 58c40d59 <main> for virtualenv-py3.10....
·· Installing 58c40d59 <main> into virtualenv-py3.10
·· Failed to build the project and import the benchmark suite.

I don't think the No module named 'libmambapy' is relevant here, I tried it in another environment and didn't see that warning but got the same failure.

[bdice]

Update: I saw #1416 (comment) and found that using asv==0.6.1 fixes the problem. 0.6.2 shows the errors seen above. (Current version is 0.6.4.)

[hchau630]

Following https://asv.readthedocs.io/en/latest/asv.conf.json.html#build-command-install-command-uninstall-command, I fixed this by replacing
"python -mpip wheel -w {build_cache_dir} {build_dir}"
under build_command in asv.conf.json with
"python -mpip wheel --no-deps --no-build-isolation --no-index -w {build_cache_dir} {build_dir}".
This seems consistent with the observation that asv==0.6.1 fixes the issue since this is modified in 0.6.2.

[HaoZeke]

@bdice if @hchau630's fix works for you I'll close this out, since the change of default build is documented (but certainly a bit surprising).

[bdice]

Thanks for the help @HaoZeke and @hchau630!

I'm a little surprised that the default command would change significantly in a patch release. I have filed a PR to try and fix signac. glotzerlab/signac#1021

I am now able to run benchmarks with asv run. I have some issues with asv publish / asv preview where it doesn't seem to find the benchmark results for the commit hashes that it ran, but that's probably a separate issue. It's been quite a while since I've tried to use asv so I'm probably forgetting something.