Gravity task and distance task QPIK test fails on windows #799
Labels
bug
Something isn't working
Comments
|
To simplify the debug:
|
|
That's the diff between the two conda envs. The main difference I see is iDynTree and osqp @traversaro 5c5
< aom 3.5.0 h63175ca_0 conda-forge
---
> aom 3.6.1 h63175ca_0 conda-forge
7c7
< assimp 5.2.5 h4dcb625_0 conda-forge
---
> assimp 5.3.1 h4dcb625_1 conda-forge
13c13
< casadi 3.6.2 py310ha06830a_0 conda-forge
---
> casadi 3.6.3 py310hc65f482_3 conda-forge
21c21
< dav1d 1.2.0 hcfcfb64_0 conda-forge
---
> dav1d 1.2.1 hcfcfb64_0 conda-forge
27c27
< ffmpeg 5.1.2 gpl_h5037a79_109 conda-forge
---
> ffmpeg 5.1.2 gpl_h8bb4bc8_112 conda-forge
42a43
> glfw 3.3.9 hcfcfb64_0 conda-forge
53c54
< idyntree 8.1.0 py310h481cdde_1 conda-forge
---
> idyntree 10.2.0 py310h042274b_0 conda-forge
58c59
< ipopt 3.14.11 ha9547d1_0 conda-forge
---
> ipopt 3.14.13 ha9547d1_0 conda-forge
69,70c70,71
< libclang 15.0.7 default_h77d9078_3 conda-forge
< libclang13 15.0.7 default_h77d9078_3 conda-forge
---
> libclang 15.0.7 default_hde6756a_4 conda-forge
> libclang13 15.0.7 default_h85b4d89_4 conda-forge
77c78
< libhwloc 2.9.1 h51c2c0f_0 conda-forge
---
> libhwloc 2.9.3 default_haede6df_1009 conda-forge
87c88
< libosqp 0.6.2 h63175ca_4 conda-forge
---
> libosqp 0.6.3 h63175ca_0 conda-forge
95c96
< libunicycle-footstep-planner 0.6.0 h63175ca_0 conda-forge
---
> libunicycle-footstep-planner 0.7.0 h63175ca_0 conda-forge
98c99
< libxml2 2.10.4 hc3477c8_0 conda-forge
---
> libxml2 2.12.4 hc3477c8_1 conda-forge
109c110
< metis 5.1.0 h63175ca_1007 conda-forge
---
> metis 5.1.1 h63175ca_2 conda-forge
121c122
< osqp-eigen 0.8.0 h63175ca_0 conda-forge
---
> osqp-eigen 0.8.1 h6d7489e_0 conda-forge
129c130
< proxsuite 0.3.7 py310h5588dad_0 conda-forge
---
> proxsuite 0.5.1 py310h5588dad_1 conda-forge
229c230
< tbb 2021.9.0 h91493d7_0 conda-forge
---
> tbb 2021.11.0 h91493d7_0 conda-forge
275c276
< USERDOMAIN_ROAMINGPROFILE=fv-az741-700
---
> USERDOMAIN_ROAMINGPROFILE=fv-az1112-453
278c279
< HOSTNAME=fv-az741-700
---
> HOSTNAME=fv-az1112-453
280c281
< GITHUB_PATH=D:\a\_temp\_runner_file_commands\add_path_13b8f5dc-a1bb-4553-ae5d-846d4ce8c280
---
> GITHUB_PATH=D:\a\_temp\_runner_file_commands\add_path_cdab868f-d3f1-45e2-95a3-55d06301913d
288,289c289,290
< GITHUB_RUN_NUMBER=4071
< RUNNER_NAME=GitHub Actions 6
---
> GITHUB_RUN_NUMBER=4072
> RUNNER_NAME=GitHub Actions 5
306c307
< USERDOMAIN=fv-az741-700
---
> USERDOMAIN=fv-az1112-453
343c344
< RUNNER_TRACKING_ID=github_687aa1cb-3cde-4d1e-aba6-9032b5f8e5bf
---
> RUNNER_TRACKING_ID=github_670ff035-73ae-4224-890a-b2457d691e6b
350c351
< GITHUB_STATE=D:\a\_temp\_runner_file_commands\save_state_13b8f5dc-a1bb-4553-ae5d-846d4ce8c280
---
> GITHUB_STATE=D:\a\_temp\_runner_file_commands\save_state_cdab868f-d3f1-45e2-95a3-55d06301913d
355c356
< GITHUB_ENV=D:\a\_temp\_runner_file_commands\set_env_13b8f5dc-a1bb-4553-ae5d-846d4ce8c280
---
> GITHUB_ENV=D:\a\_temp\_runner_file_commands\set_env_cdab868f-d3f1-45e2-95a3-55d06301913d
361c362
< GITHUB_RUN_ID=7523465014
---
> GITHUB_RUN_ID=7536153930
365c366
< GITHUB_STEP_SUMMARY=D:\a\_temp\_runner_file_commands\step_summary_13b8f5dc-a1bb-4553-ae5d-846d4ce8c280
---
> GITHUB_STEP_SUMMARY=D:\a\_temp\_runner_file_commands\step_summary_cdab868f-d3f1-45e2-95a3-55d06301913d
367c368
< COMPUTERNAME=fv-az741-700
---
> COMPUTERNAME=fv-az1112-453
376c377
< LOGONSERVER=\\fv-az741-700
---
> LOGONSERVER=\\fv-az1112-453
449c450
< GITHUB_OUTPUT=D:\a\_temp\_runner_file_commands\set_output_13b8f5dc-a1bb-4553-ae5d-846d4ce8c280
---
> GITHUB_OUTPUT=D:\a\_temp\_runner_file_commands\set_output_cdab868f-d3f1-45e2-95a3-55d06301913d |
I think the problem is metis 5.1.1 . It created problems in other packages (see conda-forge/mumps-feedstock#106) and this is why in conda-forge we reverted back to pin metis to 5.1.0 right this afternoon: conda-forge/conda-forge-pinning-feedstock#5396 . I tought we were safe from that as we did not migrated ipopt to metis 5.1.1 (see conda-forge/ipopt-feedstock#97), but I guess that ipopt Windows packages do not depend directly on metis, and so once a new mumps compiled against metis 5.1.1 version was released, we ended up with ipopt installed with metis 5.1.1 . The good news is that we are now safe, we just need to compile ipopt with the latest mumps (happening soon, the migration to 5.6.2 was merged in conda-forge/conda-forge-pinning-feedstock#5274 and the bot will start to open PRs soon) and the CI will be fixed. |
1 task
|
The nice thing is that I did metis 5.1.1 related comments and fixes the last few days, but I was not aware that it already was affecting blf! :D |
|
See https://conda-forge.org/status/#mumps_mpi562 for the migration that should as a side effect fix this. |
|
I was thinking more a problem of osqp since it is the solver we use for the IK. I don't know if osqp is affected by metis |
|
Actually, also osqp changed from 0.6.2 to 0.6.3, as @GiulioRomualdi noticed above |
|
Yes, exactly, the IK uses osqp 🤔 |
|
Ah, you are right, I was confusing this with iDynTree's IK. |
|
I reproduced the failure locally with a minimal env create as: detail env: |
|
Also forcing libosqp to 0.6.2 result in the same error being generated: |
|
An env created with: works fine instead. |
|
Ok, after a few iterations I found two environments quite similar (with same libosqp and osqp-eigen version, but one that fails, and one not. One that fails: One that is successful: Diff between the two: --- <unnamed>
+++ <unnamed>
@@ -1,10 +1,10 @@
-# Not working
+# Workingyy
aom 3.5.0 h63175ca_0 conda-forge
assimp 5.2.5 h4dcb625_0 conda-forge
boost-cpp 1.78.0 h9f4b32c_4 conda-forge
bzip2 1.0.8 hcfcfb64_5 conda-forge
ca-certificates 2023.11.17 h56e8100_0 conda-forge
-casadi 3.6.2 py311h4ef4915_1 conda-forge
+casadi 3.6.2 py311hd11293b_0 conda-forge
cmake 3.28.1 hf0feee3_0 conda-forge
dav1d 1.2.1 hcfcfb64_0 conda-forge
eigen 3.4.0 h91493d7_0 conda-forge
@@ -20,11 +20,10 @@
fonts-conda-forge 1 0 conda-forge
freetype 2.12.1 hdaf720e_2 conda-forge
gettext 0.21.1 h5728263_0 conda-forge
-glfw 3.3.9 hcfcfb64_0 conda-forge
gtest 1.14.0 h91493d7_1 conda-forge
-idyntree 9.1.0 py311h42847bd_0 conda-forge
+idyntree 8.1.0 py311he5a1352_1 conda-forge
intel-openmp 2023.2.0 h57928b3_50497 conda-forge
-ipopt 3.14.12 ha9547d1_1 conda-forge
+ipopt 3.14.11 ha9547d1_0 conda-forge
irrlicht 1.8.5 h65f4d7e_4 conda-forge
krb5 1.21.2 heb0366b_0 conda-forge
libblas 3.9.0 20_win64_mkl conda-forge
@@ -45,7 +44,7 @@
libqdldl 0.1.5 h63175ca_1 conda-forge
libsqlite 3.44.2 hcfcfb64_0 conda-forge
libssh2 1.11.0 h7dfc565_0 conda-forge
-libunicycle-footstep-planner 0.6.0 h63175ca_1 conda-forge
+libunicycle-footstep-planner 0.6.0 h63175ca_0 conda-forge
libuv 1.44.2 hcfcfb64_1 conda-forge
libxml2 2.12.4 hc3477c8_1 conda-forge
libzlib 1.2.13 hcfcfb64_5 conda-forge
|
|
So, at this point, I was wondering if there is something in idyntree. Notice that the IK uses idyntree to compute jacobians and transforms. Ipopt and casadi are not linked to the IK (afak) |
|
I also suspect some subtle ABI incompatibility doing something here, even if the perfectly repeatable error seems to indicate otherwise. There is a bunch of rebuilds being done due to new ipopt, new mumps, old metis, let's see what happens after all of that. |
|
Ok, I created an env with the latest mumps and ipopt and idyntree, and it still fails: output: |
|
I wonder if this failure is related to the change in |
Ok, indeed running that test just on a model with revolute joints solve the problem (I got the int by the fact that in robotology/idyntree#1057 we also run the IK tests in iDynTree for revolute joints). I guess the problem is indeed in some kind of bug with Jacobian on models with prismatic joints, that have been in iDynTree since prismatic joints were added, but was hidden by the fact that prismatic joints were not actually tested in getRandomModel and in KinDynComputations tests. As the problem is actually in iDynTree, I think it make sense just to test revolute joints in BLF. I guess there is nothing specifically Windows-related in the error, we were just kind of "lucky" to get this on Windows. I guess that if we run multiple times |
|
(unrelated, diff for debug, posting it not to lose it): @@ -967,6 +1003,12 @@ TEST_CASE("QP-IK [Distance and Gravity tasks Unconstrained]")
baseVelocity = ik->getOutput().baseVelocity.coeffs();
jointVelocity = ik->getOutput().jointVelocity;
+ std::cerr << "--- iteration: " << iteration << std::endl;
+ std::cerr << "--- baseVelocity:\n" << baseVelocity << std::endl;
+ std::cerr << "--- jointVelocity:\n" << jointVelocity << std::endl;
+ std::cerr << "--- world_H_target current translation norm: " << toManifPose(kinDyn->getWorldTransform(desiredSetPoints.targetFrameDistance)).translation().norm() << std::endl;
+ std::cerr << "--- world_H_target desired translation norm: " << desiredSetPoints.targetDistance << std::endl;
+
// propagate the dynamical system
system.dynamics->setControlInput({baseVelocity, jointVelocity});
system.integrator->integrate(0s, dT); |
traversaro
added a commit
to traversaro/bipedal-locomotion-framework
that referenced
this issue
Jan 21, 2024
See discussion in ami-iit#799 (comment) Workaround for ami-iit#799 The actual issue is probably in iDynTree and will be tackled in robotology/idyntree#1149
|
#800 should provide a workaround for the problem in blf, the actual issue will be tracked in robotology/idyntree#1149 . |
GiulioRomualdi
pushed a commit
that referenced
this issue
Jan 22, 2024
See discussion in #799 (comment)
|
Fixed by #800! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In #790 I noticed that Gravity task and distance task QPIK test failed on windows @EhsanRanjbari and @S-Dafarra.
As @S-Dafarra mentioned this is weird since the tests started failing recently with a huge error. This issue aims to investigate what is happening
Originally posted by @GiulioRomualdi in #790 (comment)
The text was updated successfully, but these errors were encountered: