Crash with metis 5.1.1 #106
Comments
|
It's a different segfault caught by our updated tests (that's why Here is the crash in CI, but I'll provide a more self-contained example in a bit. |
|
I have investigated the failure more, and I have arrived to the following reproducer that works on Ubuntu 23.10. First Make an environment name: mumps_bug
channels:
- conda-forge
dependencies:
- mumps-seq=5.2.1
- metis=5.1.1
- kwant=1.4.4
- valgrind
- pytest
- pip
- pip:
- pytest-valgrindDownload this test file contentsimport itertools
import numpy as np
import pytest
from pytest import raises
from numpy.testing import assert_almost_equal
import kwant
from kwant._common import ensure_rng
import kwant.solvers.sparse
import kwant.solvers.mumps
no_mumps = False
mumps_solver_options = [
{'nrhs': 10, 'ordering': 'metis'},
{'nrhs': 10, 'sparse_rhs': True, 'ordering': 'metis'},
{'nrhs': 2, 'ordering': 'metis', 'sparse_rhs': True},
]
solvers = list(itertools.chain(
[("mumps", opts) for opts in mumps_solver_options],
))
def solver_id(s):
solver_name, opts = s
args = ", ".join(f"{k}={repr(v)}" for k, v in opts.items())
return f"{solver_name}({args})"
@pytest.fixture(scope="function", params=mumps_solver_options)
def solver(request):
solver_opts = request.param
solver = kwant.solvers.mumps
solver.options(**solver_opts)
return solver
@pytest.fixture
def smatrix(solver):
return solver.smatrix
@pytest.fixture
def greens_function(solver):
return solver.greens_function
@pytest.fixture
def wave_function(solver):
return solver.wave_function
@pytest.fixture(scope="function")
def twolead_builder():
rng = ensure_rng(4)
system = kwant.Builder()
left_lead = kwant.Builder(kwant.TranslationalSymmetry((-1,)))
right_lead = kwant.Builder(kwant.TranslationalSymmetry((1,)))
for b, site in [(system, chain(0)), (system, chain(1)),
(left_lead, chain(0)), (right_lead, chain(0))]:
h = rng.random_sample((n, n)) + 1j * rng.random_sample((n, n))
h += h.conjugate().transpose()
b[site] = h
for b, hopp in [(system, (chain(0), chain(1))),
(left_lead, (chain(0), chain(1))),
(right_lead, (chain(0), chain(1)))]:
b[hopp] = (10 * rng.random_sample((n, n)) +
1j * rng.random_sample((n, n)))
system.attach_lead(left_lead)
system.attach_lead(right_lead)
return system
n = 5
chain = kwant.lattice.chain(norbs=n)
sq = square = kwant.lattice.square(norbs=n)
def test_output(twolead_builder, smatrix):
fsyst = twolead_builder.finalized()
result1 = smatrix(fsyst)
s, modes1 = result1.data, result1.lead_info
assert s.shape == 2 * (sum(len(i.momenta) for i in modes1) // 2,)
s1 = result1.submatrix(1, 0)
result2 = smatrix(fsyst, 0, (), [1], [0])
s2, modes2 = result2.data, result2.lead_info
assert s2.shape == (len(modes2[1].momenta) // 2,
len(modes2[0].momenta) // 2)
assert_almost_equal(abs(s1), abs(s2))
assert_almost_equal(np.dot(s.T.conj(), s),
np.identity(s.shape[0]))
raises(ValueError, smatrix, fsyst, out_leads=[])
modes = smatrix(fsyst).lead_info
h = fsyst.leads[0].cell_hamiltonian()
t = fsyst.leads[0].inter_cell_hopping()
modes1 = kwant.physics.modes(h, t)[0]
h = fsyst.leads[1].cell_hamiltonian()
t = fsyst.leads[1].inter_cell_hopping()
modes2 = kwant.physics.modes(h, t)[0]
raise
def test_smatrix_shape(smatrix):
chain = kwant.lattice.chain(norbs=1)
system = kwant.Builder()
lead0 = kwant.Builder(kwant.TranslationalSymmetry((-1,)))
lead1 = kwant.Builder(kwant.TranslationalSymmetry((1,)))
for b, site in [(system, chain(0)), (system, chain(1)),
(system, chain(2))]:
b[site] = 2
lead0[chain(0)] = lambda site: lead0_val
lead1[chain(0)] = lambda site: lead1_val
for b, hopp in [(system, (chain(0), chain(1))),
(system, (chain(1), chain(2))),
(lead0, (chain(0), chain(1))),
(lead1, (chain(0), chain(1)))]:
b[hopp] = -1
system.attach_lead(lead0)
system.attach_lead(lead1)
fsyst = system.finalized()
lead0_val = 4
lead1_val = 4
s = smatrix(fsyst, 1.0, (), [1], [0]).data
assert s.shape == (0, 0)
lead0_val = 2
lead1_val = 2
s = smatrix(fsyst, 1.0, (), [1], [0]).data
assert s.shape == (1, 1)
lead0_val = 4
lead1_val = 2
s = smatrix(fsyst, 1.0, (), [1], [0]).data
assert s.shape == (1, 0)
lead0_val = 2
lead1_val = 4
s = smatrix(fsyst, 1.0, (), [1], [0]).data
assert s.shape == (0, 1)
def test_reflection_no_open_modes(greens_function):
# Build system
syst = kwant.Builder()
lead = kwant.Builder(kwant.TranslationalSymmetry((-1, 0)))
syst[(square(i, j) for i in range(3) for j in range(3))] = 4
lead[(square(0, j) for j in range(3))] = 4
syst[square.neighbors()] = -1
lead[square.neighbors()] = -1
syst.attach_lead(lead)
syst.attach_lead(lead.reversed())
syst = syst.finalized()
# Sanity check; no open modes at 0 energy
_, m = syst.leads[0].modes(energy=0)
assert m.nmodes == 0
assert np.isclose(greens_function(syst).transmission(0, 0), 0)Place the file in an empty folder and activate the environment. Observe that running Furthermore, running valgrind using |
|
Also: installing |
|
Investigating the issues with Metis 5.1.1, it seems unavoidable. I suggest skipping 5.1.1 and waiting until 5.2.1 arrives to the feedstock (conda-forge/metis-feedstock#41 if it succeeds). |
|
Thanks a lot for the thorough investigation @akhmerov, I totally agree. At this point, considering also the other failures we are seeing with metis 5.1.1 (conda-forge/gtsam-feedstock#21) we could consider reverting the migration to 5.1.1 at the conda-forge level, and stick to 5.1.0 until metis 5.2.1 is available. This has the downside that dgl will not be installable side-by-side with other conda-forge packages that depend on metis, but if anyone really needs that they can invest in the work either in packaging metis 5.2.1 or ensuring that the package of interest build for both metis 5.1.0 and 5.1.1 . Any opinion on this @conda-forge/metis @conda-forge/dgl @conda-forge/mumps ? |
|
I'll try and take a look into this more tomorrow, thank you so much for investigating! |
|
As a partial update @traversaro @akhmerov we have some friends at Quansight looking into fixing the METIS 5.2.1 build (hopefully). Will keep you posted. |
|
This is becoming a blocker for using the feedstock on windows. There:
(I'm just working on a feedstock over here: conda-forge/staged-recipes#25042) |
|
The basic blocker is KarypisLab/GKlib#23 (comment). If we don't get a response soon we will do a release targeting latest sha. |
|
I'm a bit concerned about counting on that, see the evaluation of Metis 5.2 by SuiteSparse DrTimothyAldenDavis/SuiteSparse#291 (comment) |
|
Would it be an appropriate solution to have build variants for metis 5.1.0 and 5.2.1 (when it's available)? Looking at #88, the only difference is whether to apply the patch. On the other hand only packaging for a library that wasn't tested (metis 5.2.1) or even isn't releasable right now seems like a potential for a lot of pain for the users. |
|
Personally I am in favor of stopping the metis 5.1.1 migration and switching back to metis 5.1.0 here and in the other migrated feedstocks (see https://conda-forge.org/status/#metis511). Once metis 5.2.1 is ready, we can try it and if it works fine proceed with the 5.2.1 migration, I am not sure if any other @conda-forge/mumps @conda-forge/metis have any other opinion. I would be happy to do the necessary PRs to stop the migration and revert migrated repos to 5.1.0 . |
|
Since the packages are now effectively broken, I would really appreciate that. |
|
conda-forge/conda-forge-pinning-feedstock#5396 halts the migration. Rebuilds can start once that lands. Looks like only 10 packages. @Traverso feel free to ping me on any un-migrate PRs |
|
More examples of metis 5.1.1 problems in the wild: ami-iit/bipedal-locomotion-framework#799 . |
Solution to issue cannot be found in the documentation.
Issue
The fix in #88 is creating problems, even if i solved the crash in
import kwant; kwant.test().See #87 (comment) for more details.
Installed packages
.Environment info
.The text was updated successfully, but these errors were encountered: