Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failures in Python 3.12.0 on Mac M1 #889

Closed
jdebacker opened this issue Oct 24, 2023 · 4 comments
Closed

Test failures in Python 3.12.0 on Mac M1 #889

jdebacker opened this issue Oct 24, 2023 · 4 comments

Comments

@jdebacker
Copy link
Member

I recently rebuilt the ogcore-dev environment and tried to run the OG-Core testing suite. It resulted in an error when collecting tests:

(ogcore-dev) jason.debacker@JDEBACKER-7 OG-Core % pytest
============================= test session starts ==============================
platform darwin -- Python 3.12.0, pytest-7.4.2, pluggy-1.3.0
rootdir: /Users/jason.debacker/repos/OG-Core
configfile: pytest.ini
testpaths: ./tests
plugins: xdist-3.3.1
collecting 239 items                                                           Fatal Python error: Illegal instruction

Current thread 0x0000000200983240 (most recent call first):
  File "/Users/jason.debacker/anaconda3/envs/ogzaf-dev/lib/python3.11/site-packages/paramtools/sorted_key_list.py", line 53 in <lambda>
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/sortedcontainers/sortedlist.py", line 1878 in update
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/sortedcontainers/sortedlist.py", line 1750 in __init__
  File "/Users/jason.debacker/repos/OG-Core/ogcore/utils.py", line 393 in safe_read_pickle
  File "/Users/jason.debacker/repos/OG-Core/tests/test_output_plots.py", line 26 in <module>
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/assertion/rewrite.py", line 178 in exec_module
  File "<frozen importlib._bootstrap>", line 929 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1325 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1354 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1381 in _gcd_import
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/importlib/__init__.py", line 90 in import_module
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/pathlib.py", line 567 in import_path
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/python.py", line 617 in _importtestmodule
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/python.py", line 528 in _getobj
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/python.py", line 310 in obj
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/python.py", line 545 in _inject_setup_module_fixture
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/python.py", line 531 in collect
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/runner.py", line 372 in <lambda>
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/runner.py", line 341 in from_call
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/runner.py", line 372 in pytest_make_collect_report
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/runner.py", line 547 in collect_one_node
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/main.py", line 842 in genitems
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/main.py", line 675 in perform_collect
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/main.py", line 335 in pytest_collection
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/main.py", line 324 in _main
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/main.py", line 271 in wrap_session
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/main.py", line 318 in pytest_cmdline_main
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/config/__init__.py", line 169 in main
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/lib/python3.12/site-packages/_pytest/config/__init__.py", line 192 in console_main
  File "/Users/jason.debacker/anaconda3/envs/ogcore-dev/bin/pytest", line 10 in <module>

Extension modules: yaml._yaml, cytoolz.utils, cytoolz.itertoolz, cytoolz.functoolz, cytoolz.dicttoolz, cytoolz.recipes, psutil._psutil_osx, psutil._psutil_posix, markupsafe._speedups, tornado.speedups, msgpack._cmsgpack, lz4._version, lz4.block._block, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._isolve._iterative, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.linalg._flinalg, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy.optimize._minpack2, scipy.optimize._group_columns, scipy._lib.messagestream, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._cobyla, scipy.optimize._slsqp, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy.optimize.__nnls, scipy.optimize._highs.cython.src._highs_wrapper, scipy.optimize._highs._highs_wrapper, scipy.optimize._highs.cython.src._highs_constants, scipy.optimize._highs._highs_constants, scipy.linalg._interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.special._ellip_harm_2, scipy.spatial.transform._rotation, scipy.optimize._direct, _brotli, pyarrow.lib, pyarrow._hdfsio, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pandas._libs.ops, pyarrow._compute, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.indexing, pandas._libs.index, pandas._libs.internals, pandas._libs.join, pandas._libs.writers, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, scipy.interpolate._fitpack, scipy.interpolate.dfitpack, scipy.interpolate._bspl, scipy.interpolate._ppoly, scipy.interpolate.interpnd, scipy.interpolate._rbfinterp_pythran, scipy.interpolate._rgi_cython, matplotlib._c_internal_utils, PIL._imaging, matplotlib._path, kiwisolver._cext, matplotlib._image, scipy.ndimage._nd_image, _ni_label, scipy.ndimage._ni_label, scipy.integrate._odepack, scipy.integrate._quadpack, scipy.integrate._vode, scipy.integrate._dop, scipy.integrate._lsoda, scipy.special.cython_special, scipy.stats._stats, scipy.stats.beta_ufunc, scipy.stats._boost.beta_ufunc, scipy.stats.binom_ufunc, scipy.stats._boost.binom_ufunc, scipy.stats.nbinom_ufunc, scipy.stats._boost.nbinom_ufunc, scipy.stats.hypergeom_ufunc, scipy.stats._boost.hypergeom_ufunc, scipy.stats.ncf_ufunc, scipy.stats._boost.ncf_ufunc, scipy.stats.ncx2_ufunc, scipy.stats._boost.ncx2_ufunc, scipy.stats.nct_ufunc, scipy.stats._boost.nct_ufunc, scipy.stats.skewnorm_ufunc, scipy.stats._boost.skewnorm_ufunc, scipy.stats.invgauss_ufunc, scipy.stats._boost.invgauss_ufunc, scipy.stats._biasedurn, scipy.stats._levy_stable.levyst, scipy.stats._stats_pythran, scipy._lib._uarray._uarray, scipy.stats._statlib, scipy.stats._sobol, scipy.stats._qmc_cy, scipy.stats._mvn, scipy.stats._rcont.rcont, lz4.frame._frame (total: 176)
zsh: illegal hardware instruction  pytest

I've not seen this before an a quick search didn't reveal an obvious solution.

@jdebacker
Copy link
Member Author

Follow up. I've been able to run pytest tests/test_module_name.py for most modules. The only one I've noted giving an error like the above is test_parameter_plots.py. So perhaps issues with with M1, 3.12, and a package called there?

@jdebacker
Copy link
Member Author

Following up on this issue... it appears the issue is with reading files that were saved via CloudPickle in versions of Python < 3.12.

I have no problems running the model generally, but when running tests, those test modules that import objects saved via CloudPickle (e.g., Specifications objects or tax functions of the mono form) such as test_parameter_plots.py and test_ouptut_tables.py, I get a failure:

(ogcore-dev) jason.debacker@JDEBACKER-8 OG-Core % pytest tests/test_parameter_plots.py
============================= test session starts ==============================
platform darwin -- Python 3.12.3, pytest-8.2.2, pluggy-1.5.0
rootdir: /Users/jason.debacker/repos/OG-Core
configfile: pytest.ini
plugins: cov-5.0.0, anyio-4.4.0, xdist-3.6.1
collecting ... zsh: trace trap  pytest tests/test_parameter_plots.py

The lack of any traceback makes it not clear, but I if I comment out the lines reading in these CloudPickled files, I do get some tests collected and run (of course there is an error noting that I don't the the parameters object available since I didn't import it, but we can at least see that tests try to run):

(ogcore-dev) jason.debacker@JDEBACKER-8 OG-Core % pytest tests/test_parameter_plots.py
============================= test session starts ==============================
platform darwin -- Python 3.12.3, pytest-8.2.2, pluggy-1.5.0
rootdir: /Users/jason.debacker/repos/OG-Core
configfile: pytest.ini
plugins: cov-5.0.0, anyio-4.4.0, xdist-3.6.1
collected 1 item

tests/test_parameter_plots.py F                                          [100%]

=================================== FAILURES ===================================
_____________________________ test_plot_imm_rates ______________________________

    def test_plot_imm_rates():
        fig = parameter_plots.plot_imm_rates(
>           base_params.imm_rates,
            base_params.start_year,
            [base_params.start_year],
            include_title=True,
        )
E       NameError: name 'base_params' is not defined

tests/test_parameter_plots.py:46: NameError
=========================== short test summary info ============================
FAILED tests/test_parameter_plots.py::test_plot_imm_rates - NameError: name 'base_params' is not defined
============================== 1 failed in 1.83s ===============================

@jdebacker
Copy link
Member Author

We already have separate parameter files for Python 3.11 and Python 3.9/10:

if sys.version_info[1] < 11:
    base_params = utils.safe_read_pickle(
        os.path.join(CUR_PATH, "test_io_data", "model_params_baseline.pkl")
    )
else:
    base_params = utils.safe_read_pickle(
        os.path.join(
            CUR_PATH, "test_io_data", "model_params_baseline_v311.pkl"
        )
    )

Other times, we don't run certain parameterizations under a different Python version:

if sys.version_info[1] < 11:
    test_list = [
        (base_taxfunctions, 43, "DEP", "etr", True, None, None),
        (base_taxfunctions, 43, "DEP", "etr", False, None, "Test title"),
        (GS_nonage_spec_taxfunctions, None, "GS", "etr", True, None, None),
        (base_taxfunctions, 43, "DEP", "etr", True, [micro_data], None),
        (base_taxfunctions, 43, "DEP", "mtry", True, [micro_data], None),
        (base_taxfunctions, 43, "DEP", "mtrx", True, [micro_data], None),
        (mono_nonage_spec_taxfunctions, None, "mono", "etr", True, None, None),
    ]
    id_list = [
        "over_labinc=True",
        "over_labinc=False",
        "Non age-specific",
        "with data",
        "MTR capital income",
        "MTR labor income",
        "Mono functions",
    ]
else:
    test_list = [
        (base_taxfunctions, 43, "DEP", "etr", True, None, None),
        (base_taxfunctions, 43, "DEP", "etr", False, None, "Test title"),
        (GS_nonage_spec_taxfunctions, None, "GS", "etr", True, None, None),
        (base_taxfunctions, 43, "DEP", "etr", True, [micro_data], None),
        (base_taxfunctions, 43, "DEP", "mtry", True, [micro_data], None),
        (base_taxfunctions, 43, "DEP", "mtrx", True, [micro_data], None),
    ]
    id_list = [
        "over_labinc=True",
        "over_labinc=False",
        "Non age-specific",
        "with data",
        "MTR capital income",
        "MTR labor income",
    ]


@pytest.mark.parametrize(
    "tax_funcs,age,tax_func_type,rate_type,over_labinc,data,title",
    test_list,
    ids=id_list,
)
def test_plot_2D_taxfunc(
    tax_funcs, age, tax_func_type, rate_type, over_labinc, data, title
):
    """
    Test of plot_2D_taxfunc
    """
    if sys.version_info[1] < 11:
        fig = parameter_plots.plot_2D_taxfunc(
            2030,
            2021,
            [tax_funcs],
            age=age,
            tax_func_type=[tax_func_type],
            rate_type=rate_type,
            over_labinc=over_labinc,
            data_list=data,
            title=title,
        )

        assert fig
    else:
        assert True

@rickecon What do you want to do for Python 3.12? Create a third pkl file for parameters? Skip tests that require these Cloudpickled objects on 3.12? Note we may have to update this with each Python version we want to test against because CloudPickle is not really suited for long term storage due to changes in Python.

@jdebacker
Copy link
Member Author

Resolved with PR #969

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant