Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

missing neuron-nightly for macOS M1/M2 #2359

Open
ramcdougal opened this issue May 15, 2023 · 7 comments · May be fixed by #3321
Open

missing neuron-nightly for macOS M1/M2 #2359

ramcdougal opened this issue May 15, 2023 · 7 comments · May be fixed by #3321
Labels
CI Continuous Integration dev Developer Tickets enhancement macOS Issues that have a macOS-specific dimension.
Milestone
Release v9.1

Comments

@ramcdougal
Copy link
Member

neuron-nightly does not currently have any versions for M1/M2 macs.

https://pypi.org/project/NEURON-nightly/#files

Possibly related to #2358
@ramcdougal ramcdougal added dev Developer Tickets macOS Issues that have a macOS-specific dimension. labels May 15, 2023
@ramcdougal ramcdougal added this to the Release 9.0 Alpha milestone May 15, 2023
@alexsavulescu alexsavulescu added enhancement CI Continuous Integration labels May 15, 2023
@pramodk
Copy link
Member

pramodk commented May 15, 2023

@ramcdougal : one of the limitation/reason is that we don't have Apple M1 hardware freely available in Azure or GitHub CI.

A side note: we have apple m1 machine for testing/PR builds. But having self-hosted runner is another task and would require some work + discussion.

@nrnhines
Copy link
Member

I know that for the pkg installers is it straightforward to cross compile arm64, x86_64, or universal on either arch.
This is done by the nrn/bldnrnmacpkgcmake.sh script. I'm guessing that by analogy the wheel builds can accomplish the same thing if all the relevant libraries/programs are universal. eg.

lipo -archs `which python3.11`      
x86_64 arm64

@JCGoran JCGoran mentioned this issue May 8, 2024
Closed
@nrnhines
Copy link
Member

nrnhines commented Sep 26, 2024 •
edited
Loading

I started a branch (hines/universal2-wheel) that gets a long way toward creating a Mac universal2 wheel. I.e.

 build % temp.macosx-10.9-arm64-cpython-312/bin/nrniv -c 'nrnpython("import platform;print(platform.machine())")'
NEURON -- VERSION 9.0a-354-g93c7d2c22+ hines/universal2-wheel (93c7d2c22+) 2024-09-25
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2022
See http://neuron.yale.edu/neuron/credits

arm64
	1 
build % arch -x86_64 temp.macosx-10.9-arm64-cpython-312/bin/nrniv -c 'nrnpython("import platform;print(platform.machine())")'
NEURON -- VERSION 9.0a-354-g93c7d2c22+ hines/universal2-wheel (93c7d2c22+) 2024-09-25
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2022
See http://neuron.yale.edu/neuron/credits

x86_64
	1

However the bash packaging/python/build_wheels.bash osx 3.12 "" &>3.12-output errors out at the delocate part of the build

INFO:delocate.tools:Sanitize: Deleting rpath '@loader_path' from '/private/var/folders/gf/cr95gnz92l9fyz726ld3vtk00000gn/T/tmpqru_nxb_/wheel/neuron/.data/share/nrn/demo/release/arm64/special'
INFO:delocate.tools:Sanitize: Deleting rpath '/Users/hines/neuron/univwheel/build/cmake_install/lib' from '/private/var/folders/gf/cr95gnz92l9fyz726ld3vtk00000gn/T/tmpqru_nxb_/wheel/neuron/.data/share/nrn/demo/release/arm64/special'
INFO:delocate.tools:Sanitize: Deleting rpath '/Users/hines/neuron/univwheel/build/cmake_install/lib' from '/private/var/folders/gf/cr95gnz92l9fyz726ld3vtk00000gn/T/tmpqru_nxb_/wheel/neuron/.data/share/nrn/demo/release/arm64/.libs/libnrnmech.so'
Fixing: dist/NEURON_nightly-9.0a0-cp312-cp312-macosx_10_15_arm64.whl
Traceback (most recent call last):
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/bin/delocate-wheel", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/cmd/delocate_wheel.py", line 116, in main
    copied = delocate_wheel(
             ^^^^^^^^^^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/delocating.py", line 1090, in delocate_wheel
    out_wheel_fixed = _check_and_update_wheel_name(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/delocating.py", line 914, in _check_and_update_wheel_name
    new_name, problematic_files = _calculate_minimum_wheel_name(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/delocating.py", line 816, in _calculate_minimum_wheel_name
    for arch, version in _get_macos_min_version(lib):
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/delocate/delocating.py", line 619, in _get_macos_min_version
    for header in MachO(dylib_path).headers:
                  ^^^^^^^^^^^^^^^^^
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/macholib/MachO.py", line 121, in __init__
    self.load(fp)
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/macholib/MachO.py", line 131, in load
    self.load_fat(fh)
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/macholib/MachO.py", line 148, in load_fat
    self.load_header(fh, arch.offset, arch.size)
  File "/Users/hines/neuron/univwheel/nrn_build_venv312_16211565/lib/python3.12/site-packages/macholib/MachO.py", line 170, in load_header
    raise ValueError("Unknown Mach-O header: 0x%08x in %r" % (header, fh))
ValueError: Unknown Mach-O header: 0x213c6172 in <_io.BufferedReader name='/private/var/folders/gf/cr95gnz92l9fyz726ld3vtk00000gn/T/tmpqru_nxb_/wheel/neuron/.data/lib/libinterviews.a'>

@nrnhines
Copy link
Member

nrnhines commented Oct 18, 2024 •
edited
Loading

@pramodk per our Skype discussion...
matthew-brett/delocate#229 "Fat binaries of static libraries are not supported in the wheel" seems relevant. The error message above mentions libinterviews.a and

lipo -archs build/lib.macosx-10.9-arm64-cpython-312/neuron/.data/lib/libinterviews.a 
x86_64 arm64

chatgpt informs me that whl files are zip files. So, though not recommended, I removed

% unzip -t dist/*.whl|grep '.a  '
    testing: neuron/.data/lib/libfmt.a   OK
    testing: neuron/.data/lib/libinterviews.a   OK
    testing: neuron/.data/lib/libunidraw.a   OK

and "successfully"? delocated with

univwheel % nrn_build_venv312_-2632554/bin/delocate-wheel -w wheelhouse dist/*.whl

which generated

 % ls wheelhouse
NEURON_nightly-9.0a1.dev355+g7fb8eff0d-cp312-cp312-macosx_11_0_arm64.whl

@nrnhines
Copy link
Member

The hines/universal2-wheel branch now creates a wheel. E.g.

 % ls wheelhouse 
NEURON_nightly-9.0a1.dev355+g7fb8eff0d.d20241019-cp312-cp312-macosx_11_0_arm64.whl

That is installable in a python virtual environment and runnable with either a arch arm64 or ```arch x86_641 prefix. Remaining issues are

The wheel name should contain the term universal2 instead of arm64 (this was created on an Apple M1).

nrnivmodl is creating an arm64 subfolder.
nrnivmodl shouid create either arm64 or x86_64 subfolders depending on which architecture it is built for.
Or perhaps not.
Presently it is creating universal2 libnrnmech.dylib shared libraries

I'm experiencing a problem with NEURONMainMenu/Quit when launching python.

@nrnhines
Copy link
Member

nrnhines commented Oct 20, 2024 •
edited
Loading

This is informational. I was trying to see if there is a name convention for universal2 wheel architectures. Python.org itself just leaves out the machine type. e.g.

https://www.python.org/ftp/python/3.12.7/python-3.12.7-macos11.pkg

I noticed a discussion at python-pillow/pillow-wheels#359 that indicates using the token "universal2", however, that discussion ended with decision to have two wheels, arm64, and x86_64.

I found it interesting that they also mention it is possible to fuse arm64 and x86_64 wheels into a single wheel. e.g.

from delocate.fuse import fuse_wheels
fuse_wheels('Pillow-9.4.0-2-cp39-cp39-macosx_10_10_x86_64.whl', 'Pillow-9.4.0-cp39-cp39-macosx_11_0_arm64.whl', 'Pillow-9.4.0-cp39-cp39-macosx_11_0_universal2.whl')

Now called delocate-merge. See https://github.com/matthew-brett/delocate Making dual-architecture binaries

@JCGoran
Copy link
Collaborator

JCGoran commented Jan 28, 2025

Possible fix would be #3321, though the automatic uploading part (and consequently, up-to-date nightly wheels on PyPI for all platforms) hasn't been implemented yet.

@JCGoran JCGoran linked a pull request Feb 11, 2025 that will close this issue
Draft
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CI Continuous Integration dev Developer Tickets enhancement macOS Issues that have a macOS-specific dimension.
Projects
None yet
Milestone
Release v9.1
Development

Successfully merging a pull request may close this issue.

[WIP] Move CI infrastructure to GitHub Actions neuronsimulator/nrn
5 participants
@pramodk @nrnhines @ramcdougal @JCGoran @alexsavulescu