Issue with installation on M1 mac

[krittisharma]

I am following the instructions for install on mac in development mode and I get the following error, any help with resolving this would be very helpful. Thanks!

(base) krittisharma@Krittis-MacBook-Pro Pylians3 % CC=clang python setup.py install
/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:15: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. Support for replacing an already imported distutils is deprecated. In the future, this condition will fail. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
  warnings.warn(
Compiling library/MAS_library/MAS_library.pyx because it changed.
Compiling library/Pk_library/Pk_library.pyx because it changed.
Compiling library/Pk_library/bispectrum_library.pyx because it changed.
Compiling library/MAS_library/field_properties.pyx because it changed.
Compiling library/redshift_space_library/redshift_space_library.pyx because it changed.
Compiling library/smoothing_library/smoothing_library.pyx because it changed.
Compiling library/void_library/void_library.pyx because it changed.
Compiling library/integration_library/integration_library.pyx because it changed.
Compiling library/density_field_library/density_field_library.pyx because it changed.
Compiling library/sorting_library/sorting_library.pyx because it changed.
Compiling library/HI_library/HI_library.pyx because it changed.
Compiling library/HI_clusters_library/HI_clusters_library.pyx because it changed.
[ 1/12] Cythonizing library/HI_clusters_library/HI_clusters_library.pyx
[ 2/12] Cythonizing library/HI_library/HI_library.pyx
[ 3/12] Cythonizing library/MAS_library/MAS_library.pyx
[ 4/12] Cythonizing library/MAS_library/field_properties.pyx
[ 5/12] Cythonizing library/Pk_library/Pk_library.pyx
[ 6/12] Cythonizing library/Pk_library/bispectrum_library.pyx
[ 7/12] Cythonizing library/density_field_library/density_field_library.pyx
[ 8/12] Cythonizing library/integration_library/integration_library.pyx
[ 9/12] Cythonizing library/redshift_space_library/redshift_space_library.pyx
[10/12] Cythonizing library/smoothing_library/smoothing_library.pyx
[11/12] Cythonizing library/sorting_library/sorting_library.pyx
[12/12] Cythonizing library/void_library/void_library.pyx
running install
/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  self.initialize_options()
running bdist_egg
running egg_info
creating library/Pylians.egg-info
writing library/Pylians.egg-info/PKG-INFO
writing dependency_links to library/Pylians.egg-info/dependency_links.txt
writing requirements to library/Pylians.egg-info/requires.txt
writing top-level names to library/Pylians.egg-info/top_level.txt
writing manifest file 'library/Pylians.egg-info/SOURCES.txt'
Traceback (most recent call last):
  File "/Users/krittisharma/Desktop/research/frb_cosmo/Pylians3/setup.py", line 88, in <module>
    setup(
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 183, in setup
    return run_commands(dist)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
    dist.run_commands()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
    self.run_command(cmd)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 995, in run_command
    super().run_command(command)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 109, in run
    self.do_egg_install()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 167, in do_egg_install
    self.run_command('bdist_egg')
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 995, in run_command
    super().run_command(command)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 168, in run
    self.run_command("egg_info")
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 995, in run_command
    super().run_command(command)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 313, in run
    self.find_sources()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 321, in find_sources
    mm.run()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 544, in run
    self.add_defaults()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 582, in add_defaults
    sdist.add_defaults(self)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/sdist.py", line 109, in add_defaults
    super().add_defaults()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/sdist.py", line 238, in add_defaults
    self._add_defaults_ext()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/command/sdist.py", line 322, in _add_defaults_ext
    build_ext = self.get_finalized_command('build_ext')
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 303, in get_finalized_command
    cmd_obj.ensure_finalized()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/distutils/cmd.py", line 107, in ensure_finalized
    self.finalize_options()
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 198, in finalize_options
    self.check_extensions_list(self.extensions)
  File "/Users/krittisharma/opt/anaconda3/lib/python3.9/distutils/command/build_ext.py", line 362, in check_extensions_list
    raise DistutilsSetupError(
distutils.errors.DistutilsSetupError: each element of 'ext_modules' option must be an Extension instance or 2-tuple

[krittisharma]

Here's my modified setup.py file:

from platform import machine, system

import numpy
from Cython.Build import cythonize
from setuptools import Extension, find_packages, setup


is_mac = system() == "Darwin"
is_arm = machine() == "arm64"
is_m1 = is_mac and is_arm

arch_flag = "-mcpu=apple-m1"
omp_flag = "-Xpreprocessor" if is_m1 else ""
extra_compile_args = ["-O3", "-ffast-math"]
extra_compile_args_omp = extra_compile_args.copy()
extra_compile_args_omp.append(omp_flag)
extra_link_args = [omp_flag]

ext_modules = [
    Extension(
        "MAS_library.MAS_library",
        ["library/MAS_library/MAS_library.pyx", "library/MAS_library/MAS_c.c"],
        extra_compile_args=extra_compile_args_omp,
        extra_link_args=extra_link_args,
        libraries=["m"],
    ),
    Extension(
        "Pk_library.Pk_library",
        ["library/Pk_library/Pk_library.pyx"],
        extra_compile_args=extra_compile_args_omp,
    ),
    Extension(
        "Pk_library.bispectrum_library", ["library/Pk_library/bispectrum_library.pyx"]
    ),
    Extension(
        "MAS_library.field_properties", ["library/MAS_library/field_properties.pyx"]
    ),
    Extension(
        "redshift_space_library.redshift_space_library",
        ["library/redshift_space_library/redshift_space_library.pyx"],
    ),
    Extension(
        "smoothing_library.smoothing_library",
        ["library/smoothing_library/smoothing_library.pyx"],
        extra_compile_args=extra_compile_args_omp,
        extra_link_args=extra_link_args,
        libraries=["m"],
    ),
    Extension(
        "void_library.void_library",
        [
            "library/void_library/void_library.pyx",
            "library/void_library/void_openmp_library.c",
        ],
        extra_compile_args=extra_compile_args_omp,
        extra_link_args=extra_link_args,
        libraries=["m"],
    ),
    Extension(
        "integration_library.integration_library",
        [
            "library/integration_library/integration_library.pyx",
            "library/integration_library/integration.c",
            "library/integration_library/runge_kutta.c",
        ],
        extra_compile_args=extra_compile_args,
    ),
    Extension(
        "density_field_library.density_field_library",
        ["library/density_field_library/density_field_library.pyx"],
    ),
    Extension(
        "sorting_library.sorting_library",
        ["library/sorting_library/sorting_library.pyx"],
        extra_compile_args=extra_compile_args,
    ),
    Extension("HI_library.HI_library", ["library/HI_library/HI_library.pyx"]),
    Extension(
        "HI_clusters_library.HI_clusters_library",
        ["library/HI_clusters_library/HI_clusters_library.pyx"],
    ),
]


with open("README.md", "r") as f:
    documentation = f.read()

setup(
    name="Pylians",
    version="0.12",
    author="Francisco Villaescusa-Navarro",
    author_email="[email protected]",
    description="Python libraries for the analysis of numerical simulations",
    url="https://github.com/franciscovillaescusa/Pylians3",
    license="MIT",
    long_description_content_type="text/markdown",
    long_description=documentation,
    packages=find_packages(where="library/"),
    ext_modules=cythonize(
        ext_modules,
        compiler_directives={"language_level": "3"},
        include_path=[
            "library/MAS_library/",
            "library/void_library/",
            "library/integration_library/",
        ],
    ),
    include_dirs=[numpy.get_include()],
    install_requires=[
        "h5py",
        "pyfftw; platform_system!='Darwin' and platform_machine!='arm64'",
        "scipy",
        "hdf5plugin",
        "Cython<3.0.0",
    ],
    package_dir={"": "library/"},
    py_modules=[
        "bias_library",
        "CAMB_library",
        "correlation_function_library",
        "cosmology_library",
        "HI_image_library/HI_image_library",
        "HOD_library",
        "IM_library",
        "mass_function_library",
        "plotting_library",
        "readfof",
        "readgadget",
        "readsnapHDF5",
        "readsnap",
        "readsnap2",
        "readsnap_mpi",
        "readsubf",
        "routines",
        "units_library",
    ],
)
```
`

[franciscovillaescusa]

Hi,
Thanks for reporting this. I have never managed to install Pylians on Mac due to clang :(
Did you manage to do that?
If not, I would recommend using either google colab or install it using a linux virtual machine.
Sorry about this
Francisco