Make choosenim produce x86_64/arm64 proxies on Apple Silicon machines #10

heinthanth · 2024-07-22T03:37:29Z

This PR makes choosenim to produce arm64 proxies so that it works on Apple Silicon machines.
Previously, Nim was compiled as arm64 but choosenim produce only x86_64.

This PR adjusted the compileProxyexe so that it will produce both x86_64 and arm64 nim proxies and install them according to host machine CPU.

Note:

choosenim itself can be x86_64 binary depending on Nim compiler configuration that used to compile choosenim. It can be easily adjusted. But it's not the scope of this PR.
choosenim will install proxies according to the host CPU (which means even if you compile choosenim as x86_64 and run under rosetta, arm64 proxies will be installed)

Rules:

Build machine:

if build machine that build choosenim is running macOS version below Big Sur, amd64 proxies will be used. (since it can't cross-compile proxies to arm64).
Otherwise, two proxies (arm64 and amd64) will be embedded inside choosenim.

User machine:

if user machine is MacOS Big Sur and above, it check isAppleSilicon.
if user machine is AppleSilicon, choosenim will install arm64 proxies.
Otherwise, amd64 proxies will be installed.

Like this: (arm64 is installed even if choosenim is running under Rosetta - zsh)

$ file bin/choosenim
bin/choosenim: Mach-O 64-bit executable x86_64

$ arch -x64_64 /bin/zsh
$ bin/choosenim stable
...

$ file ~/.nimble/bin/nim
/Users/heinthant/.nimble/bin/nim: Mach-O 64-bit executable arm64

$ file ~/.choosenim/toolchains/nim-2.0.8/bin/*
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/atlas:      Mach-O 64-bit executable arm64
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/nim:        Mach-O 64-bit executable arm64
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/nim-gdb:    Bourne-Again shell script text executable, ASCII text
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/nim_dbg:    Mach-O 64-bit executable arm64
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/nimble:     Mach-O 64-bit executable arm64
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/nimgrep:    Mach-O 64-bit executable arm64
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/nimpretty:  Mach-O 64-bit executable arm64
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/nimsuggest: Mach-O 64-bit executable arm64
/Users/heinthant/.choosenim/toolchains/nim-2.0.8/bin/testament:  Mach-O 64-bit executable arm64

I did run a few tests on my M3 Macbook Pro.

[ heinthant@macbookpro ] choosenim $ file bin/choosenim
bin/choosenim: Mach-O 64-bit executable x86_64

[ heinthant@macbookpro ] choosenim $ bin/choosenim stable
Downloading Nim 2.0.8 from nim-lang.org
[##################################################] 100.0% 0kb/s
 Extracting nim-2.0.8.tar.xz
 Extracting nim-2.0.8.tar
   Building Nim 2.0.8
   Building koch
   Building Nim
   Building tools (nimble, nimgrep, nimpretty, nimsuggest, testament)
   Switched to Nim 2.0.8

[ heinthant@macbookpro ] choosenim $ ~/.nimble/bin/nim -v
Nim Compiler Version 2.0.8 [MacOSX: arm64]
Compiled at 2024-07-22
Copyright (c) 2006-2023 by Andreas Rumpf

active boot switches: -d:release


[ heinthant@macbookpro ] choosenim $ bin/choosenim 1.4.4
Downloading Nim 1.4.4 from nim-lang.org
[##################################################] 100.0% 0kb/s
 Extracting nim-1.4.4.tar.xz
 Extracting nim-1.4.4.tar
   Building Nim 1.4.4
   Building koch
   Building Nim
   Building tools (nimble, nimgrep, nimpretty, nimsuggest, testament)
   Switched to Nim 1.4.4

[ heinthant@macbookpro ] choosenim $ ~/.nimble/bin/nim -v
Nim Compiler Version 1.4.4 [MacOSX: arm64]
Compiled at 2024-07-22
Copyright (c) 2006-2020 by Andreas Rumpf

active boot switches: -d:release


[ heinthant@macbookpro ] choosenim $ bin/choosenim 1.6.6 
Downloading Nim 1.6.6 from nim-lang.org
[##################################################] 100.0% 0kb/s
 Extracting nim-1.6.6.tar.xz
 Extracting nim-1.6.6.tar
   Building Nim 1.6.6
   Building koch
   Building Nim
   Building tools (nimble, nimgrep, nimpretty, nimsuggest, testament)
   Switched to Nim 1.6.6

[ heinthant@macbookpro ] choosenim $ ~/.nimble/bin/nim -v
Nim Compiler Version 1.6.6 [MacOSX: arm64]
Compiled at 2024-07-22
Copyright (c) 2006-2021 by Andreas Rumpf

active boot switches: -d:release


[ heinthant@macbookpro ] choosenim $ bin/choosenim devel
    Warning Recent nightly release not found, installing latest devel commit.
Downloading Nim devel from GitHub
[##################################################] 100.0% 0kb/s
 Extracting devel.tar.gz
    Setting up git repository
   Building Nim #devel
   Building Nim using build_all.sh
   Switched to Nim #devel

[ heinthant@macbookpro ] choosenim $ ~/.nimble/bin/nim -v
Nim Compiler Version 2.1.9 [MacOSX: arm64]
Compiled at 2024-07-22
Copyright (c) 2006-2024 by Andreas Rumpf

active boot switches: -d:release


[ heinthant@macbookpro ] choosenim $ bin/choosenim #2f5cfd68298868cabcbc20967aa35bc708d507a8
Downloading Nim 2f5cfd68298868cabcbc20967aa35bc708d507a8 from GitHub
[##################################################] 100.0% 0kb/s
 Extracting 2f5cfd68298868cabcbc20967aa35bc708d507a8.tar.gz
   Building Nim #2f5cfd68298868cabcbc20967aa35bc708d507a8
   Building Nim using build_all.sh
   Switched to Nim #2f5cfd68298868cabcbc20967aa35bc708d507a8

[ heinthant@macbookpro ] choosenim $ ~/.nimble/bin/nim -v
Nim Compiler Version 2.1.9 [MacOSX: arm64]
Compiled at 2024-07-22
Copyright (c) 2006-2024 by Andreas Rumpf

active boot switches: -d:release

Here is my original PR on the original repo: dom96/choosenim#301.

Edit:

I've also addressed #9 with temporary solution as suggested here: treeform/puppy#118.

pietroppeter · 2024-07-22T17:36:03Z

hi, thanks for this!

I checked out the PR and choosenim.

It indeed compiles to a arm64 binary:

➜  choosenim git:(heinthanth/master) file bin/choosenim
bin/choosenim: Mach-O 64-bit executable arm64

after using to install a nim version (tried both 1.6.20 and devel), nim is not arm:

➜  choosenim git:(heinthanth/master) bin/choosenim devel
    Warning Recent nightly release not found, installing latest devel commit.
Downloading Nim devel from GitHub
[##################################################] 100.0% 0kb/s
 Extracting devel.tar.gz
    Setting up git repository
   Building Nim #devel
   Building Nim using build_all.sh
   Switched to Nim #devel

➜  choosenim git:(heinthanth/master) file ~/.nimble/bin/nim
/Users/pietropeterlongo/.nimble/bin/nim: Mach-O 64-bit executable x86_64

from the examples above I was expecting it to be an arm executable but maybe there is something I am doing wrong?
happy to run more diagnostics if it can help

my OS version is 14.3

heinthanth · 2024-07-23T01:39:29Z

@pietroppeter Thanks for the info! I could reproduce that too! I think the x86_64 proxies is calling arm64 nim and that's why I got arm64 in nim -v. I'll figure out and fix that.

heinthanth · 2024-07-23T01:57:58Z

@pietroppeter I pushed an update! Should be fine by now! It was caused by a bug in Apple Silicon check, forcing choosenim to use x86_64 proxies.

pietroppeter · 2024-07-23T10:07:36Z

works for me too, thanks!

pietroppeter

From what I can tell the gist of this PR was already reviewed and approved by dom (with help and additional review from @elcritch). with respect to the original PR, it does provide additional features (puppy's workaround and an additional fix), so it looks very good to me.

thanks @heinthanth for working on this and keeping push it even after quite some time after the initial reviews!

I do not feel really competent to review the details of the implementation though, so take my approval with a grain of salt (not a maintainer here) :)

ringabout · 2024-08-29T06:09:43Z

@heinthanth Hello, it seems to break choosenim building on windows

D:\a\choosenim\choosenim\src\choosenimpkg\switcher.nim(57, 39) Error: undeclared identifier: 'isRosetta'

heinthanth · 2024-08-29T06:12:59Z

@ringabout looks like there's a circular dependency on Windows. Switcher import utils and Utils import switcher on Windows. I'll import update switcher.nim to import utils only on Unix.

make choosenim produce x86_64/arm64 proxies

6d5ac0f

fix apple silicon check

26dbccf

ringabout mentioned this pull request Jul 23, 2024

add m1 matrix nim-lang/nightlies#82

Draft

Update comments

d78e520

pietroppeter approved these changes Jul 23, 2024

View reviewed changes

ringabout merged commit 680dea6 into nim-lang:master Aug 29, 2024

heinthanth mentioned this pull request Aug 29, 2024

Remove utils.nim import in switcher.nim on windows #16

Merged

ringabout mentioned this pull request Aug 29, 2024

cannot build a release on Apple Silicon (Mac M3 Pro) #9

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make choosenim produce x86_64/arm64 proxies on Apple Silicon machines #10

Make choosenim produce x86_64/arm64 proxies on Apple Silicon machines #10

heinthanth commented Jul 22, 2024 •

edited

Loading

pietroppeter commented Jul 22, 2024

heinthanth commented Jul 23, 2024

heinthanth commented Jul 23, 2024

pietroppeter commented Jul 23, 2024

pietroppeter left a comment

ringabout commented Aug 29, 2024

heinthanth commented Aug 29, 2024

Make choosenim produce x86_64/arm64 proxies on Apple Silicon machines #10

Make choosenim produce x86_64/arm64 proxies on Apple Silicon machines #10

Conversation

heinthanth commented Jul 22, 2024 • edited Loading

pietroppeter commented Jul 22, 2024

heinthanth commented Jul 23, 2024

heinthanth commented Jul 23, 2024

pietroppeter commented Jul 23, 2024

pietroppeter left a comment

Choose a reason for hiding this comment

ringabout commented Aug 29, 2024

heinthanth commented Aug 29, 2024

heinthanth commented Jul 22, 2024 •

edited

Loading