Summary: Add the Cross compilation Script for RPi (4 & 5) for Linux host machine #15014

psiddh · 2025-10-10T20:06:01Z

Test Plan:
examples/raspberry_pi/rpi_setup.sh pi5
...
[100%] Linking CXX executable llama_main
[100%] Built target llama_main
[SUCCESS] LLaMA runner built successfully

==== Extracting Bundled Libraries ====
[INFO] Extracting GLIBC libraries from toolchain... [WARNING] Use bundled GLIBC script on RPI device ONLY if you encounter a GLIBC mismatch error when running llama_main. [SUCCESS] Bundled libraries prepared in: /home/sidart/working/executorch/cmake-out/bundled-libs [INFO] On Raspberry Pi, run: sudo ./install_libs.sh

==== Verifying Build Outputs ====
[INFO] Checking required binaries...
[SUCCESS] ✓ llama_main (6.1M)
[SUCCESS] ✓ libllama_runner.so (4.0M)
[SUCCESS] ✓ libextension_module.a (89K) - static library [SUCCESS] All required binaries built successfully!

==== Setup Complete! ====

✓ ExecuTorch cross-compilation setup completed successfully!

📦 Built binaries:
• llama_main: /home/sidart/working/executorch/cmake-out/examples/models/llama/llama_main
• libllama_runner.so: /home/sidart/working/executorch/cmake-out/examples/models/llama/runner/libllama_runner.so
• libextension_module.a: Statically linked into llama_main ✅
• Bundled libraries: /home/sidart/working/executorch/cmake-out/bundled-libs/

📋 Next steps:

Copy binaries to your Raspberry Pi pi5: scp /home/sidart/working/executorch/cmake-out/examples/models/llama/llama_main pi@:~~/ scp /home/sidart/working/executorch/cmake-out/examples/models/llama/runner/libllama_runner.so pi@:~~/ scp -r /home/sidart/working/executorch/cmake-out/bundled-libs/ pi@:~/
Copy shared libraries to system location: sudo cp libllama_runner.so /lib/ # Only this one needed! sudo ldconfig
Dry run to check for GLIBC or other issues: ./llama_main --help # Ensure there are no GLIBC or other errors before proceeding.
If you see GLIBC errors, install bundled libraries: cd ~/bundled-libs && sudo ./install_libs.sh source setup_env.sh # Only do this if you encounter a GLIBC version mismatch or similar error.
Download your model and tokenizer: # Refer to the official documentation for exact details.
Run ExecuTorch with your model: ./llama_main --model_path ./model.pte --tokenizer_path ./tokenizer.model --seq_len 128 --prompt "What is the meaning of life ?"

🎯 Deployment Summary:
📁 Files to copy: 2 (llama_main + libllama_runner.so)
🏗️ Extension module: Built-in (no separate .so needed)

🔧 Toolchain saved at: /home/sidart/working/executorch/arm-toolchain/arm-gnu-toolchain-14.3.rel1-x86_64-aarch64-none-linux-gnu
🔧 CMake toolchain file: /home/sidart/working/executorch/arm-toolchain-pi5.cmake

Happy inferencing! 🚀

Summary

[PLEASE REMOVE] See CONTRIBUTING.md's Pull Requests for ExecuTorch PR guidelines.

[PLEASE REMOVE] If this PR closes an issue, please add a Fixes #<issue-id> line.

[PLEASE REMOVE] If this PR introduces a fix or feature that should be the upcoming release notes, please add a "Release notes: " label. For a list of available release notes labels, check out CONTRIBUTING.md's Pull Requests.

Test plan

[PLEASE REMOVE] How did you test this PR? Please write down any manual commands you used and note down tests that you have written if applicable.

pytorch-bot · 2025-10-10T20:06:05Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15014

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures, 1 Unrelated Failure

As of commit a42b3a8 with merge base 45336ce ():

NEW FAILURES - The following jobs have failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t 7009ec9a6deb286337d645e6444e8f556399cc95adc14c86635ea7745507419b /exec failed with exit code 1
Test CUDA Builds / export-voxtral-cuda-artifact / linux-job (gh)
RuntimeError: Command docker exec -t 7cc554a852e45ed983404fd277455240684d666921d4ee1d448f63617bdc2478 /exec failed with exit code 2
Test CUDA Builds / test-models-cuda (add_mul) / linux-job (gh)
RuntimeError: Command docker exec -t 9626bb5fbc7def25988144c4c44f0197ea8eeef0c3ea1640b79e44ca006ecff8 /exec failed with exit code 1
Test CUDA Builds / test-models-cuda (add) / linux-job (gh)
RuntimeError: Command docker exec -t 53024ef2244f22ac9e25a162cbef8454042eb58f7edf5a6d7b47f6a7c7943bb7 /exec failed with exit code 1
Test CUDA Builds / test-models-cuda (conv1d) / linux-job (gh)
RuntimeError: Command docker exec -t 7f4fddda6c5cb320502f69b811b712ee5981b5fec1f833a5e5389599d8a3bcca /exec failed with exit code 1
Test CUDA Builds / test-models-cuda (linear) / linux-job (gh)
RuntimeError: Command docker exec -t d8f9d1d48a96a96ba7a3b69e785a075018f1af4508575014cf739e6da5572a3d /exec failed with exit code 1
Test CUDA Builds / test-models-cuda (resnet18) / linux-job (gh)
RuntimeError: Command docker exec -t b9b69ee9e9ab8216b012017cd50d78cbd01eccb761f478ef4a042b936d4caa07 /exec failed with exit code 1

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2025-10-10T20:06:43Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

digantdesai · 2025-10-13T14:59:59Z

examples/raspberry_pi/setup.sh

+parse_args() {
+    while [[ $# -gt 0 ]]; do
+        case $1 in
+            pi4|pi5)


why not others?

Haven't tested, It should work theoretically

digantdesai · 2025-10-13T15:01:22Z

examples/raspberry_pi/rpi_setup.sh

+  --bundled-glibc       Bundle toolchain GLIBC libraries (default)
+  --no-bundled-glibc    Use system GLIBC (requires RPi GLIBC upgrade)


Suggested change

--bundled-glibc Bundle toolchain GLIBC libraries (default)

--no-bundled-glibc Use system GLIBC (requires RPi GLIBC upgrade)

--no-bundled-glibc Use system GLIBC (may require RPi GLIBC upgrade)

I removed the entire bundled-glibc option to make the script simple. These steps are detailed out clearly in the tutorial under trouble shooting section.

digantdesai · 2025-10-13T15:01:30Z

examples/raspberry_pi/rpi_setup.sh

+  pi4|pi5               Target Raspberry Pi model
+
+Options:
+  --bundled-glibc       Bundle toolchain GLIBC libraries (default)


why is this the default?

Removed this option completely as it is making the setup on Rpi complicated,

digantdesai · 2025-10-13T15:01:57Z

examples/raspberry_pi/rpi_setup.sh

+  --force-rebuild       Force complete rebuild of all components
+  --force-download      Force re-download of the ARM toolchain


Suggested change

--force-rebuild Force complete rebuild of all components

--force-download Force re-download of the ARM toolchain

--force-rebuild Force complete rebuild of all components

--force-download Force re-download of the ARM toolchain (default: disabled)

digantdesai · 2025-10-13T15:04:14Z

examples/raspberry_pi/setup.sh

@@ -0,0 +1,723 @@
+#!/usr/bin/env bash


s/rpi_setup.sh/setup.sh

digantdesai · 2025-10-13T15:05:18Z

examples/raspberry_pi/rpi_setup.sh

+        arm64|aarch64)
+            if [[ "$HOST_OS" == "Darwin" ]]; then
+                HOST_ARCH="x86_64"  # Use x86_64 toolchain on Apple Silicon via Rosetta
+                log_info "Using x86_64 toolchain via Rosetta on Apple Silicon"


how are we going to test this?
we can drop this tbh

digantdesai · 2025-10-13T15:06:43Z

examples/raspberry_pi/rpi_setup.sh

+        cd "$TOOLCHAIN_DIR"
+
+        # Download toolchain
+        if ! curl -L -o "$toolchain_archive" "$toolchain_url"; then


check md5sum for the tarball?

digantdesai · 2025-10-13T15:07:54Z

examples/raspberry_pi/rpi_setup.sh

+            arch_flags="-march=armv8-a -mtune=cortex-a72"
+            ;;
+        pi5)
+            arch_flags="-march=armv8.2-a+dotprod+fp16"


add mtune? Does it have fp16?

Yes, Raspberry Pi 5 fully supports FP16 and mtune too, (added mtune)

digantdesai · 2025-10-13T15:10:17Z

examples/raspberry_pi/rpi_setup.sh

+        cp "$toolchain_lib_dir"/libc.so.* "$bundle_dir/" 2>/dev/null || true
+        cp "$toolchain_lib_dir"/libm.so.* "$bundle_dir/" 2>/dev/null || true


ideally we should check et libs and see what all they depend on

Removed glibc post setup steps from the script to keep it simple

digantdesai · 2025-10-13T15:11:20Z

examples/raspberry_pi/rpi_setup.sh

+
+    log_warning "Use bundled GLIBC script on RPI device ONLY if you encounter a GLIBC mismatch error when running llama_main."
+    # Create deployment script
+    cat > "$bundle_dir/install_libs.sh" << 'EOF'


why is this not a static file?

This is no longer needed , as I removed glibc option

digantdesai · 2025-10-13T15:12:54Z

examples/raspberry_pi/rpi_setup.sh

+        if command -v nm &> /dev/null; then
+            if nm "$CMAKE_OUT_DIR/examples/models/llama/llama_main" 2>/dev/null | grep -q "extension"; then
+                log_info "📦 libextension_module.a is statically linked into llama_main"
+                static_linked=true
+            fi
+        fi


since we are building and linking it why do we have to do this? we should already know static or dynamic, no?

Yes, removed all this logic, It is staically linked by default

digantdesai · 2025-10-13T15:13:30Z

examples/raspberry_pi/rpi_setup.sh

+    echo "  • libllama_runner.so: $CMAKE_OUT_DIR/examples/models/llama/runner/libllama_runner.so"
+
+    # Smart extension module reporting
+    if [[ "$EXTENSION_STATIC_LINKED" == "true" ]]; then


let's just pick one and simplify all this?

digantdesai · 2025-10-13T15:14:14Z

examples/raspberry_pi/rpi_setup.sh

+    echo -e "\n📋 Next steps:"
+
+    echo "1. Copy binaries to your Raspberry Pi $RPI_MODEL:"
+    echo "   scp $CMAKE_OUT_DIR/examples/models/llama/llama_main pi@<rpi-ip>:~/"


change ~/ into some named dir to avoid making mess in the home dir

digantdesai · 2025-10-13T15:15:10Z

examples/raspberry_pi/rpi_setup.sh

+    fi
+
+    echo -e "\n2. Copy shared libraries to system location:"
+    echo "   sudo cp libllama_runner.so /lib/  # Only this one needed!"


why do we want to put under /lib? Just put it in the same dir ~/ and set LD_LIBRARY_PATH

digantdesai · 2025-10-13T15:15:53Z

examples/raspberry_pi/rpi_setup.sh

+# Create environment setup script
+cat > "$SCRIPT_DIR/setup_env.sh" << 'ENVEOF'
+#!/bin/bash
+export LD_LIBRARY_PATH="/usr/local/lib/executorch:$LD_LIBRARY_PATH"


/usr/local/lib should already be on LD_LIBRARY_PATH

digantdesai

Looks good, I haven't tried it on my machine yet..

…ost machine Test Plan: examples/raspberry_pi/setup.sh pi5 ... [100%] Linking CXX executable llama_main [100%] Built target llama_main [SUCCESS] LLaMA runner built successfully ==== Verifying Build Outputs ==== [INFO] Checking required binaries... [SUCCESS] ✓ llama_main (6.1M) [SUCCESS] ✓ libllama_runner.so (4.0M) [SUCCESS] ✓ libextension_module.a (89K) - static library [SUCCESS] All required binaries built successfully! ==== Setup Complete! ==== ✓ ExecuTorch cross-compilation setup completed successfully! 📦 Built binaries: • llama_main: /home/sidart/working/executorch/cmake-out/examples/models/llama/llama_main • libllama_runner.so: /home/sidart/working/executorch/cmake-out/examples/models/llama/runner/libllama_runner.so • libextension_module.a: Statically linked into llama_main ✅ • Bundled libraries: /home/sidart/working/executorch/cmake-out/bundled-libs/ 📋 Next steps: 1. Copy binaries to your Raspberry Pi pi5: scp /home/sidart/working/executorch/cmake-out/examples/models/llama/llama_main pi@<rpi-ip>:~/ scp /home/sidart/working/executorch/cmake-out/examples/models/llama/runner/libllama_runner.so pi@<rpi-ip>:~/ scp -r /home/sidart/working/executorch/cmake-out/bundled-libs/ pi@<rpi-ip>:~/ 2. Copy shared libraries to system location: sudo cp libllama_runner.so /lib/ # Only this one needed! sudo ldconfig 3. Dry run to check for GLIBC or other issues: ./llama_main --help # Ensure there are no GLIBC or other errors before proceeding. 4. Download your model and tokenizer: # Refer to the official documentation for exact details. 5. Run ExecuTorch with your model: ./llama_main --model_path ./model.pte --tokenizer_path ./tokenizer.model --seq_len 128 --prompt "What is the meaning of life ?" 🎯 Deployment Summary: 📁 Files to copy: 2 (llama_main + libllama_runner.so) 🏗️ Extension module: Built-in (no separate .so needed) 🔧 Toolchain saved at: /home/sidart/working/executorch/arm-toolchain/arm-gnu-toolchain-14.3.rel1-x86_64-aarch64-none-linux-gnu 🔧 CMake toolchain file: /home/sidart/working/executorch/arm-toolchain-pi5.cmake Happy inferencing! 🚀

psiddh · 2025-10-15T15:57:35Z

@pytorchbot cherry-pick --onto release/1.0 -c docs

…ost machine (#15014) Test Plan: examples/raspberry_pi/rpi_setup.sh pi5 ... [100%] Linking CXX executable llama_main [100%] Built target llama_main [SUCCESS] LLaMA runner built successfully ==== Extracting Bundled Libraries ==== [INFO] Extracting GLIBC libraries from toolchain... [WARNING] Use bundled GLIBC script on RPI device ONLY if you encounter a GLIBC mismatch error when running llama_main. [SUCCESS] Bundled libraries prepared in: /home/sidart/working/executorch/cmake-out/bundled-libs [INFO] On Raspberry Pi, run: sudo ./install_libs.sh ==== Verifying Build Outputs ==== [INFO] Checking required binaries... [SUCCESS] ✓ llama_main (6.1M) [SUCCESS] ✓ libllama_runner.so (4.0M) [SUCCESS] ✓ libextension_module.a (89K) - static library [SUCCESS] All required binaries built successfully! ==== Setup Complete! ==== ✓ ExecuTorch cross-compilation setup completed successfully! 📦 Built binaries: • llama_main: /home/sidart/working/executorch/cmake-out/examples/models/llama/llama_main • libllama_runner.so: /home/sidart/working/executorch/cmake-out/examples/models/llama/runner/libllama_runner.so • libextension_module.a: Statically linked into llama_main ✅ • Bundled libraries: /home/sidart/working/executorch/cmake-out/bundled-libs/ 📋 Next steps: 1. Copy binaries to your Raspberry Pi pi5: scp /home/sidart/working/executorch/cmake-out/examples/models/llama/llama_main pi@<rpi-ip>:~/ scp /home/sidart/working/executorch/cmake-out/examples/models/llama/runner/libllama_runner.so pi@<rpi-ip>:~/ scp -r /home/sidart/working/executorch/cmake-out/bundled-libs/ pi@<rpi-ip>:~/ 2. Copy shared libraries to system location: sudo cp libllama_runner.so /lib/ # Only this one needed! sudo ldconfig 3. Dry run to check for GLIBC or other issues: ./llama_main --help # Ensure there are no GLIBC or other errors before proceeding. 4. If you see GLIBC errors, install bundled libraries: cd ~/bundled-libs && sudo ./install_libs.sh source setup_env.sh # Only do this if you encounter a GLIBC version mismatch or similar error. 5. Download your model and tokenizer: # Refer to the official documentation for exact details. 6. Run ExecuTorch with your model: ./llama_main --model_path ./model.pte --tokenizer_path ./tokenizer.model --seq_len 128 --prompt "What is the meaning of life ?" 🎯 Deployment Summary: 📁 Files to copy: 2 (llama_main + libllama_runner.so) 🏗️ Extension module: Built-in (no separate .so needed) 🔧 Toolchain saved at: /home/sidart/working/executorch/arm-toolchain/arm-gnu-toolchain-14.3.rel1-x86_64-aarch64-none-linux-gnu 🔧 CMake toolchain file: /home/sidart/working/executorch/arm-toolchain-pi5.cmake Happy inferencing! 🚀 ### Summary [PLEASE REMOVE] See [CONTRIBUTING.md's Pull Requests](https://github.com/pytorch/executorch/blob/main/CONTRIBUTING.md#pull-requests) for ExecuTorch PR guidelines. [PLEASE REMOVE] If this PR closes an issue, please add a `Fixes #<issue-id>` line. [PLEASE REMOVE] If this PR introduces a fix or feature that should be the upcoming release notes, please add a "Release notes: <area>" label. For a list of available release notes labels, check out [CONTRIBUTING.md's Pull Requests](https://github.com/pytorch/executorch/blob/main/CONTRIBUTING.md#pull-requests). ### Test plan [PLEASE REMOVE] How did you test this PR? Please write down any manual commands you used and note down tests that you have written if applicable. (cherry picked from commit a0e5280)

pytorchbot · 2025-10-15T15:59:49Z

Cherry picking #15014

The cherry pick PR is at #15151 The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

psiddh requested review from kirklandsign and larryliu0820 as code owners October 10, 2025 20:06

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 10, 2025

psiddh requested a review from digantdesai October 10, 2025 20:06

psiddh changed the title ~~Summary: RaspberryPi Tutorial to deply & infer llama models (Rpi4 & R…~~ Summary: Add the Cross compilation Script for RPi (4 & 5) for Linux host machine Oct 10, 2025

psiddh force-pushed the rpi branch from 908e0df to 7120fad Compare October 10, 2025 21:45

digantdesai reviewed Oct 13, 2025

View reviewed changes

psiddh requested a review from cccclai October 13, 2025 19:19

psiddh force-pushed the rpi branch from 7120fad to c7a8a13 Compare October 14, 2025 16:06

psiddh mentioned this pull request Oct 14, 2025

Add Tutorials for CortexM / Cortex A (Rpi & Pico) #14410

Open

psiddh force-pushed the rpi branch from c7a8a13 to 84f6043 Compare October 14, 2025 21:57

psiddh force-pushed the rpi branch from 84f6043 to a42b3a8 Compare October 14, 2025 22:34

mergennachin approved these changes Oct 15, 2025

View reviewed changes

psiddh merged commit a0e5280 into pytorch:main Oct 15, 2025
130 of 138 checks passed

pytorchbot mentioned this pull request Oct 15, 2025

[v1.0.0] Release Tracker #14288

Open

		--bundled-glibc Bundle toolchain GLIBC libraries (default)
		--no-bundled-glibc Use system GLIBC (requires RPi GLIBC upgrade)

	--bundled-glibc Bundle toolchain GLIBC libraries (default)
	--no-bundled-glibc Use system GLIBC (requires RPi GLIBC upgrade)
	--no-bundled-glibc Use system GLIBC (may require RPi GLIBC upgrade)

		--force-rebuild Force complete rebuild of all components
		--force-download Force re-download of the ARM toolchain

		cp "$toolchain_lib_dir"/libc.so.* "$bundle_dir/" 2>/dev/null \|\| true
		cp "$toolchain_lib_dir"/libm.so.* "$bundle_dir/" 2>/dev/null \|\| true

Summary: Add the Cross compilation Script for RPi (4 & 5) for Linux host machine #15014

Summary: Add the Cross compilation Script for RPi (4 & 5) for Linux host machine #15014

Uh oh!

Conversation

psiddh commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

pytorch-bot bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15014

❌ 7 New Failures, 1 Unrelated Failure

Uh oh!

github-actions bot commented Oct 10, 2025

This PR needs a release notes: label

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

psiddh Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

digantdesai Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

psiddh commented Oct 10, 2025 •

edited

Loading

pytorch-bot bot commented Oct 10, 2025 •

edited

Loading

This PR needs a `release notes:` label

psiddh Oct 14, 2025 •

edited

Loading

digantdesai Oct 13, 2025 •

edited

Loading