Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpc : fix load/store misaligned addresses #7948

Merged
merged 1 commit into from
Jun 17, 2024
Merged

Conversation

ggerganov
Copy link
Owner

@ggerganov ggerganov commented Jun 15, 2024

Fix a few errors when building with LLAMA_SANITIZE_UNDEFINED=ON:

./bin/rpc-server -p 50052

Accepted client connection, free_mem=206158430208, total_mem=206158430208
/Users/ggerganov/development/github/llama.cpp/ggml-rpc.cpp:1035:32: runtime error: reference binding to misaligned address 0x000140009654 for type 'const uint64_t' (aka 'const unsigned long long'), which requires 8 byte alignment
0x000140009654: note: pointer points here
  28 10 48 6f b0 39 52 18  01 00 00 00 00 00 00 00  00 c0 80 01 00 60 00 00  00 08 00 00 02 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/ggerganov/development/github/llama.cpp/ggml-rpc.cpp:1035:32 in 
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/tuple:398:102: runtime error: reference binding to misaligned address 0x000140009654 for type 'const unsigned long long', which requires 8 byte alignment
0x000140009654: note: pointer points here
  28 10 48 6f b0 39 52 18  01 00 00 00 00 00 00 00  00 c0 80 01 00 60 00 00  00 08 00 00 02 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/tuple:398:102 in 
/Users/ggerganov/development/github/llama.cpp/ggml-rpc.cpp:1039:39: runtime error: load of misaligned address 0x000140008004 for type 'const uint64_t' (aka 'const unsigned long long'), which requires 8 byte alignment
0x000140008004: note: pointer points here
  a5 02 00 00 b0 39 52 18  01 00 00 00 40 3b 52 18  01 00 00 00 d0 3c 52 18  01 00 00 00 80 41 52 18
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/ggerganov/development/github/llama.cpp/ggml-rpc.cpp:1039:39 in 
Client connection closed
./bin/llama-cli -m ../models/tinyllama-1b/ggml-model-q8_0.gguf -p "Hello, my name is" --rpc 0.0.0.0:50052 -ngl 99 -n 32

llama_new_context_with_model: RPC[0.0.0.0:50052] compute buffer size =   148.00 MiB
llama_new_context_with_model:        CPU compute buffer size =     8.01 MiB
llama_new_context_with_model: graph nodes  = 710
llama_new_context_with_model: graph splits = 3
/Users/ggerganov/development/github/llama.cpp/ggml-rpc.cpp:604:9: runtime error: store to misaligned address 0x0003300c0004 for type 'uint64_t' (aka 'unsigned long long'), which requires 8 byte alignment
0x0003300c0004: note: pointer points here
  a5 02 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/ggerganov/development/github/llama.cpp/ggml-rpc.cpp:604:9 in 

@mofosyne mofosyne added bugfix fixes an issue or bug Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix labels Jun 16, 2024
@rgerganov rgerganov merged commit 21be9ca into master Jun 17, 2024
68 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix fixes an issue or bug Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants