[clang-repl] Last value printing fails in out of process mode

[anutosh491]

I see the following

(xeus-cpp) abhinav@Abhinavs-MacBook-Air build % ./bin/clang-repl -oop-executor -orc-runtime -Xcc="-g" -Xcc="-O0" -Xcc="-debugger-tuning=lldb"
clang-repl> 1
JIT session error: Symbols not found: [ ___clang_Interpreter_SetValueNoAlloc ]
error: dlupdate failed
clang-repl> 

In in-process mode

clang-repl> 1
(int) 1

When run using --debug-only=orc flag I see this difference

1. in-process

Looking up { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) } in [ ("main", MatchAllSymbols), ("<Platform>", MatchExportedSymbolsOnly), ("<Process Symbols>", MatchExportedSymbolsOnly) ] (required state: Resolved)
Dispatching MaterializationUnits...
Done dispatching MaterializationUnits.
Entering OL_applyQueryPhase1:
  Lookup kind: Static
  Search order: [ ("main", MatchAllSymbols), ("<Platform>", MatchExportedSymbolsOnly), ("<Process Symbols>", MatchExportedSymbolsOnly) ], Current index = 0 (entering new JITDylib)
  Lookup set: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator candidates: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator non-candidates: { }
Visiting "main" (MatchAllSymbols) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  First time visiting main, resetting candidate sets and building generator stack
  Updating candidate set...
    Remaining candidates = { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  No generators to run for this JITDylib.
Phase 1 moving to next JITDylib.
Visiting "<Platform>" (MatchExportedSymbolsOnly) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  First time visiting <Platform>, resetting candidate sets and building generator stack
  Updating candidate set...
    Remaining candidates = { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  No generators to run for this JITDylib.
Phase 1 moving to next JITDylib.
Visiting "<Process Symbols>" (MatchExportedSymbolsOnly) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  First time visiting <Process Symbols>, resetting candidate sets and building generator stack
  Updating candidate set...
    Remaining candidates = { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Running 1 remaining generators for 1 candidates
  Attempting to generate { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
EPCDynamicLibrarySearchGenerator trying to generate { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
EPCDynamicLibrarySearchGenerator lookup returned { ("__clang_Interpreter_SetValueNoAlloc": 0x55555a34ec22 [Data]) }
Defining MU <Absolute Symbols> for <Process Symbols> (tracker: 0x0, default will be used)
  { ("__clang_Interpreter_SetValueNoAlloc", [Data]) }
Entering OL_applyQueryPhase1:
  Lookup kind: Static
  Search order: [ ("main", MatchAllSymbols), ("<Platform>", MatchExportedSymbolsOnly), ("<Process Symbols>", MatchExportedSymbolsOnly) ], Current index = 2
  Lookup set: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator candidates: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator non-candidates: { }
Visiting "<Process Symbols>" (MatchExportedSymbolsOnly) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  Updating candidate set...
    Remaining candidates = { }
  No generators to run for this JITDylib.
All symbols matched.
Phase 1 succeeded.
Entering OL_completeLookup:
  Lookup kind: Static
  Search order: [ ("main", MatchAllSymbols), ("<Platform>", MatchExportedSymbolsOnly), ("<Process Symbols>", MatchExportedSymbolsOnly) ], Current index = 3
  Lookup set: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator candidates: { }
  Definition generator non-candidates: { }
Visiting "main" (MatchAllSymbols) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  Attempting to match "__clang_Interpreter_SetValueNoAlloc" (RequiredSymbol)... skipping: not present
Visiting "<Platform>" (MatchExportedSymbolsOnly) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  Attempting to match "__clang_Interpreter_SetValueNoAlloc" (RequiredSymbol)... skipping: not present
Visiting "<Process Symbols>" (MatchExportedSymbolsOnly) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  Attempting to match "__clang_Interpreter_SetValueNoAlloc" (RequiredSymbol)... matched, preparing to dispatch MU@0x55556cfeee70 (<Absolute Symbols>)
Stripping unmatched weakly-referenced symbols
Query successfully lodged
Adding MUs to dispatch:
  For <Process Symbols>: Adding 1 MUs.
Registering dependencies
Dispatching MaterializationUnits...
  Dispatching "<Absolute Symbols>"
Dispatching: Materialization task: <Absolute Symbols> in <Process Symbols>
In <Process Symbols> resolving { ("__clang_Interpreter_SetValueNoAlloc": 0x55555a34ec22 [Data]) }
Dispatching: Execute query complete callback for { ("__clang_Interpreter_SetValueNoAlloc": 0x55555a34ec22 [Data]) }

2. out of process

Looking up { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) } in [ ("main", MatchAllSymbols), ("<Platform>", MatchExportedSymbolsOnly), ("<Process Symbols>", MatchExportedSymbolsOnly) ] (required state: Resolved)
Dispatching MaterializationUnits...
Done dispatching MaterializationUnits.
Entering OL_applyQueryPhase1:
  Lookup kind: Static
  Search order: [ ("main", MatchAllSymbols), ("<Platform>", MatchExportedSymbolsOnly), ("<Process Symbols>", MatchExportedSymbolsOnly) ], Current index = 0 (entering new JITDylib)
  Lookup set: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator candidates: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator non-candidates: { }
Visiting "main" (MatchAllSymbols) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  First time visiting main, resetting candidate sets and building generator stack
  Updating candidate set...
    Remaining candidates = { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  No generators to run for this JITDylib.
Phase 1 moving to next JITDylib.
Visiting "<Platform>" (MatchExportedSymbolsOnly) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  First time visiting <Platform>, resetting candidate sets and building generator stack
  Updating candidate set...
    Remaining candidates = { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Running 1 remaining generators for 1 candidates
  Attempting to generate { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Updating candidate set post-generation
Phase 1 moving to next JITDylib.
Visiting "<Process Symbols>" (MatchExportedSymbolsOnly) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  First time visiting <Process Symbols>, resetting candidate sets and building generator stack
  Updating candidate set...
    Remaining candidates = { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Running 1 remaining generators for 1 candidates
  Attempting to generate { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
EPCDynamicLibrarySearchGenerator trying to generate { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
SimpleRemoteEPC::sendMessage: opc = CallWrapper, seqno = 34, tag-addr = 0x5555555d5a44, arg-buffer = 0x3c bytes
  LookupState captured. Exiting phase1 for now.
SimpleRemoteEPC::handleMessage: opc = Result, seqno = 34, tag-addr = 0x0, arg-buffer = 0x14 bytes
EPCDynamicLibrarySearchGenerator lookup returned { }
Entering OL_applyQueryPhase1:
  Lookup kind: Static
  Search order: [ ("main", MatchAllSymbols), ("<Platform>", MatchExportedSymbolsOnly), ("<Process Symbols>", MatchExportedSymbolsOnly) ], Current index = 2
  Lookup set: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator candidates: { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  Definition generator non-candidates: { }
Visiting "<Process Symbols>" (MatchExportedSymbolsOnly) with lookup set { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }:
  Updating candidate set...
    Remaining candidates = { (__clang_Interpreter_SetValueNoAlloc, RequiredSymbol) }
  No generators to run for this JITDylib.
Phase 1 moving to next JITDylib.
Phase 1 failed with unresolved symbols.
SimpleRemoteEPC::sendMessage: opc = CallWrapper, seqno = 35, tag-addr = 0x5555555e47b0, arg-buffer = 0x18 bytes
SimpleRemoteEPC::handleMessage: opc = Result, seqno = 35, tag-addr = 0x0, arg-buffer = 0x1 bytes
JIT session error: Symbols not found: [ __clang_Interpreter_SetValueNoAlloc ]

[anutosh491]

In summary, I think the failure is caused by the symbol not being propagated to the remote executor, is it ?

cc @lhames what should be the approach to fix something like this

For some context

1. we ParseandExecute this runtimes variable even before user input, so here's the extern "C" declaration for __clang_Interpreter_SetValueNoAlloc
2. And the implementation is here in interpretervalueprinter.cpp

So

1. Do we register it as an absolute symbol (Host keeps definition, remote only gets the address via RPC ?)

So before launching the JIT, we define an absolute symbol mapping on the remote ExecutionSession pointing to the host address of __clang_Interpreter_SetValueNoAlloc. So something like this (not sure)

auto &ProcJD = *JIT->getProcessSymbolsJITDylib();
llvm::orc::SymbolMap SM;
auto Name = JIT->mangleAndIntern("__clang_Interpreter_SetValueNoAlloc");
SM[Name] = llvm::JITEvaluatedSymbol(
    llvm::pointerToJITTargetAddress(&__clang_Interpreter_SetValueNoAlloc),
    llvm::JITSymbolFlags::Exported);

ProcJD.define(llvm::orc::absoluteSymbols(std::move(SM)));

2. or do we somehow ship it as code to the remote JIT (JIT defines it in JD).

Maybe add small IR/bitcode or object containing InterpreterValuePrinter.cpp’s symbols to the remote JIT before executing any user PTU.

// in cmake

add_library(clangInterpreterRuntime OBJECT
  InterpreterValuePrinter.cpp
)

// during jit creation
auto &ES = JITBuilder->getExecutionSession();
auto &PlatformJD = *ES.getJITDylibByName("<Platform>");
auto &ObjLayer = JITBuilder->getObjLinkingLayer();

if (auto G = llvm::orc::StaticLibraryDefinitionGenerator::Create(
        ObjLayer, "/path/to/libclangInterpreterRuntime.a"))
  PlatformJD.addGenerator(std::move(*G));
else
  return G.takeError();

Not sure this is how we address this issue but this is what I could think off. Hope i'm understanding the problem correctly !?

[anutosh491]

Relevant #156649