All selectors unrecognised when invoking Objective-C methods using the LLVM ExecutionEngine

Question

I'm running on OSX, using Clang to compile some Obj-C code that uses the OSX Cocoa classes, and I'm trying to run the result with the LLVM JIT compiler. I'm using the latest bleeding-edge version of LLVM/Clang.

There are no problems compiling or linking my code, and I can happily make C and C++ system calls without any trouble. But all my Obj-C invocations are failing miserably, and I'm out of my depth trying to work out why! The objc_msgSend() function appears to be getting called correctly, but the runtime is refusing to recognise even the simplest selectors.

I've managed to reproduce the problem using just Clang and LLI, and this is how to do it:

Create a simple file test.mm:

#include <Cocoa/Cocoa.h>
#include <cstdio>
#include <iostream>

extern "C" int main (int, char**)
{
    std::cout << "==== step 1" << std::endl;

    NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
    [pool release];

    std::cout << "==== step 2" << std::endl;

    return 0;
}

..compile it to bitcode with clang:

clang -emit-llvm test.mm -c -o test.bc

Then run it with lli:

lli -load=/System/Library/Frameworks/Foundation.framework/Versions/Current/Foundation test.bc

The output of lli looks like this:

==== step 1
objc[45353]: Object 0x101a362a0 of class __NSCFString autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
2012-04-29 20:07:35.384 lli[45353:707] -[NSAutoreleasePool init]: unrecognized selector sent to instance 0x101a35170
2012-04-29 20:07:35.386 lli[45353:707] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSAutoreleasePool init]: unrecognized selector sent to instance 0x101a35170'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff89c76fc6 __exceptionPreprocess + 198
    1   libobjc.A.dylib                     0x00007fff8c9e6d5e objc_exception_throw + 43
    2   CoreFoundation                      0x00007fff89d032ae -[NSObject doesNotRecognizeSelector:] + 190
    3   CoreFoundation                      0x00007fff89c63e73 ___forwarding___ + 371
    4   CoreFoundation                      0x00007fff89c63c88 _CF_forwarding_prep_0 + 232
    5   ???                                 0x0000000101929111 0x0 + 4321349905
    6   lli                                 0x000000010148f36b _ZN4llvm15ExecutionEngine17runFunctionAsMainEPNS_8FunctionERKSt6vectorISsSaISsEEPKPKc + 1259
    7   lli                                 0x0000000101016657 main + 3095
    8   lli                                 0x0000000101015a34 start + 52
    9   ???                                 0x0000000000000003 0x0 + 3
)
terminate called throwing an exception0  lli               0x00000001015c5b02 _ZL15PrintStackTracePv + 34
1  lli               0x00000001015c5fd9 _ZL13SignalHandleri + 633
2  libsystem_c.dylib 0x00007fff8f8bccfa _sigtramp + 26
3  libsystem_c.dylib 0x0000000000000001 _sigtramp + 18446603338107859745
4  libsystem_c.dylib 0x00007fff8f85ba7a abort + 143
5  libc++abi.dylib   0x00007fff8518a7bc abort_message + 214
6  libc++abi.dylib   0x00007fff85187fcf default_terminate() + 28
7  libobjc.A.dylib   0x00007fff8c9e71b9 _objc_terminate + 94
8  libc++abi.dylib   0x00007fff85188001 safe_handler_caller(void (*)()) + 11
9  libc++abi.dylib   0x00007fff8518805c __cxa_bad_typeid + 0
10 libc++abi.dylib   0x00007fff85189152 __gxx_exception_cleanup(_Unwind_Reason_Code, _Unwind_Exception*) + 0
11 libobjc.A.dylib   0x00007fff8c9e6e7a _objc_exception_destructor + 0
12 CoreFoundation    0x00007fff89d032ae -[NSObject doesNotRecognizeSelector:] + 190
13 CoreFoundation    0x00007fff89c63e73 ___forwarding___ + 371
14 CoreFoundation    0x00007fff89c63c88 _CF_forwarding_prep_0 + 232
15 CoreFoundation    0x0000000101929111 _CF_forwarding_prep_0 + 18446603342526043505
16 lli               0x000000010148f36b llvm::ExecutionEngine::runFunctionAsMain(llvm::Function*, std::vector<std::string, std::allocator<std::string> > const&, char const* const*) + 1259
17 lli               0x0000000101016657 main + 3095
18 lli               0x0000000101015a34 start + 52
19 lli               0x0000000000000003 start + 18446744069397718531
Stack dump:
0.  Program arguments: Release/bin/lli -load=/System/Library/Frameworks/Foundation.framework/Versions/Current/Foundation /Users/jules/Desktop/test.bc 
Abort trap: 6

As you can see in the log, it says that -[NSAutoreleasePool init] is an unrecognised selector. The same happens for any other selector, e.g. -[NSString init] or other things that clearly should work.

Any help or clues would be much appreciated! I am a bit lost as to whether this is a bug, or something I'm doing wrong, or maybe just a feature that hasn't been finished yet. I can't find any references to this issue anywhere in the LLVM docs or interwebs.

I've tried different clang options such as the legacy Obj-C fragile ABI, but had no luck. I'm no expert on either LLVM or the Obj-C runtime, and this one has got me stumped.

--EDIT--

Just a bit more info, in the hope that it might ring a bell with someone..

When I tried replacing the normal obj-C message invocation with an explicit call to objc_msgSend, I found this:

SEL s = sel_getUid ("init");
objc_msgSend (myObject, s);   // Succeeds!

SEL s = @selector (init);
objc_msgSend (myObject, s);   // Fails!

..so it seems that when clang auto-generates the SEL value, it's producing a value that's not useable by the runtime. Can anyone even suggest where in the LLVM/Clang codebase I should look to try to understand what might be going on?

Answer 1

Objective-C uses specially-named globals to refer to selectors, and the linker and ObjC runtime have special knowledge of these globals which makes everything work normally. lli has no knowledge of Objective-C; therefore, the ObjC runtime never runs its special handling for the globals in question.

Off the top of my head, I have no idea what exactly you need to do to make it work, though.

Answer 2

After a few months of research on this topic I would like to share my findings. To quote myself:

It is possible to run Objective-C and Swift code with LLVM JIT on macOS system. One way to make it work is to subclass a SectionMemoryManager used by LLVM JIT engine, intercept memory sections related to Objective-C as they get allocated in memory, find the Objective-C metadata in these sections, parse the Objective-C class information from this metadata, use a number of Objective-C Runtime API methods to register found Objective-C classes in Objective-C runtime.

Although this approach only targets Objective-C code and the Objective-C Runtime, it also seems to enable support for combined Swift and Objective-C code: given that the Objective-C classes are registered, Swift code with enabled Objective-C interoperability seems to run without any major issues in LLVM JIT.

See full article for the details:

LLVM JIT, Objective-C and Swift on macOS: knowledge dump.