Swift WHOLE_MODULE_OPTIMIZATION improves compile time, but causes lldb/Xcode crash

Question

TL;DR

Before

SWIFT_WHOLE_MODULE_OPTIMIZATION = NO

• Debug compile takes 10-15 minutes
• Release compile takes 25+ minutes
• po works fine in LLDB

After

SWIFT_WHOLE_MODULE_OPTIMIZATION = YES

• Debug compile takes 1-2 minutes
• Release compile takes ~8 minutes
• po always causes Xcode to crash

Any idea why the horrible compile times based on this info, and/or why Xcode might be crashing?

---

Deets

I'm working on a large 100% Swift project (there are 3rd party libraries in Objective-C, but all our code is Swift). We have been having atrocious compile times, usually around 10-15 minutes to compile the debug configuration and 30+ minutes to compile the release configuration.

This project has been very difficult to work with because of the horrible compile times. I've been searching for ways to improve this, particularly through build settings and for months had no luck. One thing I overlooked was SWIFT_WHOLE_MODULE_OPTIMIZATION, particularly because any mention of it claims it will increase a project's compile time.

So the other day we enabled SWIFT_WHOLE_MODULE_OPTIMIZATION and lo and behold we have a 10x improvement on compile times.

The problem is, now whenever we're debugging the project and try printing an object in lldb with po myObject Xcode immediately crashes. Here's some info from the crash log:

Process: Xcode [5860]
Path: /Applications/Xcode.app/Contents/MacOS/Xcode
Identifier: com.apple.dt.Xcode
Version: 6.4 (7720)
Build Info: IDEFrameworks-7720000000000000~8
App Item ID: 497799835
App External ID: 812725084
Code Type: X86-64 (Native)
Parent Process: ??? [1]
Responsible: Xcode [5860]

Date/Time: 2015-08-05 15:53:08.265 -0600
OS Version: Mac OS X 10.11 (15A235d)
Report Version: 11

Time Awake Since Boot: 13000 seconds

Crashed Thread: 20

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000000000008f
Exception Note: EXC_CORPSE_NOTIFY

VM Regions Near 0x8f: --> __TEXT 000000010ef62000-000000010ef63000 [ 4K] r-x/rwx SM=COW /Applications/Xcode.app/Contents/MacOS/Xcode

Application Specific Information:
ProductBuildVersion: 6E35b

Here's the stack trace on the crashed thread:

Thread 20 Crashed:: <DBGLLDBSessionThread (pid=6402)> 0   com.apple.LLDB.framework        0x0000000116b09ab4 swift::ArchetypeBuilder::resolveArchetype(swift::Type) + 68 1   com.apple.LLDB.framework        0x0000000116b0f808 std::__1::__function::__func<substConcreteTypesForDependentTypes(swift::ArchetypeBuilder&, swift::Type)::$_6, std::__1::allocator<substConcreteTypesForDependentTypes(swift::ArchetypeBuilder&, swift::Type)::$_6>, swift::Type (swift::Type)>::operator()(swift::Type&&) + 152 2   com.apple.LLDB.framework        0x0000000116bc0986 swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 54 3   com.apple.LLDB.framework        0x0000000116bc0f2b swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 1499 4   com.apple.LLDB.framework        0x0000000116bc0bbb swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 619 5   com.apple.LLDB.framework        0x0000000116bc0c0a swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 698 6   com.apple.LLDB.framework        0x0000000116b0c8f2 swift::ArchetypeBuilder::substDependentType(swift::Type) + 50 7   com.apple.LLDB.framework        0x0000000116e9554e (anonymous namespace)::LowerType::visitAnyStructType(swift::CanType, swift::StructDecl*) + 270 8   com.apple.LLDB.framework        0x0000000116e92e66 swift::Lowering::TypeConverter::getTypeLoweringForUncachedLoweredType(swift::Lowering::TypeConverter::TypeKey) + 150 9   com.apple.LLDB.framework        0x0000000116e92b39 swift::Lowering::TypeConverter::getTypeLowering(swift::Lowering::AbstractionPattern, swift::Type, unsigned int) + 2361 10  com.apple.LLDB.framework        0x0000000116f8f711 lldb_private::SwiftSILManipulator::emitLValueForVariable(swift::VarDecl*, lldb_private::SwiftExpressionParser::SILVariableInfo&) + 1521 11  com.apple.LLDB.framework        0x00000001172ac7ee (anonymous namespace)::LLDBNameLookup::emitLValueForVariable(swift::VarDecl*, swift::SILBuilder&) + 102 12  com.apple.LLDB.framework        0x0000000116ebb162 swift::Lowering::SILGenFunction::emitInitializationForVarDecl(swift::VarDecl*, swift::Type) + 98 13  com.apple.LLDB.framework        0x0000000116ebbc74 swift::ASTVisitor<(anonymous namespace)::InitializationForPattern, void, void, void, std::__1::unique_ptr<swift::Lowering::Initialization, std::__1::default_delete<swift::Lowering::Initialization> >, void, void>::visit(swift::Pattern*) + 404 14  com.apple.LLDB.framework        0x0000000116ebbc57 swift::ASTVisitor<(anonymous namespace)::InitializationForPattern, void, void, void, std::__1::unique_ptr<swift::Lowering::Initialization, std::__1::default_delete<swift::Lowering::Initialization> >, void, void>::visit(swift::Pattern*) + 375 15  com.apple.LLDB.framework        0x0000000116ebba0d swift::Lowering::SILGenFunction::visitPatternBindingDecl(swift::PatternBindingDecl*) + 45 16  com.apple.LLDB.framework        0x0000000116f0617c swift::Lowering::SILGenFunction::visitBraceStmt(swift::BraceStmt*) + 284 17  com.apple.LLDB.framework        0x0000000116ecd1c0 swift::Lowering::SILGenFunction::emitFunction(swift::FuncDecl*) + 320 18  com.apple.LLDB.framework        0x0000000116ea3966 swift::Lowering::SILGenModule::emitFunction(swift::FuncDecl*) + 246 19  com.apple.LLDB.framework        0x0000000116ea3828 swift::Lowering::SILGenModule::visitFuncDecl(swift::FuncDecl*) + 168 20  com.apple.LLDB.framework        0x0000000116ea579b swift::Lowering::SILGenModule::emitSourceFile(swift::SourceFile*, unsigned int) + 427 21  com.apple.LLDB.framework        0x0000000116ea5c22 swift::SILModule::constructSIL(swift::Module*, swift::SILOptions&, swift::SourceFile*, llvm::Optional<unsigned int>, bool, bool) + 386 22  com.apple.LLDB.framework        0x0000000116ea5d42 swift::performSILGeneration(swift::SourceFile&, swift::SILOptions&, llvm::Optional<unsigned int>, bool) + 98 23  com.apple.LLDB.framework        0x00000001172aa617 lldb_private::SwiftExpressionParser::Parse(lldb_private::Stream&, unsigned int, unsigned int, unsigned int) + 10715 24  com.apple.LLDB.framework        0x000000011706b3e8 lldb_private::ClangUserExpression::Parse(lldb_private::Stream&, lldb_private::ExecutionContext&, lldb_private::ExecutionPolicy, bool, unsigned int) + 1064 25  com.apple.LLDB.framework        0x000000011706cdb4 lldb_private::ClangUserExpression::Evaluate(lldb_private::ExecutionContext&, lldb_private::EvaluateExpressionOptions const&, char const*, char const*, lldb_private::SharingPtr<lldb_private::ValueObject>&, lldb_private::Error&, unsigned int, std::__1::shared_ptr<lldb_private::Module>*) + 628 26  com.apple.LLDB.framework        0x00000001171d1696 lldb_private::Target::EvaluateExpression(char const*, lldb_private::StackFrame*, lldb_private::SharingPtr<lldb_private::ValueObject>&, lldb_private::EvaluateExpressionOptions const&) + 376 27  com.apple.LLDB.framework        0x000000011716d75c lldb_private::SwiftLanguageRuntime::GetObjectDescription(lldb_private::Stream&, lldb_private::ValueObject&) + 668 28  com.apple.LLDB.framework        0x00000001170464e6 lldb_private::ValueObject::GetObjectDescription() + 370 29  com.apple.LLDB.framework        0x000000011548e228 lldb::SBValue::GetObjectDescription() + 76 30  com.apple.dt.dbg.DebuggerLLDB   0x00000001153f3c9e -[DBGLLDBDataValue _lldbValueObjectDescription] + 24 31  com.apple.dt.dbg.DebuggerLLDB   0x00000001153f3b7f -[DBGLLDBDataValue lldbDescription] + 29 32  com.apple.dt.dbg.DebuggerLLDB   0x00000001154023dc __87-[DBGLLDBSession printDescriptionOfDataValueToConsole:runAllThreads:completionHandler:]_block_invoke + 182 33  com.apple.dt.dbg.DebuggerLLDB   0x0000000115402e6d -[DBGLLDBSession handleNextActionWithState:withRunPending:] + 424 34  com.apple.dt.dbg.DebuggerLLDB   0x00000001153fdf44 DBGLLDBSessionThread(void*) + 980 35  libsystem_pthread.dylib         0x00007fff8ec12cb3 _pthread_body + 131 36  libsystem_pthread.dylib         0x00007fff8ec12c30 _pthread_start + 168 37  libsystem_pthread.dylib         0x00007fff8ec10419 thread_start + 13 

Seems like we may be on the forefront of new technology here because I haven't found much help on this issue yet. I'm wondering if we have some sort of unusual configuration that is causing the compile time issues, or if there is a known reason why lldb might be crashing. Its the same on multiple different machines, with El Capitan, Yosemite, Xcode 6.3, Xcode 6.4. Any help is appreciated!

Answer 1

I've worked and I am actively working on projects that are probably even larger than yours (50+ libraries, hundreds of custom classes with thousands of LoC). It takes few seconds to compile and up to 2 minutes for release build (Late 2013 MBP), even for full Swift project.

I would highly recommend to look for other causes, as what this option tells you is that your problem is somewhere else since it affects app performance, not building speed. Maybe you have your .pch file full of stuff that does not belong there or something, it can cause this kind of slowing.

Other interesting thing is use of inference. I personally declare type for every single variable just because it is easier to read for others, but there is more to it. Since the inference can get complicated, it can cause compiler to take A LOT of time to figure out what your code actually does. Read this example article, though it was more like alpha-problem. BUT MAYBE, just maybe, you have something like this in your code as well. What would be worth doing is trying to remove all the swift code, compile libraries if that works well, then you know the problem is in your swift code and it will be probably connected to this.

Also, if you provide WHAT actually slows your build time (because you can see whole build process with timestamps, and there is high probability that one step will just take forever), it can help pinpoint the problem more accurately.

Hope it helps!

Edit: Another article on this interested topic