How can I break on UBSan reports in gdb and continue?

Question

Recent versions of GCC and Clang feature Undefined Behavior Sanitizer (UBSan) which is a compile flag (-fsanitize=undefined) that adds runtime instrumentation code. On errors, a warning such as this one is shown:

packet-ber.c:1917:23: runtime error: left shift of 54645397829836991 by 8 places cannot be represented in type 'long int'

Now I would like to debug this and get a debug break on said line. For Address Sanitizer (ASAN) there is ASAN_OPTIONS=abort_on_error=1 which results in a fatal error that is catchable. The only UBSan option that seems usable is UBSAN_OPTIONS=print_stacktrace=1 which results in a call trace dump for reports. This however does not allow me to inspect the local variables and then continue the program. Use of -fsanitize-undefined-trap-on-error therefore not possible.

How should I break in gdb on UBSan reports? While break __sanitizer::SharedPrintfCode seems to work, the name looks quite internal.

Answer 1

While breaking on the detection functions (as described by @Mark Plotnick and @Iwillnotexist Idonotexist) is one option, a better approach is breaking on the functions that report these issues after detection. This approach is also used for ASAN where one would break on __asan_report_error.

Summary: You can stop on an ubsan report via a breakpoint on __ubsan::ScopedReport::~ScopedReport or __ubsan::Diag::~Diag. These are private implementation details which might change in the future though. Tested with GCC 4.9, 5.1.0, 5.2.0 and Clang 3.3, 3.4, 3.6.2.

For GCC 4.9.2 from ppa:ubuntu-toolchain-r/test, you need libubsan0-dbg to make the above breakpoints available. Ubuntu 14.04 with Clang 3.3 and 3.4 do not support the __ubsan::ScopedReport::~ScopedReport breakpoints, so you can only break before printing the message using __ubsan::Diag::~Diag.

Example buggy source code and a gdb session:

$ cat undef.c int main(void) { return 1 << 1000; } $ clang --version clang version 3.6.2 (tags/RELEASE_362/final) Target: x86_64-unknown-linux-gnu Thread model: posix $ clang -w -fsanitize=undefined undef.c -g $ gdb -q -ex break\ __ubsan::ScopedReport::~ScopedReport -ex r ./a.out  Reading symbols from ./a.out...done. Breakpoint 1 at 0x428fb0 Starting program: ./a.out  undef.c:1:27: runtime error: shift exponent 1000 is too large for 32-bit type 'int'  Breakpoint 1, 0x0000000000428fb0 in __ubsan::ScopedReport::~ScopedReport() () (gdb) bt #0  0x0000000000428fb0 in __ubsan::ScopedReport::~ScopedReport() () #1  0x000000000042affb in handleShiftOutOfBoundsImpl(__ubsan::ShiftOutOfBoundsData*, unsigned long, unsigned long, __ubsan::ReportOptions) () #2  0x000000000042a952 in __ubsan_handle_shift_out_of_bounds () #3  0x000000000042d057 in main () at undef.c:1 

---

Detailled analysis follows. Note that both ASAN and ubsan both originate from a LLVM project, compiler-rt. This is used by Clang and ends up in GCC as well. Links in the following sections point to the compiler-rt project code, release 3.6.

ASAN has made its internal __asan_report_error part of the documented public interface. This function gets called whenever a violation is detected, its flow continues in lib/asan/asan_report.c:938:

void __asan_report_error(uptr pc, uptr bp, uptr sp, uptr addr, int is_write,                          uptr access_size) {   // Determine the error type.   const char *bug_descr = "unknown-crash";   ...    ReportData report = { pc, sp, bp, addr, (bool)is_write, access_size,                         bug_descr };   ScopedInErrorReport in_report(&report);    Decorator d;   Printf("%s", d.Warning());   Report("ERROR: AddressSanitizer: %s on address "              "%p at pc %p bp %p sp %p\n",              bug_descr, (void*)addr, pc, bp, sp);   Printf("%s", d.EndWarning());    u32 curr_tid = GetCurrentTidOrInvalid();   char tname[128];   Printf("%s%s of size %zu at %p thread T%d%s%s\n",          d.Access(),          access_size ? (is_write ? "WRITE" : "READ") : "ACCESS",          access_size, (void*)addr, curr_tid,          ThreadNameWithParenthesis(curr_tid, tname, sizeof(tname)),          d.EndAccess());    GET_STACK_TRACE_FATAL(pc, bp);   stack.Print();    DescribeAddress(addr, access_size);   ReportErrorSummary(bug_descr, &stack);   PrintShadowMemoryForAddress(addr); } 

ubsan on the other hand has no public interface, but its current implementation is also much simpler and limited (less options). On errors, a stacktrace can be printed when the UBSAN_OPTIONS=print_stacktrace=1 environment variable is set. Thus, by searching the source code for print_stacktrace, one finds function MaybePrintStackTrace which is called though the ScopedReport destructor:

ScopedReport::~ScopedReport() {   MaybePrintStackTrace(Opts.pc, Opts.bp);   MaybeReportErrorSummary(SummaryLoc);   CommonSanitizerReportMutex.Unlock();   if (Opts.DieAfterReport || flags()->halt_on_error)     Die(); } 

As you can see, there is a method to kill the program on errors, but unfortunately there is no builtin mechanism to trigger a debugger trap. Let's find a suitable breakpoint then.

The GDB command info functions <function name> made it possible to identify MaybePrintStackTrace as function on which a breakpoint can be set. Execution of info functions ScopedReport::~ScopedReport gave another function: __ubsan::ScopedReport::~ScopedReport. If none of these functions seem available (even with debugging symbols installed), you can try info functions ubsan or info functions sanitizer to get all (UndefinedBehavior)Sanitizer-related functions.