Delphi - Obtain Full Stack Trace on OSX

Question

I have an application which can log a stacktrace, which can be later used for debugging.

On Windows, I've gotten by using the excellent JCLDebug unit provided by the JEDI project.

Now that my application is running on OSX, I've hit a bit of a hitch - I don't know how to obtain the correct stacktrace when an exception occurs.

I have got the basics down -

1) I can get a stacktrace using 'backtrace' (found in libSystem.dylib)

2) The resulting backtrace can be converted into line numbers using the .map file provided by Delphi's linker

The issue I'm left with is - I don't know where to call backtrace from. I know that Delphi uses Mach exceptions (on a separate thread), and that I cannot use posix signals, but that's all that I've managed to sort out.

I can get a backtrace in the 'try...except' block, but unfortunately, by that point the stack has already wound down.

How can I install a proper exception logger which will run right after the exception occurs?

Update:

As per 'Honza R's suggestion, I've taken a look at the 'GetExceptionStackInfoProc' procedure.

This function does get me 'inside' of the exception handling process, but unfortunately leaves me with some of the same issues I had previously.

First of all - on desktop platforms, this function 'GetExceptionStackInfoProc' is just a function pointer, which you can assign with your own exception info handler. So out of the box, Delphi doesn't provide any stack information provider.

If I assign a function to 'GetExceptionStackInfoProc' and then run a 'backtrace' inside of it, I receive a stacktrace, but that trace is relative to the exception handler, not the thread which caused the exception.

'GetExceptionStackInfoProc' does contain a pointer to a 'TExceptionRecord', but there's very limited documentation available on this.

I might be going beyond my depth, but how can I get a stacktrace from the correct thread? Would it be possible for me to inject my own 'backtrace' function into the exception handler and then return to the standard exception handler from there?

Update 2

Some more details. One thing to clear up - this question is about exceptions that are handled by MACH messages, not software exceptions that are handled entirely within the RTL.

Embarcadero has laid out some comments along with these functions -

    System.Internal.MachExceptions.pas -> catch_exception_raise_state_identity

    {
     Now we set up the thread state for the faulting thread so that when we
     return, control will be passed to the exception dispatcher on that thread,
     and this POSIX thread will continue watching for Mach exception messages.
     See the documentation at <code>DispatchMachException()</code> for more
     detail on the parameters loaded in EAX, EDX, and ECX.
    }

    System.Internal.ExcUtils.pas -> SignalConverter

    {
      Here's the tricky part.  We arrived here directly by virtue of our
      signal handler tweaking the execution context with our address.  That
      means there's no return address on the stack.  The unwinder needs to
      have a return address so that it can unwind past this function when
      we raise the Delphi exception.  We will use the faulting instruction
      pointer as a fake return address.  Because of the fencepost conditions
      in the Delphi unwinder, we need to have an address that is strictly
      greater than the actual faulting instruction, so we increment that
      address by one.  This may be in the middle of an instruction, but we
      don't care, because we will never be returning to that address.
      Finally, the way that we get this address onto the stack is important.
      The compiler will generate unwind information for SignalConverter that
      will attempt to undo any stack modifications that are made by this
      function when unwinding past it.  In this particular case, we don't want
      that to happen, so we use some assembly language tricks to get around
      the compiler noticing the stack modification.
    }

Which seem to be responsible for the issue I'm having.

When I do a stacktrace after this exception system has handed control over to the RTL, it looks like this - (bearing in mind, the stack unwinder has been superseded by a backtrace routine. The backtrace will hand control over to the unwinder once it is completed)

0: MyExceptionBacktracer
1: initunwinder in System.pas
2: RaiseSignalException in System.Internal.ExcUtils.pas 

Since RaiseSignalException is called by SignalConverter, I'm led to believe that the backtrace function provided by libc is not compatible with the modifications made to the stack. So, it's incapable of reading the stack beyond that point, but the stack is still present underneath.

Does anyone know what to do about that (or whether my hypothesis is correct)?

Update 3

I've finally managed to get proper stacktraces on OSX. Huge thanks to both Honza and Sebastian. By combining both of their techniques, I found something that works.

For anyone else who could benefit from this, here's the basic source. Bear in mind that I'm not quite sure if it's 100% correct, if you can suggest improvements, go ahead. This technique hooks onto an exception right before Delphi unwinds the stack on the faulting thread, and compensates for any stack frame corruption that might have taken place beforehand.

unit MyExceptionHandler;

interface

implementation

uses
  SysUtils;

var
  PrevRaiseException: function(Exc: Pointer): LongBool; cdecl;

function backtrace2(base : NativeUInt; buffer : PPointer; size : Integer) : Integer;
var SPMin   : NativeUInt;
begin
  SPMin:=base;
  Result:=0;
  while (size > 0) and (base >= SPMin) and (base <> 0) do begin

    buffer^:=PPointer(base + 4)^;
    base:=PNativeInt(base)^;

    //uncomment to test stacktrace
    //WriteLn(inttohex(NativeUInt(buffer^), 8));

    Inc(Result);
    Inc(buffer);
    Dec(size);

  end;
  if (size > 0) then buffer^:=nil;
end;

procedure UnInstallExceptionHandler; forward;

var
  InRaiseException: Boolean;

function RaiseException(Exc: Pointer): LongBool; cdecl;
var b : NativeUInt;
    c : Integer;
    buff : array[0..7] of Pointer;
begin
  InRaiseException := True;

  asm
    mov b, ebp
  end;

  c:=backtrace2(b - $4 {this is the compiler dependent value}, @buff, Length(buff));
  //... do whatever you want to do with the stacktrace

  Result := PrevRaiseException(Exc);
  InRaiseException := False;
end;

procedure InstallExceptionHandler;
var
  U: TUnwinder;
begin
  GetUnwinder(U);
  Assert(Assigned(U.RaiseException));
  PrevRaiseException := U.RaiseException;
  U.RaiseException := RaiseException;
  SetUnwinder(U);
end;

procedure UnInstallExceptionHandler;
var
  U: TUnwinder;
begin
  GetUnwinder(U);
  U.RaiseException := PrevRaiseException;
  SetUnwinder(U);
end;

initialization
  InstallExceptionHandler;
end.

Answer 1

You can use GetExceptionStackInfoProc, CleanUpStackInfoProc and GetStackInfoStringProc in Exception class you need to save stack trace in GetExceptionStackInfoProc and then retrieve it with GetStackInfoStringProc which will get called by RTL if you use StackTrace property of the Exception. Maybe you could also take look at https://bitbucket.org/shadow_cs/delphi-arm-backtrace which demonstrates this on Android.

To do this properly on Mac OS X the libc backtrace function cannot be used because Delphi will corrupt stack frame when calling the GetExceptionStackInfoProc from Exception.RaisingException. Own implementation must be used that is capable of walking the stack from different base address which can be corrected by hand.

Your GetExceptionStackInfoProc would then look like this (I used XE5 for this example the value added to EBP bellow may differ based on which compiler you use and this example was only tested on Mac OS X, Windows implementation may or may not differ):

var b : NativeUInt;
    c : Integer;
    buff : array[0..7] of Pointer;
begin
  asm
    mov b, ebp
  end;
  c:=backtrace2(b - $14 {this is the compiler dependent value}, @buff, Length(buff));
  //... do whatever you want to do with the stacktrace
end;

And the backtrace2 function would look like this (note that stop conditions and other validations are missing in the implementation to ensure that AVs are not caused during stack walking):

function backtrace2(base : NativeUInt; buffer : PPointer; size : Integer) : Integer;
var SPMin   : NativeUInt;
begin
  SPMin:=base;
  Result:=0;
  while (size > 0) and (base >= SPMin) and (base <> 0) do begin
    buffer^:=PPointer(base + 4)^;
    base:=PNativeInt(base)^;
    Inc(Result);

    Inc(buffer);
    Dec(size);
  end;
  if (size > 0) then buffer^:=nil;
end;