Detecting debugger on Mac OS X

Question

I am trying to detect whether my process is being run in a debugger or not and, while in Windows there are many solutions and in Linux I use:

ptrace(PTRACE_ME,0,0,0) 

and check its return value, I did not manage to perform the same basic check on Mac OS X. I tried to use the

ptrace(PT_TRACE_ME,0,0,0)

call but it always returns 0 even when run under gdb.

If I change the request to PT_DENY_ATTACH it correctly stops the debugging but that is not what I want to achieve. Any ideas?

Answer 1

You can just call the function AmIBeingDebugged() from Apple Technical Q&A QA1361, which is reproduced here because Apple sometimes breaks documentation links and makes old documentation hard to find:

#include <assert.h>
#include <stdbool.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/sysctl.h>

static bool AmIBeingDebugged(void)
    // Returns true if the current process is being debugged (either 
    // running under the debugger or has a debugger attached post facto).
{
    int                 junk;
    int                 mib[4];
    struct kinfo_proc   info;
    size_t              size;

    // Initialize the flags so that, if sysctl fails for some bizarre 
    // reason, we get a predictable result.

    info.kp_proc.p_flag = 0;

    // Initialize mib, which tells sysctl the info we want, in this case
    // we're looking for information about a specific process ID.

    mib[0] = CTL_KERN;
    mib[1] = KERN_PROC;
    mib[2] = KERN_PROC_PID;
    mib[3] = getpid();

    // Call sysctl.

    size = sizeof(info);
    junk = sysctl(mib, sizeof(mib) / sizeof(*mib), &info, &size, NULL, 0);
    assert(junk == 0);

    // We're being debugged if the P_TRACED flag is set.

    return ( (info.kp_proc.p_flag & P_TRACED) != 0 );
}