How can I monitor what's being put into the standard out buffer and break when a specific string is deposited in the pipe?

Question

In Linux, with C/C++ code, using gdb, how can you add a gdb breakpoint to scan the incoming strings in order to break on a particular string?

I don't have access to a specific library's code, but I want to break as soon as that library sends a specific string to standard out so I can go back up the stack and investigate the part of my code that is calling the library. Of course I don't want to wait until a buffer flush occurs. Can this be done? Perhaps a routine in libstdc++ ?

Answer 1

This question might be a good starting point: how can I put a breakpoint on "something is printed to the terminal" in gdb?

So you could at least break whenever something is written to stdout. The method basically involves setting a breakpoint on the write syscall with a condition that the first argument is 1 (i.e. STDOUT). In the comments, there is also a hint as to how you could inspect the string parameter of the write call as well.

x86 32-bit mode

I came up with the following and tested it with gdb 7.0.1-debian. It seems to work quite well. $esp + 8 contains a pointer to the memory location of the string passed to write, so first you cast it to an integral, then to a pointer to char. $esp + 4 contains the file descriptor to write to (1 for STDOUT).

$ gdb break write if 1 == *(int*)($esp + 4) && strcmp((char*)*(int*)($esp + 8), "your string") == 0

x86 64-bit mode

If your process is running in x86-64 mode, then the parameters are passed through scratch registers %rdi and %rsi

$ gdb break write if 1 == $rdi && strcmp((char*)($rsi), "your string") == 0

Note that one level of indirection is removed since we're using scratch registers rather than variables on the stack.

Variants

Functions other than strcmp can be used in the above snippets:

• strncmp is useful if you want match the first n number of characters of the string being written
• strstr can be used to find matches within a string, since you can't always be certain that the string you're looking for is at the beginning of string being written through the write function.

Edit: I enjoyed this question and finding it's subsequent answer. I decided to do a blog post about it.

Answer 2

catch + strstr condition

The cool thing about this method is that it does not depend on glibc write being used: it traces the actual system call.

Furthermore, it is more resilient to printf() buffering, as it might even catch strings that are printed across multiple printf() calls.

x86_64 version:

define stdout
    catch syscall write
    commands
        printf "rsi = %s\n", $rsi
        bt
    end
    condition $bpnum $rdi == 1 && strstr((char *)$rsi, "$arg0") != NULL
end
stdout qwer

Test program:

#define _XOPEN_SOURCE 700
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main() {
    write(STDOUT_FILENO, "asdf1", 5);
    write(STDOUT_FILENO, "qwer1", 5);
    write(STDOUT_FILENO, "zxcv1", 5);
    write(STDOUT_FILENO, "qwer2", 5);
    printf("as");
    printf("df");
    printf("qw");
    printf("er");
    printf("zx");
    printf("cv");
    fflush(stdout);
    return EXIT_SUCCESS;
}

Outcome: breaks at:

• qwer1
• qwer2
• fflush. The previous printf didn't actually print anything, they were buffered! The write syacall only happened on the fflush.

Notes:

• $bpnum thanks to Tromey at: https://sourceware.org/bugzilla/show_bug.cgi?id=18727
• rdi: register that contains the number of the Linux system call in x86_64, 1 is for write
• rsi: first argument of the syscall, for write it points to the buffer
• strstr: standard C function call, searches for submatches, returns NULL if non found

Tested in Ubuntu 17.10, gdb 8.0.1.

strace

Another option if you are feeling interactive:

setarch "$(uname -m)" -R strace -i ./stdout.out |& grep '\] write'

Sample output:

[00007ffff7b00870] write(1, "a\nb\n", 4a

Now copy that address and paste it into:

setarch "$(uname -m)" -R strace -i ./stdout.out |& grep -E '\] write\(1, "a'

The advantage of this method is that you can use the usual UNIX tools to manipulate strace output, and it does not require deep GDB-fu.

Explanation:

• -i makes strace output RIP
• setarch -R disables ASLR for a process with a personality system call: How to debug with strace -i when everytime address is different GDB already does that by default, so no need to do it again.