printf slows down my program

Question

I have a small C program to calculate hashes (for hash tables). The code looks quite clean I hope, but there's something unrelated to it that's bugging me.

I can easily generate about one million hashes in about 0.2-0.3 seconds (benchmarked with /usr/bin/time). However, when I'm printf()inging them in the for loop, the program slows down to about 5 seconds.

1. Why is this?
2. How to make it faster? mmapp()ing stdout maybe?
3. How is stdlibc designed in regards to this, and how may it be improved?
4. How could the kernel support it better? How would it need to be modified to make the throughput on local "files" (sockets,pipes,etc) REALLY fast?

I'm looking forward for interesting and detailed replies. Thanks.

PS: this is for a compiler construction toolset, so don't by shy to get into details. While that has nothing to do with the problem itself, I just wanted to point out that details interest me.

Addendum

I'm looking for more programatic approaches for solutions and explanations. Indeed, piping does the job, but I don't have control over what the "user" does.

Of course, I'm doing a testing right now, which wouldn't be done by "normal users". BUT that doesn't change the fact that a simple printf() slows down a process, which is the problem I'm trying to find an optimal programmatic solution for.

---

Addendum - Astonishing results

The reference time is for plain printf() calls inside a TTY and takes about 4 mins 20 secs.

Testing under a /dev/pts (e.g. Konsole) speeds up the output to about 5 seconds.

It takes about the same amount of time when using setbuffer() in my testing code to a size of 16384, almost the same for 8192: about 6 seconds.

setbuffer() has apparently no effect when using it: it takes the same amount of time (on a TTY about 4 mins, on a PTS about 5 seconds).

The astonishing thing is, if I'm starting the test on TTY1 and then switch to another TTY, it does take just the same as on a PTS: about 5 seconds.

Conclusion: the kernel does something which has to do with accessibility and user friendliness. HUH!

Normally, it should be equally slow no matter if you stare at the TTY while its active, or you switch over to another TTY.

---

Lesson: when running output-intensive programs, switch to another TTY!

Answer 1

Unbuffered output is very slow.

By default stdout is fully-buffered, however when attached to terminal, stdout is either unbuffered or line-buffered.

Try to switch on buffering for stdout using setvbuf(), like this:

char buffer[8192];

setvbuf(stdout, buffer, _IOFBF, sizeof(buffer));

Answer 2

You could store your strings in a buffer and output them to a file (or console) at the end or periodically, when your buffer is full.

If outputting to a console, scrolling is usually a killer.

Answer 3

If you are printf()ing to the console it's usually extremely slow. I'm not sure why but I believe it doesn't return until the console graphically shows the outputted string. Additionally you can't mmap() to stdout.

Writing to a file should be much faster (but still orders of magnitude slower than computing a hash, all I/O is slow).