tee doesn't write to screen or file

Question

I use the following command to see which VirtualHosts are getting traffic on my server (alternatives welcome).

 ngrep 'Host:' port 80 -t  -W byline -q | grep  '^Host'

I want to be able to analyse the number of hits in a time interval while also seeing what's going on. To do this I have been trying to write the output to a file while also displaying it using tee.

 ngrep 'Host:' port 80 -t  -W byline -q | grep  '^Host' | tee ~/hosts.log

However, nothing is printed to screen or logged to file.

I have tried different file locations, outside of my screen session - with no difference. Does grep need special handling?

Answer 1

There is line buffering at play on the part of both ngrep and grep. Here is how you can get aroundthe issue you’re seeing:

% sudo ngrep 'Host:' port 80 -t -W byline -d lo -q -l |grep --line-buffered '^Host:' |tee foo.log

Notice that ngrep is using -l to make stdout line buffered, and grep uses --line-buffered for the same.

(I added the -d lo just for my own testing.)

Answer 2

You might use the stdbuf command, like this:

ngrep 'Host:' port 80 -t  -W byline -q | stdbuf -o0 grep '^Host' | tee /tmp/hosts

I'm setting the stdout buffer of grep to zero.

stdbuf is a general hammer that works for programs which are linked against glibc since output/input buffering happens in the libc. The ouput of a program which is writing to a terminal will getting line buffered, the output of a program which is outputting to a pipe or a file will getting block buffered - unless the program itself calls flush(). You can use stdbuf to modify this behaviour.

Btw, ngrep and tee seem to call flush() (ngrep on stdout, tee on stdin) since we didn't needed to use stdbuf for them.

---

I would accept MicahElliot's answer since --line-buffered is more specific for grep and should be used here. stdbuf is a nice alternative for programs which does not offer such an option (and for guys like me which didn't read grep's man page before ;) )