BASH output column formatting

Question

First time posting. HELLO WORLD. Working on my first script that just simply checks if a list of my websites are online and then returns the HTTP code and the amount of time it took to return that to another file on my desktop.

-- THIS SCRIPT WILL BE RUNNING ON MAC OSX --

I would like to amend my script so that it formats its output into 3 neat columns.

currently

#!/bin/bash
file="/Users/USER12/Desktop/url-list.txt"
printf "" > /Users/USER12/Desktop/url-results.txt
while read line
do
    printf "$line" >> /Users/USER12/Desktop/url-results.txt
    printf "\t\t\t\t" >> /Users/USER12/Desktop/url-results.txt
    curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line" >> /Users/USER12/Desktop/url-results.txt
    printf "\n" >> /Users/USER12/Desktop/url-results.txt
done <"$file"

which outputs in the following format

google.com              200 0.389
facebook.com                200 0.511
abnormallyLongDomain.com                200 0.786

but i would like to format into neat aligned columns for easy reading

DOMAIN_NAME                 HTTP_CODE   RESPONSE_TIME
google.com                  200         0.389
facebook.com                200         0.511
abnormallyLongDomain.com    200         0.486

Thanks for the help everyone!!

Answer 1

column is very nice. You are, however, already using printf which gives you fine control over the output format. Using printf's features also allows the code to be somewhat simplified:

#!/bin/bash
file="/Users/USER12/Desktop/url-list.txt"
log="/Users/USER12/Desktop/url-results.txt"
fmt="%-25s%-12s%-12s\n"
printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME > "$log"
while read line
do
    read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
    printf "$fmt" "$line" "$code" "$time" >> "$log"
done <"$file"

With the above defined format, the output looks like:

DOMAIN_NAME              HTTP_CODE   RESPONSE_TIME
google.com               301         0.305
facebook.com             301         0.415
abnormallyLongDomain.com 000         0.000

You can fine-tune the output format, such as spacing or alignment, by changing the fmt variable in the script.

Further Refinements

The above code opens and closes the log file with each loop. This can be avoided as Charles Duffy suggests, simply by using exec to redirect stdout to the log file before the first printf statement:

#!/bin/bash
file="/Users/USER12/Desktop/url-list.txt"
exec >"/Users/USER12/Desktop/url-results.txt"
fmt="%-25s%-12s%-12s\n"
printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME
while read line
do
    read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
    printf "$fmt" "$line" "$code" "$time"
done <"$file"

Alternatively, as Chepner suggests, the print statements can be grouped:

#!/bin/bash
file="/Users/USER12/Desktop/url-list.txt"
fmt="%-25s%-12s%-12s\n"
{
printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME
while read line
do
    read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
    printf "$fmt" "$line" "$code" "$time"
done <"$file"
} >"/Users/USER12/Desktop/url-results.txt"

An advantage of grouping is that, after the group, stdout is automatically restored to its normal value.

Answer 2

Shortened a bit

#!/bin/bash

file="./url.txt"
fmt="%s\t%s\t%s\n"
( printf "$fmt" "DOMAIN_NAME" "HTTP_CODE" "RESPONSE_TIME"
while read -r line
do
    printf "$fmt" "$line" $(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
done <"$file" ) | column -t > ./out.txt

Don't need redirect every printf but you can enclose the part of your script into (...) and run it in an subshell a redirect it's output. Print every field separated with one tab and use the column command to format it nicely.

Anyway, usually is better don't put filenames (nor headers) into the script and reduce it to

#!/bin/bash

while read -r line
do
    printf "%s\t%s\t%s\n" "$line" $(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
done | column -t

and use it like:

myscript.sh < url-list.txt >result.txt

this allows you use your script in pipes, like:

something_produces_urls | myscript.sh | grep 200 > somewhere.txt