I am trying to create a BASH script what would extract the data from HTML table. Below is the example of table from where I need to extract data:
<table border=1>
<tr>
<td><b>Component</b></td>
<td><b>Status</b></td>
<td><b>Time / Error</b></td>
</tr>
<tr><td>SAVE_DOCUMENT</td><td>OK</td><td>0.406 s</td></tr>
<tr><td>GET_DOCUMENT</td><td>OK</td><td>0.332 s</td></tr>
<tr><td>DVK_SEND</td><td>OK</td><td>0.001 s</td></tr>
<tr><td>DVK_RECEIVE</td><td>OK</td><td>0.001 s</td></tr>
<tr><td>GET_USER_INFO</td><td>OK</td><td>0.143 s</td></tr>
<tr><td>NOTIFICATIONS</td><td>OK</td><td>0.001 s</td></tr>
<tr><td>ERROR_LOG</td><td>OK</td><td>0.001 s</td></tr>
<tr><td>SUMMARY_STATUS</td><td>OK</td><td>0.888 s</td></tr>
</table>
And I want the BASH script to output it like so:
SAVE_DOCUMENT OK 0.475 s
GET_DOCUMENT OK 0.345 s
DVK_SEND OK 0.002 s
DVK_RECEIVE OK 0.001 s
GET_USER_INFO OK 4.465 s
NOTIFICATIONS OK 0.001 s
ERROR_LOG OK 0.002 s
SUMMARY_STATUS OK 5.294 s
How to do it?
So far I have tried using the sed, but I don't know how to use it quite well. The header of the table(Component, Status, Time/Error) I excluded with grep using grep "<tr><td>
, so only lines starting with <tr><td>
will be selected for next parsing (sed).
This is what I used: sed 's@<\([^<>][^<>]*\)>\([^<>]*\)</\1>@\2@g'
But then <tr>
tags still remain and also it wont separate the strings. In other words the result of this script is:
<tr>SAVE_DOCUMENTOK0.406 s</tr>
The full command of the script I'm working on is:
cat $FILENAME | grep "<tr><td>" | sed 's@<\([^<>][^<>]*\)>\([^<>]*\)</\1>@\2@g'
$? is the exit status of the most recently-executed command; by convention, 0 means success and anything else indicates failure. That line is testing whether the grep command succeeded. The grep manpage states: The exit status is 0 if selected lines are found, and 1 if not found.
In this tutorial, we will explain two of the most useful bash expansions used in shell scripts: $() – the command substitution. ${} – the parameter substitution/variable expansion.
$_ (dollar underscore) is another special bash parameter and used to reference the absolute file name of the shell or bash script which is being executed as specified in the argument list. This bash parameter is also used to hold the name of mail file while checking emails. $@
These are positional arguments of the script. Executing ./script.sh Hello World. Will make $0 = ./script.sh $1 = Hello $2 = World. Note. If you execute ./script.sh , $0 will give output ./script.sh but if you execute it with bash script.sh it will give output script.sh .
You may use html2text
command and format the columns via column
, e.g.:
$ html2text table.html | column -ts'|'
Component Status Time / Error
SAVE_DOCUMENT OK 0.406 s
GET_DOCUMENT OK 0.332 s
DVK_SEND OK 0.001 s
DVK_RECEIVE OK 0.001 s
GET_USER_INFO OK 0.143 s
NOTIFICATIONS OK 0.001 s
ERROR_LOG OK 0.001 s
SUMMARY_STATUS OK 0.888 s
then parse it further from there (e.g. cut
, awk
, ex
).
In case you'd like to sort it first, you can use ex
, see the example here or here.
Go with (g)awk
, it's capable :-), here is a solution, but please note: it's only working with the exact html table format you had posted.
awk -F "</*td>|</*tr>" '/<\/*t[rd]>.*[A-Z][A-Z]/ {print $3, $5, $7 }' FILE
Here you can see it in action: https://ideone.com/zGfLe
Some explanation:
-F
sets the input field separator to a regexp (any of tr
's or td
's opening or closing tag
then works only on lines that matches those tags AND at least two upercasse fields
then prints the needed fields.
HTH
You can use bash xpath
(XML::XPath perl module) to accomplish that task very easily:
xpath -e '//tr[position()>1]' test_input1.xml 2> /dev/null | sed -e 's/<\/*tr>//g' -e 's/<td>//g' -e 's/<\/td>/ /g'
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With