Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWK: execute CURL on each line and parse result

Tags:

shell

curl

awk

given an input stream with following lines:

123
456
789
098
...

I would like to call

curl -s http://foo.bar/some.php?id=xxx

with xxx being the number for each line, and everytime let an awk script fetch some information from the curl output which is written to the output stream. I am wondering if this is possible without using the awk "system()" call in following way:

cat lines | grep "^[0-9]*$" | awk '
    {
        system("curl -s " $0 \
        " | awk \'{ #parsing; print }\'")
    }'
like image 524
cbix Avatar asked Jul 18 '11 21:07

cbix


1 Answers

You can use bash and avoid awk system call:

grep "^[0-9]*$" lines | while read line; do
    curl -s "http://foo.bar/some.php?id=$line" | awk 'do your parsing ...'
done
like image 120
Foo Bah Avatar answered Sep 28 '22 06:09

Foo Bah