Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Split Header and Content in curl

curl -L -i google.com

I want to split the HEADER and CONTENT from the response in two variables

curl -I google.com
curl -L google.com

I cant use these two becouse im going to use it with 10000+ links

Both Header and Content can have three or more blank lines, so spliting blank lines wont work everytime


I found the answer

b=$(curl -LsD h google.com)
h=$(<h)
echo "$h$b"

This code works too

curl -sLi google.com | 
awk -v bl=1 'bl{bl=0; h=($0 ~ /HTTP\/1/)} /^\r?$/{bl=1} {print $0>(h?"header":"body")}'

header=$(<header)
body=$(<body)
like image 296
JavaOdd Avatar asked Oct 31 '22 12:10

JavaOdd


1 Answers

You can use this script:

curl -sLi google.com | 
   awk -v bl=1 'bl{bl=0; h=($0 ~ /HTTP\/1/)} /^\r?$/{bl=1} {print $0>(h?"header":"body")}'

header=$(<header)
body=$(<body)
like image 96
anubhava Avatar answered Nov 02 '22 10:11

anubhava