Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Regex a string with unknown number of parameters

Let say I have millions of string in a text file in this format:

st=expand&c=22&t=button&k=fun HTTP

This is a string we can look at as a hash with keys st, c, t and k. Some of the strings in the text file might not have a given &KEY=VALUE present and might thus look like this:

st=expand&k=fun HTTP

How would one use sed to change the string to following

expand,,,fun

that is, even thought the key=value isn't present, we still add a comma. We can assume that we have a fixed key set [st,c,t,k].

What I've tried is something like (just an idea!!)

sed 's/\(st=\|c=\|t=\|k=\)\([\(^\&\|HTTP\)])\(\&\|HTTP\)/\3,/g' big_file

but obviously, if c isn't there, it isn't adding a comma since it doesn't find any. Any ideas how to approach this? Using awk might also be acceptable (or any other fast text-processing utility)

Thanks!


Input data example

st=expand&c=22&t=button&k=fun HTTP
c=22&t=button&k=fun HTTP
st=expand&c=22&t=party&k=fun HTTP
st=expand&c=22&k=fun HTTP
st=expand HTTP
 HTTP

Output data

expand,22,button,fun
,22,button,fun
expand,22,party,fun
expand,22,,fun
expand,,,
,,,
like image 855
Ingo Avatar asked Apr 22 '16 16:04

Ingo


1 Answers

You can use this sed:

sed -E 's/(st=([^& ]*)|)(.*c=([^& ]*)|)(.*t=([^& ]*)|)(.*k=([^& ]*)|) HTTP/\2,\4,\6,\8/' file

expand,22,button,fun
,22,button,fun
expand,22,party,fun
expand,22,,fun
expand,,,
,,,

Sed Demo

RegEx Demo

like image 190
anubhava Avatar answered Oct 05 '22 02:10

anubhava