Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove duplicate words after stripping punctuation

Let's say I have a file with the following contents:

VSDmaMapInfo
VSDmaMapInfo::
VSDmaMapInfo;
VSPortErr
VSPortErr,
VSPortErr::

and after sorting I wanted the output to be

VSDmaMapInfo
VSPortErr

Is there any way to do it using either grep, awk, uniq,or any other tools etc

Thanks a lot for your help.

like image 480
Marc Spencer Avatar asked Dec 09 '25 23:12

Marc Spencer


2 Answers

$ awk -F'[[:punct:]]' '{print $1}' file | sort -u
VSDmaMapInfo
VSPortErr
like image 130
Ed Morton Avatar answered Dec 11 '25 13:12

Ed Morton


Code for sorted content with GNU sed

sed -r '$!N;/(\w+)\W*\n\1\W*/!{s/(\w+).*/\1/;P};D' file
like image 43
captcha Avatar answered Dec 11 '25 12:12

captcha



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!