Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Piping to grep and using regex

Tags:

regex

bash

Basically what I want to do is parse lines in a file and return usernames. Usernames are always surrounded in < and >, so I want to use regex to match eveything before (and including) the < and everything after (and including) the >, and then invert my match. I understand that grep -vE should be able to do this.

My script looks a little something like this so far:

#!/bin/bash
while read line; do
        echo $line | grep -vE '(.*<)|(>.*)'
done < test_log

And test_log consists of the following:

Mar  1 09:28:08 (IP redacted) dovecot: pop3-login: Login: user=<emcjannet>, method=PLAIN, rip=(IP redacted), lip=(IP redacted)
Mar  1 09:27:53 (IP redacted) dovecot: pop3-login: Login: user=<dprotzak>, method=PLAIN, rip=(IP redacted), lip=(IP redacted)
Mar  1 09:28:28 (IP redacted) dovecot: imap-login: Login: user=<gconnie>, method=PLAIN, rip=(IP redacted), lip=(IP redacted), TLS
Mar  1 09:27:25 (IP redacted) dovecot: imap-login: Login: user=<gconnie>, method=PLAIN, rip=(IP redacted), lip=(IP redacted), TLS

However, when running my script, nothing is returned, despite when I test the regex in something like regexpal with an inverse match it does exactly what I want. What am I doing wrong?

like image 605
Skyline969 Avatar asked Mar 01 '13 16:03

Skyline969


People also ask

Can you use regex with grep?

GNU grep supports three regular expression syntaxes, Basic, Extended, and Perl-compatible. In its simplest form, when no regular expression type is given, grep interpret search patterns as basic regular expressions. To interpret the pattern as an extended regular expression, use the -E ( or --extended-regexp ) option.

Can you pipe to grep?

grep is very often used as a "filter" with other commands. It allows you to filter out useless information from the output of commands. To use grep as a filter, you must pipe the output of the command through grep . The symbol for pipe is " | ".

What is pipe used for in regex?

A pipe symbol allows regular expression components to be logically ORed. For example, the following regular expression matches lines that start with the word "Germany" or the word "Netherlands". Note that parentheses are used to group the two expressive components.

Can you use regex in Linux command line?

Remember that you can use regexes with many Linux commands. We're just using grep as a convenient way to demonstrate them. The first part of the file is displayed. Each line that contains the search pattern is displayed, and the matching letter is highlighted.


2 Answers

try this grep line:

grep -Po "(?<=<)[^>]*"

or more secure:

grep -Po "(?<=user=<)[^>]*"

EDIT

short explanation

-P perl-regex
-o only matching
you can get above info from man page
(?<=foo)bar look-behind assertion. matches bar, only if bar is following foo.
[^>]* any not > characters.
like image 99
Kent Avatar answered Oct 22 '22 09:10

Kent


Actually, I like @Kent's answer too and it is correct, but sometimes it is difficult to remember switches like "-Po" for "grep" utility. Usually if you don't remember exact flag you may ask grep utility to refresh your memory in a following way:

$ grep --help | grep regex
  -E, --extended-regexp     PATTERN is an extended regular expression (ERE)
  -G, --basic-regexp        PATTERN is a basic regular expression (BRE)
  -P, --perl-regexp         PATTERN is a Perl regular expression
  -e, --regexp=PATTERN      use PATTERN for matching
  -w, --word-regexp         force PATTERN to match only whole words
  -x, --line-regexp         force PATTERN to match only whole lines

As we can see, there also another possible options, like "-E".

like image 20
Victor Signaevskyi Avatar answered Oct 22 '22 09:10

Victor Signaevskyi