Find String Pattern Match in Pandas Dataframe and Return Matched Strin

Question

I have a dataframe column with variable comma separated text and just trying to extract the values that are found based on another list. So my dataframe looks like this:

col1 | col2
-----------
 x   | a,b


listformatch = [c,d,f,b]
pattern = '|'.join(listformatch)

def test_for_pattern(x):
    if re.search(pattern, x):
        return pattern
    else:
        return x

#also can use col2.str.contains(pattern) for same results

The above filtering works great but instead of returning b when it finds the match it returns the whole pattern such as a|b instead of just b whereas I want to create another column with the pattern it finds such as b.

Here is my final function but still getting UserWarning: This pattern has match groups. To actually get the groups, use str.extract." groups, use str.extract.", UserWarning) I wish I can solve:

def matching_func(file1, file2):
    file1 = pd.read_csv(fin)
    file2 = pd.read_excel(fin1, 0, skiprows=1)
    pattern = '|'.join(file1[col1].tolist())
    file2['new_col'] = file2[col1].map(lambda x: re.search(pattern, x).group()\
                                             if re.search(pattern, x) else None)

I think I understand how pandas extract works now but probably still rusty on regex. How do I create a pattern variable to use for the below example:

df[col1].str.extract('(word1|word2)')

Instead of having the words in the argument, I want to create variable as pattern = 'word1|word2' but that won't work because of the way the string is being created.

My final and preferred version with vectorized string method in pandas 0.13:

Using values from one column to extract from a second column:

df[col1].str.extract('({})'.format('|'.join(df[col2]))

Answer 1

You might like to use extract, or one of the other vectorised string methods:

In [11]: s = pd.Series(['a', 'a,b'])

In [12]: s.str.extract('([cdfb])')
Out[12]:
0    NaN
1      b
dtype: object