Use Pandas string method 'contains' on a Series containing lists of strings

Question

Given a simple Pandas Series that contains some strings which can consist of more than one sentence:

In:
import pandas as pd
s = pd.Series(['This is a long text. It has multiple sentences.','Do you see? More than one sentence!','This one has only one sentence though.'])

Out:
0    This is a long text. It has multiple sentences.
1                Do you see? More than one sentence!
2             This one has only one sentence though.
dtype: object

I use pandas string method split and a regex-pattern to split each row into its single sentences (which produces unnecessary empty list elements - any suggestions on how to improve the regex?).

In:
s = s.str.split(r'([A-Z][^\.!?]*[\.!?])')

Out:
0    [, This is a long text.,  , It has multiple se...
1        [, Do you see?,  , More than one sentence!, ]
2         [, This one has only one sentence though., ]
dtype: object

This converts each row into lists of strings, each element holding one sentence.

Now, my goal is to use the string method contains to check each element in each row seperately to match a specific regex pattern and create a new Series accordingly which stores the returned boolean values, each signalizing if the regex matched on at least one of the list elements.

I would expect something like:

In:
s.str.contains('you')

Out:
0   False
1   True
2   False

<-- Row 0 does not contain 'you' in any of its elements, but row 1 does, while row 2 does not.

However, when doing the above, the return is

0   NaN
1   NaN
2   NaN
dtype: float64

I also tried a list comprehension which does not work:

result = [[x.str.contains('you') for x in y] for y in s]
AttributeError: 'str' object has no attribute 'str'

Any suggestions on how this can be achieved?

Answer 1

you can use python find() method

>>> s.apply(lambda x : any((i for i in x if i.find('you') >= 0)))
0    False
1     True
2    False
dtype: bool

I guess s.str.contains('you') is not working because elements of your series is not strings, but lists. But you can also do something like this:

>>> s.apply(lambda x: any(pd.Series(x).str.contains('you')))
0    False
1     True
2    False