Remove all articles, connector words, etc., from a string in Python

Question

I have a list that contains many sentences. I want to iterate through the list, removing from all sentences words like "and", "the", "a", "are", etc.

I tried this:

def removearticles(text):


articles = {'a': '', 'an':'', 'and':'', 'the':''}
for i, j in articles.iteritems():
    text = text.replace(i, j)
return text

As you can probably tell, however, this will remove "a" and "an" when it appears in the middle of the word. I need to remove only the instances of the words when they are delimited by blank space, and not when they are within a word. What is the most efficient way of going about this?

Answer 1

I would go for regex, something like:

def removearticles(text):
  re.sub('(\s+)(a|an|and|the)(\s+)', '\1\3', text)

or if you want to remove the leading whitespace as well:

def removearticles(text):
  re.sub('\s+(a|an|and|the)(\s+)', '\2', text)