Remove stopwords from a string in Java

Question

I have a string with a lot of words that I need to count.

But I want to avoid some words without significancy to the context.

So, I have a file with all the words I will ignore. I open this file and create a list I call

ArrayList<String> stopWordsList;

Now I have the string and need to clean it, eliminating the stopWords from the list.

I've tried like this:

String example = "Job in a software factory. Work with Agile, Spring, Hibernate, GWT, etc.";

for(String stopWord : stopWordsList){
    example = example.replaceAll(" "+ stopWord + " ", " ");
}

After this, string example should be:

"Job software factory. Work Agile, Spring, Hibernate, GWT, ."

The problem is that "etc." was not remove it, because of the dot after the word.

Then I tried:

for(String stopWord : stopWordsList){
    example = example.replaceAll(" "+ stopWord + " ", " ");    
    example = example.replaceAll(" "+ stopWord + ",", ",");     
    example = example.replaceAll(" "+ stopWord + ".", ".");
}

But, this is not right, it does not do what I need.

Can anybody help me finding a way to clean this string, including words that comes before punctuations or blankspaces.

PS: I can not just do

 example = example.replaceAll(stopWord, " ");   

because this can break some words like "initial". It will remove "in" and leave me "itial".

Answer 1

The easiest way could be to split the String along word boundaries and add back everything but stop words.

StringBuilder result = new StringBuilder(example.length());
for (String s : result.split("\\b")) {
    if (!stopWordsSet.contains(s)) result.append(s);
}