Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Package tm: removeWords How do I avoid removing CERTIAN (negations specifically) "english" stopwords if specified?

I would like to use the removeWords (stopwords("english")) function via: corpus <- tm_map(corpus,removeWords, stopwords("english")) but some words like "not", and other negations I'd like to keep.

Is it possible to use the removeWords, stopwords("english") function BUT exclude certain words in that list if specified?

How could I prevent the removal of "not" for example?

(Secondary) is it possible to set this type of control list to all "negations"?

I'd rather not resort to creating my own custom list with only the words from that stoplist that I'm interested in.

like image 260
Robert Avatar asked Dec 01 '25 15:12

Robert


1 Answers

You can create a custom list of stopwords by taking the difference between stopwords("en") and the list of words you want to exclude:

exceptions   <- c("not")
my_stopwords <- setdiff(stopwords("en"), exceptions)

If you need to remove all the negations, you can grep them from the stopwords() list:

exceptions <- grep(pattern = "not|n't", x = stopwords(), value = TRUE)
# [1] "isn't"     "aren't"    "wasn't"    "weren't"   "hasn't"    "haven't"   "hadn't"    "doesn't"   "don't"     "didn't"   
# [11] "won't"     "wouldn't"  "shan't"    "shouldn't" "can't"     "cannot"    "couldn't"  "mustn't"   "not"
my_stopwords <- setdiff(stopwords("en"), exceptions)
like image 197
Duf59 Avatar answered Dec 04 '25 05:12

Duf59



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!