Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is Vader SentimentIntensityAnalyzer Multilingual?

I'm stuck in sentiment analysis and I found Vader solution which is the best I could find so far. My issue is that I don't find any doc on how to feed it with languages other than English.

like image 832
Curcuma_ Avatar asked Jul 24 '17 08:07

Curcuma_


2 Answers

The short answer is "no".

The README file on the github page states

if you have access to the Internet, the demo has an example of how VADER can work with analyzing sentiment of texts in other languages (non-English text sentences).

but if you take a look at what is actually done for this demonstration (beginning at line 552 in the current version of vaderSentiment.py), this is based entirely on using a machine-translation web service to automatically translate the text into English. As such, the results are reliant not only upon the accuracy of the sentiment analysis tool but also upon the accuracy of whichever translation tool you use to create the English version of the input.

Vader only performs sentiment analysis on English texts, but that workaround (automatic translation) may be a viable option. Sentiment analysis is less sensitive to common machine translation problems than other usages*, but you'll certainly still have to keep the limitations in mind if you choose to use that workaround.

*To give an example, the service used in the demo translates "Das Internet funktioniert heute nicht. Ist eine Störung bekannt?" to "The Internet was not working today. Is a disorder known?", which would be more accurately translated as "The internet isn't working today. Is a disruption known?". It's got the tense wrong in the first sentence, and while there are several legitimate translations of "Störung" in this context, "disorder" is an awkward choice at best. Nevertheless, while this makes it quite a bad translation in general, the errors are unlikely to affect a sentiment analysis significantly.

like image 50
Chris H Avatar answered Sep 25 '22 23:09

Chris H


I tried NLTK Vader in another language. It works fairly well with German - after all, the languages are not too far from each other.

There is some work involved - we can't just translate the lexicon:

  • Change the vader_lexicon.txt
  • Change the NEGATE words in the code
  • Change the BOOSTER words in the code
  • Change SPECIAL_CASE_IDIOMS in the code

In general, negations work, but there are cases which involve some additional work which I haven't figured out yet.

like image 26
FrancoSwiss Avatar answered Sep 21 '22 23:09

FrancoSwiss