Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python langdetect: choose between one language or the other only

I'm using langdetect to determine the language of a set of strings which I know are either in English or French.

Sometimes, langdetect tells me the language is Romanian for a string I know is in French.

How can I make langdetect choose between English or French only, and not all other languages?

Thanks!

like image 355
vandernath Avatar asked May 15 '16 08:05

vandernath


2 Answers

Option 1

One option would be using the package langid instead. Then you can simply restrict the languages with a method call:

import langid
langid.set_languages(['fr', 'en'])  # ISO 639-1 codes
lang, score = langid.classify('This is a french or english text')
print(lang) # en

Option 2

If you really want to use the langdetect package, you can copy the package folder (if you're not sure where it is, use python -m site --user-site) and remove the profiles you don't need from the folder langdetect\profiles.

This is not a very dynamic solution though.

like image 73
Philip Bergström Avatar answered Sep 20 '22 17:09

Philip Bergström


The way I'd do this is to use detect_langs, which returns a list of Language objects with probabilities, and then iterate through this list, returning the language if one of the options is English or French, or None if this isn't the case. This function works well for this purpose:

from langdetect import detect_langs

def englishOrFrench(string):
    res = detect_langs(string)
    for item in res:
        if item.lang == "fr" or item.lang == "en":
            return item.lang
    return None

print(englishOrFrench("Bonjour"))              # fr
print(englishOrFrench("The quick brown fox"))  # en
print(englishOrFrench("Hallo, mein Freund"))   # None
like image 27
Aaron Christiansen Avatar answered Sep 17 '22 17:09

Aaron Christiansen