Menu
I have downloaded a .bin FastText model, and I use it with gensim as follows:
model = FastText.load_fasttext_format("cc.fr.300.bin")
I would like to continue the training of the model to adapt it to my domain. After checking FastText's Github and the Gensim documentation it seems like it is not currently feasible appart from using this person's proposed modification (not yet merged).
Am I missing something?
734
To train a Word2Vec model takes about 22 hours, and FastText model takes about 33 hours. If it's too long to you, you can use fewer "iter", but the performance might be worse.
FastText uses a simple and efficient baseline for sentence classification( represent sentences as bag of words (BoW) and train a linear classifier). It uses negative sampling , hierarchical softmax and N-gram features to reduce computational cost and improve efficiency.
What is fastText? FastText is an open-source, free, lightweight library that allows users to learn text representations and text classifiers. It works on standard, generic hardware. Models can later be reduced in size to even fit on mobile devices.
The model can be stored/loaded via its save() and load() methods, or loaded from a format compatible with the original Fasttext implementation via load_facebook_model() .
You can continue training in some versions of Gensim's fastText (for example, v.3.7.*). Here is an example of "Loading, inferring, continuing training"
from gensim.test.utils import datapath
model = load_facebook_model(datapath("crime-and-punishment.bin"))
sent = [['lord', 'of', 'the', 'rings'], ['lord', 'of', 'the', 'semi-groups']]
model.build_vocab(sent, update=True)
model.train(sentences=sent, total_examples = len(sent), epochs=5)
For some reason, the gensim.models.fasttext.load_facebook_model() is missing on Windows, but exists on Mac's installation. Alternatively, one can use gensim.models.FastText.load_fasttext_format() to load a pre-trained model and continue training.
Here are various pre-trained Wiki word models and vectors (or here).
Another example. "Note: As in the case of Word2Vec, you can continue to train your model while using Gensim's native implementation of fastText."
200
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
