Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

creating word2vec model syn1neg.npy extension

When creating model,there is not any more model with extension finish

.syn1neg.npy

syn0.npy

My code is below:

corpus= x+y
tok_corp= [nltk.word_tokenize(sent.decode('utf-8')) for sent in corpus]
model = gensim.models.Word2Vec(tok_corp, min_count=1, size = 32)
model.save('/home/Desktop/test_model')

model = gensim.models.Word2Vec.load('/home/kafein/Desktop/chatbot/test_model')

There is only 1 model file

test_model

Which part i am wrong ?

like image 668
Tomas Ukasta Avatar asked Apr 24 '17 12:04

Tomas Ukasta


1 Answers

Gensim's native .save() only saves off parts of the model into such separate files (like test_model.syn1neg.npy etc) if they are larger than a certain threshold. When they're small, they get "pickled" up into the single model save file.

So there's no problem/error here. If you start training a larger model with more words, you may see those other files re-appear. (When you do, be sure to keep them alongside the main test_model file, if copying/moving them elsewhere – all the files together are needed to re-load() the model.)

like image 104
gojomo Avatar answered Sep 24 '22 16:09

gojomo