I have X as a csr_matrix that I obtained using scikit's tfidf vectorizer, and y which is an array
My plan is to create features using LDA, however, I failed to find how to initialize a gensim's corpus variable with X as a csr_matrix. In other words, I don't want to download a corpus as shown in gensim's documentation nor convert X to a dense matrix, since it would consume a lot of memory and the computer could hang.
In short, my questions are the following,
Gensim has a semi-well-hidden function that can kind of do this for you:
http://radimrehurek.com/gensim/matutils.html#gensim.matutils.Sparse2Corpus
"class gensim.matutils.Sparse2Corpus(sparse, documents_columns=True) Convert a matrix in scipy.sparse format into a streaming gensim corpus."
I've had some success with it using a corpus extracted with CountVectorizer, then loaded into gensim.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With