I built LDA model using Gensim and I want to get the topic words only How can I get the words of the topics only no probabilities and no IDs.words only
I tried print_topics() and show_topics() functions in gensim but I can't get clean words !
This is the code I used
dictionary = corpora.Dictionary(doc_clean)
doc_term_matrix = [dictionary.doc2bow(doc) for doc in doc_clean]
Lda = gensim.models.ldamodel.LdaModel
ldamodel = Lda(doc_term_matrix, num_topics=12, id2word = dictionary, passes = 100, alpha='auto', update_every=5)
x = ldamodel.print_topics(num_topics=12, num_words=5)
for i in x:
print(i[1])
#print('\n' + str(i))
0.045*تعرض + 0.045*الماضية + 0.045*السنوات + 0.045*وءسرته + 0.045*لءحمد
0.021*مصر + 0.021*الديمقراطية + 0.021*حرية + 0.021*باسم + 0.021*الحكومة
0.068*المواطنة + 0.068*الطاءفية + 0.068*وانهيارات + 0.068*رابطة + 0.005*طبول
0.033*عربية + 0.033*انكسارات + 0.033*رهابيين + 0.033*بحقوق + 0.033*ل
0.007*وحريات + 0.007*ممنهج + 0.007*قواءم + 0.007*الناس + 0.007*دراج
0.116*طبول + 0.116*الوطنية + 0.060*يكتب + 0.060*مصر + 0.005*عربية
0.064*قيم + 0.064*وهن + 0.064*عربيا + 0.064*والتعددية + 0.064*الديمقراطية
0.036*تضامنا + 0.036*الشخصية + 0.036*مع + 0.036*التفتيش + 0.036*الءخلاق
0.052*تضامنا + 0.052*كل + 0.052*محمد + 0.052*الخلوق + 0.052*مظلوم
0.034*بمواطنين + 0.034*رهابية + 0.034*لم + 0.034*عليهم + 0.034*يثبت
0.035*مع + 0.035*ومستشار + 0.035*يستعيدا + 0.035*ءرهقهما + 0.035*حريتهما
0.064*للقمع + 0.064*قريبة + 0.064*لا + 0.064*نهاية + 0.064*مصر
I tried show_topics and it gave the same output
y = np.array(ldamodel.show_topics(num_topics=12, num_words=5))
for i in y[:,1]:
#if i != '%d':
#print([str(word) for word in i])
print(i)
If I have the topic ID how can I access its words and other informations
Thanks in Advance
To decide on a suitable number of topics, you can compare the goodness-of-fit of LDA models fit with varying numbers of topics. You can evaluate the goodness-of-fit of an LDA model by calculating the perplexity of a held-out set of documents. The perplexity indicates how well the model describes a set of documents.
Passes is the number of times you want to go through the entire corpus. Below are a few examples of different combinations of the 3 parameters and the number of online training updates which will occur while training LDA.
pyLDAvis is an open-source python library that helps in analyzing and creating highly interactive visualization of the clusters created by LDA. In this article, we will see how to use LDA and pyLDAvis to create Topic Modelling Clusters visualizations.
I think the below code snippet should give you a list of tuples containing the each topic(tp) and corresponding list of words(wd) in that topic
x=ldamodel.show_topics(num_topics=12, num_words=5,formatted=False)
topics_words = [(tp[0], [wd[0] for wd in tp[1]]) for tp in x]
#Below Code Prints Topics and Words
for topic,words in topics_words:
print(str(topic)+ "::"+ str(words))
print()
#Below Code Prints Only Words
for topic,words in topics_words:
print(" ".join(words))
The other answer was giving a string with weights associated with each word. But if you want to get each word in a topic separately for further work. Then you can try this. Here topic no is the key to the dictionary and the value is a single string containing all words in that topic separated by space
x=ldamodel.show_topics()
twords={}
for topic,word in x:
twords[topic]=re.sub('[^A-Za-z ]+', '', word)
print(twords)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With