Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to access topic words only in gensim

I built LDA model using Gensim and I want to get the topic words only How can I get the words of the topics only no probabilities and no IDs.words only

I tried print_topics() and show_topics() functions in gensim but I can't get clean words !

This is the code I used

dictionary = corpora.Dictionary(doc_clean)
doc_term_matrix = [dictionary.doc2bow(doc) for doc in doc_clean]
Lda = gensim.models.ldamodel.LdaModel
ldamodel = Lda(doc_term_matrix, num_topics=12, id2word = dictionary, passes = 100, alpha='auto', update_every=5)
x = ldamodel.print_topics(num_topics=12, num_words=5)
for i in x:
    print(i[1])
    #print('\n' + str(i))

0.045*تعرض + 0.045*الماضية + 0.045*السنوات + 0.045*وءسرته + 0.045*لءحمد
0.021*مصر + 0.021*الديمقراطية + 0.021*حرية + 0.021*باسم + 0.021*الحكومة
0.068*المواطنة + 0.068*الطاءفية + 0.068*وانهيارات + 0.068*رابطة + 0.005*طبول
0.033*عربية + 0.033*انكسارات + 0.033*رهابيين + 0.033*بحقوق + 0.033*ل
0.007*وحريات + 0.007*ممنهج + 0.007*قواءم + 0.007*الناس + 0.007*دراج
0.116*طبول + 0.116*الوطنية + 0.060*يكتب + 0.060*مصر + 0.005*عربية
0.064*قيم + 0.064*وهن + 0.064*عربيا + 0.064*والتعددية + 0.064*الديمقراطية
0.036*تضامنا + 0.036*الشخصية + 0.036*مع + 0.036*التفتيش + 0.036*الءخلاق
0.052*تضامنا + 0.052*كل + 0.052*محمد + 0.052*الخلوق + 0.052*مظلوم
0.034*بمواطنين + 0.034*رهابية + 0.034*لم + 0.034*عليهم + 0.034*يثبت
0.035*مع + 0.035*ومستشار + 0.035*يستعيدا + 0.035*ءرهقهما + 0.035*حريتهما
0.064*للقمع + 0.064*قريبة + 0.064*لا + 0.064*نهاية + 0.064*مصر

I tried show_topics and it gave the same output

y = np.array(ldamodel.show_topics(num_topics=12, num_words=5))
for i in y[:,1]:
    #if i != '%d':
    #print([str(word) for word in i])
    print(i)

If I have the topic ID how can I access its words and other informations

Thanks in Advance

like image 745
Muhammed Eltabakh Avatar asked Oct 03 '17 01:10

Muhammed Eltabakh


People also ask

How do you choose the number of topics in topic modeling?

To decide on a suitable number of topics, you can compare the goodness-of-fit of LDA models fit with varying numbers of topics. You can evaluate the goodness-of-fit of an LDA model by calculating the perplexity of a held-out set of documents. The perplexity indicates how well the model describes a set of documents.

What is passes in LDA Gensim?

Passes is the number of times you want to go through the entire corpus. Below are a few examples of different combinations of the 3 parameters and the number of online training updates which will occur while training LDA.

What is pyLDAvis?

pyLDAvis is an open-source python library that helps in analyzing and creating highly interactive visualization of the clusters created by LDA. In this article, we will see how to use LDA and pyLDAvis to create Topic Modelling Clusters visualizations.


2 Answers

I think the below code snippet should give you a list of tuples containing the each topic(tp) and corresponding list of words(wd) in that topic

x=ldamodel.show_topics(num_topics=12, num_words=5,formatted=False)
topics_words = [(tp[0], [wd[0] for wd in tp[1]]) for tp in x]

#Below Code Prints Topics and Words
for topic,words in topics_words:
    print(str(topic)+ "::"+ str(words))
print()

#Below Code Prints Only Words 
for topic,words in topics_words:
    print(" ".join(words))
like image 198
oldmonk Avatar answered Sep 19 '22 09:09

oldmonk


The other answer was giving a string with weights associated with each word. But if you want to get each word in a topic separately for further work. Then you can try this. Here topic no is the key to the dictionary and the value is a single string containing all words in that topic separated by space

x=ldamodel.show_topics()

twords={}
for topic,word in x:
    twords[topic]=re.sub('[^A-Za-z ]+', '', word)
print(twords)
like image 35
Sreehari P.V Avatar answered Sep 21 '22 09:09

Sreehari P.V