How to make term frequency matrix in python

Question

I am working on a sentiment analysis problem, I have to prepare a document frequency matrix. For example, I have three words(data) with the sentiment

He is a good person | Positive Sense

He is bad student | Negative Sense

He is hardworking | Positive Sense

There are the following words in the unique vocabulary.

He,is,a,good,person,bad,student,hardworking

Based on the vocabulary and data I will have 3X8 Matrix given bellow

For 1st sentence: 1,1,1,1,1,0,0,0

For 2nd sentence: 1,1,0,0,0,1,1,0

For 3rd sentence: 1,1,0,0,0,0,0,1

Please suggest any best and efficient way to achieve this in python.

Answer 1

Since you tagged the question with machine-learning, I suggest you use sklearn.CountVectorizer:

import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer

corpus = ['He is a good person',
          'He is bad student',
          'He is hardworking']
df = pd.DataFrame(data=corpus, columns=['sentences'])

vectorizer = CountVectorizer(vocabulary=['he', 'is', 'a', 'good', 'person', 'bad', 'student', 'hardworking'], min_df=0,
                             stop_words=frozenset(), token_pattern=r"(?u)\b\w+\b")
X = vectorizer.fit_transform(df['sentences'].values)
result = pd.DataFrame(data=X.toarray(), columns=vectorizer.get_feature_names())
print(result)

Output

   he  is  a  good  person  bad  student  hardworking
0   1   1  1     1       1    0        0            0
1   1   1  0     0       0    1        1            0
2   1   1  0     0       0    0        0            1