print header-value from dataframe matrix in python

Question

I have a matrix obtained with pandas.dataframe in this way:

tfidf = TfidfVectorizer()
x = tfidf.fit_transform(corpus)
df_tfidf = pd.DataFrame(x.toarray(),columns=tfidf.get_feature_names())

It seems like the matrix below:

My matrix has more columns and more rows. It has 7180 rows and 10390 columns. Is there a way to print the header of col and its value is this value is greater than 0 ? somethink like this and: 0.511859, document: 0.46,0.68 ..

I tried in this way but it take a lot of time:

for col in df_tfidf.columns:
   for row in df_tfidf.index:
     if df_tfidf[col][row] > 0:
        print str(df_tfidf[col][row]) + ' ' + col.encode('utf8')

Is there a way to do this faster ?

Answer 1

You can use boolean masking with numpy array to filter positive values inside a dict comprehension:

r = {c: s[s > 0] for c, s in zip(df, df.T.to_numpy())}

EDIT: DataFrame.to_numpy() is available in pandas version >= 0.24, if you are using pandas version below 0.24 then use:

r = {c: s[s > 0] for c, s in zip(df, df.T.values)}

Example:

# Sample dataframe
       col0      col1      col2
0  0.392938 -0.427721 -0.546297
1  0.102630  0.438938 -0.153787
2  0.961528  0.369659 -0.038136
3 -0.215765 -0.313644  0.458099
4 -0.122856 -0.880644 -0.203911

# Result
{'col0': array([0.39293837, 0.10262954, 0.9615284 ]),
 'col1': array([0.43893794, 0.36965948]),
 'col2': array([0.45809941])}