Can TF/IDF take classes in account

Question

Using a classsication algorythm (for example naive bayes or SVM), and StringToWordVector, would it be possible to use TF/IDF and to count terms frequency in the whole current class instead of just looking in a single document?

Let me explain, I would like the computation to give high score to words that are very frequent for a given class (not just for a given document) but not very frequent in the whole corpus.

Is it possible out of the box or does this need some extra developments?

Thanks :)

Answer 1

I would like the computation to give high score to words that are very frequent for a given class (not just for a given document) but not very frequent in the whole corpus.

You seem to want supervised term weighting. I'm not aware of any off-the-shelf implementation of that, but there's a host of literature about it. E.g. the weighting scheme tf-χ² replaces idf with the result of a χ² independence test, so terms that statistically depend on certain classes get boosted, and there are several others.

Tf-idf itself is by its very nature unsupervised.

Answer 2

I think you're confusing yourself here---what you're asking for is essentially the feature weight on that term for documents of that class. This is what the learning algorithm is intended to optimise. Just worry about a useful representation of documents, which must necessarily be invariant to the class to which they belong (since you won't know what the class is for unseen test documents).