What exactly does 'use_idf' do when creating a TfidfTransformer in sklearn?

Question

I am using the TfidfTransformer from the sklearn package in Python 2.7.

As I was getting comfortable with the arguments, I became a bit confused about use_idf, as in:

TfidfVectorizer(use_idf=False).fit_transform(<corpus goes here>)

What exactly does use_idf do when false or true?

Since we are generating a sparse Tfidf matrix, it doesn't make sense to have an argument to choose a sparse Tfidif matrix; that seems redundant.

This post was interesting but didn't seem to nail it.

The documentation says only, Enable inverse-document-frequency reweighting, which isn't very illuminating.

Any comments appreciated.

EDIT I think I figured it out. It's real simple:
Text --> counts
Counts --> TF, meaning we just have raw counts or Counts --> TFIDF, meaning we have weighted counts.

What was confusing me was...since they called it TfidfVectorizer I didn't realize that was true only if you chose it to be a TFIDF. You could have also use it to create just a TF.

Answer 1

Typically, the tf-idf weight is composed by two terms: the first computes the normalized Term Frequency (TF), aka. the number of times a word appears in a document, divided by the total number of words in that document; the second term is the Inverse Document Frequency (IDF), computed as the logarithm of the number of the documents in the corpus divided by the number of documents where the specific term appears.

TF: Term Frequency, which measures how frequently a term occurs in a document. TF(t) = (Number of times term t appears in a document) / (Total number of terms in the document)

IDF: Inverse Document Frequency, which measures how important a term is. While computing TF, all terms are considered equally important. However it is known that certain terms, such as "is", "of", and "that", may appear a lot of times but have little importance. Thus we need to weigh down the frequent terms while scale up the rare ones, by computing the following:

IDF(t) = log_e(Total number of documents / Number of documents with term t in it).

If you give use_idf=False, you will score using only TF.