What is the 11pt average precision metric?

Question

I have two questions

1- what is the "11pt average precision metric"?

2- and how to use it in information retrieval?

Thanks

Answer 1

This comes mainly from the book "Introduction to Information Retrieval" by Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze.

In ranked retrieval, you have precision and recall. Precision is how accurate the system is in the documents it returns (1.0 means it only returned relevant documents). Recall is what percentage of the relevant documents the system found (1.0 means it found them all). So you can adjust the recall of the system by returning more and more documents, but there is a tradeoff with precision, because you are likely to admit more error.

Interpolated precision is where you pick a recall level r and for all recall levels P(r') >= P(r), where P(r) is the precision at rank r. It is the best precision you can achieve.

Then in 11-pt interpolated average precision, you are looking at 11 recall levels (0.0, 0.1, 0.2, ..., 1.0) and finding the interpolated precision at each point. You average these scores across all of the different queries or information needs (as the Manning & Schuetze book put it) to find your system's score.

The typical usage in information retrieval is for evaluation. It helps you compare one system to another, but where it really helps is in letting you compare how your system is changing as you tweak parameters. Does changing this setting improve or damage my 11-pt interp avg prec? The benefit is, it measures across all information needs in your collection, so it helps to give you a "big picture" view. But, be careful about relying too heavily on these metrics since they are only as good as the evaluation set you are using.

Answer 2

Here is an online book, Introduction to Information Retrieval
and there is a reference to 11-point interpolated average precision use, evaluation of ranked retrieval results.