What is Mean Average Precision (MAP)

How It Works

Walk down the ranked list. Each time a relevant result appears at position k, record precision at k: the relevant results seen so far divided by k. Add those values and divide by the total number of relevant results for the query, including any the system never returned. That is AP for one query, and MAP is the mean of AP over the evaluation set.

Worked Example

• Query 1 has 3 relevant results, found at positions 1, 3 and 6: AP = (1/1 + 2/3 + 3/6) / 3 = 0.722
• Query 2 has 2 relevant results, and only one is returned, at position 2: AP = (1/2) / 2 = 0.25
• MAP = (0.722 + 0.25) / 2 = 0.486
• The relevant result query 2 never returned still counts in its denominator, which is why AP also reflects recall

Retrieval MAP vs Object Detection mAP

• Retrieval MAP judges a ranked list of documents against relevance labels
• Detection mAP judges predicted boxes: a box counts as a hit when its intersection over union (IoU) with a labeled box passes a threshold
• [email protected] uses one IoU threshold of 0.5; COCO mAP averages over thresholds from 0.5 to 0.95 in steps of 0.05
• Both compute average precision the same way and differ in what counts as relevant

Measuring MAP in Mixpeek

Mixpeek retriever evaluations compute MAP at each cutoff in `k_values`, next to precision, recall, F1, NDCG and MRR. AP is divided by the total number of relevant documents in the ground truth, so a retriever that misses relevant documents scores lower even when its top results are right. Run `POST /v1/retrievers/{retriever_id}/evaluations` with `"metrics": ["map"]`. See retriever evaluations.

Common Pitfalls

• Comparing MAP@10 from one system with MAP over the full list from another
• Ground truth that lists only some of the relevant results, which makes AP look better than it is
• Using MAP when relevance comes in grades; NDCG handles graded labels
• Confusing retrieval MAP with detection mAP when reading a model card