What is F1 Score
F1 Score - The harmonic mean of precision and recall
The F1 score is the harmonic mean of precision and recall: F1 = 2 × precision × recall / (precision + recall). It ranges from 0 to 1 and is high only when both precision and recall are high, so a system cannot raise it by returning everything (high recall, low precision) or by returning one safe answer (high precision, low recall). F1@K computes it over the top K results.
How It Works
Precision is the share of returned results that are relevant. Recall is the share of all relevant results that were returned. The harmonic mean pulls toward the smaller of the two, so precision 0.9 with recall 0.1 gives F1 0.18, while the plain average would be 0.5. See precision and recall for the two inputs.
Worked Example
- A query has 8 relevant results. The top 10 results contain 6 of them
- Precision@10 = 6 / 10 = 0.60 and recall@10 = 6 / 8 = 0.75
- F1@10 = 2 × 0.60 × 0.75 / (0.60 + 0.75) = 0.667
- F2@10, which weights recall more, = 5 × 0.60 × 0.75 / (4 × 0.60 + 0.75) = 0.714
Technical Details
- General form: F_beta = (1 + beta²) × P × R / (beta² × P + R). F1 sets beta = 1
- beta above 1 favors recall (F2 for search where a missed result is costly, such as compliance review); beta below 1 favors precision
- F1 is 0 whenever precision or recall is 0
- F1 ignores order inside the top K; use NDCG, MAP or MRR when rank matters
Measuring F1 in Mixpeek
Mixpeek retriever evaluations report F1 and F2 at each cutoff in `k_values`. Precision@K divides by the number of results actually returned, so a retriever that returns 3 of 3 relevant results scores precision 1.0 at K = 20 rather than 0.15. Run `POST /v1/retrievers/{retriever_id}/evaluations` with `"metrics": ["precision", "recall", "f1", "f2"]`. See retriever evaluations.
Common Pitfalls
- Reporting one F1 without the cutoff K it was computed at
- Averaging F1 per query (macro) and computing F1 from pooled counts (micro) give different numbers; state which
- Using F1 when one type of error costs far more than the other; pick a beta that reflects it
- Treating F1 as a ranking metric
Put multimodal search to work
Connect a bucket and Mixpeek runs the whole multimodal search pipeline for you: extraction, indexing, and search over your own objects. No models to wire up, nothing to host.
Start with ManagedAlready have vectors?
Keep your embeddings on your own cloud and run dense, sparse, and BM25 search directly on object storage. From $25/mo.
Start with MVS