7 Gemini Flash Generations, 143 Product Photos, and a Tie Worth Trusting
Summary
We ran seven generations of Gemini Flash over the same 143 real e-commerce product images: write the caption, tags and product type, scored against each merchant's own product title with Mixpeek's F2 metric (verified identical to the server's F2AtK on all 999 predictions). The headline is a near tie: the F2 spread across all seven models is 0.035 (0.207 to 0.241), too narrow to crown a winner. What does separate: Gemini 3.7 Flash leads recall at 0.291, the axis that decides whether an asset is findable at all, and Gemini 3.5 Flash Lite names the exact product type 61% of the time against the newest full model's 42%, at 1502ms versus 3063ms per image with the set's best precision. Generation number is a weak predictor: 2.5 Flash still beats three newer models. On the embedding side only one Gemini generation is scoreable on video retrieval: gemini-embedding-001 is text-only, the -2-preview id returns byte-identical vectors to GA (cosine 1.000000), and gemini-embedding-2 scores NDCG@10 0.764 on our public 20-video benchmark. Repo: github.com/mixpeek/video-embedding-benchmark
About this video
We ran seven generations of Gemini Flash over the same 143 real e-commerce product images: write the caption, tags and product type, scored against each merchant's own product title with Mixpeek's F2 metric (verified identical to the server's F2AtK on all 999 predictions). The headline is a near tie: the F2 spread across all seven models is 0.035 (0.207 to 0.241), too narrow to crown a winner. What does separate: Gemini 3.7 Flash leads recall at 0.291, the axis that decides whether an asset is findable at all, and Gemini 3.5 Flash Lite names the exact product type 61% of the time against the newest full model's 42%, at 1502ms versus 3063ms per image with the set's best precision. Generation number is a weak predictor: 2.5 Flash still beats three newer models. On the embedding side only one Gemini generation is scoreable on video retrieval: gemini-embedding-001 is text-only, the -2-preview id returns byte-identical vectors to GA (cosine 1.000000), and gemini-embedding-2 scores NDCG@10 0.764 on our public 20-video benchmark. Repo: github.com/mixpeek/video-embedding-benchmark