Search & Discovery

8 min read

Updated 2026-10-12

# Why Does My Content Moderation Flag So Many False Positives?

Content moderation flags too much when one confidence threshold does every job. Default cutoffs are low (Amazon Rekognition returns any moderation label at 50% confidence unless you set one), harmful content is rare so even an accurate classifier flags more safe items than real ones, and a video checked frame by frame gives every frame a chance to misfire. Set a threshold per category from your own labeled sample, send a middle band to review, and judge video by segment with context.

![Ethan Steininger](/people/ethan.webp)

By [Ethan Steininger](/blog/authors/ethan), Founder & CEO, Mixpeek

Content Moderation

Trust and Safety

Video

Classification

Thresholds

[Run this on your own data. Try it as you read.Search your own archive →](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=guide_cta&utm_campaign=mvs_onramp)

[One-click templateVideo Moderation Extraction FoundationSegment a video corpus, extract a keyframe record for every segment, and match brand marks against a reference set you upload. Face and public-figure matching run on a dedicated or customer-hosted deployment. This deploy is the extraction layer a moderation policy sits on, and the page carries a state on every part so you can see which policy pieces are yours to add.Deploy it](/templates/video-moderation)

## Why does my content moderation flag so many false positives?

  
  
Usually because one confidence threshold is doing every job. The default cutoffs in moderation APIs are low: Amazon Rekognition returns any moderation label at 50% confidence or higher unless you set a minimum. Harmful content is rare, so even an accurate classifier flags more safe items than harmful ones. And a video checked frame by frame gives each frame its own chance to misfire. The fix is a threshold per category, set from a labeled sample of your own content, a review band between "allow" and "block", and video judged by segment with the context the classifier cannot see.  
  

## What do moderation APIs actually return?

  
  
Each API returns a score per category, and the decision about what counts as a violation is left to you.  
  

API

What it returns

Default or scale

[Amazon Rekognition DetectModerationLabels](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_DetectModerationLabels.html)

Labels with a confidence from 0 to 100

Labels at 50% or more are returned unless you set MinConfidence

[Google Cloud Vision SafeSearch](https://cloud.google.com/vision/docs/detecting-safe-search)

A likelihood for adult, spoof, medical, violence and racy

Five buckets, VERY\_UNLIKELY to VERY\_LIKELY

[Azure AI Content Safety](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/harm-categories)

A severity for hate, sexual, violence and self-harm

Text on a 0 to 7 scale; images on 0, 2, 4 and 6

  
  
Checked 2026-10-12. A score of 50 or LIKELY is the model's confidence that a category is present. It says nothing about whether the content breaks your policy, which is the decision you need.  
  

## Why does a good classifier still flag mostly safe content?

  
  
Because harmful content is rare. Suppose 1 upload in 1,000 breaks your policy, and your classifier wrongly flags 1% of safe uploads. Out of 100,000 uploads, about 100 are harmful and 99,900 are safe, so the classifier flags about 999 safe uploads alongside at most 100 harmful ones. Roughly nine in ten flags are wrong, from a model that is right about 99% of safe content.  
  
The rarer the violation, the worse this gets, and it differs by category. Weapons may be rare on a cooking site and common on a hunting forum, so one threshold across every category is too strict for some and too loose for others.  
  

## Why does video moderation flag even more?

  
  
Each sampled frame is a separate chance to misfire. A 10-minute video sampled at one frame per second is 600 checks. If each frame has a 0.1% chance of a false flag, the chance that at least one of the 600 fires is 1 - 0.999^600, about 45%. Nearly half of perfectly safe videos would get flagged somewhere.  
  
Frames also lose context. A single frame of a surgery, a boxing match, a war documentary or a museum statue looks like the thing a policy bans, and the scene around it is what makes it acceptable.  
  

## Why doesn't similarity search work as a moderation filter?

  
  
A common shortcut is to embed each video segment and flag anything close to a text description such as "a gun being fired". In our own test of that approach, no similarity cutoff separated the footage: a revolver fired at the camera scored 0.33 against the description, below a folk dance at 0.34, and real logo and product footage scored 0.03 to 0.05. Similarity is good for ranking which segments to look at, and a separate judge has to decide whether each one shows the thing.  
  

## How do I cut false positives?

  
  
1\. **Label a sample of your own content.** A few hundred items per category, drawn from what users actually upload, with the flags you got and the right answer for each. 2. **Set a threshold per category from that sample.** Pick the score where precision is acceptable for that category, and accept that some categories need a much higher bar than others. 3. **Use two thresholds per category.** Allow below the lower one, block above the upper one, and send the band between them to review. Most volume never needs a person, and the uncertain cases get one. 4. **Judge video by segment.** Require a label on several consecutive frames, or score the scene as a whole, before acting on it. 5. **Give the review band context.** A vision language model that sees the segment, its transcript and your written policy can tell a surgery from an assault far better than a single-frame score. 6. **Measure precision every week.** Content drifts. Re-check a sample of flags and blocks and move thresholds when the mix changes.  
  

## How do I do this with Mixpeek?

  
  
Mixpeek reads your files where they already live, in S3, GCS or any bucket, and turns what's in them into data your software can search, classify and moderate. A retriever can find candidate segments with a [feature\_search](https://mixpeek.com/docs/retrieval/stages/feature-search) stage and then pass each one to an [llm\_filter](https://mixpeek.com/docs/retrieval/stages/llm-filter) stage with `include_media` on, so Gemini watches the segment and keeps it only when your written criteria hold. Other providers see the segment's thumbnail.

\[
  {
    "stage\_name": "candidates",
    "stage\_type": "filter",
    "config": {
      "stage\_id": "feature\_search",
      "parameters": {
        "searches": \[
          {
            "feature\_uri": "mixpeek://multimodal\_extractor@v1/vertex\_multimodal\_embedding",
            "query": { "input\_mode": "text", "value": "a weapon being fired" },
            "top\_k": 50
          }
        \]
      }
    }
  },
  {
    "stage\_name": "judge",
    "stage\_type": "filter",
    "config": {
      "stage\_id": "llm\_filter",
      "parameters": {
        "criteria": "Keep the segment only if a real firearm is fired at a person or toward the camera. Do not keep target shooting at a range, toy guns, or footage from films or games.",
        "include\_media": true,
        "include\_reasoning": true
      }
    }
  }
\]

The reasoning recorded for each decision gives your reviewers the why, and it is what you read when tuning the criteria. Prices are on the [pricing page](/pricing). To compare moderation services, see the [best AI content moderation tools](/curated-lists/best-ai-content-moderation-tools), the [best video moderation tools](/curated-lists/best-video-moderation-tools) and the [best NSFW detection APIs](/curated-lists/best-nsfw-detection-apis). For advertiser rules, see [brand safety vs brand suitability](/guides/brand-safety-vs-brand-suitability-video).  
  

## Frequently Asked Questions

  
  

### Why does my content moderation flag so many false positives?

  
  
Because one threshold does every job. Default cutoffs are low, harmful content is rare so even accurate classifiers flag more safe items than harmful ones, and frame-by-frame video checks give every frame a chance to misfire. Set thresholds per category from your own labeled content and send a middle band to review.  
  

### What confidence threshold should I use for content moderation?

  
  
There is no single right number. Label a few hundred of your own items per category, then pick the score where precision is acceptable for that category. Use two thresholds: allow below the lower one, block above the upper one, and review what falls between.  
  

### Why does moderation flag safe videos more than safe images?

  
  
A video is checked many times. At one frame per second a 10-minute video is 600 checks, and with a 0.1% false-flag rate per frame about 45% of safe videos get flagged at least once. Requiring a label across consecutive frames, or scoring the whole scene, removes most of those.  
  

### Can an LLM reduce moderation false positives?

  
  
Yes, when it judges the uncertain middle band rather than everything. A vision language model that sees the segment, its transcript and your written policy can use context a single-frame classifier cannot, such as telling medical footage from violence. It is slower and costs more per item, so run it after a cheaper first pass.

## Run this on your own data

Point Mixpeek at the storage you already have and search your video, images, audio, and documents the way this guide describes. Build starts at $25/mo for up to 1M vectors.

[Search your own archive](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=guide_cta&utm_campaign=mvs_onramp)[Read Docs](/docs)

[Explore MVS](/mvs)[See pricing](/pricing)

## Related guides

[

Search & Discovery

### How Do I Keep Footage of Minors Out of a Video Training Dataset?

To keep footage of minors out of a video training dataset, screen at ingest: detect and group faces across cuts, estimate each person's age, and send anyone near the cutoff or too unclear to age to human review. On our 650-face evaluation, a cutoff of 18 caught 82% of minors.

Read guide →](/guides/how-to-keep-footage-of-minors-out-of-a-video-training-dataset)[

Search & Discovery

### How Do I Find Out Where a Video Came From?

Someone sent you a clip and you want to know who filmed it, when, and whether it is what it claims to be. Reverse image search on a frame is the first move and it fails more often on video than people expect. This covers what each method actually catches, how to pull the right frames, how to spot recycled footage and AI-generated video, and what to do when the clip is cropped or re-encoded.

Read guide →](/guides/how-do-i-find-out-where-a-video-came-from)[

Search & Discovery

### Brand Safety vs Brand Suitability: How AI Classifies Video for Advertisers

Safety is a universal floor; suitability is a per-brand tolerance curve over graded risk tiers: and the GARM-style taxonomy remains the reference for both even after the organization wound down. How multimodal classification actually works on video (visual + speech + on-screen text + adjacency), why treatment beats topic, why moderation APIs can't express per-brand tiers, and the query-time reclassification pattern that avoids re-paying analysis.

Read guide →](/guides/brand-safety-vs-brand-suitability-video)