Write your content moderation rules in plain English

Every platform with user content ends up with the same list of rules nobody's tooling knows about.

"Does this listing have a phone number in it?" Easy, until someone writes five five five, one two three four instead of 555-1234.
"Is this seller trying to take the payment outside our app?" A keyword filter bans everyone who types "Discord", including the people who sell Discord Nitro on your own marketplace.
"Is this profile photo too revealing for our audience?" Every app draws that line somewhere different.

Off-the-shelf moderation APIs give you fixed categories: nudity, hate, violence. Useful, but most of the moderation work we've seen on real platforms happens in the rules those categories don't cover.

That's what we're building with Detectivision.

What it does

One API. You send text, an image or a video, and you get back which of your rules it breaks, each with a confidence score.

The rules are the interesting part: each one is a plain yes/no question, in your own words.

Does the message share a phone number, Instagram handle or email, even if it's written out in words or split with spaces?
Does the message ask for payment outside the platform, like a bank transfer or IBAN?
Is the comment insulting or abusive, including slang and misspelled swear words?
Does the photo show alcohol or tobacco?

A rule can also say where the line is, in two more sentences: what counts and what doesn't.

contact_info
  Does the message share a phone number, Instagram handle or email, even if it's written out in words or split with spaces?
  Yes: digits, numbers written out in words ("five five five, one two three"), handles ("insta: jane.shop"), "jane at gmail".
  No:  prices, order numbers, the platform's own support address.

Each rule becomes a flag on your account, next to the built-in ones (hate, insult, profanity, sexual, scam, violence and so on). No custom classifier training, no category mapping. If your trust & safety person can say the rule out loud, it can be a flag.

Text, images and video are all live. Video is checked frame by frame at the rate you pick (0.5 to 10 fps), and each finding includes its timestamp.

Calling it

One endpoint for every type. The key goes in the Api-Key header:

curl https://api.detectivision.ai/api/v1/moderation \
  -H "Api-Key: $DETECTIVISION_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "text_moderation",
    "input": "my number is five five five, one two three four, text me on whatsapp",
    "mode": "detect",
    "flags": ["contact_info", "insult"]
  }'

On an account with the contact_info rule above:

{
  "id": 1050,
  "type": "text_moderation",
  "mode": "detect",
  "status": "completed",
  "flagged": true,
  "flags": [{ "flag": "contact_info", "confidence": 0.96 }],
  "checked": ["contact_info", "insult"],
  "unchecked": [],
  "tookMs": 640
}

• There's no action field on purpose. Whether a flag means block, hold for review or let it through is your call, in your code.
• Images and videos take a URL as input ("type": "image_moderation" or "video_moderation", plus fps for video).

Full reference: detectivision.ai/docs.

Running in production: live chat moderation for Kick

We run the same API in front of live chat. Our Kick integration checks every chat message as it arrives and removes flagged messages or times out the sender within seconds, with a live console where the streamer's moderators can review each decision. Streamers connect their channel with OAuth and moderation runs on their own Detectivision account.

It's open to Kick streamers today at streaming.detectivision.ai; Twitch support is coming next.

Try it

We're opening free trials for platforms that want to test their own rules on real content: marketplaces, dating apps, communities, live streams. The Basic plan is free for the first two months.

• Site: detectivision.ai
• Docs: detectivision.ai/docs
• Chat moderator for Kick streamers: streaming.detectivision.ai
• 30-second demo video: YouTube

If you run a platform with user content and want to try your own rules on real data, reach out through the site. We'd like to hear which rule you'd start with.