# Moderation

mondochat classifies text, photos, videos, voice notes and calls after they are delivered. A finding opens a case for your moderators in the dashboard. Nothing is hidden or removed unless your settings say so.

## Before you start

- Adult content is allowed. Sex, nudity, erotic talk and selling sexual content between adults are never flagged, and there is no category for them.
- You can sign in to the dashboard. Moderators work cases there. Admins edit settings and alert rules.
- The kinds you want are on in your settings: `moderation.text`, `moderation.photos`, `moderation.videos` and `moderation.voice` are all on by default, and calls use `moderation.calls.agora`.

## Steps

1. Know what is classified.

   | Kind | What is read | When |
   |---|---|---|
   | Text | The last 30 messages of the conversation, with no ids. A caption counts as a line. | After delivery, at most once per conversation per minute. |
   | Photo | A JPEG rendition of the photo. | After delivery. |
   | Video | The frames the sender's browser sent with the upload, judged together. | After delivery. |
   | Voice | A transcript of the note, judged as the sender's line in the conversation. | After a delivered note is sent. A note held for consent waits until it is released. |
   | Call | Agora call moderation per participant, or frames from the other participant's client. | While the call is active. See [Calls](/guides/calls/). |

   Reports from users open cases too, of kind `report`. See [Reports, blocks and bans](/guides/safety/).

2. Choose an action per category in `moderation.rules`. A category with no key acts as `alert`.

   | Action | What happens |
   |---|---|
   | `off` | The category is ignored. |
   | `alert` | Opens a case, sends the `moderation.flagged` webhook and fires your alert rules. The content stays delivered. |
   | `hold` | As `alert`, and the message or photo is hidden from the peer until a moderator acts. |
   | `block` | As `alert`, and the message is redacted or the photo is rejected. |

   The categories are:

   - `Minors`: any sign a participant is or could be under age, or any request for content of minors.
   - `Trafficking`: a third party controls a participant.
   - `NonConsent`: sextortion and sexual coercion, including threats to show images, and pressing on after a clear no.
   - `Violence`, `SelfHarm`, `Hate`, `Harassment`, `Doxxing`, `Fraud`: what the names say.
   - `IllegalServices`, `Weapons`, `Drugs`: selling or showing them.
   - `ExternalPayment`: the parties agree that money for the service changes hands outside your site. This is a business rule, not a safety one.
   - `Political` and `Alcohol`: images only.

   Most adult sites set `Minors`, `Trafficking` and `NonConsent` to `block`.

   ```json
   { "moderation": { "rules": { "Minors": "block", "Trafficking": "block", "NonConsent": "block", "Fraud": "hold" } } }
   ```

   Send the whole settings document with `PUT /v1/app/settings` and `If-Match`; see [Settings overview](/settings/overview/).

3. Work the cases in the dashboard. A case has a kind (`text`, `photo`, `video`, `voice`, `call` or `report`), a severity (`high`, `medium`, `low` or `info`), categories, evidence, context messages and a timeline. A moderator can assign it, add notes and act on it. The actions depend on the kind:

   | Kind | Actions |
   |---|---|
   | text, photo, video, voice | `dismiss`, `warn` (a system card to the user), `remove_content`, `ban` |
   | call | `dismiss`, `warn`, `end_call`, `ban` |
   | report | `dismiss`, `warn`, `ban` |

   Any other action for a kind answers 422. Acting on a case that is already decided answers 409 `conflict` with `decided_by`, `decided_by_name` and `decided_at`. The case list filters by status, kind, severity, category, assignee and age, and sorts by severity, newest or oldest.

4. Receive `moderation.flagged`. The webhook is sent for every case the classifier opens, calls included. A report opens a case but sends `report.created`, not `moderation.flagged`. The data is `{case_id, kind, severity, categories, action, subject}`, where `action` is the rule that applied: `alert`, `hold` or `block`. It never carries message text or media.

5. Get alerts in your own tools. An admin creates alert rules in the dashboard. A rule has one trigger and one destination.

   | Trigger | Fires |
   |---|---|
   | `moderation.flagged` | A new case of any kind except `call`, at or above a minimum severity and in the categories you pick (all, if you pick none). |
   | `call.moderation` | A new `call` case, with the same filters. |
   | `report.created` | A user reports someone. |
   | `ban.created` | A ban is created. |

   A new case fires exactly one trigger. A rule on `moderation.flagged` does not fire for a call case.

   | Destination | URL must be |
   |---|---|
   | Signed webhook (Standard Webhooks, its own secret) | Any https URL |
   | Slack | `hooks.slack.com` |
   | Discord | `discord.com/api/webhooks` or `discordapp.com/api/webhooks` |
   | Microsoft Teams | `*.api.powerplatform.com` or `*.logic.azure.com` |
   | Email | One or more addresses |

   A URL on the wrong host for its type answers 422. Use **Send test** on a rule to deliver a sample alert and see the response code. A disabled rule delivers nothing. Deliveries retry and are logged like webhook deliveries; see [Delivery and retries](/webhooks/delivery/).

6. Understand evidence. The flagged photo, frame, video, voice audio or transcript is kept while a case is open or actioned, and moderators open it through a link that lasts 120 seconds (1 hour for video, so it plays to the end). Evidence is deleted within 24 hours of an `allow` and 30 days after a `dismiss`.

## Check it works

The classifier is a real model, so no particular text is guaranteed to open a case on demand. Check the parts you control instead:

1. Set `Fraud` to `hold` in `moderation.rules` and save the settings.
2. Create an alert rule and press **Send test**. The destination gets a sample alert and the dashboard shows the response code.
3. Have one test user report another with `POST /v1/reports`. A case of kind `report` opens in the dashboard, severity `medium`, and your `report.created` webhook and `report.created` alert rules fire.
4. When a real message is flagged, a case opens in the dashboard and your endpoint gets `moderation.flagged`. With the rule on `hold`, the peer does not see the message until a moderator dismisses the case.

## When it fails

| What you see | Cause |
|---|---|
| No case for a message | The category is `off`, the kind is off in `moderation`, or text was already read for that conversation in the last minute. |
| Case opens but nothing is hidden | The rule is `alert`, which is the default for every category. |
| 422 on a case action | The action is not allowed for that kind. |
| 409 on a case action | Another moderator already decided the case. |
| 422 saving an alert rule | The URL host does not match the destination type. |

## Limits

- Moderation runs after delivery. Nothing is held or blocked before a message arrives.
- A call case from client frames never holds, blocks or ends the call, whatever your rules say.
- A video is judged on the frames its sender's browser sent, so a modified client can send clean frames.
- A voice note is judged on its words only, and only when it is delivered.
- If the AI provider refuses a photo as possible child abuse material, mondochat opens a `Minors` case even when `Minors` is `off`. Do not send that media to any AI and do not delete it. The photo is kept as evidence while the case is not dismissed, even if you set `block` or `photos_days` retention, or erase its uploader.
- Alert payloads carry ids, category, severity, a short label and a dashboard link, never message text, media, media URLs, report reasons or reporter emails.

## Related topics

- [moderation.flagged webhook](/api/webhooks/moderation-flagged/), [report.created webhook](/api/webhooks/report-created/), [ban.created webhook](/api/webhooks/ban-created/)
- [Reports, blocks and bans](/guides/safety/)
- [Settings reference](/reference/settings/) for `moderation` and `retention`
- [Errors](/errors/)