# CLAUDE Source: https://docs.moderationapi.com/CLAUDE # Mintlify documentation ## Working relationship * You can push back on ideas-this can lead to better documentation. Cite sources and explain your reasoning when you do so * ALWAYS ask for clarification rather than making assumptions * NEVER lie, guess, or make up information ## Project context * Format: MDX files with YAML frontmatter * Config: docs.json for navigation, theme, settings * Components: Mintlify components ## Content strategy * Document just enough for user success - not too much, not too little * Prioritize accuracy and usability of information * Make content evergreen when possible * Search for existing information before adding new content. Avoid duplication unless it is done for a strategic reason * Check existing patterns for consistency * Start by making the smallest reasonable changes ## docs.json * Refer to the [docs.json schema](https://mintlify.com/docs.json) when building the docs.json file and site navigation ## Frontmatter requirements for pages * title: Clear, descriptive page title * description: Concise summary for SEO/navigation ## Writing standards * Second-person voice ("you") * Prerequisites at start of procedural content * Test all code examples before publishing * Match style and formatting of existing pages * Include both basic and advanced use cases * Language tags on all code blocks * Alt text on all images * Relative paths for internal links ## Git workflow * NEVER use --no-verify when committing * Ask how to handle uncommitted changes before starting * Create a new branch when no clear branch exists for changes * Commit frequently throughout development * NEVER skip or disable pre-commit hooks ## Do not * Skip frontmatter on any MDX file * Use absolute URLs for internal links * Include untested code examples * Make assumptions - always ask for clarification * Expose underlying dependencies, technology, or internal APIs. Describe user-facing behavior only — no vendor or model names (OpenAI, GPT, Anthropic, Claude, etc.), no infra (Redis, Postgres, xxHash), no frameworks or SDKs (tRPC, React, Tailwind, Zustand, Prisma), and no internal endpoints or procedure names. Only the public REST endpoints under `/api-reference/*` should be referenced by name. Frame limits and constraints around reliability and UX, not cost or token counts. # Manage team Source: https://docs.moderationapi.com/account/adding-team-members How to add team members to your organization Visit the [team management page](https://dash.moderationapi.com/team) to add team members to your organization. You can add team members by entering their email address. They will receive an email invitation to join your organization. ## Roles There are three roles in an organization: * **Owner**: The owner has full access to the organization and can manage team members and billing. Only one owner can exist per organization and cannot be removed or changed. * **Admin**: The admin has full access to the organization and can manage team members and billing. * **Developer**: The developer can access and change the projects, models, and review queues, but cannot manage team members or billing. * **Moderator**: The moderator can access and use the review queues, but cannot manage team members, billing, projects, or models. You can limit the access of moderators to specific queues. # Quota usage Source: https://docs.moderationapi.com/account/quota-usage Understanding how quota usage works and how to manage it is important for managing your costs. ## Included quota All Moderation API plans include a certain amount of quota. This quota is used when you submit content for moderation. The included quota is different for each plan. See the [pricing page](https://moderationapi.com/pricing) for more information. Custom plans can be created with a different quota for use cases where you need more quota, so please [reach out](https://moderationapi.com/sales) if you need more quota. ## Exceeding the included quota By default, the API will block requests when you've exceeded the included quota and repond with a `429 status code`. If you want to continue submitting content, you can **upgrade** your plan or enable **pay-as-you-go**. ### Upgrade to a higher plan If you upgrade to a higher plan, your quota will immediately be reset to the included quota for the new plan. Upgrade to a higher plan in the dashboard here. ### Pay-as-you-go If you enable pay-as-you-go, you will be billed for the quota you use on top of your included quota. This is useful if you want to avoid being blocked when you exceed the included quota. Set up pay-as-you-go limits in the dashboard here. #### When is pay-as-you-go billed? Pay-as-you-go is billed monthly at the same time as your subscription. ## Viewing quota usage You can view your quota usage in the dashboard. View your quota usage in the dashboard here. ## When does the quota reset? Your quota resets at the beginning of your billing cycle. For example, if you have a monthly plan, your quota will reset on the day that you started your plan. You can also see the start date of your billing cycle in the dashboard on the [billing page](https://dash.moderationapi.com/billing). ## How is the quota usage calculated? The quota usage is calculated based on the number of requests you make to the API, which models you use, and the plan your account is on. You can see the quota usage for a request in the response. * Pre-built text models use 1 quota unit per request no matter how many models you use. * Pre-built image models use 3 quota units per image. * Custom models use 1 quota unit per request per model. * Some exceptions might apply. #### Example: * Your project is configured with 5 pre-built text models and 1 custom model. * 5 pre-built text models = 1 quota unit per request * 1 custom model = 1 quota unit per request * Total quota usage = 2 quota units per request * Submit 10 requests to the API = 10 x 2 quota units = 20 quota units ## Please reach out if you need a custom quota solution If your use case does not fit into the default price model, please [reach out](https://moderationapi.com/sales) and we can discuss a custom plan or another solution. # Manage plan Source: https://docs.moderationapi.com/account/upgrading Get more quota and features by upgrading your plan If you need more quota or features, you can upgrade your plan at any time. Additional quota will be available immediately. ## How to change plan Visit the [billing page](https://dash.moderationapi.com/billing) and click on the "Manage billing" button. You will be redirected to the billing portal.
Here you can: * Update your plan * Cancel your subscription * Change your payment method * Download your invoices # Author actions Source: https://docs.moderationapi.com/actions/author-actions Block, suspend, and manage authors with built-in and custom moderation actions. Moderation actions provide direct moderation controls for managing problematic users in your community. Common scenarios for moderation actions: * **Block persistent violators** identified through [trust levels](/users/trust-levels) or [fraud detection](/users/fraud-detection) * **Remove spam accounts** posting promotional content repeatedly * **Suspend users** to send clear signals about acceptable behavior Actions can be executed manually from the [user dashboard](https://dash.moderationapi.com/project/latest/authors) or programmatically via the [Author API](/api-reference/author). They complement content moderation by addressing the user behind the content, helping you stop persistent abuse and enforce community standards. ## Built-in actions The system provides three essential moderation actions for comprehensive account management: Permanently prevents a user from submitting content for moderation. **When to use:** * Spam accounts with no legitimate content * Users repeatedly violating major community guidelines * Accounts identified as malicious through fraud detection * Coordinated attacks or bot networks **Technical behavior:** * User status set to `"blocked"` * All future content submissions are rejected * `block.until` field set to `null` (permanent) * `block.reason` stores the reason provided Temporarily prevents a user from submitting content for a specified period. **When to use:** * First-time serious violations that warrant a cooling-off period * Users who might reform with temporary consequences * Escalating enforcement before permanent blocks * Accounts needing investigation time **Technical behavior:** * User status set to `"suspended"` * Content submissions rejected until suspension ends * `block.until` field set to suspension end time (timestamp) * `block.reason` stores the reason provided * Automatic reinstatement when period expires Removes existing blocks or suspensions, restoring normal access. **When to use:** * Appeals that have been approved * Mistaken blocks that need correction * Policy changes requiring user reinstatement * Early release from suspensions for good behavior **Technical behavior:** * User status reset to `"enabled"` * Content submission access restored * Block fields cleared from user record ## Action customization You can customize how actions appear and behave: **Value customization:** * **Predefined values**: Add custom dropdown options like "Spam", "Harassment", "Fraud", "Policy Violation" * **Free text option**: Enable "Other" option that shows a text input field for custom reasons * **Combined approach**: Mix predefined categories with free text for flexibility ### Adding custom actions Beyond built-in actions, create custom author-level actions for specialized workflows that show up alongside standard actions. **Use cases for custom actions:** * **Warning systems**: Formal warnings that don't restrict access * **Verification requirements**: Require additional account verification * **Feature restrictions**: Limit specific platform features * **Escalation triggers**: Automatically escalate to senior moderators To create custom actions, choose "Author level action" when creating a new action in the dashboard. These appear alongside built-in actions on user detail pages. ## Executing actions 1. Navigate to the user detail page in the [user dashboard](https://dash.moderationapi.com/project/latest/authors) 2. Select the appropriate action (Block, Suspend, or Enable) 3. Choose from predefined reason options or select "Other" for free text 4. For suspensions: select duration from dropdown or set custom end time 5. Confirm the action **Reason field importance:** * **Required for documentation** and potential appeals * **Visible to moderators** in user timeline * **Returned in API responses** for application logic * **Helps identify patterns** in violation types **Timeline** User timeline showing content submissions and moderation actions All moderation actions are automatically tracked in the user's timeline in the dashboard: * **Action history**: Complete chronological record of all actions taken * **Moderator attribution**: Which team member executed each action * **Reason logging**: Full context for every moderation decision * **Audit trail**: Comprehensive record for compliance and appeals This timeline helps moderators understand user history and make informed decisions about future actions. ```javascript theme={"theme":"nord"} // Block a user permanently const blockResponse = await fetch( "https://api.moderationapi.com/v1/actions/execute", { method: "POST", headers: { Authorization: "Bearer YOUR_API_KEY", "Content-Type": "application/json", }, body: JSON.stringify({ actionKey: "AUTHOR_BLOCK", authorIds: ["{authorId}"], value: "Repeated spam violations", }), } ); // Suspend a user temporarily const suspendResponse = await fetch( "https://api.moderationapi.com/v1/actions/execute", { method: "POST", headers: { Authorization: "Bearer YOUR_API_KEY", "Content-Type": "application/json", }, body: JSON.stringify({ actionKey: "AUTHOR_BLOCK_TEMP", authorIds: ["{authorId}"], value: "Harassment of other users", duration: 7 * 24 * 60 * 60 * 1000, // 7 days in milliseconds }), } ); // Unblock a user (remove block/suspension) const unblockResponse = await fetch( "https://api.moderationapi.com/v1/actions/execute", { method: "POST", headers: { Authorization: "Bearer YOUR_API_KEY", "Content-Type": "application/json", }, body: JSON.stringify({ actionKey: "AUTHOR_UNBLOCK", authorIds: ["{authorId}"], value: "Appeal approved", }), } ); ``` See the [Author API documentation](/api-reference/author) for complete endpoint details. ## Technical implementation Moderation actions modify several fields returned in API responses, allowing seamless integration with your application logic. ### Status fields Monitor user status through API responses: ```javascript theme={"theme":"nord"} { "id": "user123", "status": "suspended", // "enabled", "blocked", "suspended" "block": { "until": 1705320000000, // timestamp in ms, null for permanent blocks "reason": "Harassment of other users" } } ``` ### Content submission integration The system automatically handles blocked/suspended users when content is submitted to [moderation endpoints](/api-reference/moderate/analyze-text), but you can also check status proactively in your application by querying the user in the Author API. ### Webhooks Receive real-time notifications when moderation actions are executed through [webhook integration](/actions/webhooks): **Common webhook use cases:** * **Sync with external systems** like customer support platforms * **Trigger email notifications** to affected users * **Update internal user databases** with moderation status * **Log actions for compliance** and audit requirements * **Alert security teams** about fraud-related blocks Configure webhooks to receive instant notifications whenever blocks, suspensions, or other moderation actions are executed. *** ## FAQ Moderation actions only prevent content submission for moderation. They don't affect content viewing unless you implement additional restrictions in your application using the status fields. Existing content remains in your system and review queues. moderation actions only prevent new submissions. You may want to review existing content from blocked users separately. The system provides the suspension information through API responses, but displaying this to users depends on your application implementation. Use the `user.block.until` and `user.block.reason` fields to show appropriate messages. Use the Enable action to reinstate users after successful appeals. The action history timeline maintains a record of the original block and subsequent reinstatement for transparency. You can use the Enable action to end suspensions early, or apply a new Suspend action to change the period. Each action creates a new timeline entry for full audit tracking. Yes, moderation actions complement these systems perfectly. High fraud risk scores or low trust levels often indicate when moderation actions are needed. The systems work together to provide comprehensive user management. # Content actions Source: https://docs.moderationapi.com/actions/content-actions Reject, allow, and custom actions that run on individual queue items. Review queue for reviewing and improving automated moderation Content actions run against a single queue item. They're how moderators (and your application code) decide what happens to flagged content. ## Built-in actions Two are always available: * **Reject** — the item is rejected. Fires the `queue_item.rejected` [webhook event](/actions/webhooks). * **Allow** — the item is approved. Fires the `queue_item.allowed` event. ## Custom actions Beyond reject and allow, create custom content actions for any workflow you need — "Hide post", "Send to legal", "Escalate to senior moderator", "Republish to homepage". Custom actions appear in the queue alongside the built-ins and fire the generic `queue_item.action` webhook event. We recommend using actions for things like: * Publish approved content * Remove rejected content * Route items into dedicated views (see [Routing with views](#routing-with-views)) * Anything your moderation flow needs Configure actions under [**Configure → Actions**](https://dash.moderationapi.com/project/latest/configure/actions). ## Resolving an item when an action runs An action can be configured to resolve the item automatically when it runs, removing it from the queue. This saves moderators a step — they don't have to click "resolve" after taking an action. ## Adding a value Actions can carry a value — a reason ("spam", "harassment"), a category, a duration, or any free-text input. Configure the available values per action: * **Predefined values** — show a dropdown of options * **Free text** — show a text input * **Both** — predefined options plus an "Other" free-text fallback The value is included in the [webhook payload](/actions/webhooks) so your application can branch on it. ## Routing with views You can use actions to route content into dedicated [views](/review/content-views). Create a view with an **Action taken** filter for a specific action, and any item that gets that action will appear in the view automatically — this is how escalation and user-report workflows are built. ## Running actions programmatically Trigger any action — built-in or custom — from your own systems via the [execute action endpoint](/api-reference/actions/execute-moderation-action). ## Integrating with your app Pair actions with [webhooks](/actions/webhooks) to react in real time, or use one of our [platform plugins](https://moderationapi.com/integrations) for direct integrations with Discord, Slack, WordPress, and more. # Actions Source: https://docs.moderationapi.com/actions/overview Actions are the verbs of moderation — what moderators (and automation) do to content and users. An **action** is anything that can run against a piece of content or a user: rejecting a comment, blocking an author, sending a warning, removing a profile from your app. Each action is configurable, can carry a value (a reason, a category, a duration), and can fire a [webhook](/actions/webhooks) so your application reacts in real time. Actions live in two flavors: Run on individual queue items. Built-in **reject** and **allow**, plus any custom actions you create — "Hide post", "Send to legal", "Flag for review", etc. Run on the user behind the content. Built-in **block**, **suspend**, and **enable**, plus custom author-level actions like warnings or feature restrictions. ## How actions are used * **In the dashboard.** Moderators run actions from the queue or user detail pages. Custom actions appear alongside the built-ins. * **Programmatically.** Call the [execute action endpoint](/api-reference/actions/execute-moderation-action) to run any action — built-in or custom — against content or authors from your own systems. * **As webhook triggers.** Every action that runs fires a [webhook event](/actions/webhooks) (`queue_item.rejected`, `author.blocked`, `queue_item.action`, etc.) so you can sync state back to your application. ## Built-in vs. custom Built-in actions cover the common cases — rejecting content, blocking authors, and so on. If they don't cover what you need, you can create custom actions to fit your workflow. | | Built-in | Custom | | ----------------- | ------------------------------------------------------- | ----------------------------------------------- | | **Created by** | The system | You | | **Examples** | Reject, allow, block, suspend, unblock | "Hide post", "Send warning email", "Escalate" | | **Webhook event** | Dedicated event per action (e.g. `queue_item.rejected`) | Generic `queue_item.action` or `author.action` | | **Configurable** | Possible values, queue visibility | Everything: name, level, values, queue behavior | Configure actions under [**Configure → Actions**](https://dash.moderationapi.com/project/latest/configure/actions) in the dashboard. ## What's next Reject, allow, and custom item-level actions. Block, suspend, enable, and custom user-level actions. Receive real-time events when actions run. # Webhooks Source: https://docs.moderationapi.com/actions/webhooks Receive HTTP callbacks when moderation events happen — queue items resolved, authors blocked, custom actions performed, and more. Webhooks let you react to moderation events in real time. Whenever an event you subscribe to fires, we'll `POST` a JSON payload to your URL. A single webhook can subscribe to **any combination of events** — you don't need a separate URL per event type. Configure webhooks under [**Configure → Webhooks**](https://dash.moderationapi.com/project/latest/configure/webhooks) in the dashboard. ## How it works 1. Add a webhook in the dashboard, set its URL, and tick the events you care about. 2. Whenever any of those events fire, we POST the event to your URL. 3. Your endpoint returns a `2xx` within 5 seconds to acknowledge. 4. Anything else (timeout, `4xx`, `5xx`) triggers retries with exponential backoff — up to **5 attempts** total. ## Request headers Every delivery includes: | Header | Description | | ------------------ | --------------------------------------------------------------------------------------------------- | | `webhook-version` | Payload envelope version. Currently always `v2`. | | `webhook-event-id` | Stable event ID (matches `id` in the body). Use this to dedupe retries. | | `modapi-signature` | HMAC-SHA256 signature of the raw request body. See [Verifying signatures](#verifying-signatures). | | `User-Agent` | `ModAPI/1.0`. Useful for allow-listing in security tools — see [Troubleshooting](#troubleshooting). | | `Content-Type` | `application/json`. | ## Payload envelope Every event shares the same outer envelope. The event-specific data lives in `data.object`. Stable event ID, prefixed with `evt_`. Identical across retries — use it to dedupe. The event type, e.g. `queue_item.rejected` or `author.blocked`. See the [event catalog](#event-catalog). Always `v2`. ISO 8601 timestamp of when the event was emitted. Wraps the event-specific payload. The resource that triggered the event. The shape depends on `type` — see each event below. ```json Envelope shape theme={"theme":"nord"} { "id": "evt_clxxx...", "type": "queue_item.rejected", "api_version": "v2", "created": "2026-05-08T12:34:56.789Z", "data": { "object": { /* event-specific resource */ } } } ``` Need typed payloads? The full OpenAPI schema for every event lives under `components.schemas` in our [OpenAPI spec](https://app.stainless.com/api/spec/documented/moderation-api/openapi.documented.yml) — look for `WebhookEvent` (a discriminated union over `type`) and the per-event schemas like `QueueItemRejectedEvent`, `AuthorBlockedEvent`, etc. ## Event catalog | Event | Fires when | `data.object` | | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------- | | [`queue_item.resolved`](/api-reference/webhooks/queue-item-resolved) | A moderator marks a queue item as resolved (checked off the queue). | The item, plus optional `author` and `queue` references. | | [`queue_item.action`](/api-reference/webhooks/queue-item-action) | A **custom** moderation action runs on a queue item. | The action that ran, with the related `item`, `author`, and `queue`. | | [`queue_item.rejected`](/api-reference/webhooks/queue-item-rejected) | The built-in **reject** action runs on a queue item. | Same shape as `queue_item.action`. | | [`queue_item.allowed`](/api-reference/webhooks/queue-item-allowed) | The built-in **allow** action runs on a queue item. | Same shape as `queue_item.action`. | | [`author.blocked`](/api-reference/webhooks/author-blocked) | An author transitions to the `blocked` status. | The action that drove the transition, with the affected `author`. | | [`author.unblocked`](/api-reference/webhooks/author-unblocked) | An author transitions back to `enabled`. May fire automatically when a temporary suspension expires. | Same shape as `author.blocked`. | | [`author.suspended`](/api-reference/webhooks/author-suspended) | An author is suspended for a finite period. The nested `author.block.until` indicates when the suspension lifts. | Same shape as `author.blocked`. | | [`author.updated`](/api-reference/webhooks/author-updated) | Public author fields (name, email, profile picture, metadata, etc.) change. Status transitions don't trigger this. | The updated `author`. | | [`author.trust_level_changed`](/api-reference/webhooks/author-trust-level-changed) | An author's resolved trust level transitions to a new value. Doesn't fire for no-op recomputes. | The updated `author`. | | [`author.action`](/api-reference/webhooks/author-action) | A custom action runs against an author. | Same shape as `author.blocked`. | **Reject and allow are first-class events.** If you want to capture every moderation action firing on a queue item, subscribe to `queue_item.action` **and** `queue_item.rejected` **and** `queue_item.allowed`. Built-in actions don't fire under `queue_item.action`. ## Routing by content type Queue-item events carry the item's `meta_type` — a high-level classifier you set when submitting content (or inherit from the channel). Use it to fan a single webhook out to the right handler per entity: | `meta_type` | Typical use | | ----------- | --------------- | | `profile` | User profiles | | `message` | DMs / chat | | `post` | Long-form posts | | `comment` | Replies | | `event` | Event listings | | `product` | Marketplace | | `review` | Ratings/reviews | | `other` | Anything else | ```ts Webhook router theme={"theme":"nord"} import type { WebhookEvent } from "@moderation-api/sdk"; const route = (event: WebhookEvent) => { switch (event.type) { case "queue_item.rejected": switch (event.data.object.item?.meta_type) { case "event": return rejectEvent(event); case "profile": return rejectProfile(event); case "review": return rejectReview(event); } break; case "queue_item.allowed": switch (event.data.object.item?.meta_type) { case "event": return allowEvent(event); case "profile": return allowProfile(event); case "review": return allowReview(event); } break; } return null; }; ``` Set `meta_type` per submission via the `type` field on the [moderation endpoint](/api-reference/moderate/submit-content), or configure it as the default for a channel. ## Verifying signatures Each delivery is signed with HMAC-SHA256 using your project's webhook secret. Find the secret under [API Keys in the dashboard](https://dash.moderationapi.com/project/latest/configure/api-keys). To verify, compute `HMAC_SHA256(rawRequestBody, webhookSecret)` as a hex digest and compare it to the `modapi-signature` header. Always use a constant-time comparison. ```js Node.js (SDK) theme={"theme":"nord"} import ModerationAPI from "@moderation-api/sdk"; // Reads MODAPI_SECRET_KEY by default const client = new ModerationAPI(); export async function POST(request) { const rawBody = await request.text(); const signatureHeader = request.headers.get("modapi-signature") ?? ""; // Verifies the signature with MODAPI_WEBHOOK_SECRET (or pass it explicitly // as a third arg). Throws if the signature is invalid. const event = client.webhooks.constructEvent( Buffer.from(rawBody), signatureHeader, ); // `event` is typed as the WebhookEvent discriminated union — switching on // `event.type` narrows `event.data.object` to the right resource. switch (event.type) { case "queue_item.rejected": // event.data.object is the action_performed record break; case "author.blocked": // event.data.object.author is the blocked author break; // ... } return Response.json({ received: true }); } ``` ```js Node.js (manual) theme={"theme":"nord"} import crypto from "crypto"; import { buffer } from "micro"; export const config = { api: { bodyParser: false } }; export default async function handler(req, res) { const rawBody = (await buffer(req)).toString("utf8"); const signatureHeader = req.headers["modapi-signature"]; if (!signatureHeader) { return res.status(400).json({ error: "Missing modapi-signature" }); } const expected = crypto .createHmac("sha256", process.env.MODAPI_WEBHOOK_SECRET) .update(rawBody) .digest("hex"); const sig = Buffer.from(signatureHeader, "utf8"); const dig = Buffer.from(expected, "utf8"); if (sig.length !== dig.length || !crypto.timingSafeEqual(sig, dig)) { return res.status(401).json({ error: "Invalid signature" }); } const event = JSON.parse(rawBody); switch (event.type) { case "queue_item.rejected": // handle reject break; case "author.blocked": // handle block break; // ... } return res.json({ received: true }); } ``` ```python Python (Flask) theme={"theme":"nord"} import hmac import hashlib import os from flask import Flask, request, abort, jsonify app = Flask(__name__) SECRET = os.environ["MODAPI_WEBHOOK_SECRET"].encode() @app.post("/webhooks/moderation") def moderation_webhook(): raw = request.get_data() signature = request.headers.get("modapi-signature", "") expected = hmac.new(SECRET, raw, hashlib.sha256).hexdigest() if not hmac.compare_digest(signature, expected): abort(401) event = request.get_json() if event["type"] == "queue_item.rejected": ... # handle reject elif event["type"] == "author.blocked": ... # handle block return jsonify(received=True) ``` ### Preventing replay attacks The envelope `created` timestamp lets you reject events older than your tolerance window (e.g. 5 minutes). Combine that with the stable `webhook-event-id` header to dedupe retries — store recently-seen IDs and ignore repeats. ## Retries and delivery * **Success:** any `2xx` response within 5 seconds closes the delivery. * **Failure:** any non-`2xx`, timeout, or network error triggers a retry. * **Limits:** up to **5 attempts** total, with exponential backoff between tries. * **Permanent failure:** after the final attempt fails, we'll email the project's admin. You can inspect every delivery attempt — request headers, payload, response status, and response body — under [**Events log**](https://dash.moderationapi.com/moderation/events) in the dashboard. ## Troubleshooting ### Check the events log If webhooks aren't behaving as expected, the [events log](https://dash.moderationapi.com/moderation/events) is the first place to look. It shows every delivery attempt with full request and response details. ### Allow Moderation API through your firewall Some hosting providers and security services block webhook traffic by default. Cloudflare's Bot Fight Mode, for example, will challenge the request instead of letting it through. ### Common blocking scenarios * Bot Fight Mode / Super Bot Fight Mode is enabled * WAF Managed Rules active * Custom security rules ### Choose your solution based on your plan **✅ Use Security Rules (Recommended)** 1. Go to your Cloudflare dashboard 2. Select your domain 3. Navigate to **Security → Security rules** 4. Create a custom rule: * **Name**: "Moderation API Webhooks" * **Field**: User Agent * **Operator**: starts with * **Value**: `ModAPI/` * **Action**: Skip `All Super Bot Fight Mode Rules` and other rules that might interfere. 5. Make sure to place this rule before other rules 6. Deploy the rule **❌ Security Rules won't work** Bot Fight Mode cannot be bypassed using Skip actions in WAF custom rules or Page Rules. Skip, Bypass, and Allow actions only apply to Ruleset Engine rules. **✅ Solutions:** * **Option 1**: Upgrade to a Pro plan to use Super Bot Fight Mode (which can be bypassed) * **Option 2**: Create a rule to allow Moderation API's IP address — see [Cloudflare's guide here](https://developers.cloudflare.com/waf/tools/ip-access-rules/create/), and reach out to support at [support@moderationapi.com](mailto:support@moderationapi.com) to get the IP address used for your account. **Common providers with security blocking:** * SiteDistrict * DigitalOcean App Platform * Vercel * Netlify * AWS CloudFront * Azure Front Door **General configuration steps:** 1. Access your provider's security/firewall settings 2. Look for "allow rules", "exceptions", or "bypass rules" 3. Add a User-Agent rule for `ModAPI/` (preferred) 4. Or contact support to get IP addresses allowlisted for your account 5. Save and deploy the configuration **Where to find settings:** * **Vercel**: Project settings → Security * **Netlify**: Site settings → Build & deploy → Post processing * **DigitalOcean**: App settings → Security section * **AWS CloudFront**: WAF & Shield → Web ACLs * **Azure Front Door**: Rules engine or WAF policies Each provider uses different terminology. Look for "firewall rules", "security exceptions", or "allow rules" in your provider's documentation. ### Still having issues? 1. Check your hosting provider's security logs for blocked `ModAPI/` requests. 2. Verify your endpoint returns a `2xx` status code within 5 seconds. 3. Test with a minimal endpoint (just respond `200 OK`) to isolate the issue. 4. Make sure firewall allow rules don't conflict with deny rules earlier in the chain. Need a hand? Email [support@moderationapi.com](mailto:support@moderationapi.com) and we'll help you configure or allowlist for your account. # Get account details Source: https://docs.moderationapi.com/api-reference/account/get-account-details get /account Get account details # Create an action Source: https://docs.moderationapi.com/api-reference/actions/create-an-action post /actions Create an action. # Delete an action Source: https://docs.moderationapi.com/api-reference/actions/delete-an-action delete /actions/{id} Delete an action and all of its webhooks. # Execute an action Source: https://docs.moderationapi.com/api-reference/actions/execute-an-action post /actions/{actionId}/execute Execute an action on a set of content items in a queue. # Execute moderation action Source: https://docs.moderationapi.com/api-reference/actions/execute-moderation-action post /actions/execute Execute a moderation action on one or more content items. # Get an action Source: https://docs.moderationapi.com/api-reference/actions/get-an-action get /actions/{id} Get an action by ID. # List moderation actions Source: https://docs.moderationapi.com/api-reference/actions/list-moderation-actions get /actions List all available moderation actions for the authenticated organization. # Update an action Source: https://docs.moderationapi.com/api-reference/actions/update-an-action put /actions/{id} Update an action. # Authentication Source: https://docs.moderationapi.com/api-reference/authentication How to authenticate with the API The Moderation API uses [API keys](/content-moderation/manage-api-key) to authenticate requests. Each project has its own API key. You can view and manage your API keys in the [dashboard](https://moderationapi.com/dashboard) -> Project -> Configure. Your API keys carry many privileges, so be sure to keep them secure. Don't share your secret API keys in publicly accessible areas such as GitHub, client-side code, and so forth. Authentication to the API is performed with the Authorization header when making a HTTP request. Provide your API key prefixed with `Bearer` as the value of the Authorization header. You must make all API calls over HTTPS. Calls that you make over plain HTTP will fail. API requests without authentication will also fail. We recommend creating two projects in your dashboard, one for development and one for production. This way you can create tokens for both environments. # Create a new author Source: https://docs.moderationapi.com/api-reference/author/create-a-new-author post /authors Create a new author. Typically not needed as authors are created automatically when content is moderated. # Delete an author Source: https://docs.moderationapi.com/api-reference/author/delete-an-author delete /authors/{id} Delete a specific author # Get author details Source: https://docs.moderationapi.com/api-reference/author/get-author-details get /authors/{id} Get detailed information about a specific author including historical data and analysis # List authors (Enterprise) Source: https://docs.moderationapi.com/api-reference/author/list-authors get /authors Get a paginated list of authors with their activity metrics and reputation # Update author details Source: https://docs.moderationapi.com/api-reference/author/update-author-details put /authors/{id} Update the details of a specific author # Create an action Source: https://docs.moderationapi.com/api-reference/create-an-action post /actions Create an action. # Delete an action Source: https://docs.moderationapi.com/api-reference/delete-an-action delete /actions/{id} Delete an action and all of its webhooks. # Errors Source: https://docs.moderationapi.com/api-reference/errors Learn about the error codes and messages that Moderation API returns. Moderation API uses conventional HTTP response codes to indicate the success or failure of an API request. In general: Codes in the 2xx range indicate success. Codes in the 4xx range indicate an error that failed given the information provided (e.g., a required parameter was omitted or is formatted incorrectly, etc.). Codes in the 5xx range indicate an error with Moderation API's servers. | Error Code | Meaning | | ---------- | ------------------------------------------------------------------------------------------------------------------------ | | 400 | Bad Request -- The request was unacceptable, often due to missing a required parameter. | | 401 | Unauthorized -- No valid API key provided. | | 403 | Forbidden -- The API key doesn't have permissions to perform the request, or your quota is used and you need to upgrade. | | 404 | Not Found -- The specified resource could not be found. | | 405 | Method Not Allowed -- You tried to access a resource with an invalid method. | | 429 | Too Many Requests -- See [rate limits](/api-reference/rate-limits) | | 500 | Internal Server Error -- We had a problem with our server. Try again later. | | 503 | Service Unavailable -- We're temporarily offline for maintenance. Please try again later. | ## Response signature Error responses are returned with a JSON object with the following signature: The error code. `BAD_REQUEST`, `UNAUTHORIZED`, `FORBIDDEN`, `NOT_FOUND`, `METHOD_NOT_ALLOWED`, `TOO_MANY_REQUESTS`, `INTERNAL_SERVER_ERROR`, `SERVICE_UNAVAILABLE` A human-readable message providing more details about the error. An array of issues that caused the error. For example input fields that are invalid. ## Handling errors Our API can return errors for many reasons, such as a authentication errors, and network unavailability. We recommend writing code that gracefully handles all possible API errors to avoid making your own service unavailable. # Execute moderation action Source: https://docs.moderationapi.com/api-reference/execute-an-action post /actions/execute Execute a moderation action on one or more content items. ```bash cURL theme={"theme":"nord"} curl -X POST 'https://api.moderationapi.com/v1/actions/execute' \ -H 'Authorization: Bearer ' \ -H 'Content-Type: application/json' \ -d '{ "contentIds": [ "60c9e1c0e4e7e1001c7a0e1e", "60c9e1c0e4e7e1001c7a0e1f" ], "actionId": "suspend-author", "value": "Spam" }' ``` ```javascript Node.js theme={"theme":"nord"} import ModerationAPI from "@moderation-api/sdk"; // Uses environment variable MODAPI_SECRET_KEY const moderationApi = new ModerationAPI(); // Or pass key explicitly // const moderationApi = new ModerationAPI({ secretKey: '' }); await moderationApi.actions.execute({ contentIds: ["60c9e1c0e4e7e1001c7a0e1e", "60c9e1c0e4e7e1001c7a0e1f"], actionId: "suspend-author", value: "Spam", }); ``` ```ruby Ruby theme={"theme":"nord"} require 'moderation_api' ModerationApi.configure do |config| config.access_token = '' end api = ModerationApi::QueueActionsApi.new api.actions_execute( content_ids: [ '60c9e1c0e4e7e1001c7a0e1e', '60c9e1c0e4e7e1001c7a0e1f' ], action_id: 'suspend-author', value: 'Spam' ) ``` ```python Python theme={"theme":"nord"} import requests url = "https://api.moderationapi.com/v1/actions/execute" headers = { "Authorization": "Bearer ", "Content-Type": "application/json" } payload = { "contentIds": [ "60c9e1c0e4e7e1001c7a0e1e", "60c9e1c0e4e7e1001c7a0e1f" ], "actionId": "suspend-author", "value": "Spam" } response = requests.post(url, headers=headers, json=payload) ``` ```php PHP theme={"theme":"nord"} ", "Content-Type: application/json" ); $data = array( "contentIds" => array( "60c9e1c0e4e7e1001c7a0e1e", "60c9e1c0e4e7e1001c7a0e1f" ), "actionId" => "suspend-author", "value" => "Spam" ); $ch = curl_init($url); curl_setopt($ch, CURLOPT_POST, true); curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data)); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); $response = curl_exec($ch); curl_close($ch); $result = json_decode($response, true); ?> ``` ```java Java theme={"theme":"nord"} import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.net.URI; String url = "https://api.moderationapi.com/v1/actions/execute"; String requestBody = """ { "contentIds": [ "60c9e1c0e4e7e1001c7a0e1e", "60c9e1c0e4e7e1001c7a0e1f" ], "actionId": "suspend-author", "value": "Spam" } """; HttpClient client = HttpClient.newHttpClient(); HttpRequest request = HttpRequest.newBuilder() .uri(URI.create(url)) .header("Authorization", "Bearer ") .header("Content-Type", "application/json") .POST(HttpRequest.BodyPublishers.ofString(requestBody)) .build(); HttpResponse response = client.send(request, HttpResponse.BodyHandlers.ofString()); System.out.println(response.body()); ``` ```go Go theme={"theme":"nord"} package main import ( "bytes" "encoding/json" "fmt" "net/http" ) func main() { url := "https://api.moderationapi.com/v1/actions/execute" data := map[string]interface{}{ "contentIds": []string{ "60c9e1c0e4e7e1001c7a0e1e", "60c9e1c0e4e7e1001c7a0e1f", }, "actionId": "suspend-author", "value": "Spam", } jsonData, _ := json.Marshal(data) req, _ := http.NewRequest("POST", url, bytes.NewBuffer(jsonData)) req.Header.Set("Authorization", "Bearer ") req.Header.Set("Content-Type", "application/json") client := &http.Client{} resp, err := client.Do(req) if err != nil { panic(err) } defer resp.Body.Close() var result map[string]interface{} json.NewDecoder(resp.Body).Decode(&result) fmt.Println(result) } ``` # Get an action Source: https://docs.moderationapi.com/api-reference/get-an-action get /actions/{id} Get an action by ID. # Introduction Source: https://docs.moderationapi.com/api-reference/introduction The Moderation API is organized around REST. Our API has predictable resource-oriented URLs, accepts JSON-encoded request bodies, returns JSON-encoded responses, and uses standard HTTP response codes, authentication, and verbs. ## Just getting started? Check out our quickstart guide to get up and running. Get set up with Moderation API in 5 minutes. ## Not a developer? If you're not a developer, you can still use the API. We have many integrations and no-code solutions that allows you to create projects and moderate content without writing a single line of code. See our integrations and no-code solutions. # List moderation actions Source: https://docs.moderationapi.com/api-reference/list-actions get /actions List all available moderation actions for the authenticated organization. # Submit content Source: https://docs.moderationapi.com/api-reference/moderate/submit-content POST /moderate Submit text, image, video, audio, or object content for moderation analysis. Submit content for moderation and receive analysis results immediately. Supports text, image, object, video, and audio content types. For detailed usage examples and content metadata options, see [Submitting content](/content-moderation/submit-content). # Real-time voice stream Source: https://docs.moderationapi.com/api-reference/moderate/voice-stream GET /stream Open a WebSocket to moderate live voice audio in real time and receive a verdict per spoken utterance. Moderate live voice and call audio over a WebSocket. You send a `start` frame, stream `media` frames as audio arrives, then `stop`; the server transcribes speech and returns a moderation verdict for each finalized utterance. For the full walkthrough and code examples, see [Real-time voice](/content-moderation/real-time-voice). # Rate limits Source: https://docs.moderationapi.com/api-reference/rate-limits Learn about API rate limits and how to work with them. Different plans have different rate limit tiers based on your subscription plan. You can view your organization's current rate limit tier in your [dashboard](https://dash.moderationapi.com/usage). To request an increased rate limit, please [contact our sales team](https://moderationapi.com/sales). ## Rate limit headers The API returns rate limit information in the response headers for every request: | Header | Description | | ----------------------- | ------------------------------------------------------------------------------------- | | `X-RateLimit-Limit` | The maximum number of requests you can make in the current time window | | `X-RateLimit-Remaining` | The number of requests remaining in the current time window | | `X-RateLimit-Reset` | Unix timestamp (in seconds) when your rate limit will reset | | `Retry-After` | Number of seconds to wait before retrying (only included when rate limit is exceeded) | ### Example response headers ``` X-RateLimit-Limit: 60 X-RateLimit-Remaining: 45 X-RateLimit-Reset: 1699564800 ``` ## Handling rate limits A basic technique for integrations to gracefully handle limiting is to watch for `429` status codes and build in a retry mechanism. The retry mechanism should follow an exponential backoff schedule to reduce request volume when necessary. When you exceed your rate limit, the API will return a `429` error with a message indicating how long to wait before retrying. Use the `Retry-After` header to determine when to make your next request. ### Example error response When rate limited, you'll receive a response like: ```json theme={"theme":"nord"} { "error": "Rate limit exceeded. Try again in 10 seconds." } ``` The response will include the `Retry-After` header indicating how many seconds to wait. ## Checking your rate limit You can check your current rate limit in your [dashboard under usage](https://dash.moderationapi.com/usage). # Get a queue Source: https://docs.moderationapi.com/api-reference/review-queues/get-a-queue get /queue/{id} Get a queue # Get queue items Source: https://docs.moderationapi.com/api-reference/review-queues/get-queue-items get /queue/{id}/items Get paginated list of items in a moderation queue with filtering options # Get queue statistics Source: https://docs.moderationapi.com/api-reference/review-queues/get-queue-statistics get /queue/{id}/stats Get detailed statistics about a moderation queue including review times, action counts, and trends # Resolve a queue item Source: https://docs.moderationapi.com/api-reference/review-queues/resolve-a-queue-item post /queue/{id}/items/{itemId}/resolve Mark a queue item as resolved with a specific moderation action # Unresolve a queue item Source: https://docs.moderationapi.com/api-reference/review-queues/unresolve-a-queue-item post /queue/{id}/items/{itemId}/unresolve Mark a previously resolved queue item as unresolved/pending # Update an action Source: https://docs.moderationapi.com/api-reference/update-an-action put /actions/{id} Update an action. # Author action Source: https://docs.moderationapi.com/api-reference/webhooks/author-action openapi.json webhook author.action Fired when a custom action is performed on an author. # Author blocked Source: https://docs.moderationapi.com/api-reference/webhooks/author-blocked openapi.json webhook author.blocked Fired when an author transitions to the blocked status. # Author suspended Source: https://docs.moderationapi.com/api-reference/webhooks/author-suspended openapi.json webhook author.suspended Fired when an author is temporarily suspended. # Author trust level changed Source: https://docs.moderationapi.com/api-reference/webhooks/author-trust-level-changed openapi.json webhook author.trust_level_changed Fired when an author's resolved trust level transitions to a new value. # Author unblocked Source: https://docs.moderationapi.com/api-reference/webhooks/author-unblocked openapi.json webhook author.unblocked Fired when an author transitions back to the enabled status. # Author updated Source: https://docs.moderationapi.com/api-reference/webhooks/author-updated openapi.json webhook author.updated Fired when an author's profile or metadata changes. # Queue item action Source: https://docs.moderationapi.com/api-reference/webhooks/queue-item-action openapi.json webhook queue_item.action Fired when a custom action is performed on a queue item. # Queue item allowed Source: https://docs.moderationapi.com/api-reference/webhooks/queue-item-allowed openapi.json webhook queue_item.allowed Fired when a queue item is allowed. # Queue item rejected Source: https://docs.moderationapi.com/api-reference/webhooks/queue-item-rejected openapi.json webhook queue_item.rejected Fired when a queue item is rejected. # Queue item resolved Source: https://docs.moderationapi.com/api-reference/webhooks/queue-item-resolved openapi.json webhook queue_item.resolved Fired when a queue item is resolved. # Add words to wordlist Source: https://docs.moderationapi.com/api-reference/wordlist/add-words-to-wordlist post /wordlist/{id}/words Add words to an existing wordlist # Get embedding status Source: https://docs.moderationapi.com/api-reference/wordlist/get-embedding-status get /wordlist/{id}/embedding-status Get the current embedding progress status for a wordlist # Get wordlist Source: https://docs.moderationapi.com/api-reference/wordlist/get-wordlist get /wordlist/{id} Get a specific wordlist by ID # List wordlists Source: https://docs.moderationapi.com/api-reference/wordlist/list-wordlists get /wordlist List all wordlists for the authenticated organization # Remove words from wordlist Source: https://docs.moderationapi.com/api-reference/wordlist/remove-words-from-wordlist delete /wordlist/{id}/words Remove words from an existing wordlist # Update wordlist Source: https://docs.moderationapi.com/api-reference/wordlist/update-wordlist put /wordlist/{id} Update a wordlist # Managing your casebook Source: https://docs.moderationapi.com/casebook/managing Import decisions you already have, resolve disagreements, forget cases, and audit what the casebook settles ## The ledger The **Cases** table on the Casebook page lists everything the casebook has learned. Search it, or filter by **Ruling** (Allowed / Rejected), **Label**, or **Topic** — including *No topic*. Each row shows the content, the ruling, the labels, who decided it, and **Times applied** — how many items that one decision has settled. Open a case to see the full content and jump to the item it was decided on. ## Importing cases The import dialog offering two sources: past review decisions, or a CSV upload A casebook doesn't have to start from zero. **Import** on the Casebook page offers two sources: * **Use past review decisions.** Replays the last 90 days of your review-queue decisions — each human allow or reject becomes a case. Decisions the casebook already holds are skipped. * **Upload a CSV.** Bring content and rulings from another tool. You pick the content column, and map your own ruling values to *allow* or *reject* — nothing is guessed for you. If the file has no ruling column, you can apply one ruling to every row. Imports run in the background — close the wizard and keep working; the toolbar shows a progress light while a run is live. Rows that can't be imported are counted and reported without stopping the rest, and imported cases spend a short time learning before they start matching content. ## Disagreements Two cases that rule opposite ways on similar content block the casebook from ruling on content like them — it applies neither until someone picks which ruling stands. The **Disagreements** card counts them, and **Resolve** steps you through each group: The disagreements stepper: conflicting cases on the left, and the decision panel with Should be allowed, Should be rejected, and Keep both rulings * **Should be allowed** / **Should be rejected** — one choice settles the whole group. The losing side stops applying, and the casebook rules on content like this again. * **Keep both rulings** — every case stands. Repeats of each message follow their own ruling, and new similar content still goes to review. * **Keep as is** — hold an individual case out of the group decision. ## Forgetting a decision The delete dialog showing how many items the case has decided before you confirm Deleting a case makes the casebook forget it: similar content goes back to being judged by your policies and rules alone. Before you confirm, the dialog shows how much weight the case carries — how many items it has decided. Items it already decided keep their outcome; deleting only changes the future. ## Auditing what it settles A queue filter bar combining Casebook allowed with Casebook overruled The queue's [filters](/review/content-views) ask two casebook questions that combine: * **Casebook** — what it said: *Casebook allowed*, *Casebook rejected*, or *No casebook match*. * **Casebook vs severity** — where it stood against severity triage: *Casebook overruled* when they disagreed, *Casebook agreed* when they said the same. Set both to narrow. *Casebook allowed* plus *Casebook overruled* is every item severity triage would have held back that the casebook let through — the false positives your moderators would otherwise re-approve by hand. **Handled for you** on the Casebook page opens the same slice, and it's the one worth a periodic look once the casebook is settling work for you: it's your team's own past judgement being applied, so check that you still agree with it. # Casebook Source: https://docs.moderationapi.com/casebook/overview The casebook remembers your team's review decisions and settles similar content automatically The Casebook page with stats, topics grouped by theme, and the ledger of learned cases Review teams decide the same things over and over. A rejected scam template, an approved profile update, a corrected resubmission — each comes back with small edits and gets reviewed again from scratch. The casebook is your project's memory of review decisions. Every human allow or reject in [review](/review/overview) is remembered as a **case**, and when new content arrives that reads like something your team already decided, the casebook applies the same ruling — so nobody reviews it again. ## How cases are created Every human **Allow** or **Reject** in the review inbox becomes a case automatically. An action that neither allows nor rejects is not added, and only humans set precedent: automated decisions and author actions like block or warn don't become cases. When a [lookup](/casebook/rulings#test-your-casebook) finds nothing, **Add this as a case** lets you teach a decision directly — paste the content, choose Allow or Reject. Similar content follows from then on. Seed a new casebook from the last 90 days of your review history, or upload a CSV of content and rulings from another tool. See [importing cases](/casebook/managing#importing-cases). ## What a case holds A case dialog showing the content, ruling, labels, topic, channel, reviewer, and times applied A case is a snapshot of one decision: the content, the **ruling** — always allow or reject — the **labels** your policies flagged at the time, the channel, who decided it and when, and **times applied** — how many later items this case has settled. From the case you can open the original item it was decided on. A new case spends a short moment being committed to memory before it can match content — the ledger marks these and tells you how many are left. ## How to enable The casebook is on by default: new projects remember review decisions, and new channels start with the casebook rule enabled. Two settings control it: * **Configure → Casebook** turns the casebook on or off for the whole project. Off means nothing is remembered and no lookups run. * The **Casebook lookup** rule in each channel's [rules](/content-moderation/rules) decides whether the casebook may settle content on that channel. Turn it off where you want the casebook to watch and learn but not decide. Applying rulings is available on the Growth plan and above. Remembering works on every plan, so your casebook keeps learning in the meantime and the rule works the moment you upgrade. ## Reading the stats The four cards at the top of the Casebook page: * **Awaiting review** — what's sitting in your review queue right now. * **Handled for you** — the share of would-be reviews the casebook settled instead, with the item count. Press it to open those items in the queue. While no channel enforces, it reads **Casebook could have handled** — the reviews you'd have been spared. * **Casebook topics** — how many [topics](/casebook/topics) your cases are grouped into. * **Disagreements** — cases that rule opposite ways on similar content. Until someone [resolves them](/casebook/managing#disagreements), the casebook applies neither ruling to content like theirs. ## Where to go next * [How the casebook rules](/casebook/rulings) — when it decides, when it deliberately declines, and how to test it. * [Topics](/casebook/topics) — how the casebook organizes what it remembers, and rules that act on a whole topic. * [Managing your casebook](/casebook/managing) — imports, disagreements, and making the casebook forget a decision. # How the casebook rules Source: https://docs.moderationapi.com/casebook/rulings How remembered decisions settle content, when the casebook deliberately declines, and how to test it A casebook match always rules **allow** or **reject** — never review. Whatever it can't settle goes to your [review inbox](/review/overview) as usual, so the casebook only ever removes work, never hides it. ## Two ways to a ruling * **Recognition.** The exact message has been ruled on before, so the same ruling applies without a review. * **Agreement among close cases.** The content is very similar to several existing cases, and enough of them agree — "96% of its close cases agree, and that is enough to rule." A lookup result: the casebook would reject the message, with the closest cases and their similarity Every ruling shows its evidence: the closest cases, each with a similarity percentage, its ruling, who decided it, and its topic. ### Cases vote only on what they were flagged for A case's ruling is scoped by the labels your policies flagged on it. A rejection flagged for spam only counts as precedent for content flagged for spam — in a lookup's evidence, a case that didn't participate reads *sat out — flags differ*, and one that's too far off reads *not close enough to count*. This is what makes corrected resubmissions safe: content close to rejected cases but missing the flag that got them rejected goes to a person instead of inheriting the rejection. ### Cautious in one direction Overriding your policies takes more evidence than agreeing with them. The casebook is quick to learn rules that are *stricter* than your policies — things your team rejects that they let through — and deliberately slow to allow content they flagged hard. ## When it declines to rule Declining is a feature, not a failure. The casebook sends content to review when: | Situation | What the casebook says | | --------------------------- | -------------------------------------------------------------- | | Nothing similar exists | "Your casebook has not seen anything like this." | | Matches aren't close enough | "No cases close enough to apply." | | Evidence is thin | "Close cases agree, but there are not enough of them yet." | | Cases conflict | "Your cases disagree about content like this." | | A new flag appeared | "Close to allowed cases, but flagged for something new." | | The old reason is gone | "Close to rejected cases, but their reason no longer applies." | A lookup that declines to rule: close cases agree but there are not enough of them yet, 58% of the way to ruling on its own When evidence is thin, the result shows how far along the casebook is — "58% of the way to ruling on its own" — so you know a few more decisions will settle content like this. If the casebook can't answer within its time budget, moderation simply proceeds without it. It can only ever add a ruling, never delay one. ## Test your casebook **Look up** on the Casebook page opens a test dialog: paste any message, pick a channel, and see how your casebook would rule — the verdict, what your policies flagged, and the closest cases. Testing never teaches it anything. When the lookup finds nothing, **Add this as a case** teaches the decision on the spot. ## Precedent while you review The casebook bookmark pulled out beside an item, showing the ruling and the closest cases In the item detail view, a **Casebook** tab sits at the right edge. Pull it out and it shows what your casebook makes of the item on screen — the same answer the lookup gives, with each case one click from the item it was decided on. Leave it out and it answers for every item you open. It reads and never writes: your queue decisions are still what the casebook learns from. ## In the API response When the casebook has a verdict for a request, the moderation response carries a `casebook` object — the verdict, how close and how unanimous the supporting cases were, and the topic they belong to. See [checking the casebook](/content-moderation/acting-on-responses#check-the-casebook) for the field reference and code examples. Whether the verdict was *enforced* depends on the channel's Casebook lookup rule — when it was, `recommendation.reason_codes` contains `casebook_match`, and the item shows **Casebook lookup** as its outcome in the dashboard. # Casebook topics Source: https://docs.moderationapi.com/casebook/topics Your precedent, organized into named themes you can browse, pin, and write rules against Topics are the table of contents of your casebook. As cases accumulate, they're grouped into plainly named themes — "Telegram spam", "Insults between players", "Heated product feedback" — so you can see what kinds of content your team is ruling on, and how. The topics browser: a topic tree on the left, and the selected topic's cases, keywords, and ruling tendency on the right Each topic shows its case count, its ruling tendency — *Always rejected*, *Always allowed*, or a split like *42% rejected* — its keywords, and recent cases. **View all cases** filters the ledger below to the topic. A split tendency isn't a defect — it's often the insight. A team that rejects cannabis sales while allowing other sales will see exactly that split, and can dive into the topic to check the boundary is where they want it. The last row is always **No topic yet**: cases still being committed to memory, waiting for the next organizing pass, or one-offs too unlike anything else to file. ## Young casebooks reorganize, grown ones settle Topics appear on their own after about ten cases, and the page tells you which of two phases you're in: * **While the casebook is small**, every organizing pass re-reads the whole of it and re-authors the topics. Yesterday's names aren't binding — a topic that was really just the first week's traffic gets renamed or dropped as the picture fills in. * **As it grows**, topics stop moving and passes only add new ones, so the vocabulary your team has been reading for weeks stays put. Organizing runs in the background — the footer shows when the last pass ran, and **Reorganize now** asks for a fresh one. ## Pinning Pinning a topic is how you say "keep this". A pinned topic keeps its name and survives every organizing pass; new cases can still be filed into it. You can also **Rename topic** — renaming or pinning tells automation to leave the topic alone. Writing a rule about a topic pins it automatically, so nothing your rules depend on can be reshaped underneath you. ## Rules that act on a topic Any [content rule](/content-moderation/rules) can be scoped to a casebook topic — *if the content matches this topic, then take this action*. That lets you act on a whole kind of content before enough precedent has accumulated for the casebook to settle it case by case: send a borderline topic to review, or reject a scam topic outright. ## The weekly digest When your casebook identifies a new topic, you get a weekly summary email. Nothing is sent in a week with no new topics, and you can turn it off in your notification settings under **New casebook topics**. # Understand and act on API responses Source: https://docs.moderationapi.com/content-moderation/acting-on-responses Learn what the API response contains and how to act on it. The moderation API returns a structured response that helps you decide how to handle content. Here's an example response: ```json Example response theme={"theme":"nord"} { "content": { "id": "message-123", "masked": true, "modified": "This is a test, my email is {{ email hidden }}" }, "author": { "id": "auth_abc123", "external_id": "user-123", "status": "enabled", "block": null, "trust_level": { "level": 2, "manual": false } }, "evaluation": { "flagged": true, "flag_probability": 0.87, "severity_score": 0.3, "unicode_spoofed": false }, "recommendation": { "action": "review", "reason_codes": ["severity_review"] }, "casebook": { "verdict": "reject", "similarity": 0.94, "agreement": 0.92, "confidence": 0.78, "case_count": 6, "topic": { "id": "clx8k2p00000", "label": "Competitor spam" } }, "policies": [ { "id": "personal_information", "type": "entity_matcher", "probability": 0.95, "flagged": true, "flagged_fields": [], "matches": [ { "probability": 0.95, "match": "test@example.com", "span": [28, 44] } ] }, { "id": "toxicity", "type": "classifier", "probability": 0.12, "flagged": false, "labels": [ { "id": "toxic", "probability": 0.12, "flagged": false } ] } ], "insights": [ { "id": "sentiment", "type": "insight", "probability": 0.78, "value": "neutral" }, { "id": "language", "type": "insight", "probability": 0.99, "value": "en" } ], "meta": { "status": "success", "timestamp": 1735902168566, "channel_key": "default", "usage": 1, "processing_time": "245ms" } } ``` How you act on the response is up to you: * Block the content and return an error message to your user if it gets flagged * Store it to your database and let a human review it using [review queues](/review/overview) * Do something in between and only review content when the AI is not confident in its decision *** ## Use the recommendation The easiest way to handle moderation responses is to use the `recommendation` object. It provides a clear action based on your channel configuration, severity scores, and author status. ```javascript theme={"theme":"nord"} switch (response.recommendation.action) { case "reject": // Block the content, show an error to the user throw new Error("Content not allowed"); case "review": // Save the content but flag for manual review await saveContent(content, { needsReview: true }); break; case "allow": // Content is approved, proceed normally await saveContent(content); break; } ``` The `reason_codes` array tells you why a particular recommendation was made: | Reason code | Description | | ----------------- | ------------------------------------------------------------------------------------------------------------ | | `rule_match` | A [content rule](/content-moderation/rules) matched — the action came from that rule | | `rule_fallback` | No rule matched, the severity score triage fallback assigned the action | | `rule_default` | No rule matched and triage is disabled — fell back to the channel default | | `severity_reject` | Content severity score exceeded the rejection threshold | | `severity_review` | Content severity score exceeded the review threshold | | `author_block` | The author is blocked or suspended | | `casebook_match` | Your [casebook](#check-the-casebook) recognised the content and applied your earlier ruling | | `trusted_allow` | A rule allowed the content because the author's trust level is high | | `client_override` | A [`clientAction`](/content-moderation/submit-content#clientaction) you submitted changed the recommendation | | `dry_run` | Dry run mode is enabled (always returns `allow`) | *** ## Check if flagged For simple use cases, you can check the `evaluation.flagged` field. This boolean indicates if any of your enabled policies detected something that triggered a flag. ```javascript theme={"theme":"nord"} if (response.evaluation.flagged) { // Content was flagged by at least one policy } ``` ### Severity score The `severity_score` gives you more granular control. Higher scores indicate more severe violations: ```javascript theme={"theme":"nord"} if (response.evaluation.flagged && response.evaluation.severity_score > 0.7) { // High severity - reject immediately } else if (response.evaluation.flagged) { // Lower severity - send to review queue } ``` We recommend using the `recommendation.action` field instead of implementing your own threshold logic. Configure thresholds in your channel settings for easier management. *** ## Check the casebook Your casebook is the record of how your team has ruled on content in the past. When you submit something close to a case it already holds, the response carries a `casebook` object describing what it found. ```javascript theme={"theme":"nord"} if (response.casebook) { console.log( `Casebook says ${response.casebook.verdict}, based on ${response.casebook.case_count} past cases` ); } ``` | Field | Description | | ------------ | ------------------------------------------------------------------------------------------- | | `verdict` | The ruling your past decisions point to: `allow` or `reject` | | `similarity` | How close the nearest backing case is, 0-1. `1` means identical content | | `agreement` | How unanimous the backing cases are, 0-1 — ignores how many there are | | `confidence` | How strongly the casebook holds the ruling, 0-1 — agreement scaled by evidence | | `case_count` | How many of your past cases backed this ruling | | `topic` | The topic the closest backing case is filed under, or `null` if it has not been grouped yet | ### Agreement vs. confidence Both are 0-1 strength numbers, and they answer different questions: **`agreement`** is unanimity, ignoring volume. Two cases that both say reject score the same as fifty that do. It is always at least `0.8` when a verdict is returned — below that the casebook reports a disagreement rather than picking a side — so treat it as a description of how clean the consensus was, not as a threshold to re-apply yourself. **`confidence`** is that agreement scaled by how much evidence backs it, so a handful of close, recent cases outweighs a single distant one. Older cases count for less, halving in weight roughly every 180 days. This is the number to use when you want a strength condition, and it is the one available in [rules](/content-moderation/rules) as `casebook.confidence`. A high `agreement` with a low `confidence` means your cases were unanimous but there were few of them, or they were old. That is the casebook telling you it has an opinion it does not hold strongly. ### When `casebook` is null The field is `null` whenever the casebook has nothing to say, which covers several situations you do not need to tell apart: * Nothing in your casebook was close enough to the content. * The matching cases contradicted each other, so the casebook declined to pick a side. * Casebook lookups are not switched on for the channel handling the request. * Your plan does not include casebook lookups. Always guard on it rather than reading through: ```javascript theme={"theme":"nord"} // casebook may be null if (response.casebook?.verdict === "reject") { // ... } ``` `casebook` reports what your casebook *found*. Whether it decided the outcome is a separate question — check `recommendation`, where a higher-priority rule may have settled the item first. When the casebook did decide, `recommendation.reason_codes` contains `casebook_match`. *** ## Work with individual policies The `policies` array contains results from each policy enabled in your channel, sorted by highest probability. Each policy includes: | Field | Description | | ---------------- | ---------------------------------------------------------------- | | `id` | The policy identifier (e.g., `toxicity`, `personal_information`) | | `type` | Either `classifier` or `entity_matcher` | | `probability` | Model confidence level (0-1) | | `flagged` | Whether this policy triggered based on your thresholds | | `flagged_fields` | For object submissions, which fields triggered the flag | ### Classifier policies Classifier policies (like `toxicity`, `spam`, `hate`) analyze content and return a probability score: ```javascript theme={"theme":"nord"} const toxicityPolicy = response.policies.find(p => p.id === "toxicity"); if (toxicityPolicy?.flagged) { console.log(`Toxicity detected with ${toxicityPolicy.probability * 100}% confidence`); } // Check specific labels within a classifier if (toxicityPolicy?.labels) { const severeLabel = toxicityPolicy.labels.find(l => l.id === "severe"); if (severeLabel?.flagged) { // Handle severe toxicity differently } } ``` ### Entity matcher policies Entity matcher policies (like `personal_information`, `url`) detect and extract specific entities: ```javascript theme={"theme":"nord"} const piiPolicy = response.policies.find(p => p.id === "personal_information"); if (piiPolicy?.matches?.length > 0) { console.log("Found PII:"); piiPolicy.matches.forEach(match => { console.log(` - "${match.match}" at position ${match.span[0]}-${match.span[1]}`); }); } ``` *** ## Handle masked content If you have PII masking enabled, the API can automatically redact sensitive information. Check the `content` object: ```javascript theme={"theme":"nord"} if (response.content.masked) { // Use the modified content with masked values const safeContent = response.content.modified; await saveToDatabase(safeContent); } else { // No masking applied, use original content await saveToDatabase(originalContent); } ``` This is useful for: * Anonymizing content before storing in your database * Preventing users from seeing personal information * Compliance with data protection regulations *** ## Check author status If you're using [author management](/users/overview), the response includes author information: ```javascript theme={"theme":"nord"} const { author } = response; if (author?.status === "blocked") { // Author is permanently blocked throw new Error("Your account has been blocked"); } if (author?.status === "suspended") { const until = new Date(author.block.until); throw new Error(`Your account is suspended until ${until.toLocaleDateString()}`); } // Optionally adjust behavior based on trust level if (author?.trust_level.level >= 3) { // Trusted author - maybe skip certain checks } ``` *** ## Use insights The `insights` array provides additional analysis that doesn't affect flagging: ```javascript theme={"theme":"nord"} const sentimentInsight = response.insights.find(i => i.id === "sentiment"); const languageInsight = response.insights.find(i => i.id === "language"); console.log(`Language: ${languageInsight?.value}`); // e.g., "en" console.log(`Sentiment: ${sentimentInsight?.value}`); // "positive", "neutral", or "negative" // Route negative sentiment to priority review if (sentimentInsight?.value === "negative" && response.evaluation.flagged) { await addToPriorityQueue(content); } ``` *** ## Detect unicode spoofing Spammers sometimes use look-alike characters to bypass moderation (e.g., `mоney` with a Cyrillic "о" instead of Latin "o"). The API detects and normalizes these characters. Check the `unicode_spoofed` field: ```javascript theme={"theme":"nord"} if (response.evaluation.unicode_spoofed) { console.log("Content contains look-alike characters"); // The policies ran on normalized text for accurate detection console.log(response.content.modified); // the cleaned-up message } ``` The field is omitted when neither normalization nor the Unicode spoofing policy is enabled on the channel, since nothing analyzed the text. To flag and route spoofed content rather than just observe it, enable the [Unicode spoofing policy](/policies/unicode-spoofing). *** ## Handle errors Check `meta.status` for the overall request status: ```javascript theme={"theme":"nord"} if (response.meta.status === "partial_success") { // Some policies failed - check the errors array response.errors?.forEach(error => { console.warn(`Policy ${error.id} failed: ${error.message}`); }); } ``` The `errors` array contains details about any policies that encountered issues during processing. # Analyze audio Source: https://docs.moderationapi.com/content-moderation/analyze-audio How audio moderation works, supported formats, limits, and transcription quality settings. ## How it works Audio files are automatically transcribed to text using speech recognition, then the transcript is analyzed by all enabled text-based policies. This means any policy that works on text (toxicity, hate, PII, wordlists, guidelines, etc.) also works on audio with zero additional configuration. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "audio", url: "https://example.com/audio.mp3", }, }); ``` ## Supported audio formats Any format FFmpeg can decode is supported. All audio is internally converted to 16 kHz mono WAV before transcription. | Format | Extensions | | ------ | ---------------------- | | MP3 | `.mp3` | | WAV | `.wav` | | AAC | `.aac`, `.m4a` | | OGG | `.ogg`, `.oga` | | Opus | `.opus` | | FLAC | `.flac` | | WebM | `.webm` | | AMR | `.amr` | | WMA | `.wma` | | MP4 | `.mp4`, `.m4a`, `.mov` | ## Limits | Constraint | Value | | -------------------- | ------------------------- | | Max file size | 50 MB | | Max audio duration | 10 minutes | | Processing timeout | 30 seconds | | URL schemes | `http`, `https` only | | Private/internal IPs | Blocked (SSRF protection) | ## Transcription quality You can configure transcription quality per channel in the dashboard under **Content > Audio > Transcription quality**. | Setting | Label | Use case | Relative speed | | --------------- | -------- | ------------------------------------ | -------------- | | SPEED (default) | Fast | Real-time moderation, high volume | Fastest | | BALANCED | Balanced | General purpose, good accuracy | \~2x slower | | ACCURACY | Accurate | Noisy audio, critical content review | \~3x slower | ## Usage and billing Each audio moderation request costs **2 units**: 1 for transcription + 1 for policy analysis. # Analyze images Source: https://docs.moderationapi.com/content-moderation/analyze-images Image models are enabled on an account basis. If you don't have access to image models, please contact our support team. # Analyze videos Source: https://docs.moderationapi.com/content-moderation/analyze-videos How video moderation works, supported formats, limits, and frame sampling settings. ## How it works Frames are sampled from the video at a configurable interval, then each frame is analyzed by all enabled image-based policies (NSFW, toxicity, etc.). If any sampled frame is flagged, the video is flagged. This means any policy that works on images also works on video with zero additional configuration. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "video", url: "https://example.com/video.mp4", }, }); ``` ## Supported video formats | Format | Extensions | | --------- | ---------- | | MP4 | `.mp4` | | QuickTime | `.mov` | | M4V | `.m4v` | | MPEG | `.mpeg` | ## Limits | Constraint | Value | | ------------------ | ---------- | | Max file size | 100 MB | | Max duration | 10 minutes | | Processing timeout | 10 minutes | Videos longer than 10 minutes are rejected with an error rather than partially analyzed. Split longer videos into segments of 10 minutes or less before submitting them. ## Frame sampling You can configure how frames are sampled per channel in the dashboard under **Content > Video > Video sampling**. | Setting | Default | Description | | -------------------- | ------------------ | ---------------------------------------------------------------------------- | | Capture interval | 1 second (min 0.5) | Time between sampled frames. | | Maximum frames | 10 | Caps the number of frames analyzed per video. Use this to control max spend. | | Spread frames evenly | On | Distributes frames across the video duration instead of from the start. | ## Usage and billing Each analyzed frame costs **3 units**. Total usage for a video is the number of sampled frames × 3 — for example, a video sampled at 8 frames costs 24 units. Lower the **Maximum frames** setting to cap usage per video. # Optimizing accuracy Source: https://docs.moderationapi.com/content-moderation/optimizing-accuracy The Moderation API is designed to work for a broad range of use cases, but you might need to make adjustments for optimal accuracy in your context. ## False positives and negatives When you’re relying on a content moderation system, a key concept to understand is the trade-off between false positives and false negatives: * **False positives** occur when a system flags content as violating policy when it is actually compliant. * **False negatives** occur when a system fails to flag content that does violate policy. It’s crucial to decide which type of error is more costly to your specific use case. Some use cases demand minimal tolerance for any potentially harmful content (prioritizing avoiding false negatives), while others might tolerate a small amount of non-harmful content being flagged incorrectly (i.e., false positives). Balancing these is an ongoing process. Consider the following strategies: * Adjust thresholds (covered below) to tightly align with your standards and risk appetite. * Supplement automated checks with human moderation for borderline cases to reduce both types of errors. ## Adjusting flagging thresholds The Moderation API allows configuration for how strictly it flags content. By experimenting with threshold scores, you can tweak the sensitivity: 1. **Lowering the threshold:** This will reduce false negatives (because more content is flagged), but potentially increase false positives. 2. **Raising the threshold:** This will reduce false positives (because the system flags fewer pieces of content), but potentially increase false negatives. It’s helpful to analyze your actual data for patterns. If you notice consistent misclassifications in content that truly violates your policies, lower the threshold. Conversely, if you see a flood of harmless content getting flagged, consider raising it. In production, you might set different thresholds per category (e.g., hateful vs. spam content) based on your tolerance. Read more about [how to adjust thresholds](/content-moderation/thresholds). ## Adding or removing models Your workflow or platform could be improved by adding specialized models or removing unnecessary ones: * **Adding specialized models**: If your use case deals with a specific domain (e.g., medical or legal), you might consider training or deploying a model tuned for that domain. This model could work in tandem with the general Moderation API to reduce domain-specific false flags. * **Removing unneeded models**: If your current pipeline includes multiple checks from overlapping or redundant models, rationalizing them may reduce complexity and potential conflicts in results. It can also streamline moderation decisions. ## Writing custom guidelines When pre-built policies don't quite fit your platform, [Guidelines](/policies/guidelines) let you describe what isn't allowed in plain English. Each guideline is evaluated by an LLM with your project context in scope, so the rule is interpreted against what your platform is and who uses it. Guidelines work well for: * Platform-specific rules that aren't covered by the standard categories. * Cases where intent matters more than the literal words used. * Iterating quickly — you can change a guideline by editing one sentence rather than retraining a model. ## Letting reviews set precedent Human review is most valuable when a decision counts more than once. The [casebook](/casebook/overview) remembers every allow and reject your team makes in review and applies the same ruling to similar content, so a correction made once doesn't need making again. Both error types benefit: false positives your team keeps re-approving and false negatives they keep rejecting become precedent instead of repeat work. ## Enabling context awareness Many moderation challenges arise when the system doesn’t understand the broader context behind the text: * Certain terms might be acceptable in an educational or reclaiming context (e.g., quoting a slur to explain its original meaning). * Cultural or community-specific language usage might not translate well with a general-purpose model. Some ways to incorporate context: * Provide additional metadata or preceding conversation snippets along with your text, to give the Moderation API or other classifiers a better understanding of what’s being said. * Enable Context Awareness in your project settings and include `authorId` and `conversationId` in your requests so the system can reference previous messages. See [Submitting content to Moderation API](/content-moderation/submit-content) for more details. ## Training custom models If you find that standard models are not meeting your performance requirements, consider building a custom fine-tuned model. This can help if: * You have a substantial dataset specific to your industry or type of content. * You need higher precision for borderline or ambiguous cases. * You want to reduce reliance on manual moderation for specialized content. With careful curation of training data and setting well-defined moderation policies, a custom model can further reduce both false positives and false negatives, fine-tuning decisions for your exact use case. ## Get help from our team If you need help deciding how to set up moderation, fine-tuning or advanced configurations, reach out to our support teams. We can help you: * Determine the right thresholds for your workflow. * Explore sample code to integrate moderation into your application. * Identify potential pitfalls with domain-specific moderation. Working together, we can ensure that your content moderation solution is well-aligned with your application’s requirements—minimizing harm, maximizing safety, and providing users with the best possible experience. # Automated content moderation Source: https://docs.moderationapi.com/content-moderation/overview Learn to set up projects and submit content for moderation ## Analyzing content After you've created a project, you can submit content for moderation. How to analyze content. ## Optimization As you analyze content, you can use the results to improve the accuracy of the moderation and stay up to date with the latest content. How to improve the accuracy of the moderation. # Real-time voice Source: https://docs.moderationapi.com/content-moderation/real-time-voice Moderate live voice and call audio in real time, with per-utterance verdicts grouped into conversations. Real-time voice moderation is available to customers on custom plans. If you're interested in using it, please [reach out here](https://moderationapi.com/sales). ## How it works Voice moderation analyzes live voice and call audio as it happens. You open a streaming connection and send call audio; the speech is transcribed and each finalized utterance is moderated by your enabled text policies—toxicity, hate, PII, wordlists, guidelines, and the rest—with no extra configuration. You receive a moderation result for every utterance as it's spoken. Unlike [audio file moderation](/content-moderation/analyze-audio), which analyzes a complete recording after the fact, voice moderation works on a live stream and returns a verdict for each utterance during the call. ## Conversations A voice call is a **conversation**: a single live session with a start and an end, where every utterance belongs to the same thread. This lets you review an entire call as one unit instead of a series of disconnected messages. * **Bring your own id.** Supply a `conversationId` to link the call to a record in your own system. If you don't, one is generated for you and returned when the session starts—every utterance in the call shares it. * **Filter by type.** Voice utterances are tagged with the `voice` content type, so you can separate them from messages, posts, and other content. Real-time voice is in early access and the streaming interface may still change. Coordinate with us before building a production integration so we can confirm the current contract and your account's limits. ## Connecting Open a WebSocket connection to the streaming endpoint, authenticating with your API key on the upgrade request and requesting the `moderationapi.v1` subprotocol. ``` wss://voice.moderationapi.com/v1/stream Authorization: Bearer Sec-WebSocket-Protocol: moderationapi.v1 ``` A missing or malformed key closes the connection with code `4401`. ### Start the session Send a `start` frame as the first message. It declares the conversation, the audio format, and the tracks you'll stream (for example a caller and an agent), each with an optional author id. ```json theme={"theme":"nord"} { "event": "start", "conversationId": "your-call-id", "channel": "your-channel-key", "mediaFormat": { "encoding": "audio/x-mulaw", "sampleRate": 8000 }, "tracks": [ { "name": "inbound", "authorId": "caller-123" }, { "name": "outbound", "authorId": "agent-456" } ], "emitPartials": false, "metadata": { "crmTicket": "T-9912", "region": "eu" } } ``` * `conversationId` — optional. Omit it to have one generated and returned in `session.started`. * `channel` — optional. Selects which channel's policy configuration applies. * `tracks` — stream **one or both** tracks. Send both `inbound` and `outbound` to moderate the full call with each side attributed to its own author, or just one track (for example only `inbound`) if that's all you have access to. Audio for any track you don't declare is ignored. * `mediaFormat.encoding` — `audio/x-mulaw` (`PCMU`), `audio/x-alaw` (`PCMA`), linear PCM (`audio/l16`, `linear16`), or common encoded containers (`wav`, `mp3`, `ogg`, `flac`). `sampleRate` may be 8000–48000 Hz. Audio is passed through without resampling. The spoken language is detected automatically. * `emitPartials` — optional. Set `true` to also receive interim, non-final transcripts. * `metadata` — optional, arbitrary JSON attached to the conversation. Put anything you want to associate with the call here (your own ids, tags, context); it's stored on the conversation and not interpreted by moderation. The server replies with `session.started`: ```json theme={"theme":"nord"} { "v": 1, "event": "session.started", "conversationId": "your-call-id", "sessionId": "…", "tracks": ["inbound", "outbound"] } ``` ### Stream audio Send `media` frames as audio arrives, one per track, with the audio chunk base64-encoded in `payload`: ```json theme={"theme":"nord"} { "event": "media", "media": { "track": "inbound", "payload": "" } } ``` ### End the session Send a `stop` frame to end the call gracefully (or simply disconnect). The server drains any in-flight utterances, emits `session.ended`, and closes. ```json theme={"theme":"nord"} { "event": "stop" } ``` ### Using Twilio or another telephony provider Telephony providers like Twilio stream call audio but can't consume the moderation verdicts the gateway streams back, so they don't connect to the gateway directly. Instead, run a thin bridge in your own backend: 1. Accept the provider's media stream (for Twilio, its `connected` / `start` / `media` messages). 2. Open this WebSocket and map those onto the `start` and `media` frames above—pass your call id as `conversationId` and the caller/agent identifiers as each track's `authorId`. 3. Relay the `utterance.final` verdicts back to your application to act on them. ## Events you receive Every outbound message carries `"v": 1` and an `event` field. | Event | When | | ------------------- | ------------------------------------------------------------ | | `session.started` | After your `start` frame is accepted. | | `utterance.partial` | Interim transcript (only if `emitPartials` was `true`). | | `utterance.final` | A finalized utterance, with its moderation result. | | `warning` | Non-fatal condition (e.g. a transient transcription hiccup). | | `session.error` | Fatal error; the connection closes. | | `session.ended` | The call ended; includes summary stats. | The key event is `utterance.final`—the transcribed text plus the standard moderation result (`evaluation`, `recommendation`, and `policies`), in the same shape as every other moderation response: ```json theme={"theme":"nord"} { "v": 1, "event": "utterance.final", "conversationId": "your-call-id", "contentId": "…", "track": "inbound", "authorId": "caller-123", "text": "transcribed speech for this utterance", "startMs": 0, "endMs": 2000, "sttConfidence": 0.95, "evaluation": { "flagged": false }, "recommendation": { "action": "allow" }, "policies": [] } ``` Use the `recommendation.action` (`allow`, `review`, or `reject`) to decide what to do—see [Acting on responses](/content-moderation/acting-on-responses). When the call ends you receive `session.ended` with a summary: ```json theme={"theme":"nord"} { "v": 1, "event": "session.ended", "conversationId": "your-call-id", "sessionId": "…", "stats": { "durationMs": 125000, "utterances": 42, "actions": { "allow": 39, "review": 2, "reject": 1 } } } ``` ### Close codes | Code | Meaning | | ------ | -------------------------------------- | | `1000` | Normal close | | `1011` | Server error | | `4400` | Bad request (e.g. a malformed `start`) | | `4401` | Authentication failed | | `4403` | Not authorized for voice | | `4429` | Concurrency limit reached—retry later | ## Example A minimal Node.js client that opens a session, streams audio from your telephony source, and acts on each verdict: ```javascript theme={"theme":"nord"} import WebSocket from "ws"; const ws = new WebSocket("wss://voice.moderationapi.com/v1/stream", "moderationapi.v1", { headers: { Authorization: `Bearer ${process.env.MODERATION_API_KEY}` }, }); ws.on("open", () => { ws.send( JSON.stringify({ event: "start", conversationId: "call-abc-123", channel: "support-calls", mediaFormat: { encoding: "audio/x-mulaw", sampleRate: 8000 }, tracks: [{ name: "inbound", authorId: "caller-123" }], emitPartials: false, metadata: { crmTicket: "T-9912" }, }), ); }); // Forward audio as it arrives from your telephony source (base64-encoded chunks). function sendAudio(base64Chunk) { ws.send(JSON.stringify({ event: "media", media: { track: "inbound", payload: base64Chunk } })); } ws.on("message", (raw) => { const msg = JSON.parse(raw.toString()); switch (msg.event) { case "session.started": console.log("session started:", msg.conversationId); break; case "utterance.final": console.log(`[${msg.track}] ${msg.text} -> ${msg.recommendation.action}`); if (msg.recommendation.action === "reject") { // act on it — flag the call, alert an agent, etc. } break; case "session.ended": console.log("session ended:", msg.stats); break; } }); // When the call ends, close the session gracefully. function endCall() { ws.send(JSON.stringify({ event: "stop" })); } ``` ## Limits | Constraint | Value | | ----------------- | ---------------------------------------------------------------------------- | | Language | Detected automatically | | Mode | Observe and report—verdicts are returned, the live call is not interrupted | | Max call duration | 1 hour | | Concurrent calls | Per-account limit; [contact us](https://moderationapi.com/sales) to raise it | # Content rules Source: https://docs.moderationapi.com/content-moderation/rules Set up rules that decide whether content is allowed, sent to review, or rejected based on policy flags, severity, author trust, and language. Each content rule reads as *if these conditions are true, then take this action*. Rules run top-to-bottom for every request, and the first match determines the `recommendation.action` returned by the API. They give you one place to express the moderation logic for a channel. ## Where to find it Open your project, pick a channel, and go to **Rules**. You'll see your rules listed in evaluation order, with the **Severity score triage** fallback at the bottom. Content rules page showing four rules in evaluation order with the severity score triage fallback at the bottom ### Rules you start with Every new channel is created with two author rules already enabled: | Order | Rule | Condition | Action | | ----- | --------------------- | --------------------------------- | ------ | | 1 | Allow trusted authors | Trust Level is at least `Trusted` | Allow | | 2 | Block banned authors | Status is not `Enabled` | Reject | Both are ordinary rules — edit, reorder, disable, or delete them like any other. Everything below them falls through to severity score triage until you add more. Note the order: because the first match wins, **Allow trusted authors** takes precedence over **Block banned authors**. If you [manually promote authors](/users/trust-levels#allowlist-a-specific-author) to `Trusted`, blocking one of them won't reject their content until you either move the block rule to the top or clear the trust level override. Rules decide the **recommendation**, not whether content is analyzed. Every request runs through the channel's policies first, so a matching Allow rule still returns the full policy scores and `flagged` still reflects what the analysis found. What changes is `recommendation.action`. ## Common use cases ### Auto-approve established users Return `allow` for authors who've earned a track record on your platform, without waiting on the severity thresholds. * **If** Trust Level is at least `Regular` * **Then** Allow `Regular` is reached automatically — 90+ days on your platform, 50+ clean items, and a violation rate at or below your threshold — so this covers the authors who proved themselves through their own behavior. Use `Member` to extend it to newer accounts, and see [trust levels](/users/trust-levels) for what each tier requires. The rule shipped with each channel is stricter, matching only `Trusted`. Lowering it to `Regular` is the usual first adjustment once you trust the automatic tiers. `Trusted` is manual-only, so leave the threshold there if you want the rule to apply to nobody but the authors you [promote by hand](/users/trust-levels#allowlist-a-specific-author). ### Block banned users immediately Reject anything from authors you've already disabled. * **If** Status is not `Enabled` * **Then** Reject Move this rule **above** your trust level Allow rule if you use manual trust level overrides. In the default order the Allow rule comes first, and since the first match wins, a blocked author who was manually promoted to `Trusted` still gets `allow`. Automatic trust levels don't have this problem — an author whose content gets flagged drops out of the upper tiers on their own. ### Always reject the worst categories If a category is zero-tolerance for your platform, send it straight to reject regardless of the severity score. * **If** Illicit is `Flagged` * **Then** Reject ### Catch risky links from new accounts Spam and phishing often arrive as a fresh account dropping a link. Combining trust level with URL Risk lets you hold suspect new accounts without slowing down established users. * **If** Trust Level is at most `New` * **And** URL Risk is `Flagged` * **Then** Review ### Tighten moderation for new accounts Trigger review when a new account hits any flag, even if the severity score wouldn't normally cross your threshold. * **If** Trust Level is at most `New` * **And** Any Policy Flagged is `true` * **Then** Review ### Route by language Send anything you can't review in-house to a separate action. * **If** Language is not `English` * **Then** Review *** ## How rules work ### Conditions Each rule has one or more conditions. Conditions inside a rule are joined with **AND**: every condition must be true for the rule to match. A condition has three parts: * **Field**: the signal you're matching on (Trust Level, Toxicity Score, Sentiment, ...) * **Operator**: how to compare it (`is`, `is not`, `at least`, `is one of`, ...) * **Value**: what you're comparing it to Add more conditions with **+ Add condition** to make the rule narrower. Expanded rule card with two conditions joined by AND and a Review action ### Available signals | Group | Field | Examples | | -------- | ------------------ | ------------------------------------------------ | | Author | Trust Level | Untrusted, New, Basic, Member, Regular, Trusted | | Author | Status | Enabled, Blocked, Temporarily Blocked | | Severity | Severity Score | A number between `0` and `1` | | Insights | Sentiment | Positive, Neutral, Negative | | Insights | Language | English, Spanish, French, ... | | Policies | Any Policy Flagged | `true` / `false` | | Policies | `` Flagged | `true` / `false`, per enabled policy | | Policies | `` Score | A number between `0` and `1`, per enabled policy | Policy fields appear automatically based on which policies are enabled in the channel. If you disable a policy that a rule still references, the rule is highlighted so you can update or remove it. ### Order matters Rules run top-to-bottom and the **first** rule that matches wins. Drag rules by the handle on the left to reorder. A typical ordering: 1. **Reject** rules for blocked users, so nothing below can approve them 2. **Allow** rules for trusted users, to short-circuit the rest of the list 3. **Reject** rules for zero-tolerance categories 4. **Review** rules for borderline cases 5. Severity score triage fallback at the bottom Put the narrowest, most certain rules at the top. Anything that should hold regardless of the other signals — a blocked author, a zero-tolerance category — belongs above the broad Allow rules. ### Actions Each rule resolves to one of three actions, returned as `recommendation.action`: | Action | Use when | | -------- | -------------------------------------------------- | | `allow` | Content is fine, publish it | | `review` | Hold or send to a [review queue](/review/overview) | | `reject` | Block it | ### Severity score fallback If no rule matches, the channel falls back to **Severity score triage**. This is a built-in step at the bottom of the list that assigns Allow, Review, or Reject based on the severity score thresholds set just below the rules. Click the row to see the current zones. You can disable triage if you'd rather have non-matching content default to Allow without any severity check. See [Thresholds](/content-moderation/thresholds#severity-score-triage) for how to tune the cutoffs and use the calibration helper. ### Casebook lookup Channels also carry a **Casebook lookup** row in the cascade — it applies the ruling your team gave to similar past content, and abstains when precedent is weak or conflicting. It's draggable and has its own switch like any rule; by default it runs just above the severity fallback. See [Casebook](/casebook/overview). *** ## Simulate before saving Click **Simulate** to run your draft rules against recent moderation history for this channel. The result shows how the action mix shifts: * How many recent items would now be Allowed, Reviewed, or Rejected * The delta versus what actually happened * The sample size and time range used Use this to catch over-aggressive rules before they hit production. Simulation reads your unsaved changes, so you can iterate on the rule set safely. Simulation results showing Allowed, Reviewed, and Rejected counts with deltas after running draft rules against recent moderation history *** ## What you see in the API response When a rule matches, the moderation response reflects it: ```json theme={"theme":"nord"} { "recommendation": { "action": "review", "reason_codes": ["rule_match"] } } ``` Reason codes you may see when rules are configured: | Reason code | Meaning | | ----------------- | --------------------------------------------------------------------------------------------------------------------- | | `rule_match` | A configured rule matched, and the action came from that rule | | `rule_fallback` | The severity score triage fallback matched | | `rule_default` | No rule matched and triage is disabled, so the channel default was used | | `client_override` | A [`clientAction`](/content-moderation/submit-content#clientaction) on the request changed the action after rules ran | Read more about acting on the response in [Understanding API responses](/content-moderation/acting-on-responses). If you submit a [`clientAction`](/content-moderation/submit-content#clientaction) with a request — your own recommendation from a blocklist or external tool — it's applied **after** your rules and the severity fallback. Depending on its `behavior`, it can escalate or override the action they produced. *** ## Tips * **Start broad and tighten over time.** The two default author rules plus the severity fallback get most channels 80% of the way there. Add narrower rules as you spot patterns in the queue. * **Don't be afraid to lean on severity score triage.** It does a good job out of the box: blocking the obvious stuff, sending borderline items to review, and letting clean content through. Rules are best for the cases triage can't express on its own (author trust, specific categories, language routing). * **Give rules a real name.** They show up in dashboards and audit logs, so "Reject high toxicity" is more useful than "Rule 4". The pencil icon on an expanded rule lets you rename it. * **Run Simulate after every change.** A condition that looks safe on paper can quietly shift hundreds of items between Review and Reject, and Simulate is the fastest way to spot it. * **Toggle off instead of deleting.** When you're not sure if a rule is still pulling its weight, disable it. You keep the configuration around and can flip it back on if the data says otherwise. # Submitting content to Moderation API Source: https://docs.moderationapi.com/content-moderation/submit-content How to submit content for moderation and which content types you can analyze with Moderation API. To analyze content, send a POST request to the `/moderate` endpoint. The API accepts different content types (text, image, object, video, audio) and returns results immediately. *** ```typescript Node.js example theme={"theme":"nord"} import ModerationAPI from "@moderation-api/sdk"; // Configure with environment variable MODAPI_SECRET_KEY const moderationApi = new ModerationAPI(); // Text moderation const textResult = await moderationApi.content.submit({ content: { type: "text", text: "Hello world!", }, // Optional content data contentId: "text-1", authorId: "user-123", conversationId: "room-456", metadata: { customField: "value", }, }); // Use the API's recommendation if (textResult.recommendation.action === "reject") { // Block the content } else if (textResult.recommendation.action === "review") { // Send to moderation queue } else { // Content approved - add to database } // Image moderation const imageResult = await moderationApi.content.submit({ content: { type: "image", url: "https://example.com/image.jpg", }, // Optional content data contentId: "image-1", authorId: "user-123", metadata: { customField: "value", }, }); // Simple flagged check if (imageResult.evaluation.flagged) { // Return error to user etc. } else { // Add to database etc. } ``` ```javascript Fetch example theme={"theme":"nord"} const response = await fetch("https://api.moderationapi.com/v1/moderate", { method: "POST", headers: { Authorization: `Bearer ${API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ content: { type: "text", text: "Hello world!", }, // Optional content data contentId: "text-1", authorId: "user-123", conversationId: "room-456", metadata: { customField: "value", }, }), }); const result = await response.json(); if (result.evaluation.flagged) { // Return error to user etc. } else { // Add to database etc. } ``` *** ## Content metadata You can add metadata to the content you send for moderation. Some fields are used by the moderation pipeline to improve accuracy, while others enhance the dashboard experience. ### contentId Specify a `contentId` to associate the request with specific content. This is typically the content's unique identifier from your database. If you don't specify a `contentId`, the API generates a random ID for the content. When you include a `contentId`, submitting the same ID again updates the existing content. This is useful when using [review queues](/review/overview) - you can update content in the queue without creating duplicate items. The `contentId` can also be used to execute actions in the review queue programmatically. For example, you can allow users to report content on your platform and then add it to a review queue. ### conversationId Use `conversationId` to group related content together, such as messages in a chatroom or comments on a post. If you're using [review queues](/review/overview), the `conversationId` can filter the queue to show content from a specific conversation. Enable [context awareness](#context-awareness) to improve moderation accuracy using the `conversationId`. ### authorId Use `authorId` to identify the user who created the content. This enables user-level moderation in [review queues](/review/overview) and allows filtering by specific users. Enable [context awareness](#context-awareness) to improve moderation accuracy using the `authorId`. ### metaType Use `metaType` to specify what kind of content you're moderating. This helps the API apply appropriate analysis: | Value | Use case | | --------- | ------------------------------- | | `message` | Chat messages, direct messages | | `post` | Forum posts, social media posts | | `comment` | Comments on posts or articles | | `review` | Product or service reviews | | `profile` | User profile information | | `product` | Product listings | | `event` | Event descriptions | | `other` | Any other content type | ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "text", text: "Great product, highly recommend!", }, metaType: "review", authorId: "user-123", }); ``` ### channel Use `channel` to route content to a specific channel configuration. If not provided, the project's default channel is used. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "text", text: "Hello world!", }, channel: "high-risk-content", }); ``` ### metadata Use `metadata` to attach any additional information to the request. This object can contain custom key-value pairs. Metadata is displayed in [review queues](/review/overview) and included in webhooks. If you add a link in metadata, it will be clickable from the review queue. This is useful for linking back to the original content in your application. *** ## Submit your own moderation recommendation If you already run your own flagging — a blocklist, a third-party tool, or your own logic — you can submit that recommendation alongside the content and have it combined with the API's own analysis. Without this, your own flags live outside Moderation API, so content you've flagged but our analysis considered clean never shows up where your moderators work. ### clientAction Use `clientAction` to submit your recommendation. It's not a log of an action you took — it's an input the API weighs alongside its own analysis to produce the final recommendation. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "text", text: "Check out my site: example.com", }, contentId: "post-789", authorId: "user-123", clientAction: { action: "review", behavior: "escalate", source: "ip-blocklist", reason: "Author IP is on our internal blocklist", }, }); // recommendation.action is now at least "review", and // recommendation.reason_codes includes "client_override" ``` The `clientAction` object accepts: | Field | Required | Description | | ---------- | -------- | ------------------------------------------------------------------------------------- | | `action` | Yes | Your recommendation: `allow`, `review`, or `reject` | | `behavior` | No | How your recommendation combines with ours (see below). Defaults to `escalate` | | `source` | No | Where your recommendation came from, e.g. `"ip-blocklist"`. Shown in the review queue | | `reason` | No | A human-readable explanation, shown in the review queue | **Behavior** controls how your recommendation combines with the API's own recommendation: | Behavior | What happens | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `escalate` (default) | Your recommendation is applied only when it's *stricter* than ours — `reject` is stricter than `review`, which is stricter than `allow`. This lets you push content into review or reject, but never loosens the API's own decision. | | `override` | Your recommendation replaces the API's recommendation entirely, even if that means loosening it. | When your recommendation changes the outcome, the response includes the `client_override` [reason code](/content-moderation/acting-on-responses#use-the-recommendation). `clientAction` sets `recommendation.action` and adds the `client_override` reason code, but it does **not** change `evaluation.flagged` — that always reflects the API's own analysis. Content you escalate to `review` this way still enters the [inbox](/review/overview) — a review recommendation is exactly what the inbox collects. If you work from a [content view](/review/content-views) filtered by flag status, make sure its filter includes these items, since `evaluation.flagged` can still be `false`. *** ## Context awareness Enable `Context awareness` in your channel settings, then include `authorId` and/or `conversationId` in API requests. This allows models to analyze previous messages for improved accuracy. When context awareness is enabled, models analyze the current message alongside recent conversation history. The API retrieves previous messages with the same `conversationId` or `authorId` and provides them to the model sequentially, allowing it to understand the full context before making a decision. LLM-based policies can use the `conversationId` to see previous messages in the same conversation, and `authorId` to see previous messages from the same author. This can prevent unwanted content spread across multiple messages: ``` msg 1 -> f msg 2 -> u msg 3 -> c msg 4 -> k [FLAGGED with context awareness] ``` It also helps understand messages in the context of a conversation: ``` user 1 -> What's the worst thing you know? user 2 -> European people [FLAGGED with context awareness] ``` *** ## Content types The `/moderate` endpoint accepts different content types through the `content` object: Analyze text content Analyze image content Analyze mixed content types Analyze video content Analyze audio content ### Text Text moderation is the most common type. Use it for: * Chat messages * Forum posts * Comments * Reviews * Product descriptions * Profile bios ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "text", text: "Hello world!", }, }); ``` If you're analyzing chat messages or thread-based content, enable [context awareness](#context-awareness) for better accuracy. ### Image Image moderation analyzes visual content to detect inappropriate or harmful images, including nudity, violence, or other objectionable content. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "image", url: "https://example.com/image.jpg", }, }); ``` ### Object Object moderation analyzes multiple fields at once, useful for moderating entire entities like user profiles or product listings. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "object", data: { title: { type: "text", text: "Product name" }, description: { type: "text", text: "Product description" }, image: { type: "image", url: "https://example.com/product.jpg" }, }, }, metaType: "product", }); ``` The response includes `flagged_fields` in each policy result, showing which specific fields triggered the flag. ### Audio Audio files are automatically transcribed to text, then analyzed by all enabled text-based policies. Useful for podcasts, voice messages, or other audio content. See [Analyze audio](/content-moderation/analyze-audio) for supported formats, limits, and transcription quality settings. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "audio", url: "https://example.com/audio.mp3", }, }); ``` ### Video Frames are sampled from the video and analyzed by all enabled image-based policies, detecting inappropriate scenes such as nudity or other objectionable content. See [Analyze videos](/content-moderation/analyze-videos) for supported formats, limits, and frame sampling settings. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "video", url: "https://example.com/video.mp4", }, }); ``` *** ## Opt out of content store Set `doNotStore` to `true` to prevent the content from being stored. The content will still be analyzed but won't appear in the dashboard or review queues. ```javascript theme={"theme":"nord"} const result = await moderationApi.content.submit({ content: { type: "text", text: "Hello world!", }, doNotStore: true, }); ``` Setting `doNotStore` to `true` will make parts of the moderation dashboard less useful, as content won't be available for review or analysis. Do not disable content storage if you want to train or optimize models based on your data. # Thresholds Source: https://docs.moderationapi.com/content-moderation/thresholds Tune what counts as flagged for each policy and what action the channel takes by default. These are your two main dials for the false positive / false negative trade-off. A channel has two layers of thresholds: 1. **Per-policy thresholds** decide *what counts as flagged* for each policy. 2. **Severity score triage** decides *what action to take* when content is flagged and no [content rule](/content-moderation/rules) matches. Both live on the channel and can be tuned independently. Most teams start with the defaults, watch a few hundred items go through, then nudge the dials based on what they see in the review queue. *** ## Severity score triage When a moderation request doesn't match any content rule, the channel falls back to **severity score triage**. Triage takes the request's `severity_score` (a 0–1 number summarizing how problematic the content is) and assigns Allow, Review, or Reject based on two thresholds. ### Where to find it Open your project, go to **Rules**, and scroll to **Severity score triage**. If the project has more than one channel, pick the channel from the switcher at the top of the page. ### The two thresholds | Threshold | Default | What it does | | ---------------- | ------- | ------------------------------- | | Review threshold | `50%` | Below this, content is allowed | | Block threshold | `90%` | Above this, content is rejected | Anything between the two is sent to **review**. The card shows a histogram of recent severity scores in this channel, color-coded by which zone each item falls into at the current settings. You can see how a slider change would shift your review queue load and reject rate before you save. Severity score triage card with histogram and dual-handle slider ### Presets Use the preset dropdown for a starting point: | Preset | Review / Block | When to use | | ----------------- | -------------- | ------------------------------------------------------------ | | Strict | `40% / 70%` | Conservative platforms that want to minimize false negatives | | Balanced | `50% / 90%` | Default for most general-purpose moderation | | Forgiving | `70% / 95%` | Communities tolerant of edge cases | | Skip reviewing | `75% / 75%` | Auto allow or reject, no review queue | | Always review | `50% / 100%` | Send borderline content to review, never auto-reject | | Review everything | `0% / 100%` | Manual moderation, every flagged item goes to a human | | Allow everything | `100% / 100%` | Observe-only mode, log scores but never block | The [casebook](/casebook/overview) runs alongside triage and can overrule it where your team's past decisions disagree with it. The queue's **Casebook overruled** filter shows where — a steady stream of overrulings on one side is a sign a threshold sits in the wrong place. ### Disable triage Toggle the switch on the **Severity score triage** row in the rules list to turn the fallback off completely. With triage disabled, anything that doesn't match a rule is allowed. This is useful when: * You've moved all your decision logic into [content rules](/content-moderation/rules) and want a single source of truth. * You're observing the system in shadow mode and want every request to pass through unless a rule says otherwise. ### Calibration helper The **Calibrate** button on the severity triage card pulls items your team has already resolved in the [review queue](/review/overview) and recommends thresholds that maximize agreement between your pipeline and your moderators' decisions. The flow is: 1. Pull a sample of resolved items (allowed and rejected) from the queue. 2. Compare the severity scores of allowed vs. rejected items to find the natural review zone. 3. Preview the suggested thresholds against your data before applying. You'll get the most useful recommendations after your team has resolved a few hundred items in the queue. *** ## Per-policy thresholds Each classifier policy returns a `probability` between `0` and `1` for every request. The **detection threshold** is the cutoff at which that probability flips the policy's `flagged` flag from `false` to `true`. A flagged policy contributes to the overall severity score, can be referenced directly in [content rules](/content-moderation/rules#available-signals) (e.g. *Toxicity is Flagged*), and surfaces in the review queue. ### Where to find it Open your project, go to **Policies**, pick the category (Toxicity, NSFW, Illicit, ...), expand the policy, and select the **Threshold** tab. If the project has more than one channel, pick the channel from the switcher at the top of the page. Policy threshold slider with histogram and would-flag / would-not-flag counts ### How to tune it The slider shows a histogram of recent confidence scores for that policy. As you move the slider: * **Bars to the left** of the slider are below threshold → not flagged * **Bars to the right** are at or above threshold → flagged Two KPI cards show how many recent items would have flipped sides. Use this to size the impact before saving. A general rule: * **Lower the threshold** if you're seeing false negatives: content you'd want flagged is slipping through. * **Raise the threshold** if you're seeing false positives: borderline content is being flagged unnecessarily. Click **Reset** to revert to the system default for that policy. ### Flag vs. shadow flag Each policy also has a **Flagging** tab with two options: | Option | Behavior | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | Flag content when detected | Standard. Crossing the threshold sets `flagged: true`, contributes to severity, and the policy can be referenced in rules | | Shadow flag when detected | The policy still scores content but doesn't mark it flagged. Items still appear in the queue under the *shadow flagged* filter | Flagging tab showing 'Flag content when detected' and 'Shadow flag when detected' options for the Toxicity policy Shadow flagging is useful for testing a new policy in production without changing what your users experience. You can watch how it would score real traffic, validate it against your moderators' decisions, then switch it on for real. *** ## How the two layers interact For a single moderation request, here's the order things happen in: Every enabled policy returns a `probability`. Per-policy thresholds decide which policies set `flagged: true`. Policies that scored contribute to the overall `severity_score`. The more severe the violation, the higher the score. Your [rules](/content-moderation/rules) run top-to-bottom. The first match decides the recommended action. If no rule matches, the severity score triage thresholds decide the action: Allow, Review, or Reject. So per-policy thresholds control *what gets flagged* and shape the severity score. Severity triage thresholds control *what action is taken* when no rule explicitly handles the case. If you want a category to never auto-reject regardless of score, write a [rule](/content-moderation/rules) that catches it. Rules run before triage, so a matching rule wins. *** ## Tips * **Change one dial at a time.** If you raise a policy threshold *and* lower the review cutoff in the same save, you won't know which one moved the queue. * **Watch the histogram, not just the numbers.** A 5-point slider move can shift hundreds of items between zones if your traffic clusters around that score. * **Hold off on Calibrate until you have a meaningful sample.** A handful of resolved items isn't enough to tune from, so let the queue accumulate first. * **Pair shadow flagging with [Simulate](/content-moderation/rules#simulate-before-saving).** Shadow flagging previews how a policy would score real traffic; Simulate previews how the rules would route it. Neither touches what your users see. # Adaptive model optimization Source: https://docs.moderationapi.com/custom-models/adaptive-model-optimization Learn how to use the review queue to improve your custom models automatically. Please reach out to our support team to enable adaptive model optimization for your account. Adaptive models can be improved over time by using the review queue. In machine learning, this is called reinforcement learning. This is a way to improve the model by providing feedback on its performance. With Moderation API, you do this using the review queue. Here you can remove incorrect labels, add missing labels, and provide feedback to the model. **Adaptive models can be useful for** * Improving accuracy over time * Adapting to changes in culture or language * Getting started with a small dataset and improving it over time ## How to provide feedback When viewing content in the review queue, you can correct the model's decision by removing labels or adding new ones. Correcting model decision Next time you train a custom model, the feedback will be used for improving the model. You can also create a new model and import the corrected data to start with accurate training data. ## Automatically retraining models If you already have a custom model, you can enable adaptive model optimization. This will automatically retrain the model on an ongoing basis with the feedback you provide in the review queue. Retraining will happen in the background, and you will be notified when the model has been updated. This way, you can ensure that your model is always up-to-date and accurate. # Custom models Source: https://docs.moderationapi.com/custom-models/overview Create custom models in the dashboard. Tailored to your specific needs and improve accuracy for special use cases. Illustration of blocks Create custom models tailored to your specific needs and improve accuracy for special use cases. Training a custom model may seem daunting at first, but the workflow is userfriendly and designed so anyone can train a model with minimal effort. You can get started in less than 10 minutes. **Custom models helps you** * Improve accuracy for special use cases * Improve models over time with self-learning * Classify your data with new labels * Bake all your policies into a single model Create a model in less than 15 minutes in the dashboard. ## How it works For example "Allowed" and "Blocked". Upload a CSV or use content you've already submitted to Moderation API. If your training data is missing labels, you can annotate it using our annotation tools. Start the training and wait for it to finish. Once your model is trained, you can use it your projects in Moderation API. ## Training data It is recommended that you have some examples of your content, ideally 200+ examples, but you can get started with as little as 20 examples. If you don't have any data to upload, you can start collecting it today with Moderation API. Everything submitted using the [text moderation endpoint](/api-reference) will we available as training data for your custom model. ## Self-learning models Custom models are self-learning, which means that they will improve over time as you use them. This is done by using the [review queue](/review-queues) and correcting the model when it makes a mistake. ## Need help getting started? If you need any guidance for creating a custom model please [send us a message](https://moderationapi.com/sales) any time. We're happy to help with anything from advice on creating a dataset for your business to implementing the model in your product. # Using a custom model Source: https://docs.moderationapi.com/custom-models/using When a model training has completed, the model will show up alongside all the pre-built models in your project, and you can [add it like any other model](/content-moderation/adding-models). ### Response signature Custom models have a similar response signature to the [pre-built classifiers](/models/classifier-overview). Each model returns an object with the detected label and the respective scores. This object is added to the API response under the model's key. The most probable label. This will always be the label with the highest score. Returns null if the analyzer fails. The score of the label. From 0-1 score with 1 meaning a high probability of being correct. An object containing all the label scores. # Concepts Source: https://docs.moderationapi.com/get-started/concepts How organizations, projects, channels, and policies fit together in Moderation API. Moderation API is organized as a four-level hierarchy. Most work happens inside a project; the other levels matter when you start scaling — adding teammates, separating environments, or running multiple products on the same plan. ```mermaid theme={"theme":"nord"} flowchart TB subgraph O["Organization"] direction TB subgraph P["Project"] direction TB A["Authors"] Q["Content views"] subgraph C["Channel"] direction TB PO["Policies"] end end end style O fill:#eff6ff,stroke:#2563eb,stroke-width:1.5px,color:#1e3a8a style P fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#1e3a8a style A fill:#bfdbfe,stroke:#2563eb,stroke-width:1.5px,color:#1e3a8a style Q fill:#bfdbfe,stroke:#2563eb,stroke-width:1.5px,color:#1e3a8a style C fill:#bfdbfe,stroke:#2563eb,stroke-width:1.5px,color:#1e3a8a style PO fill:#2563eb,stroke:#1e40af,stroke-width:1.5px,color:#ffffff ``` ## Organization The organization is the top-level container. It owns: * Billing and plan (quota, plan tier, invoices). * Team members who can access the organization's projects. * Custom models, trained once and reusable across any channel in any project. * Wordlists, managed in one place and enabled on the channels that need them. A user can belong to multiple organizations, and an organization can contain multiple projects. ## Project Projects scope your content and authors. Everything you submit to the API (content, [authors](/users/overview), [review](/review/overview), [actions](/actions/overview), and channels) belongs to a single project. Most teams use projects to separate environments like production and staging, or to keep distinct products apart. Each project has its own API key. New project flow with steps for project name, domain, type, and review When you create a project you fill in: * A **name** for the dashboard. * A **domain**: the website or app this project moderates. The AI uses it as context, which sharpens the accuracy of LLM-based policies like [Guidelines](/policies/guidelines). * A **type**: pick a template to preconfigure channels and content views, or start from scratch. Templates available today: | Template | What it sets up | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | Marketplace | `listings`, `messages`, and `reviews` channels, plus views for unreviewed listings, flagged reviews, auto-rejected listings, and bad actors | | Game | `chat` and `usernames` channels | | Blog | a channel for blog post comments | | Community | channels for forum-style posts and comments | | Blank | nothing pre-configured | Templates create the channels and views. You decide which policies to enable on each channel. ## Channel Channels live inside a project and represent a specific surface or content type. Common examples are `chat`, `reviews`, `listings`, `usernames`, and `profile-bio`. Each channel is configured independently, so you can apply strict rules to user-to-user messages while keeping product listings looser. Channel settings showing name, key, content types, and policies A channel owns: * **Settings.** Name, key (used to route requests, see below), description, and the [content types](/content-moderation/submit-content#content-types) it accepts (text, image, video, audio). * **Review thresholds.** The cutoffs at which content gets flagged. * **Policies.** The rules that score content. Toggle each one on or off per channel. See [Policies](/policies/overview) for the full list. * **Insights.** Opt-in signals like sentiment and language. They come back alongside policy results but don't affect flagging. * **Dry-run mode.** When on, API responses always return `flagged: false` so production traffic isn't blocked, but content is still analyzed and still lands in review. Use it to preview a channel before going live. ### Routing requests to a channel You authenticate with a project's API key. To pick which channel handles each request, the API checks, in order: 1. The `channel` field in the request body (see [Submitting content](/content-moderation/submit-content#channel)). 2. A channel whose content type matches the request's `metaType`. For example, `metaType: "review"` routes to a channel configured for reviews. 3. Otherwise, the newest channel in the project. You can also set a channel as the project's default route to override step 3. ## Policies A policy is a single rule that runs against submitted content. Each enabled policy returns one entry in the API response's `policies` array. Because policies are enabled per channel, the same project can run [Toxicity](/policies/toxicity) on chat messages while only running [Privacy](/policies/privacy) on profile bios. Policies fall into two types: | Type | What it does | | ---------------- | ------------------------------------------------------------------------------------ | | `classifier` | Scores content against labels and returns a probability (e.g. Toxicity, NSFW). | | `entity_matcher` | Extracts specific entities like URLs, emails, or phone numbers (e.g. Privacy, Spam). | See the [Policies overview](/policies/overview) for the full catalog. A few pieces sit above the channel and get attached to it: * **Custom models** are trained at the organization level and can be enabled on any channel. * **Wordlists** are also organization-level, then enabled per channel. * **Guidelines** are written per channel, since the rules usually depend on the surface. ## Authors Authors (your end users) are scoped to the project, not the channel. A single author can post into multiple channels. For example, the same user might send chat messages, write reviews, and update their profile bio. The author's [trust level](/users/trust-levels) and moderation history aggregate across every channel in the project. ## Review and content views Most content is handled automatically; what needs a person goes to [review](/review/overview). The inbox collects those items across all channels. [Content views](/review/content-views) are saved, filtered slices of your content — by channel, label, severity, or the action that was taken — and aren't limited to what the inbox holds, so a view can also cover content that was auto-rejected. Views are project-scoped, and a project can have as many as it has distinct reviewer workflows. Decisions made in review are remembered by the project's [casebook](/casebook/overview), which settles similar content automatically the next time it arrives. ## Quota Quota is shared across the organization. Each request to the `/moderate` endpoint consumes quota. Most requests cost one unit, though some content types and policies cost more. See [Quota & usage](/account/quota-usage) for the per-policy cost table. # Get started Source: https://docs.moderationapi.com/get-started/introduction Moderation API is a full stack content moderation solution. ## Quickstart Learn how to get started with Moderation API in 5 minutes. } href="/quickstart/typescript" > Get started with TypeScript and npm Get started with Python and pip Get started with Go Get started with Ruby Get started with PHP Get started with Java Get started with .NET ## Not a developer? If you're not a developer, you can still use the API. We have many integrations and no-code solutions that allows you to create projects and moderate content without writing a single line of code. See our integrations and no-code solutions. ## Guides Can't find your use case? [Contact us](https://moderationapi.com/sales) and we'll help you get started. Automate your moderation with the API. Learn how to use review queues to moderate content. Define custom rules in natural language for your channel. Get an overview of the pre-built models available. Learn how to create a custom model in the dashboard. # Content moderation for chats Source: https://docs.moderationapi.com/guides/chat-content-moderation This page is under construction.
Please reach out at [support@moderationapi.com](mailto:support@moderationapi.com) to get help with this feature. # Creating a custom model Source: https://docs.moderationapi.com/guides/creating-a-custom-model Learn how to create a custom model using Moderation API Head over to the Model Studio and press the models tab. Here you can create a new custom model. [Model Studio](https://dash.moderationapi.com/models). ## 1. Select your type of model The first step is to select your type of model. In this guide we’re focused on creating a classifier so that’s what you want to pick. Select your type of model ## 2. Define your labels Firstly, you should determine the specific categories or labels you intend to detect. It is advisable to add a limited number of labels as it enhances the performance of the model by reducing complexity. In case you encounter a complex situation, you may consider developing two models instead of a single model with several labels. Add your labels When creating your labels you’ll be asked to add a description for the label. This can help your organisation remember the labeling rules later on, but more importantly, it also helps the AI understand how you want to label your data. Note, once you’ve trained your model, you can no longer change or update your labels. ## 3. Add your training data To train your model, let’s add some data by providing examples of previously seen text, and the corresponding label that needs to be assigned to them. With Moderation API, this process is made easy, and you get the infrastructure to avoid common pitfalls like duplicate or skewed data. We have a few guidelines for getting the best results: * 100 examples or more per. category, though you may get by with less for simple problems. * Clean your data. Incorrect examples will result in poor performance. * Diversify the examples to help the model generalise better. * If the model produces an inaccurate prediction, try to add a few more examples for the given problem. We provide you with 3 options to add data: 1. **Upload a CSV or excel document.** 2. **Use previous requests stored in moderation API.** 3. **Add or update examples manually.** Add your data Let's go through the use cases for each of the options. ### Upload CSV of Excel document To upload a large number of examples to your model, you can create a CSV or Excel document and upload it. For instance, you could extract a CSV file from the latest 5000 user posts on your website and upload it here. When uploading the file, you will be prompted to indicate which column contains the text example, and optionally, which column contains the label. If the data is already labeled, the label column should contain a number that corresponds to the label, not the actual label name. For example, if there are 3 labels, the label column should contain 0, 1, 2, or be empty if there is no label. If your data is not labeled yet, we will help you annotate it in the next step. ### Use previous requests stored in moderation API If you've already been using moderation API for a while, and have enabled request logging, you have the option to import these requests for your model. This is the fastest way to add training data to your model. This is also an excellent way of continously add data and re-train your model to keep it up to date with the real world. For example, once per month you can add the latest requests from your project, and re-train and update your model with the new knowledge. You can choose to import from any of your projects or from all of your projects. For example if you have a project used for development and one for production, you can import from production only. ### Add or update examples manually This option should only be used if you don't have any training data, or very little training data. You can manually type a text and choose the correct label. This way it is possible to create a bunch of examples and get started with your model. In some simple cases with few training examples you might be able to create a decent model this way. The option can also be used to inspect and update existing examples that have been added using any of the other methods. ## 4. Annotate your data If your data has already been labeled prior to uploading, you can skip this step. If not, now is the time to annotate your examples with labels. Data labeling can be extremely time-consuming and often presents a significant challenge for businesses in creating custom machine learning models. With our user-friendly annotation tools, this process becomes significantly easier. You have two options for annotating your data: 1. **Automated annoation** 2. **Manual annotation** ### Automated Annotation This feature uses an LLM to annotate your examples based on the labels and descriptions you've previously provided. Auto annotate your data Because the auto-annotation uses your label descriptions, tweaking your labels will also change the annotations. You might want to start by auto-annotating 10 examples -> review the result -> tweak the label descriptions -> run auto-annotation again. Your subscription plan includes a specific number of automated annotations per month, and you can view the remaining allocated amount before initiating the auto annotation process. Once the auto annotation is complete, you will receive a notification via email. ### Manual annotation Or semi-manual annotation provides you with an interface to manually label each example. Manually annotate your data This tool is composed of several parts, each designed to help you with the annotation process: 1. The current example you are labeling is displayed at the top. 2. Your progress is shown with the count of examples you have already labeled. 3. The example text is provided, which can be altered to your preference. 4. The appropriate label can be selected from this section. The tool will attempt to predict the correct label upon loading the example. 5. If the prediction is accurate, you can save and proceed to the next example. 5. An estimated similarity score between the labels and example is viewable here. The tool will suggest the label with the highest similarity, although please note that this similarity score does not reflect the performance of your model. ## 5. Start the training After annotating your data, proceed to the training process. ### Reviewing training data Before beginning the training, review the model’s details, such as the total amount of training data, annotated examples, and the number of examples per label. This helps you identify any underrepresented labels and allows you to adjust them as needed. Aim for a balanced distribution of examples across all labels. Keep in mind that the quality and quantity of your training data significantly affect your model’s performance. You always have the option to add more data and retrain the model. If your data evolves over time, we recommend updating the model regularly. When retraining, the system replaces the old model with the newly trained one upon completion. ### Time to train The training may take some time, depending on the amount of data and the complexity of your model. Throughout the training process, the system learns from your annotated examples and develops a model capable of classifying unseen text based on the labels you defined earlier. Once the training is complete, you will receive an email notification. ## 6. Reviewing your model Once your training is complete, you'll want to integrate it into your project, but first, take a moment to review the performance. Start the training ### F1-score Start by checking the F1-score, a popular metric for gauging classification models. It combines precision and recall into one value for a balanced assessment of your model's abilities. The F1-score ranges from 0 to 1, with 1 being the ideal balance between precision and recall. A perfect F1-score of 1 is uncommon for real-world problems. A 0.7 score can be great for certain problems. The minimum acceptable F1-score depends on the specific situation, so it can't be universally defined. If your model has a relatively low F1-score, try adding more data to improve its performance. Focus on examples the model struggles with to better capture patterns. Alternatively, if more data isn't possible, adjust the label probabilities threshold. For instance, if your threshold is at 0.5, consider raising it to 0.7 or higher. This will only assign a label if its predicted probability passes the new threshold, resulting in potentially better performance in real-world situations. ### Test run You can also test-run the model to quickly test various examples. This is useful if you want to make sure it handles certain scenarios before you implement it in your project. Test your model ## All done! That's it! You've created your first custom model. Now it's time to integrate it into your project. If you need any help or have any questions, please don't hesitate to reach out to us. # Content moderation for marketplaces Source: https://docs.moderationapi.com/guides/marketplace-content-moderation This page is under construction.
Please reach out at [support@moderationapi.com](mailto:support@moderationapi.com) to get help with this feature. # Designing a moderation workflow Source: https://docs.moderationapi.com/guides/moderation-workflow This page is under construction.
Please reach out at [support@moderationapi.com](mailto:support@moderationapi.com) to get help with this feature. # How to set up self-learning models Source: https://docs.moderationapi.com/guides/self-learning This page is under construction.
Please reach out at [support@moderationapi.com](mailto:support@moderationapi.com) to get help with this feature. # Handling User-Flagged Content Source: https://docs.moderationapi.com/guides/user-flagged-content Learn how to handle user-flagged content with the Moderation API