Concepts
A quick overview of the concepts used in the moderation API.
Projects
A project is used to configure how you want to analyze content. You do this by adding models to your project and adjust the project settings to your needs.
Each project has its own API key, so you can create multiple projects for example to separate different types of content or to separate your production and development environments.
Models
Models are used to analyze content. You can add multiple models to a project. Each model has its own settings and can be used to analyze content in a different way.
3 types of models
Ready-made models
These are pre-built models that can be used to analyze content without any additional training.
Custom models
These are models that are trained on user-provided data to analyze content specific to their use case.
AI agents
AI agents is a way of creating your own model, without prior training data. Just provide the AI with your own set of rules.
| Model | Needs training data | Self-learning | Customizable | Accuracy | Effort |
|---|---|---|---|---|---|
| Ready-made models | No | No | No | High | Low |
| AI agents | No | No | ✅ | Medium | Medium |
| Custom models | ✅ | ✅ | ✅ | High | High |
Review queues
The queues are optional to use, but they are useful as your own customizable moderation dashboard.
Review queues are used to manually review and act on content submitted to your projects. You can create multiple queues and configure what content you want in a queue by setting up filters.
How you use the review queue is up to you. For example, you can use it to review content before it is published, or to review content that has already been published.
Queue actions
When you review content in the review queue, you can take actions on the content.
You can create your own custom actions and assign them to the queues you want.
Actions can be used to, for example, delete content, block a user, send content to a different queue, or send content to a human moderator.
Self-learning
We never use your data to train our own models. Data is only used to improve your own models when you use the self-learning feature.
You can use the review queue to train and improve your custom models. For example, you can use the review queue to review content that has been flagged by a model, and then use the feedback to improve your models.
Quota
Depending on your plan you have a certain amount of quota.
Each request to your project will use a certain amount of quota. Usually this is 1 quota per request, but some models use more quota.
Wordlists
Some models can be configured to use wordlists. Wordlists are used to add custom words to the models vocabulary.
The wordlists are smart and can be used to add words with different variations. For example, if you add the word run to a wordlist, the model will also recognize running, ran and ruuuun etc.
Was this page helpful?