Unsung Heroes: Moderators on the Frontline of Internet Security

One might ask what exactly does a content moderator do? To answer that question, let’s start from the beginning.

What is content moderation?

Although the word Moderation Often misinterpreted, the central goal is clear: to assess user-generated content for potential harm to others. When it comes to content, moderation is the act of preventing extreme or malicious behavior such as offensive language, exposure to graphic images or videos, and user fraud or exploitation.

There are six types of content moderators:

1. No moderation: No content moderation or intervention where bad actors can harm others.
2. Pre-moderation: Content is screened according to predetermined guidelines before it goes live.
3. Post-measurement: The content will be checked after it is live and if it is deemed inappropriate, it will be removed
4. Retrospective: Content is only filtered if other users report it.
5. Automated moderation: Content is actively filtered and removed using AI-powered automation
6. Distributed moderation: Inappropriate content will be removed based on votes from multiple community members

Why is content moderation important for companies?

Malicious and illegal behavior by bad actors puts companies at significant risk in the following ways:

• Loss of credibility and brand name
• Exposing vulnerable audiences such as children to harmful content
• Failure to protect customers from fraudulent activity
• Losing customers to competitors who can provide safer experiences
• Allowing false or fraudulent identification

The critical importance of content moderation goes beyond protecting businesses. Managing and removing sensitive and serious content is important for every age group.

As many third-party trust and security services professionals will attest, mitigating the widest range of risks requires a multi-pronged approach. Content moderators should use both preventive and proactive measures to maximize user safety and maintain brand trust. In today’s highly online environment of political and social issues, taking a “passive” approach of waiting and waiting is no longer an option.

“The virtue of justice consists in moderation as it is directed by wisdom. – Aristotle

Why are human content moderators so important?

Many forms of content moderation involve human intervention at some point. However, reactive moderation and distributed moderation are not ideal approaches, as harmful content is not addressed after exposure to users. PostMeasurement offers an alternative approach, where an AI-powered algorithm monitors content for certain risk factors and then alerts a human moderator to ensure that certain posts, images or videos are indeed harmful and should be removed. With machine learning, the accuracy of these algorithms improves over time.

Although it would be nice to avoid the interest of human content moderators given the nature of the content exposed (including child sexual abuse, graphic violence, and other harmful online behavior), this is unlikely. Human understanding, comprehension, interpretation and empathy cannot easily be replicated artificially. These human qualities are important to maintain honesty and integrity in communication. In fact, 90% of consumers say authenticity is important when deciding which brands they like and support (up from 86% in 2017).