During the last few years, content moderation powered by Artificial Intelligence has grown exponentially. It has developed in ways that were unimaginable just a decade ago, breaking concepts and widening the horizons of what’s possible. In this article we’ll have a look at the future of content moderation.
Automated content moderation has been fueled by ever-evolving machine-learning algorithms that constantly improve in accuracy and speed. Just 10 years ago, image recognition was only able to classify and detect basic objects and shapes. Now, thanks to the advancements of deep learning, image recognition algorithms for instant detection of all types of inappropriate visual content are a reality.
Automated (also referred to as semi-automated) content moderation thus offers important new capabilities for businesses of different venues that need effective screening of digital content. The AI moderation platforms address a number of key challenges that online platforms and companies face, including:
- Huge amounts of user-generated content need to go online immediately, but still have to be monitored for appropriateness, safety, and legality. This can make it difficult for online platforms to grow and scale internationally if they don’t have an effective way to screen all postings — textual, visual, and even live streaming. Without moderation, these businesses risk great reputational harm, along with a list of other negative consequences.
- Content moderation has to happen in real-time, which is especially difficult for live streaming and video that are becoming the most popular content formats. The complexity of screening visuals, texts and moving images at the same time is tremendous.
- User safety and especially the protection of vulnerable groups is becoming a priority in legislation that covers digital platforms. This means that in many places across the globe, online businesses are required by law to have solid Trust and Safety programs and protection mechanisms based on content moderation. This is necessary not only to ensure the upholding of their internal principles and guidelines, but to safeguard consumers.
- The stress and harmful effects on human content moderators from exposure to shocking, violent and disturbing content is significant. Digital businesses aim to minimise these negative consequences and to protect their moderating teams from the worst content.
- Digital platforms have to be able not only to scale in terms of countries and amounts of content that goes live, but also to adapt to quickly changing circumstances and norms for content appropriateness.
- Public manipulation, political propaganda, disinformation through fake news, and the rise of DeepFakes are disturbing yet prevailing new phenomena online. Both official authorities and online platforms need an effective way to fight them, and machine learning algorithms are the key to that.
Contents
Research challenges
While holding great potential and already showing impressive results, there are challenges that AI-powered content moderation is facing.
One of the major issues with which automated content moderation is struggling is recognizing context. Machine learning algorithms can find it difficult to differentiate between subtle cultural and social trends and phenomena. For example, if the algorithm is set to remove all nudity, this is what it would do — even if the nudity is related to art or important news pieces. A prominent example was the case from 2016 when Facebook removed the photo of the iconic Vietnamese ‘napalm girl’ who is naked.
Another important challenge that AI platforms need to overcome is multilingual moderation. While they are getting better at it and are surely improving how content moderation in different languages is conducted, there are still obstacles on the way. The process is not only about acknowledging the direct meaning of words and phrases, but their social and cultural connotations that may make them offensive and inappropriate. In this respect, the more feedback machine learning algorithms receive, the better they can become at spotting the nuances in content — which is definitely not a mission impossible, but simply a gradual process that takes processing large amounts of data.
Live streaming and live video are another interesting challenge for AI-based content moderation platforms. They generate such a substantial amount of data per second that manual moderation is simply an impossible task. Moreover, applying AI on each frame of the live broadcast generates high platform costs. A fast and accurate AI needs to be developed to overcome these hurdles to efficient and cost-effective moderation of live streaming and videos.
Societal and Media Industry Drivers
Merry is a fact-checking manager at an online newspaper with a solid and long-standing reputation for trustworthiness. Her job is to ensure every piece of information that gets published through the media’s channels is accurate. She needs to check not only facts and textual references, but also visuals. In a way, Merry is a modern-day fighter against disinformation — the plague of today’s digital world.
But doing all of this on her own — and even with her fact-checking team — is a monstrous task. The process simply takes enormous amounts of time and effort. That’s why Merry needs a viable and scalable solution to checking accuracy and preventing the spread of fake news and visuals.
This is especially important during the current national election. Politicians, ordinary citizens, and people who want to spread fake news are all posting about the topic constantly — and it’s very difficult to sift through what’s true and what’s not. In particular, a scandalous story about financial misdealings that one of the party leader’s is involved in is spreading online — mostly fueled by his political opponents. Merry has to figure out how true the lead is and whether to publish it.
With the help of an AI-powered content moderation platform, Merry can screen various materials around the topic for authenticity. She can catch textual references, as well as photos that have already been posted, for example.
Daniel is a content editing manager at an online news outlet. He’s in charge of guaranteeing that all published content complies with the standards of the media and the legal framework. His most challenging task is to ensure the compliance of live streamings. Catching inaccurate and harmful video content in real time is a tough nut to track. Without technical support, it is a burdensome task to monitor live streaming content as it occurs.
This is where a content moderation platform based on machine learning algorithms can kick in. It processes all live video streamings, checking them for inappropriate verbal and visual elements. If there are such, the platform can immediately signal to the editors.
Daniel can test the capabilities of the AI platform in practice during an important live streaming with a local politician at a public rally. The situation is uncontrollable, as it’s a place full of people where anyone can appear and take over the stage. With the help of the moderation platform, Daniel can have the live streaming screened for problematic content throughout the whole event.
Future Trends for the Media Sector
The role of content moderation in the media sector cannot be overestimated — in fact, it’s crucial for its wholesome development on a couple of levels:
- As illustrated in the vignette, content moderation algorithms are the key to fighting the widely spread online disinformation. They can spot inaccuracies in textual data and fake or old visuals and videos. These capabilities can be a gamechanger for the media sector that direly needs adequate fact-checking in the oceans of information that get published.
- On the basis of the massive volumes of content that machine learning algorithms process, they can also make trend predictions about the types of content that needs to be moderated. This can be of huge importance for combating harmful tendencies in user-generated content sharing.
- Content moderation, if done in-depth, can help in detecting the intent of disinformation in order to differentiate between positive (for example, to keep state secrets) and negative (to influence opinions and harm society).
Goals for Next 10 or 20 Years
The long-term vision for AI-powered content moderation is a truly ambitious one.
First and foremost, content moderation would need to leave the semi-automatic status it currently has. To be fully useful, scalable, and powerful, it should be more autonomous. The vision is that AI-powered content moderation platforms would become a monitor that is always on and oversees any and all content that goes online. It would screen for all types of abnormalities to ensure protection of users from harmful and illegal content and a safe online environment without fake news — thus taking care of everything from violence and nudity to propaganda, radicalization and disinformation, and all that’s in between.
The second big goal for content moderation’s evolution is self-learning — which is already in motion, but can reach new heights. With the data that is being fed in the moderation platform in real time, the machine learning algorithm becomes better and better. It expands its knowledge base with practical examples and input from moderators. With time, this is how the AI can become more independent from humans in terms of feedback loop. In the foreseeable future, this can reach a point where the moderation platform becomes an autonomous machine that identifies and filters content accurately and effectively with no human input.
A third long-term goal for content moderation is the creation of instant and efficient on-device moderation. Nowadays, moderation is executed on the server end, only after a piece of content has already gone live. This means that harmful content can sneak in for a moment and be accidentally shown to users. In the near future, moderation would be possible on the customer device itself. This would happen before the content has gone live. This advancement would enable the prevention of illegal and disturbing content appearing on the device level, thus ensuring full protection for end users.