Memes can be used for spreading hate speech, which is why Facebook is now scanning them with the help of an Artificial Intelligence (AI) mechanism. The machine learning system, called Rosetta, is already working and extracting text from around 1 billion videos and images posted on Instagram and Facebook.
How does Rosetta work?
More than a billion images and video frames are posted on Facebook and Instagram each day, making it unimaginably difficult to review the texts—which appear in a variety of language and styles– on all those images or video. This is why Facebook is deploying AI for the job. It claims that Rosetta is fully capable of performing the job meticulously and efficiently.
Facebook explained how Rosetta is going to scan the memes for hate speech. The machine learning system works by extracting text from a billion public Instagram and Facebook video frames and images daily and in real time (in several languages) and putting it into a text recognition model that is trained on classifiers to understand the meaning of the image as well as the text.
What’s in store for Rosetta and AI for the future?
Many machine learning models are adding text extraction as a feature for improving the quality and relevance of photo search and for recognizing content that violates hate speech policy.
Facebook says that the problem of understanding text in pictures is different from the problem addressed by optical character recognition (OCR) systems. This is because of the large volume of images shared daily on Instagram and Facebook, and because traditional optical character recognition (OCR) systems recognize the characters but do not understand the context of the associated image.
While talking about future plans of its Rosetta AI, Facebook said, “The rapid growth of videos as a way to share content, the need to support many more languages, and the increasing number of ways in which people share content make text extraction from images and videos an exciting challenge that helps push the frontiers of computer vision research and applications.”
