Meta Ordered To Clean Up AI-Generated Porn By Oversight Board

Meta Platforms has been ordered to clarify its rules over banning sexually explicit AI-generated depictions of real people.

The ruling on Thursday came from Meta’s so called Supreme Court, the Oversight Board, which is the ultimate arbiter of content moderation decisions. Although funded by Meta, it operates independently of the social networking giant.

The ruling come after two cases of sexually explicit AI images that resembled female public figures from India and the United States.

AI porn images

The Board’s recommendations “seek to make Meta’s rules on this type of content more intuitive and to make it easier for users to report non-consensual sexualised images.”

In the first case, an Instagram account that shared only AI-generated or manipulated images of Indian women posted a picture of the back of a nude woman with her face visible, as part of a set of images.

This set also featured a similar picture of the woman in beachwear, most likely the source material for the explicit AI manipulation, the Oversight Board stated.

The second case also involves an explicit AI-generated image resembling a female public figure from the United States.

In this image, posted to a Facebook group for AI creations, the nude woman was being groped. The famous figure she resembles is named in the caption.

In the first case (Indian public figure), a user apparently reported the content to Meta for pornography but as the report was not reviewed within 48 hours, it was automatically closed. The user then appealed to Meta, but this was also automatically closed. Finally, the user appealed to the Board.

As a result of the Board selecting this case, Meta determined that its original decision to leave the content on Instagram was in error and the company removed the post for violating the Bullying and Harassment Community Standard. Meta also subsequently disabled the account that posted the content and added the explicit image to a Media Matching Service (MMS) bank.

In the second case (American public figure), the explicit image had already been added to an MMS bank for violating Meta’s Bullying and Harassment policy and so was automatically removed.

Board Concerns

MMS banks automatically find and remove images that already have been identified by human reviewers as breaking Meta’s rules.

The user who posted the AI-generated image appealed but this was automatically closed. The user then appealed to the Board to have their post restored.

The Oversight Board said it was concerned about the auto-closing of appeals for image-based sexual abuse. Even waiting 48 hours for a review can be harmful given the damage caused, it said.

So in summary, in the first case (Indian public figure), the Board overturned Meta’s original decision to leave up the post.

In the second case (American public figure), the Board upheld Meta’s decision to take down the post.

Board recommendations

The Oversight Board said Meta’s rules were “not sufficiently clear” in barring sexually explicit AI-generated depictions of real people and called for changes to stop such imagery from circulating on its platforms.

The Board also recommended that Meta:

• Move the prohibition on “derogatory sexualized photoshop” into the Adult Sexual Exploitation Community Standard.
• Change the word “derogatory” in the prohibition on “derogatory sexualized photoshop” to “non-consensual.”
• Replace the word “photoshop” in the prohibition on “derogatory sexualized photoshop” with a more generalized term for manipulated media.
• Harmonise its policies on non-consensual content by adding a new signal for lack of consent in the Adult Sexual Exploitation policy: context that content is AI-generated or manipulated.

Meta said it would review the board’s recommendations and provide an update on any changes adopted.

Supreme Court

Meta had created the Oversight Board in September 2019, with the mission to review Facebook and Instagram’s decisions on taking down or leaving up certain content .

It also issues rulings on whether to uphold or overturn Meta’s content moderation actions.

One of the Oversight Board’s most high profile rulings involved the decision to reinstate the Instagram and Facebook accounts of former US President Donald Trump, after he was suspended for two years for his role in inciting a mob of his supporters to storm the US Capitol building on Wednesday 6 January 2021, which resulted in the deaths of at least five people.

In December 2023 the Oversight Board also reversed Meta’s content decision to remove graphic videos on Israel-Gaza from Facebook.

One video had been posted on Instagram, and was said to have showed the aftermath of an airstrike near al-Shifa hospital in Gaza, including children, injured or killed.

The second video was posted on Facebook, which showed an Israeli woman begging her kidnappers not to kill her as she was taken hostage.