Meta Finds Facebook Propaganda Network Using Generative AI

Meta Platforms said it disrupted an influence network on Facebook and Instagram designed to sway public opinion in Israel’s favour over the war in Gaza and using material that appeared to be generated using artificial intelligence (AI) as part of its operations.

It is the first time the company has disclosed finding an influence operation using text-based generative AI since the technology came to prominence in late 2022, although it has previously found AI being used to generate profile photos.

Meta said the fake profiles that formed the network were found to be posting “likely AI-generated” content posing as user comments on posts from news organisations and US lawmakers.

The company’s policy director for threat disruption, David Agranovich, said at a briefing with journalists that the fake comments were “often met with critical responses from authentic users calling them propaganda“.

AI tactics

“So far, we have not seen novel Gen AI-driven tactics that would impede our ability to disrupt the adversarial networks behind them,” he said.

The influence network in question targeted people in the US and Canada with accounts posing as “Jewish students, African Americans and ‘concerned’ citizens” and sharing posts that praised Israel’s military actions and criticised the United Nations Relief and Works Agency (UNRWA) and college campus protests.

The accounts also shared Islamophobic comments in Canada.

Meta attributed the network to STOIC, a political marketing and business intelligence firm based in Tel Aviv.

The STOIC network was also active on X and YouTube and ran websites focused on the Gaza war and Middle Eastern politics, Meta said.

Covert influence networks

The company said it disrupted the network before it gained a large audience and that many of the fake accounts were disabled by its automated systems.

The accounts reached about 500 followers on Facebook and about 2,000 on Instagram.

The quarterly report highlighted six covert influence operations in the first quarter of 2024, including an Iran-based network focused on Gaza that did not appear to use generative AI.