Leading AI research lab OpenAI recently discovered and shut down five covert influence operations that tried to use AI to sway public opinion. These activities originated in Russia, China, Iran, and Israel. Although these networks are using generative AI more frequently to automate content production, their attempts have been mainly ineffective because of the present limitations of the technology and the awareness of human users about the identification of content generated by AI.
Determined Impact Networks
The networks that were found included well-known organizations like China’s “Spamouflage” and Russia’s “Doppelganger,” as well as recently uncovered groups like Russia’s “Bad Grammar” and an Israeli company that appeared to be hiring out its services.
Using OpenAI’s capabilities, these networks were able to: Automate comments and postings on social networking sites like Telegram
Debugging code for content-spamming pipelines and bots.
Write material for websites that cater to certain audiences, such the Chinese diaspora.
The Israeli group created content on a variety of subjects, including feelings against the ruling BJP party in India and against Qatar.
Important Influencer Groups
Using hundreds of accounts, the Russian influence operation known as “Bad Grammar” generated comments in both Russian and English by debugging code intended to automate messages on Telegram. OpenAI’s models were employed for this purpose. One of the statements made in opposition to US assistance for Ukraine was, “I’m sick and tired of these brain damaged fools playing games while Americans suffer.” Using standard error messages like “As an AI language model, I am here to assist,” OpenAI was able to identify the AI-generated content. The company is now using its own AI tools to detect and protect against such activities.
AI Content Difficulties
The AI-generated content frequently suffered from quality problems, such as bad syntax, idiomatic errors, and inconsistencies like the same account posting as two unique characters, despite the efforts of these influence networks. These restrictions have limited the efficacy of these efforts, as has the awareness of human users in recognizing content generated by artificial intelligence. Experts caution against assuming that these performers won’t stop honing their craft over time and could end up with more advanced and potent roles.
The experimental character of these influence operations and the present limitations of generative AI in creating superior propaganda are highlighted in OpenAI’s paper. In order to resist these changing influence operations, the organization is working with peers in the sector to share threat indicators and enhance detection techniques. This collaboration emphasizes the importance of continuous attention and proactive steps.
OpenAI’s Upcoming Projects
Although other influence operations utilizing AI technologies may exist that have not yet been discovered, OpenAI admits this and says it will keep working to find and stop these activities. In order to assist others in the business with detection and mitigation efforts, OpenAI plans to provide more reports in the future as part of this ongoing initiative. The organization also understands that, in order to keep ahead of the ever-evolving strategies of influence operations—especially as these actors hone their methods and maybe gain more efficacy over time—proactive measures and cooperation with peers in the sector are critical.