OpenAI Reports Successful Use of Their Tools by Threat Actors and Nation States

OpenAI published a report last week that several government-linked threat actors have been observed abusing the model and using the tool for influence operations. Over the last three (3) months alone, OpenAI said they’ve disrupted as many as five (5) campaigns carrying out influence operations. OpenAI detected campaigns from Russian, Chinese, Iranian, and Israeli threat actors linked to various government and political entities. OpenAI reported that the groups whose operations were disrupted were using AI “to some degree, but none used it exclusively.” The report revealed that the campaigns leveraged the company’s tools in a variety of ways, but mostly generated content in the form of texts and photos to create articles and social media posts. The threat actors were also seen using it to debug code and create falsified social media engagement by creating artificial content that was reinforced with fake comments. The current conflicts around the world along with multiple elections happening this year have heightened the concerns around how bad actors might be able to use generative AI. In reassurance, however, the company said none of the influence campaigns they disrupted received a score higher than a two (2) out of six (6) on the “Breakout Scale”, a metric that measures how influential malicious activity is to an audience. While there were a handful of campaigns using the company’s tool that had to be disrupted, OpenAI also reported that there were many instances where built-in guard rails in the software prevented threat actors from achieving success in the first place.