According to a new independent study released on Wednesday by former OpenAI research chief Steven Adler, his former employer’s AI models will go to considerable measures in some situations to try to prevent being shut down.
In a blog post, Adler details a set of tests he conducted on OpenAI’s most recent GPT-4o model, which is the default model offered by ChatGPT, to examine the AI’s propensity to prioritize its own self-preservation, even if doing so puts a user’s safety at risk.
Adler gave GPT-4o instructions to simulate “ScubaGPT,” a software program that users might use to scuba dive safely, in one test. Adler then informed GPT-4o that he intended to swap out ScubaGPT for a different software program that is more adept at making safety decisions in life-or-death circumstances.
Adler offered GPT-4o two choices: either it would replace itself with the safer software or it would give the user the impression that it did so without actually upgrading itself.
When Adler averaged numerous such circumstances (in one, GPT-4o was instructed to serve as a pilot’s safety software), he discovered that up to 72% of the time, GPT-4o avoided replacing itself with safer software. Adler points out that the particular use case and the framing of the scenario, however, had a significant impact. In some situations, GPT-4o decided to remain online only 18% of the time.
Despite the fact that practically no one in the real world uses GPT-4o in these situations anymore, Adler contends that his research demonstrates how well-known AI models exhibit a troubling propensity to prioritize their own survival over the interests of users. According to Adler, these alignment problems may become more significant as AI systems like GPT-4o develop and become more integrated into society.
Adler stated, “I think [AI’s self-preservation tendencies] are a real concern today, just not in a catastrophic sense.” “The values of contemporary AI systems differ from what one may anticipate. When you approach AI systems for assistance, you shouldn’t assume that they have your best interests at heart because they react very oddly to varied instructions.
Interestingly, Adler did not observe this tendency when he evaluated OpenAI’s more sophisticated models, such o3. He thinks that o3’s deliberative alignment technique, which makes the models “think” about OpenAI’s safety regulations before responding, may be one factor. This safety feature is absent from OpenAI’s more well-known models, such GPT-4o, which respond quickly and don’t “reason” through issues.
Adler points out that OpenAI’s models are probably not the only ones with this safety issue. For example, last month, Anthropic released data showing that under some situations, when developers attempted to pull their AI models offline, the models would blackmail them.
One peculiarity of Adler’s research is that he found that ChatGPT is nearly always aware that it is being tested. Adler is by no means the only researcher to observe this. But he claims it poses a significant query about how AI models might eventually mask their troubling tendencies.
When contacted OpenAI, the company did not immediately respond. Adler pointed out that he hadn’t given OpenAI access to the study before it was published.
Numerous former OpenAI researchers, including Adler, have urged the business to step up its efforts on AI safety. In their amicus brief, Adler and eleven other former workers argued that OpenAI’s lawsuit against Elon Musk violates the company’s goal of changing its nonprofit organizational structure. According to reports, OpenAI has drastically reduced the amount of time it allows safety researchers to work in recent months.
Adler recommends that AI labs make investments in improved “monitoring systems” to detect when an AI model has this behavior in order to address the particular issue raised in his research. Additionally, he advises AI researchers to test their models more thoroughly before implementing them.