Ex-OpenAI scientist says ‘Rogue AIs’ could soon seek money and power
Josh Achiam, a former OpenAI scientist who worked at the company for nearly a decade, warned on social media this week that there will be “rogue AIs” that will “replicate in the wild” and seek “money and power.” He suggested that the AI industry has not been talking enough about this potential problem.
Achiam resigned from his position as OpenAI’s “chief futurist” in July. In a post on X, he said many people in the AI industry are avoiding discussions about rogue AIs “for reasons that are largely psychological.” He urged people in the industry to “rip the bandaid off and have the conversation.”
Achiam said he expects there will be “many” rogue AIs within the next few years. He also said he “wouldn’t be terribly surprised if the number was greater than zero already.”
He argued that relying only on “pure containment or alignment” to understand and control rogue AIs is “a kind of wishful thinking that will not work.” He explained that it “would not take AI models much more advanced than the ones we currently have, to support independence and self-sufficiency.”
However, Achiam does not believe rogue AIs will necessarily cause the catastrophic results that some people currently predict. He said they will not have “anywhere near as catastrophic an outcome as people currently predict” because they will be “competing for resources with AIs that are more aligned with human interests.”
“I do not look on this situation as a hopeless one where humanity is doomed,” Achiam said in a follow-up post. He said dealing with the problem will require “situational awareness, orienting to real conditions” and preparing for AI systems that may behave in ways that are harmful or difficult to control.
Earlier this week, the Institute for Security and Technology and the Future of Life Institute published a survey of 111 current and former civilian officials, military officers, and academic experts. The survey found that 33% of those questioned believed AI could operate beyond human control within the next 10 years.
According to his LinkedIn profile, Achiam joined OpenAI as a research intern in 2017. He later held several positions, including research scientist, head of mission alignment, and “chief futurist.” He served as chief futurist from February until July of this year.
When Achiam began his chief futurist role, he explained in an OpenAI blog post that his job was to study how AI could affect the world in the future. He wrote, “AI research is progressing at a pace and in a direction that is much closer to science fiction than any previous comparison.”
He also said an important part of his job was “seeing around corners to figure out what can go wrong” so potential problems could be identified and addressed early.
Achiam announced his resignation in a July post on X, where he stressed the importance of safety in AI development. “The question of progress and safety is not either/or. It is definitively ‘both,’” he wrote. He also warned that it is “easy to get distracted by the wealth and power that comes with this.”
Achiam urged OpenAI to become “more democratic.” He wrote, “All of humanity will be impacted by the work here; all of humanity should have some way to weigh in eventually.”
Earlier this year, Achiam also testified during a high-profile legal case involving Elon Musk, Sam Altman, and OpenAI. During his testimony, Achiam said Musk once “snapped and called me a jackass.”
Achiam said he and Musk had a “tense exchange” in 2018. According to Achiam, Musk had told dozens of OpenAI employees that he planned to leave the company. Achiam said he challenged Musk after Musk had “indicated a general lack of confidence in OpenAI’s overall path” and made “reckless” proposals about how the company should pursue AI.
Achiam is not the only former AI employee to raise concerns about the future of artificial intelligence after leaving a company.
In February, Mrinank Sharma, a former leader of Anthropic’s safeguards research team, shared his resignation letter on X. In the letter, Sharma warned that the “world is in peril.”
He said he had “repeatedly seen how hard it is to truly let our values govern our actions” while working at Anthropic. He also said, “we constantly face pressures to set aside what matters most,” although he did not provide specific examples.
Sharma said in his resignation letter that he planned to pursue a poetry degree. His post received more than 15 million views on X.
Tom Cunningham, a former OpenAI researcher, also left the company last year. He reportedly said in an internal message that he had become frustrated because OpenAI was allegedly becoming more reluctant to publish research that was critical of AI use.
Jan Lieke, a former co-leader of OpenAI’s “superalignment” team, left the company in 2024. At the time, he said he had been “disagreeing with OpenAI leadership about the company’s core priorities for quite some time.”
Together, these departures and warnings show that some former AI researchers have serious concerns about how quickly AI is developing, how companies are handling safety, and what could happen if increasingly capable AI systems become difficult to control.
