Associated Press Technology Writer Matt O’Brien
A former Openai employee is urging top law enforcement officials in California and Delaware to stop them from changing the management of artificial intelligence technology from nonprofit charities to commercial businesses.
They are worried about what will happen if CHATGPT makers meet their ambitions to build AI that surpasses humans, but are no longer accountable for the public mission of preventing their technology from causing disastrous harm.
“Ultimately, I’m worried who will own and control this technology once it’s created,” said Page Hedley, former policy and ethics advisor for OpenAI, in an interview with the Associated Press.
Supported by three Nobel Prize winners, other supporters and experts, Hedley and nine other former Opene workers wrote to two state attorney generals this week.
The coalition calls on both Democrats, California Attorney General Rob Bonta and Delaware Attorney General Kathy Jennings, to use their powers to protect open charitable purposes and prevent planned restructuring. Openai is incorporated into the state of Delaware and operates in San Francisco.
“Changes to existing structures will help the wider public benefit from AI,” Openai said. Although for-profits say it will become a public benefits company, like other AI labs such as humanity and technology billionaire Elon Musk’s Xai, Openai still preserves its non-profit sector.
“This structure continues to ensure that as for-profits succeed and grow, nonprofits will also grow and achieve their mission,” the company said in a statement.
The letter is the second petition this month against state officials. The final came from a group of labor leaders and nonprofits that focus on protecting Openai’s multi-billion dollar charity assets.
Jennings said last fall that he would “review such transactions to ensure that the interests of the public are properly protected.” Bonta’s office asked for more information from Openai late last year, but said it could not comment on whether it was investigating or even denying it.
Openai co-founders, including current CEO Sam Altman and Musk, have started it as a non-profit research lab on its mission to safely build artificial general information, or what is known as AGI, for the benefit of humanity. Almost ten years later, Openai reports a market value of $300 billion, counting 400 million users per week for its flagship product, ChatGPT.
Openai already has for-profit organizations, but it faces many challenges in transforming its core governance structure. One is a lawsuit from Musk, which blames the company and Altman for betraying the establishment principle that led Tesla’s CEO to invest in charity.
While some of the signatories of this week’s letter support Musk’s lawsuit, Hedley said others were “underlyingly cynical” because Musk also runs his own rival AI company.
Signatories include two Nobel Prize-winning economists, Oliver Hart and Joseph Stiglitz, and AI pioneer and computer scientists Jeffrey Hinton and Stuart Russell, who won last year’s Nobel Prize in Physics.
“I like the open mission of ‘ensure that artificial general information benefits all humanity’ and I hope they will carry out that mission rather than enriching investors,” Hinton said in a statement Wednesday. “We are pleased to see that we have an effort to retain Openai on missions that do not involve Elon Musk.”
The conflict over the purpose of the opening has been simmering for a long time at the San Francisco Institute, contributing to the resignation of masks in 2018, the short-lived expulsion of Altman in 2023, and other prominent departures.
Hedley, a training lawyer, worked for Openai in 2017 and 2018. This was a time when nonprofits were still navigating the best ways to manage the technology they still wanted to build. Recently in 2023, Altman said that Advanced AI held its promises but also warned of extraordinary risks, from dramatic accidents to social disruptions.
However, in recent years, Hedley said he has been viewing with concern that Openai, which is backed by ChatGpt’s success, is cutting corners with safe testing and rushing to new products to get ahead of business competitors.
“The cost of these decisions continues to rise as technology becomes more powerful,” he said. “I don’t think there’s anyone who can tell them no longer that they can tell the new structure that Openai wants to have the incentive to hurry to make those decisions.
Software engineer Anish Tondwalkar said until last year, a former member of Openai’s technical team, that the key assurance in Openai’s non-profit charter was a “stop assist clause” that directs Openai to pause and assist if another organization is approaching achieving better Human’s AI.
“If Openai is permitted to become a for-profit organization, these safeguards, and Openai’s obligations to the public, could disappear overnight,” Tondwalkar said in a statement Wednesday.
Another former worker who signed the letter put it more openly.
“Openai may one day build technology that can kill us all,” said Nisan Stiennon, an AI engineer who worked for Openai from 2018 to 2020.
Associated Press and OpenAI have licenses and technology agreements that allow OpenAI to access some of the AP’s text archives.
Original issue: April 23, 2025 10:36am EDT
