How the AI industry will regulate itself when it comes to AI safety

AI safety has become a global headline, but the latest news from Washington this week means the future of policing this technology will be in the hands of those creating it. President Trump hosted a high profile lunch with some of the most prominent names in artificial intelligence at the White House on Tuesday. The big takeaway, AI safety will be coming from the companies themselves. The president saying he wants no new laws and a new accord was signed by Meta's Mark Zuckerberg, SpaceX and XAI's Elon Musk, Nvidia's Jensen Huang and OpenAI's co-founder Greg Brockman, as well as Anthropic's Dario Amodei. I think the technology has very real risks and the mechanism, how we address those risks, is still under discussion. But we all need to work, we all need to work together to make sure that we can win and we can win safely. If we do this right, we work with the president and everyone here, we can win safely. It is a voluntary safety framework, and it includes oversight from independent auditors. Is this accord that everybody signed, is it binding in any way? I think it's morally binding. Yeah. President Trump is also rebranding the tech, signing a document that he says officially changes the name of this technology to super intelligence. We've changed the name, Steve, officially to S.I. Okay. S.I. like sigh, S.I. instead of AI. Do you know what that means? Super intelligence. In the meantime, a lack of regulation doesn't mean that there's no legal enforcement. The Federal Trade Commission has now opened an investigation into OpenAI, Anthropic and other AI companies looking at whether they can be held responsible when autonomous AI agents cause consumers harm. And a nonprofit is now suing OpenAI over its high profile hack of Hugging Face. I did also sit down with OpenAI CEO Sam Altman at the company's developer day in San Francisco, and I asked him about some of these recent safety issues. He told me OpenAI already feels some pressure to self-police and then hold back versions of this technology if they don't pass the bar on safety. Everybody wants great models. Well, actually, not everybody. Many people want great models. Not everybody? We want great models. But we have to do these in a way where we can make very confident safety cases and claims so that people don't have a bunch of anxiety and can rely on us for the models we're putting out into the world. I did talk to Altman before this accord was signed at the White House, but here's what he had to say about possible action from Washington. What is your expectation of what's possible in D.C. and what are the conversations looked like behind the scenes, especially on international coordination? I don't want to be too optimistic because I have been wrong before when I thought something was going to happen in D.C. and it didn't. But I do feel like people are taking it seriously this time, and I'm optimistic. I'm, in spite of previous experience, I'm optimistic. Altman also told me he expects a new liability framework around AI. So say one of these startups here builds a great app using OpenAI and something goes terribly wrong, whose fault is that? The liability framework is being debated a lot in the world right now. I suspect, like other industries, there will be a sort of multi-level thing. Like if you think about the auto industry, for example, if there's faulty parts, you know, you can be liable in some ways there. If the car manufacturer puts them together in a way that's outside of the spec of what they're supposed to do, then the car manufacturer could be. And if someone's driving drunk, it's their fault. So, depending on whether the fault is in the model and how it's used or someone misusing it intentionally, I assume we'll have like a new liability framework. And finally, did also sit down with OpenAI's CFO, Sarah Friar. She weighed in as well on this entire safety debate. Safety and alignment is at the core of what we do. It's always been at the core of what we do. There's no change to that. I think we're being more vocal at the moment because of what we've seen happen out there in the world. Astra is our most intelligent model, but also our most aligned model. AI has to be built for people. It has to be good for people. Part of that is there has to be safe. It has to be always under human control. It has to do what people want. And you want to be able to like, use it and be confident that nothing bad is going to happen. As the models have gotten more capable, the safety cases that we now use have to be held to a higher and higher standard.