Is AI safety even a solvable problem? The White House’s new AI czar and Sam Altman’s comments

uh AI safety, right? We have to wait a couple months for Avengers doomsday, but the Avengers taking on the AI doomsday is already here. Uh the White House announced that there's an AISAR. Um a lot on their plate from, you know, concerns about AI safety to potential doomsday, but I want to start with an even broader question. Happy to open it up to the group. Is this even a solvable problem? Is this a possible test? Is this a fool's errand? Because this tech move so quickly, the people who are building it don't seem to know how to unpack how to, you know, build safety guardrails. How is the US government going to figure it out? >> I mean, I think if it was a solvable problem, the kind of current people who are being put in charge of it, they might not be the best placed to do so. Like, uh, when I when I think of people who would be very well placed to solve this, unfortunately, they're probably the same ones who are either developing the technology or screaming about the fact that, you know, there's a 10% chance it's going to kill us all. So I I think that uh you know is a is a bit of a a hangup here in terms of getting to a solution. I remember last time I was here at Yahoo it was Caleb Silver who said you know we didn't really put seat belts in cars until there were deaths. We didn't get you know drunk driving laws until there were deaths. So it probably means the best deterrent after a certain point of time is hey here's how badly we're going to punish you if things go wrong. But I think you know that would need the things going wrong part to happen first. >> Right. Lauren, what do you think? >> I think technology always moves way faster than government. And I think Jay coming in and making this binary rule of 120 days, what happens after the 120 days? The 120 days that doesn't mean that anything will be enforced. That doesn't mean that regulation will be an oversight. So really what what is the 120 days and do we need regulation over AI to a capacity? But we also know that AI always just moves a lot faster than what government could ever do. As a journalist, key to setting deadlines is make them a little bit loose and ambiguous so that if you don't hit them, you know, it's fine. Sean, I want to bring you in. What's your what's your take on the AISAR and and the task ahead? >> Listen, the AI principle already stepped in. Nvidia, Nvidia is a perfect example of a company trying to turn the problem into uh an actual infrastructure opportunity. And you saw that last week. OpenShell is going to create this kind of software boundary around the agent while Sentry is going to run on Bluefield 4. But regardless, you're seeing that all right, Jetson knows this is going to become an issue and rather than just waiting for someone else to create these guard rails. We're going to create the opportunity for everyone else and that's why you're seeing cyber security companies just continue to go up to the right. But yeah, like you guys mentioned, I mean, we have to make sure the narrative doesn't stick on slowing the AI buildout, just making sure it becomes more secure and letting the progress continue. I think that Jensen stepped in at the right time. >> Oh yeah, sure. >> Right. There's so far from from what I've seen this is like the first uh real steps or things we're taking towards in terms of trying to regulate the the outputs. So far like it seems everything's been controlling the inputs, people not liking data centers in their community and so forth. So this seems like it's a really kind of nent tiptoeing into an area that's so far kind of been wild west which means you know we're probably going to get things wrong but also means I think uh mostly what we've seen from the president is there's absolutely no desire to fall behind. If you had to kind of pick one single message from the president and AI, I think would be your leadership and anything that threatens that is bad. >> Well, also I think that right now for AI like all the AI issues have crossed the boundary of like that shouldn't have happened. They've been oopsies. They haven't been intentional. So I think that's where like you have to make sure you have some guard rails before it becomes real external threats where someone's actually trying to attack these systems and these autonomous systems are getting more advanced. Like I think it's really important right now to do it before uh whatever financial health care system all of a sudden gets violated and now we have to go back five or 10 years and who knows who was behind that attack aka >> some other country or entity. >> I love AI oopsies. I love it. I can just picture the OpenAI blog. We had an oopsie. Sorry. [laughter] But I think that gets to a bigger question, right, about this like what level of comfort do we have sacrificing some safety in for the sake of of innovation? Luke, you brought up like the the the um seats, you know, seat belts with cars. Um you know, and I'll bring in Sam Alman here. We could talk a lot about him. He had a big Vanity Fair piece um today. But kind of one of the many things that he said recently um you know is you need to accept some bad things happening to reap the benefits. So it's the old like you got to crack a couple eggs to make an omelette. I I I do agree with that. But at the same time if you know way back when when they were building the Model T if Henry Ford was like listen some of our cars are just going to blow up. It's you know it's going to happen. I think a lot less people will be in cars. So, I guess, you know, Lauren, where where do you see the balance as far as an acceptance of taking some amount of risk versus not kind of throwing the baby out with the bath water, so to speak? >> I [snorts] mean, I think the question is what is considered bad, >> you know, and and how do we go from there? Because is it bad? Is it bad for the consumer or is it bad or or is it good for the investor? But but the consumer is the one that reaps the the terrible not even terrible benefits but gets the the bad things happening to them, >> right? >> Well, is it bad if if is it bad if my like my address my data gets exposed from someone who's doing a search on me or something like that? Like I I would consider that bad if an agent did that. I would say hey we should probably punish that agent or I should get some kind of remuneration from that agent. So like yeah like I I see the scope already for things that have been you know bad and yet still the underlying technology might be you know very useful from a consumption point of view. You know Muse is the one who uh you know let me know when my emails comes in of the of the topics even for for what we're discussing today. So you know kind of always useful having the the personal assistant. >> Right. Yeah. Se I don't know if you have thoughts on like where what's your comfort level as far as taking a risk taker. [snorts] Listen, I think AI AI has such a PR issue and the unfortunate part is that the two speakers are children. Like Sam and Daario are just children. They should not be like the figures of this once in a generation movement. But unfortunately, it is. So I think that for Sam's comment, for example, specifically like he was just talking about zero risk AI versus manage risk AI, but the way he presented it was just so poor and the standard is just like not he was like people are going to use AI for scams. Nobody's going to ever hack something with it. Of course, they're going to. So, I think that there is going to be case studies, but it's going to be the minority stake. Like, it's not going to be the bulk of them. Like fire. Like, are we not going to create fire just because someone might uh launch something that's going to cause a catastrophic boom? No. You just have to contain it and make sure there's things in place in order to make sure it doesn't become catastrophic. >> So, I want to um to the the regulatory aspect of it and and how we kind of manage the risk. I'm gonna do a little uh jump in the in the time machine. Go back to 2010 and I'll make this story quick, but there's this thing called the flash crash. Some of you might be familiar with it, might have remembered it. Market kind of drops and then comes back in a couple seconds. To make a long story short, people freaked out. They were like, "Oh, what's going on with this algorithmic training and a lot of regulation, a lot of things happen. Michael Lewis wrote a book about it." But specifically this thing came called regulation at came out by the CFTC and part of it was we want all the source code for all the algorithmic traders and we want to be able to hold it so we can understand who are these people playing in these markets. Now the reason I bring this up is because people had big issue with it at the time right this is proprietary stuff that these firms were making a lot of money on. How this relates to AI is if there's some type of regulation and the regulation becomes well we need to really look inside your models. We need to understand these black boxes. I wonder how comfortable, especially some of these people that have been screaming to say we need more regulation are going to be to hand over the keys to what are according to their investors trillion dollar valuation type companies. I I that's where I think you know what I brought up in the beginning. I think there's a real issue as far as being able to actually regulate this stuff because ultimately you're going to have to give the keys to the castle to people that are the castle's worth trillions and trillions of dollars. But I don't maybe I'm not looking at it right. What do you think? >> Isn't Isn't this kind of what they're asking for though? Like isn't this what isn't this the point of Dario's pacing the frontier like please please invite someone in make it someone you know independent but probably third length kind of at least government sponsored in some way or who's looking for government backing of this to to come in and basically say like give us give us the all clear serve as the red team effectively in terms of you know testing any new models we might want to come up with poke holes in what we've done be embedded it seems like you know to a large extent this is what they're asking for whether or not they'll they'll get it from Jay Clayton another remains to be seen, >> right? Yeah. I don't know, Shai or Lauren, you have your thoughts on on how to kind of manage this risk or how to regulate this. >> Well, I I think uh Jensen's doing it for us, acquiring Hugging Face, pushing on open source. Like again, Sam and Dario, I don't want to pick on them, but like they're doing this out of defense. Like if the open source narrative gains momentum, like they have a duopoly right now in a frontier model that has no ceiling. So I think that right now it's really important to put more focus on technical standards that make these companies actually implement these type of um AI issues because if you can create rules around agent permissions, cyber security testing, external enforcement layers for example, like doesn't matter about giving up the juice that anthropic openai has, all that matters is we cannot slow down American companies from moving quickly on an AI race. That's the number one northstar for across the board. But Dario and Sam, they're they're doing a road show right now for their IPO. So, they are saying some more self-fulfilling uh prophecies. But Jensen stepping in again uh and making sure you can see the smoke between the fire. >> So, is your your pitch is that open weight's the future, open weights the way to kind of avoid a lot of these these issues. Basically, [snorts] >> it will be because we're going to see the weight and parameters of these models. There's going to essentially you can customize it per enterprise use case. You're going to see all the nitty-gritty of it. And that's how every revolution has gone by anyway. like software specifically cloud like everyone starts with open source then creates these managed source applications that's benefit for your own company but right now there's the exponential rate of AI anthropic and open AI have gone from 0 billion of revenue to hundred [snorts] billion in a couple years that's never happened in existence before in any revolution so I think that right now things are happening too quickly and there is somewhat of a prisoners dilemma that's happening that they aren't caring as much as they should as the simplicity of what should be good for AI or not.