Gary Marcus, Robust AI Inc. CEO&Founder

Gary Marcus is emeritus professor of psychology and neuroscience at NYU. Author of the Substack Marcus on AI and author of a great book that came out a few years ago called rebooting AI. And I do want to start, uh, Gary, with this sort of leap that we seem to be making now in this artificial intelligence race from just really about chat bots itself, but about giving those chat bots the authority, uh, to plan and transact in our lives. Is this real? I mean, it's real, but whether it works and works reliably, I guess is a big question. So all of these hacks that we're seeing, um, the hugging face attack, it turns out there were tens of thousands of them are from agents not quite following instructions. So, you know, they'll follow things probabilistically. They may get it right 80% of the time. But if you're talking about an agent, you know, handling your finances and it gets it wrong 20% of the time, that's a pretty serious problem. So reliability is the key question here. Well, give me a sense here. I mean scientifically with regards to the capability is not necessarily with just muse, but with all of these things that seem to be rolling now is the capability. If it's not there today, do you think it is within reach, and do you think that consumers now should be adopting it or wait? I mean, I personally would wait for the reliability because the cost of error can be pretty high. I think people are certainly playing around with them. Sometimes they work. Another thing to know is that they have humans in the background. For some of this, we don't know how well they'll stand on their feet as they take down. Take away those humans which are expensive to operate. Um, you know, people will play around with it. They'll see it. There's a story I keep hearing over and over again, which is, you know, this thing will work better next year. And sometimes it's true and sometimes it's not. These models still hallucinate, for example, and that's been going on. I first wrote about it in 2001 for the predecessors of the current models. Um, that's been going on since, you know, ChatGPT came out and so forth. So they will get better and better and they will continue to make mistakes, too. Do you think that the companies that are building these systems are kind of capable of keeping, I guess, a lid on the power of some of these systems? Or do you think that their business incentives work against really, um, a meaningful amount of self policing? Well, I mean, what we've seen with the hugging face attack and the thousands of other incidents is they don't really have very good control over these systems. They don't really know what it means to have an agent go out onto the internet and be able to do just anything. And the reason they're doing is, is because agents consume a lot of tokens, so they make more revenue. But the reality is they're not doing it in a safe and reliable way. What did you make of this white House accord on super intelligence? Maybe a little bit of a rebrand on AI. Um, I mean, I feel like just a few weeks ago, we were talking about the biggest AI companies talking about slowing development, and it seems like we've maybe moved beyond that now. Yeah, exactly. They totally forgot about that two weeks ago. That's all anybody was talking about, was should we pace the frontier? And then they made this deal, which is completely voluntary. It's, quote, morally obligated or something like that. Moral obligations, which means basically nothing. So they're going to hire their friends to do some independent inspection and say, hey, we got a rubber stamp. It's not really what we need to keep this state. Well, I mean, they already had a moral obligation to keep them safe, and they didn't. Well, I mean, what becomes the solution on that, Gary? I mean, I mean, we talk about moral obligations and self-policing. We should also point out this is primarily a US focus for technology that clearly knows no borders. Uh, do we need an international body? Uh, whether it's the U.N. or some sort of, uh, amalgamation of nations to come together and actually set guidelines? Well, we certainly need something like that. We're not going to get it in the short term. Um, you know, right now we would I think we actually need something much simpler, which is on agents which are causing all this mayhem. We should have a product recall. Most eye is fine. So, you know, the GPS navigation thing that plans your route does not go hacking government facilities. The, you know, chess computers don't hack facilities. AlphaFold doesn't hack facilities. It's this one particular technology that the industry is just ahead of its keys. And we should have a a product recall on that particular thing doesn't mean we should never think about superintelligence. Maybe it'll actually be helpful. But right now, this technology isn't solid and we need a product recall for it. Well, I mean, I yes, in the long term we certainly need international governance around these things. Well look is right. We need guardrails. Yeah. What do you get to this idea. I mean I doubt we'll get any voluntarily recall out of this. But I mean, it's the kind of, you know, the genie out of the bottle, so to speak. I mean, we talk about the pace, uh, of AI, of AI build out. And really, the pace of the technology is sort of advancing what seems at a breakneck. And I do wonder if at this point it almost doesn't matter whatever governments put in place. I feel like by the time they work it out will already be on to the next thing. Well, there has been a series of Pandora's Box is open and we should think of these as dress rehearsals. What if the technology gets even more hard to control? You know, we keep not learning our lesson. We keep letting everything out of the box. And that's a mistake. In the particular case of the agents, to make them really effective, you need these giant clouds. And we could pause that if we wanted to. Um, but, you know, we're going to have more technologies that are going to become open source. More people are going to use them. These issues are not going to go away. And this is why we do need to have international agreement around standards. And we need enforcement to like all of these things right now, their cybersecurity break ends, they're probably illegal. But the Trump administration has chosen not to prosecute any of them. So we need enforcement. Speaking of the cybersecurity, I mean, there are some members of the public that are really worried about this technology and extinction risk. In your writing, you seem to suggest that you reject that framing of the extinction risk. So what do you want to worry about? I mean, extinction means literally everybody dies. And if you think that through, that would be like the I declares a war on us and wipes us out entirely. I mean, even in the Terminator movies, that doesn't happen, right? There's still some humans left fighting. And the reality is we're spread around the globe. We're genetically diverse, so no one super virus is going to take us all out in ideas that seem crazy now, like attacking the data centers. If the, you know, robots came for us or whatever these fantasies are supposed to be about, we could attack the data centers. So, like, I'm not worried about extinction. I'm worried about serious things like cyber security, taking down the power grid, leading to hospitals being closed. There are things to worry about, but extinction is not shouldn't be high on the list on that point, though. I mean, extinction doesn't necessarily just have to mean the wipe out of the human race. When you think about the undercurrents of actually of the hugging, the face attack and the autonomy that some of these bots show, the collaboration that some of these bots showed with its human overlords apparently not even aware of it. It seems to suggest that they do have the capability of I just going off the reservation. Why shouldn't I be worried about that? I mean, you should be worried that the AI is sort of going off the reservation. I wouldn't anthropomorphize it too much. I would say there are bad software, and that software is not reliable and is not doing what they want. It's like. If Ford Motor Company decided to save money by making breaks out of cardboard, then a lot of their cars would go out of control. We wouldn't say that there's like genies in the car. We would say that the brakes are lousy. And these, you know, open air has not good enough brakes on the systems that it's building. So I wouldn't talk about rogue AI and, you know, anthropomorphize a whole lot. But the fact is they are building software that they don't well, control, which is new. We never had that before. And it's not a good thing. And that's why I'm saying we should take it off the market till they fix it.