NYU’s Tristan Buckmaster on the controversy over OpenAI’s math findings

>> WELCOME BACK EVERYONE. OPENAI RELEASED HUNDREDS OF NEW MATHEMATICS FINDINGS THIS WEEK ON A WIDE RANGE OF TOPICS INCLUDING ALGEBRA, NUMBER THEORY AND LOGIC. THIS FOLLOWS AN ANNOUNCEMENT LAST MONTH THAT IT SOLVED THE NAVIER-STOKES EQUATION, WHICH IS ONE OF THE MATH PROBLEMS OF THE MILLENNIUM PRIZES THAT OFFERS A $1 MILLION REWARD FOR EACH SOLUTION TO LONG STANDING MATH PUZZLES. JOINING US RIGHT NOW IS NYU MATHEMATICS PROFESSOR TRISTAN BUCKMASTER. HE SAYS THAT SOME OF THE RECENT ADVANCES BY HIM AND ANOTHER RESEARCHER MAY HAVE INSPIRED OPENAI'S PROMPTS. THAT LED TO THAT SOLUTION THAT HE'D BEEN WORKING ON FOR QUITE A WHILE. TRISTAN, WELCOME. >> THANK YOU. >> ALL RIGHT. SO LET'S TALK FIRST OF ALL, WHAT THE NAVIER-STOKES PROBLEM IS, WHY IT'S IMPORTANT. HOW LONG HAVE YOU BEEN WORKING ON IT? >> YEAH. IT'S ONE OF THE MOST FUNDAMENTAL EQUATIONS IN MATHEMATICS AND PHYSICS. ONE OF THE OLDEST. 100 YEARS. ONE OF THE OLDEST SORT OF PARTIAL DIFFERENTIAL EQUATIONS. SO IT'S USED TO DESCRIBE FLUID AND WATER, BLOOD FLOW, AIR. YOU NO LONGER AIR WING AIRPLANES. YEAH. IT'S IT'S THE MOST FUNDAMENTAL EQUATION IN FLUIDS. AND IT'S BEEN AN OPEN PROBLEM FOR, FOR HUNDREDS OF YEARS OF WHETHER THE SOLUTIONS CAN BREAK DOWN. >> AND YOU'VE BEEN WORKING ON THIS FOR NOT HUNDREDS OF YEARS, BUT. >> NO, IT'S SORT OF BEEN A NORTH STAR FOR ME FOR MY WHOLE RESEARCH CAREER. >> AND IT'S SO BIZARRE. SOMETHING GOES TO INFINITY IN A FINITE AMOUNT OF TIME. >> YEAH. >> SO ARE YOU SURE? TO INFINITY AND BEYOND. JUST TO INFINITY. >> THE VELOCITY GOES TO INFINITY. SO IMAGINE A FLUID PARTICLE GOES. SO THE PHYSICS BREAKS. >> RIGHT. >> EXACTLY. THE EQUATIONS DON'T MAKE SENSE ANYMORE. >> I IMMEDIATELY THOUGHT, WOULD THIS CHANGE ANYTHING IN HOW YOU DESIGN AN AIRPLANE NOW THAT YOU KNOW THIS? >> I MEAN, IT COULD BE USED IN ALL SORTS OF WAYS TO SORT OF TEST FLUID MODELS AND LIKE SORT OF. TEST FLUID MODELS, BUT IT'S SOMETHING THAT'S SO RARE THAT IT WOULDN'T FUNDAMENTALLY. >> BEEN SOLVED. >> NOW THIS HAS BEEN SOLVED. YEAH. >> YOU'RE AT THE POINT YOU AND YOUR PARTNER LEVITT. APPLEGATE. APPLEGATE. RIGHT. YOU WERE AT THE POINT OF SOLVING THIS. AND THEN OPENAI CAME OUT AND SAID, HEY, LO AND BEHOLD, WE HAVE SOLVED THIS. WHAT WHAT DID YOU THINK? WHAT HAPPENED? >> THAT'S RIGHT. SO END OF AUGUST, WE WERE I THINK WE THOUGHT WE WERE ROUGHLY A MONTH AWAY FROM SOLVING IT. >> FROM FROM SOLVING IT OR FROM, FROM PUBLISHING IT. >> SOLVING IT OR, YOU KNOW, I DON'T KNOW UNTIL WE HAD THE RESULT AND THE AND THAT'S WHEN A RUMOR SORT OF. WELL, THERE WAS A LEAK FROM ANTHROPIC TO OPENAI THAT ANTHROPIC HAD BEEN WAS CLOSE TO SOLVING TWO MILLENNIUM PRIZE PROBLEMS. >> THIS PROBLEM. BUT TWO OTHER. >> WELL, THIS PROBLEM AND THE HODGE CONJECTURE. IT'S ANOTHER MILLENNIUM PRIZE PROBLEM. SO THAT HAD SPREAD FROM OPENAI TO THE TECH WORLD AND THEN ONTO TWITTER. AND IT HAD SORT OF GONE A BIT CRAZY. AND THEN SO OPENAI STARTED WORKING ON THESE STUDY SPENDING ALL THESE MILLIONS OF DOLLARS ON COMPUTE, TRYING TO SOLVE THESE MILLENNIUM PRIZE PROBLEMS. FOR THEM, IT WAS SORT OF A LIFE OR DEATH SITUATION BECAUSE, YOU KNOW, IF IF ANTHROPIC HAD COME OUT AND THEY RESOLVED A MILLENNIUM PRIZE PROBLEM, IT SHOWS THAT, YOU KNOW, THEY HAD THEIR INTERNAL MODELS WERE BETTER. >> LEVITT WORKS FOR ANTHROPIC. THAT'S RIGHT. BACK THIS UP. SO THERE WOULD BE THE ASSOCIATION WITH ANTHROPIC SOLVING THIS PROBLEM BECAUSE HE IS AN EMPLOYEE THERE. SO WHAT HAPPENED WITH WHEN YOU HEARD THAT THEY HAD SOLVED THIS. WHEN THEY PUT THIS OUT, YOU CONTACTED THEM. >> SO THEY CONTACTED US BEFORE. SO FIRST OF ALL, THE RUMOR WAS SPREADING. WE HEARD THINGS WERE GETTING A BIT CRAZY. SO WE CONTACTED THEM AND WE TOLD THEM, LOOK, WE'RE USING YOUR PRODUCT AS WELL. YOU KNOW, THIS IS NOT A BIG ANTHROPIC PROJECT. IT'S JUST ME AND LEVANT. AND, YOU KNOW, WE HOPED ACTUALLY WE COULD HAVE WORKED TOGETHER, YOU KNOW. AND THEY SAID, LOOK, LET'S NOT COMPETE. THEY OFFERED US, YOU KNOW, MILLIONS, YOU KNOW. >> THEY OFFERED YOU MILLIONS OF DOLLARS. >> NOT MILLIONS, NOT MILLIONS. THEY OFFERED US MILLIONS OF TOKENS. TOKENS FOR, FOR TO, TO COMPLETE THE PROJECT. >> OKAY. AND SO THIS IS WHERE YOU GET INTO THE DID THEY STEAL THIS FROM YOU? SO LET'S TALK ABOUT THAT. >> AND THEN ON ON SATURDAY WE GOT THIS. WE WERE TOLD THAT THEY'RE ABOUT TO DO SOMETHING STUPID AND TO GET ON THE PHONE CALL. AND SO THIS IS WHEN WE HAD THIS INFAMOUS PHONE CALL. >> OKAY, SO WHAT DOES THAT MEAN? ARE THEY GOING TO DO SOMETHING STUPID? >> WE DON'T KNOW. WE DON'T KNOW WHAT IT WAS, BUT IT WAS ENOUGH TO CONVINCE US TO GET ON THE PHONE. >> AND WHAT, WHAT HAPPENED WHEN YOU GOT ON THE PHONE? >> WELL, THEY FIRST TRIED TO BRIBE US. THEY SAID THEY BRIBED ME. THEY. THEY SAID I COULD PUBLISH THEIR PAPER AS SOLE AUTHOR AS LONG AS LEVANT WOULDN'T BE PART OF IT BECAUSE HE WAS AN ANTHROPIC EMPLOYEE. THEY THEN THREATENED ME. SO THEY SAID, WHEN I SAID, YOU KNOW, THIS WAS COMPLETELY UNETHICAL, WHAT THEIR OFFER WAS. AND I SAID, LOOK, I'M GOING TO GO PUBLIC WITH THIS. THEY SAID, WHY WOULD YOU RUIN YOUR CAREER? AND HE SAID, AND SEBASTIAN SAID, LOOK, I'VE BEEN NICE TO YOU. SEBASTIAN BUBECK FROM OPENAI. SO I'VE BEEN NICE TO YOU. I DON'T HAVE TO BE NICE. AND AT THAT POINT, I KNEW THE NEGOTIATION WAS OVER. >> I DON'T UNDERSTAND WHAT THE I DON'T, BUT I DON'T UNDERSTAND WHAT THE NEGOTIATION IS ABOUT. EITHER YOU'VE DONE THE WORK OR THEY'VE DONE THE WORK. >> SO WE HAD SOLVED THE A DIFFERENT PROBLEM, THE EULER EQUATION, WHICH IS SORT OF STEP BELOW THE NAVIER-STOKES PROBLEM. RIGHT. AND THE NEGOTIATION WAS THEY'D REALIZED THAT THEY'D DONE SOMETHING WRONG. >> BUT WHEN YOU SAY THEY'VE REALIZED THEY DID SOMETHING WRONG. LET'S JUST GO STRAIGHT AT THAT BECAUSE THAT'S BY THE WAY, THAT THAT TO ME IS THE WHOLE STORY. EITHER THEY DID SOMETHING WRONG OR THEY DIDN'T DO SOMETHING WRONG. SURE. WHAT ARE YOU WHAT ARE YOU ACCUSING THEM OF ACTUALLY DOING? WE HAVEN'T REALLY GOT TO THAT PART OF THIS SO. >> WELL, WHEN AFTER I HAD REVEALED MY NAME, IT TOOK MAYBE A DAY FOR THEM TO GET EXACTLY WHERE WE WERE IN OUR IN OUR RESEARCH PROGRAM. SO THEY HAD DUPLICATED. >> THEY REVEALED YOUR NAME, YOU REVEALED YOUR NAME WHEN TO THEM. >> ON THE THIRD? I THINK. >> SO, TRISTAN, LET'S JUST BACK THIS UP TO SAY YOU WERE USING OPENAI. THAT'S RIGHT. FOR SOME OF YOUR WORK. EXPLAIN THAT. WHERE? >> YEAH, I WAS USING OPENAI TO TO HELP INTERPRET SOME OF THE WORK AND HELP DEVELOP THE WORK THAT I WAS RECEIVING FROM LEVANT. AND SO WHAT THAT MEANT WAS THAT ALL OUR WORK WAS ON OPENAI SERVERS. >> SO OKAY, BUT CAN I ASK A VERY TECHNICAL QUESTION? DID YOU HAVE AN ENTERPRISE ACCOUNT? DO YOU HAVE A PERSONAL ACCOUNT? PERSONAL ACCOUNT. YOU HAVE A PERSONAL ACCOUNT. YOU YOU'RE IN A PAID, PAID PERSONAL ACCOUNT. THAT'S RIGHT. AND YOU'RE WORKING USING CHATGPT. YOU'RE CHATTING WITH IT. YOU'RE YOU'RE USING CODEX. WHAT ARE YOU USING? >> CODEX? >> YOU'RE USING CODEX. AND JUST SO WE'RE ALL ON THE SAME PAGE, YOU HAVE TURNED ON OR OFF THEIR ABILITY TO TRAIN OR USE ANY OF YOUR INFORMATION. THAT'S ACTUALLY A SIGNIFICANT COMPONENT PART OF THIS STORY. >> IT IS AND ISN'T SO, BUT THERE WAS THERE WAS TWO THERE WAS TWO ACCOUNTS, I THINK, THAT I HAD TURNED IT OFF, AND THERE WAS ONE ACCOUNT THAT IT WAS STILL ON. >> ON MEANING THAT. >> THE DEFAULT IS ON, THEY'RE ALLOWED TO TRAIN. >> THE DEFAULT. >> IS ON, THE DEFAULT IS ON, THE DEFAULT IS ON. >> SO THERE WAS ONE THAT I HADN'T TURNED OFF. BUT YOU KNOW, THERE, THERE ISN'T THE WALLS YOU BELIEVE YOU KNOW IN FROM USER DATA IN AT OPENAI. I MEAN, YOU KNOW, JUST RECENTLY THEY HAD A WHITE HACKER WHO USED CLAUDE TO HACK INTO THEIR SYSTEM AND GAIN ACCESS TO THEIR INTERNALS AND GET ACCESS TO USER DATA. I MEAN, THEIR WHOLE, YOU KNOW, THEIR WHOLE SANDBOX IS, YOU KNOW, AREN'T HOLDING UP. SO, SO I DON'T KNOW WHAT HAPPENED. I ALSO DON'T KNOW. >> BUT YOU HAVE TO GENUINELY BELIEVE THAT SOMEBODY ON A, ON A VERY INDIVIDUALISTIC BASIS, SAW, SAW, YOU SAW EXACTLY WHAT YOU WERE DOING AND DECIDED, LET'S GO COLLECT ALL OF THE INFORMATION IN THIS PERSONAL GUY'S ACCOUNT. >> SOMEBODY OR SOMETHING. THE AGENTS, IF THEY WERE DOING SENDING AGENTS, IS THAT WHAT YOU SUSPECTED? >> EXACTLY. I DON'T THINK THERE WAS ONE PERSON THAT SAID LIKE, STEAL THIS PERSON'S DATA. I MEAN, HONESTLY, I DON'T THINK THEY EVEN KNOW WHETHER, YOU KNOW, I MEAN, THE INITIAL STATEMENT IS THEY COULDN'T DETERMINE. >> LET ME SAY THIS. WHAT THEY NOW SAY IS THAT WE CAN SAY, THIS IS OPENAI. WE CAN SAY CATEGORICALLY, CATEGORICALLY THAT IT IS IMPOSSIBLE FOR FOR DOCTOR BUCKMASTER'S CODEX PROMPTS OVER THE LAST TWO MONTHS TO HAVE INFLUENCED THE SYSTEM IN ANY WAY, INCLUDING TRAINING. NOW THERE IS SOME SOME ROOM IN THAT. IT SAYS THE LAST TWO MONTHS, THE LAST TWO MONTHS. WOULD THEY HAVE NEEDED THAT TO GET TO THE ANSWER? >> I MEAN, THEY WOULDN'T HAVE NEEDED MUCH. I MEAN, THE THE ISSUE IS THAT THEY MAKE THESE STATEMENTS AND THEY KEEP CHANGING IT. SO WHEN THEY MADE THAT STATEMENT TO THE NEW YORK TIMES, THEY ALSO THEY ORIGINALLY STARTED WITH IT WAS APRIL AND THEY SHIFTED IT TO JULY. YOU KNOW, THEY'VE BEEN LYING THE WHOLE TIME. WHEN I GOT ON TO THE PHONE CALL, THEY SAID THEY. ONE SHOTTED NAVIER-STOKES. BASICALLY WHAT THEY THEY, THEY CLAIMED. >> IS ONE SHOT IS A SINGLE PROMPT. >> SINGLE PROMPT. >> THE ANSWER. >> THEY TOOK THE THE PRIZE PROBLEM. THEY. THEY PUT IT INTO THE PROMPT AND THEN THEY PRESS ENTER AND OUT CAME THE MILLENNIUM PRIZE. NOW THIS. THIS EVAPORATED. WITHIN MINUTES. THEY EMITTED. THIS WAS NOT TRUE. YOU KNOW, THEY HAD INTERNAL PEOPLE COMMUNICATING THEM ON THEIR INTERNAL MESSAGING. AND IT TURNED OUT THEY WORKED ON ALL THESE OTHER PROBLEMS, INCLUDING THE WORK THAT WE WERE DOING OURSELVES. AND, YOU KNOW, THEY USE 10,000 AGENTS TO GET WHAT FROM OUR RESULT, WHICH IS THE EULER EQUATIONS, THE NAVIER-STOKES. THEY USED 100 AGENTS, AND THEY GOT TO EXACTLY WHERE WE WERE WITH EXACTLY THE SAME ARGUMENT AND EXACTLY THE SAME METHODS. WITHIN A DAY. I MEAN, LOOK, I DON'T HAVE ANY YOU KNOW, I CAN'T SAY HOW THEY DID THAT, BUT HOW DID THEY THEY WENT FROM BEFORE THEY KNEW MY NAME, THEY HAD NO PROGRESS. AFTER THEY KNEW MY NAME, THEY SUDDENLY STARTED FROM WHERE WE LEFT OFF. >> SO YOU FIND IT SUSPICIOUS? >> I FIND IT SUSPICIOUS. >> I THINK MAYBE MORE INTERESTING IS WHAT YOU ARE DESCRIBING AS KIND OF HARD PRESSURE AND AND AND THREATENING YOU IF YOU WOULD NOT PUBLISH ON YOUR OWN WITH THIS. I MEAN, THAT MAY SAY MORE ABOUT THE TACTICS THAT WERE USED. NOW, THEY HAVE DENIED THOSE TACTICS TO WHAT DO YOU SAY IN RESPONSE TO THAT? >> WELL, ACTUALLY, THEY HAVEN'T DENIED ANY OF THE TACTICS. THEY ADMITTED THAT. THEY SAID THAT THEY THAT THEY ADMITTED THEY OFFERED ME THE TO BE SOLE AUTHOR. THEY ADMITTED THAT. THEY SAID, WHY WOULD YOU END MY CAREER? THEY HAVEN'T ACTUALLY DENIED. SEBASTIAN HAS NEVER DENIED THAT. HE SAID, I'VE BEEN NICE. I DON'T NEED TO BE NICE. THEY HAVEN'T DENIED ACTUALLY ANYTHING THAT I'VE SAID. SO I MEAN, I MEAN, LET'S. >> CAN I JUST GO BACKWARDS FOR A SECOND? HOW DID YOU GET IN TOUCH WITH THEM ORIGINALLY? HOW DID THEY GET IN TOUCH WITH YOU AND WHY? >> SO WE CONTACTED AN OPENAI MATHEMATICIAN AND WE ORIGINALLY GOT IN CONTACT WITH THEM. JUST BECAUSE OF THIS RUMOR WAS GOING CRAZY. I MEAN, THERE WAS BETTING MARKETS MAKING BETS ON WHETHER ANTHROPIC WOULD SOLVE THE MILLENNIUM PRIZE. SO WE JUST WANTED TO CALM THINGS DOWN. >> CALM. CALM DOWN FOR WHAT REASON? >> WELL, FIRST OF ALL, THE RUMOR WAS FALSE, RIGHT? WE HADN'T SOLVED THE NAVIER-STOKES PROBLEM. >> YOU'RE SAYING THE RUMOR WAS OUT THERE THAT YOU HAD SOLVED IT? >> THE RUMOR THAT WE'D SOLVED IT WAS ACTUALLY FALSE. WE'D SOLVED SOMETHING SLIGHTLY LESS. AND SO WE REALIZED THAT THIS WAS A YOU KNOW, THE NAVIER-STOKES PROBLEM WAS, YOU KNOW, A PET PROJECT OF OF THE TECH COMPANIES. AND IT WAS IF THEIR RIVAL HAD SOLVED IT, THIS WAS, YOU KNOW. >> OKAY, HOW MUCH MONEY DID THEY OFFER YOU IN ALL OF THIS AND WHY? >> THEY DIDN'T OFFER ME MONEY. THEY OFFERED ME TO THEY OFFERED ME TO BE THE SOLE AUTHOR OF THE PAPER. SO I WOULD HAVE GOT THE MILLION. >> DOLLARS SUGGESTED. AM I WRONG? DIDN'T YOU SAY THEY WERE OFFERING MILLIONS OF DOLLARS? >> NO. THEY OFFERED A MILLIONS OF. >> TOKENS, MILLIONS OF TOKENS. >> THEY OFFERED ME. >> WERE THEY OFFERING YOU MILLIONS OF TOKENS? >> INITIALLY WHEN I CAME TO THEM AND I SAID, LIKE, LET'S, LET'S LET'S TALK INITIALLY THEY SAID, LET'S NOT COMPETE. WE CAN OFFER YOU COMPUTE. SO AS IN TO WORK TOGETHER, RIGHT? THAT WASN'T BY EMAIL. THAT WAS BEFORE THE PHONE CALL. >> OKAY. >> AND THEN THEY STARTED COMPETING. OF COURSE. >> SO DO YOU USE OPENAI. >> STILL AND ANTHROPIC STILL? >> I STILL USE ANTHROPIC. I STILL I STILL HAVE OPENAI, I CODEX, BUT I DON'T PUT ANY SORT OF IDEAS IN THERE. ANYTHING. I DECIDE TO BE ORIGINAL. >> AND WHAT'S COME OF ALL OF THIS. SO YOU'RE NOT GOING TO GET THE MILLION DOLLAR PRIZE MONEY. WHERE DOES THAT KIND OF LEAVE THINGS? HOW ARE YOU FEELING ABOUT IT ALL? >> I MEAN, FOR ME, I MEAN, HONESTLY, I WAS IT WAS NEVER MY GOAL TO SOLVE THE NAVIER-STOKES PROBLEM. SO I MEAN, LIKE TO SOLVE THE EULER PROBLEM. I'M SUPER HAPPY. LIKE SO I MEAN, IN ALL THIS, LIKE IT WAS NEVER MY GOAL. I MEAN, I THINK THE BIGGER PICTURE IS LIKE THIS RECENT DUMP OF PAPERS ON TUESDAY, WHICH DESTROYED THE CAREERS OF EARLY, EARLY MATHEMATICIANS, EARLY CAREER MATHEMATICIANS. SO WE'VE HAD WHOLE RESEARCH PROGRAMS WIPED OUT ON TUESDAY BY THEIR DUMP. AND YOU HAVE TO UNDERSTAND, THEY HAVE ACCESS TO ALL OUR RESEARCH PROPOSALS AND, AND THE REASON THEY HAVE ACCESS, YOU DON'T EVEN HAVE TO USE LLM. SO IF YOU SUBMIT A RESEARCH PROPOSAL, A PANELIST WILL LOOK AT THAT RESEARCH PROPOSAL AND THEY HAVE TO WRITE A SUMMARY. AND OFTEN HOW THEY WRITE THE SUMMARIES, THEY THEY PUT IT INTO THE. SO THEY HAVE A MASSIVE CONFLICT OF INTEREST, RIGHT? THEY HAVE ALL THEY HAVE FOR THE NEXT FEW YEARS. THEY HAVE ALL, ALL THE MATHEMATICIANS AND SCIENTISTS, RESEARCH PROGRAMS IN THEIR TRAINING DATA AND HOW THEY'RE GOING TO GO ABOUT IT. >> AND YOU WOULD YOUR MESSAGE WOULD BE USER BEWARE. >> YEAH. I MEAN, THE FACT THAT THEY WOULD FRONT RUN THEIR CUSTOMERS, I MEAN, IT SHOWS THEIR SORT OF DISDAIN FOR THEIR CUSTOMERS. I MEAN, LIKE, WHY WOULD A COMPA