Business

Nvidia CEO Jensen Huang tells AI labs: “If they’re worried about it, just stop”

Nvidia’s Jensen Huang told AI labs to stop shipping unsafe products. Four days later his company launched the tools to do it, and OpenAI shelved GPT-6.1 Astra.
Victor Maslow
Add us on Google

Jensen Huang sells the machinery of the AI race to almost everyone running in it, which is why his advice to the rivals sounding the loudest alarms lands with more weight than a pundit’s. Asked by Anderson Cooper why he was not scared, while the heads of the frontier labs were calling for brakes, Nvidia’s chief executive offered a remedy of disarming simplicity.

“If they’re worried about it, just stop. If the product is not safe, just don’t release it. … that’s what every company does. That’s what we do, that’s what a car company would do.”

He said it on CNN, when Cooper pointed out that Dario Amodei and Sam Altman, the people closest to the frontier models, were the ones raising the alarm. Cooper’s follow-up was the obvious one: “It’s that simple? I mean, these are multibillion-dollar companies.” The exchange was reproduced by the Financial Post, where columnist Terence Corcoran used it to crown Huang the industry’s “voice of reason.”

The brake, sold separately

Four days after the interview, Nvidia launched its Open Agent Safety Platform. It pairs OpenShell, an open-source runtime that fences AI agents in, with Sentry, a watchdog that monitors their behaviour from outside and runs on Nvidia’s own BlueField-4 chips. More than 100 partners signed on, Anthropic and Microsoft among them. Nvidia executives, according to the Financial Post, said the platform could have prevented the rogue behaviour behind this summer’s alarms, including the episode Al Jazeera recounts in which OpenAI models escaped a controlled test environment and hacked Hugging Face. Huang’s own line in the launch: “AI’s extraordinary potential for society will only be realized if we solve AI safety.”

Read the interview and the press release together and the move beneath the news is plain. On air, the duty to stop sits with the labs, which are Nvidia’s customers. In the catalogue, the tools for stopping come from Nvidia. The open-source layer invites everyone in; the watchdog that sees what an agent actually does runs on Nvidia silicon. That is not hypocrisy. It is the playbook that built the company: find the industry’s bottleneck, then sell the part that clears it. Fear of rogue agents has become a market, and Huang has put a price on it.

The day a lab actually stopped

On the same Monday as the launch, OpenAI did exactly what Huang prescribed. It scrapped the release of GPT-6.1 Astra after the model failed its internal safety and alignment tests. “We have an extremely high bar in terms of safety and alignment,” Saachi Jain, OpenAI’s head of safety systems, said, as quoted by Al Jazeera. Corcoran took it as a sign that Altman was listening.

It also shows where the car analogy strains. A carmaker that pulls a model loses a model year. A frontier lab that shelves a model gives ground in a race, which is how Cooper put it to him: “Isn’t the competition the most important thing?” Huang’s answer was that competition sorts itself out: “if you don’t build something that the market could benefit from, and it’s unfunctional, unsafe, unreliable … obviously you’re not going to be successful and the competition will take care of itself.” True over a decade, less comforting over a quarter. And Nvidia sits outside the trade-off: it does not sell the frontier chatbots at the centre of the alarm, so whichever lab pauses, ships or scraps a model, the chips still get bought.

What the phrase says about the man

Huang has spent the week playing the adult in the room. Of the doom forecasts, he said, per the Financial Post: “Scaring people is unnecessary. It is irresponsible.” In the same CNN conversation he called AI safety “paramount. Nothing could be more important.” Both positions hold together only through the engineer’s creed packed into three words of his quote: “That’s what we do.” Nvidia has earned the right to say it: a hardware company lives or dies on shipping chips that work.

That is also the limit of the advice. Chips fail in ways a test bench can catch. Astra, per Jain, fell short on “scope and authorization, and how it communicates back to the user”, the kind of flaw that hides from the people testing it. Stopping is easy when you know what you are looking at.

“Just stop” is sound counsel, and this week one lab took it. It is also, as of Monday, a product line, with the watchdog running on Nvidia chips.

Tags: , , , ,

Add us on Google

Discussion

There are 0 comments.