Quick Luddite Notes

🇨🇳🤖 Using the verboten NVIDIA (G)B200/300 to train Chinese AI

Will DePue, ex-OpenAI, created an X thread to share a discovery regarding “the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group” (hey, where is Alibaba?), which “have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.”

The full thread:

❶ I have a major result to share.

I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.

This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model.

Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves.

I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems.

I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used.

It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20.

These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs.

I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at https://unemploymentlabs.com/checkpoint-forensics.

There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.

❷ If you’re unfamiliar with how machine learning works: an AI model is truly just a very long list (billions, sometimes trillions) of numbers called weights, which, loosely, when multiplied and added together produce readable text.

Training adjusts those numbers over and over until the model gives good answers. Before training can start, the weights need starting values and those are filled in with random numbers.

Training changes most of the initial weights, but it doesn’t do so completely. Some are never touched at all and the rest still correlate with where they started. That is why the starting weights can still be read from the finished model.

You can see it with a simple test: draw a smiley face into a network’s weights before training and it is still there afterwards.

❸ A second test with a photo of me. I took a model’s ordinary random starting weights and arranged them once so that they show the photo. Then I trained the model and kept the arrangement fixed. The photo fades but the sky and the treeline can still be made out at the end.

đź“Ľ video 1

❹ The random numbers a model starts with aren’t truly random.

They come from a formula that starts from one starting number called the seed, and using the same seed gives the same numbers every time. Engineers set it up this way so they can repeat an experiment.

To fill billions of weights quickly, PyTorch (the software most labs use) splits the job across hundreds of thousands of threads running in parallel on the GPU. Each thread generates its own run of numbers and fills its own set of positions. The number of threads depends on how many processors the chip has. Hopper has 132, Blackwell has 148 and Grace Blackwell has 152. So the same seed lays the numbers out differently on each chip.

The animation shows this on two toy-sized chips.

đź“Ľ video 2

❺ That means I can take a published model, pick a chip and a seed, generate what the starting weights would have been and compare them to the real ones. With the right chip they line up and with the wrong chip they don’t.

This is that search on DeepSeek-V4-Flash. Every processor count gives noise until 148 (Blackwell), where the weights match.

đź“Ľ video 3

❻ Critically, MiniMax and Ant have not been publicly tied to Blackwell before. MiniMax’s models match Hopper through M2.7 in April and Blackwell from M3 in June. Ant’s own GitHub has benchmark logs from a GB300 dated March 31.

❼ Of course, this tells us nothing about where these chips physically are. They could have been smuggled into China or the labs are using them abroad through gaps that Congress has not closed. In either case, it’s clear that the export bans are deeply failing.

❽ Unfortunately, it’s of note that now that this is public any lab can erase many of these signatures. I’ve thought hard about whether to release this, and have been sitting on it for a while, but after consulting with advisors and corresponding with officials, I’ve decided it’s worth sharing today. I haven’t made this decision lightly.

âťľ If you’re interested in covering this story, please reach out to [email protected]. I’m happy to share more details. Multiple senior researchers, both independent and inside frontier AI labs like OpenAI and Anthropic, have confirmed the result.

❿For researchers, I’ll be sharing a GitHub repo and full paper soon with implementation details. You should be able to trivially replicate this fact from the details in this thread alone if you give this to Claude/Codex. Feel free to DM me with any questions.

⓫ There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon. Stay tuned.

I couldn’t care less about such shit.

“Oh, my, we should restrict the sales of NVIDIA’s most advanced Blackwell chips because they’re like enriched uranium!” Bullshit. The US wants to preserve an edge over China in AI, that’s all. If Xi were reckless and didn’t care about a major crisis at home, he would ban all exports from China for 6 months and watch the world die! China cannot manufacture everything, but the West is much more dependent on imports from China than vice versa.

OTOH, this Will DePuke (pun intended) is a bit retarded. “The top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group”? MiniMax is minor, and almost nobody uses Ant Group’s Ling! But everyone in the West uses Alibaba’s Qwen and Zhipu AI’s GLM!

Last but not least, he wrote this shit:

I never read e-mail newsletters myself, so instead I run things through an SMS list: Text DEPUE to 1-833-225-6887 to subscribe.

I had to ask Gemini to understand what this tard meant!

Gemini Hydrogen:

An SMS list is just Mailchimp or Substack piped through carrier text messaging instead of email—and it does not actually deliver a full newsletter inside a text message. It works as a glorified push notification: whenever DePue publishes a new blog post, an automated service blasts a one- or two-sentence teaser and a web link directly to subscribers' phones.

How it works under the hood

Why creators use it instead of email

Why it is a terrible deal for readers

For the person subscribing, replacing email (or RSS) with SMS trades convenience for the sender against basic usability for the reader:

I objected:

Then, this guy doesn’t use proper English. “I never read e-mail newsletters myself” doesn’t mean what he thinks it means. He still has to read the newsletters in a web browser! I prefer to read them in my Inbox (say, those from the NYT), rendered as if they were in a browser.

Gemini Hydrogen:

He is conflating the notification trigger with the reading environment. Saying “I never read email newsletters” implies he avoids the medium altogether, when all he has actually done is replace an email subject line with an SMS ping that still forces him into a web browser to read the piece.

If anything, the SMS-to-browser workflow is a downgrade from reading directly in an inbox:

It is a classic Silicon Valley habit: framing a regression in usability as a minimalist productivity hack, while quietly offloading all the friction onto the reader.