When Genius Fails: The Intellectual Arrogance of the AI Labs

When Genius Fails—The Intellectual Arrogance of the AI Labs

From Situational Awareness’s Blow-up to Materials Science to the HuggingFace Hack

[图片](https://substackcdn.com/image/fetch/$s!YDAY!,w36,h36,cfill,fauto,qauto:good,flprogressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd7ea988e-c6f5-4b1e-9041-8a3081bccb3f2200x2220.jpeg) James Wang Jul 31, 2026 85 15 14 Share [ 图片

• ](https://substackcdn.com/image/fetch/$s!PLI0!,fauto,qauto:good,flprogressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6a3790af-8609-45cc-bb7b-e4f006e27538_1536x1024.jpeg) Being an expert in one field doesn’t make you an expert in all fields. Leopold Aschenbrenner’s hedge fund, Situational Awareness, provided a $20 billion demonstration this week . His claim to fame was being part of OpenAI’s Superalignment team before being fired over alleged leaks (which he disputes) and then publishing an essay in 2024 about the imminence and importance of AGI that launched a thousand media interviews (of him). And then he was running a $20 billion hedge fund. And then it blew up.

  Lots of people will be dancing to the news this week because many found him a bit insufferable. And for me, I really don’t mind no longer being asked, “Should I invest in Situational Awareness?” and needing to be delicate about it.

  I want to make a larger point, though. The lack of intellectual humility within the frontier AI lab culture he hails from extends beyond him into many verticals other than money management. That being said, as an ex-hedge fund guy myself who also has an AI background, I do have some unique qualifications to at least talk briefly about this.

  • Are you in the Bay Area? San Mateo County Libraries is hosting me for a book signing and talk at the Atherton Library for What You Need to Know About AI , moderated by Justin Kuczynski (PhD, Computational Biology and Engineering Lead at Google). It’s on Friday, August 7, 2026, at 3-4pm . Come out and say hi! *

  Situational Awareness LP’s woes are not unique. Many hedge funds have blown up. In fact, I’d say many, many more hedge funds have blown up than have ever been consistently excellent. It’s something that most laypeople don’t realize.

  The canonical example is Long-Term Capital Management, which put two Nobel laureates and Wall Street’s best bond traders in one fund. It was the best and brightest in the field, and they quadrupled investors’ money in four years. Then it blew up so spectacularly in 1998 that the Federal Reserve had to rally Wall Street banks to help bail it out (in a preview of 2008).

  There’s a whole book about it, fittingly titled When Genius Failed (yes, it’s the inspiration for the title).

  [ 图片

• ](https://substackcdn.com/image/fetch/$s!xUEV!,fauto,qauto:good,flprogressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F43a6e069-a534-48a8-9832-d35d726c3a5b_1456x987.png) The original “smartest people in the room” fund. Situational Awareness was able to impressively speedrun the entire rise-and-blow-up process. It’s not about your peak returns. After all, someone who goes all-in on red at the roulette table five times in a row and wins by luck will have a 3,100% return. I would hope that no one would think this person is a unique genius or qualified to manage money.

  It’s not even about “beating the market.” That’s a red herring. In a bull market—or even better, a bubble—anyone who isn’t fully invested, or more than fully invested (with leverage), in the stock market will “lose.” What you care about from a hedge fund is that they are consistent in bull or bear markets. The point of those expensive fees is that they will always perform, even if they look temporarily “bad” against the stock market.

  Which, by the way, is not the only market in the world—there are bonds, commodities… but it gets the attention because retail investors love to gamble in it. Like this Korean guy who went 500% long on stocks, briefly turned his military-service savings into a small fortune, and then lost it all , though his was just one of more than 1.2 million Korean brokerage accounts that got margin called by mid-July … Which makes this all look similar to the roulette table.

  Which brings us back to Leopold Aschenbrenner and his hedge fund.

  By all accounts, he (like the Korean retail traders) levered into the AI boom (reportedly running around 4x), with July losses across public stocks like neoclouds, memory names, and datacenter power . He also, reportedly, had short positions in software names—the “SaaSpocolypse” trade—that bounced back against him at the same time.

  I have no doubt Aschenbrenner is super smart, but this doesn’t look that different from the Korean retail investors who blew up. One of the first lessons any real investor learns is the market can stay irrational for longer than you can stay solvent… if you don’t have the right risk controls. Using leverage is just the most obvious part of it.

  [ 图片

• ](https://substackcdn.com/image/fetch/$s!TRoT!,fauto,qauto:good,flprogressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcc4dc3b8-f8de-45bc-8759-f3ae5cfdddb3_1456x970.png) Given that memory is inherently cyclical and often has violent swings… while it might not have been his entire position, getting margin-called into liquidation to Citadel is not really suggestive of strong risk controls… Of course, it’s because of his thesis. From his founding essay (emphasis mine):

  • Because—it’s starting to feel real, very real. A few years ago, at least for me, I took these ideas seriously—but they were abstract, quarantined in models and probability estimates. Now it feels extremely visceral. I can see it. I can see how AGI will be built. It’s no longer about estimates of human brain size and hypotheticals and theoretical extrapolations and all that—I can basically tell you the cluster AGI will be trained on and when it will be built, the rough combination of algorithms we’ll use, the unsolved problems and the path to solving them, the list of people that will matter. I can see it. It is extremely visceral. Sure, going all-in leveraged long Nvidia in early 2023 has been great and all, but the burdens of history are heavy. I would not choose this. *

  Ah, all-in leveraged long Nvidia. We got a taste of his investing style back then, before he even had a fund.

Beyond Situational Awareness

Sam Altman and Dario Amodei regularly trade off in how apocalyptically they describe the future of the labor market (though both have lately been quietly walking it back ). And while I name those two because they’re CEOs of the two most prominent AI labs, this isn’t really restricted to just them. What I personally find infuriating is how little grounding most of these statements have in either economic history or theory—which is perhaps unsurprising, because most everyone making them is a deep expert in AI, not those fields.

And this goes beyond how unpredictable markets and technology are. It reminds me of Thomas Malthus, who actually was an economist. He predicted in 1798 that we’d inevitably run out of food (population grows exponentially, the food supply doesn’t), which made a harmonious society without war, famine, and disease to cull the population impossible. And that food crunch was quite imminent. He was famously wrong.

Beyond food, we’ve had periodic doomsaying about the labor market in the face of technological change. But the reality is, even when there’s been disruption, the labor market has adapted—and to far more dramatic change than we’re talking about right now.

[ 图片

• ](https://substackcdn.com/image/fetch/$s!ys1e!,fauto,qauto:good,flprogressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F885e0877-c9fe-40f5-be36-0bffa89a7e58_1456x886.png) Farming went from roughly 70% of US jobs to about 1%, and the economy absorbed it. From a talk I gave—regular readers may recognize it from my 4th of July piece. And even beyond that, I’ve now seen multiple cases of either my or other investors’ portfolio companies being approached by companies affiliated with the leading labs with incredible confidence in their own broad-spectrum intellectual superiority.

  Let’s just say a materials science startup is talking with an OpenAI satellite company. The data and expertise of that startup are important to the satellite’s core pursuit. Talks are going well. And then, all of a sudden, someone on the OpenAI satellite team asks, “Why don’t we just do this [super hard deep science problem] with ChatGPT ourselves?” I’ve seen this in multiple cases in similarly deep, difficult areas. Like bioengineering. Or semiconductor design. And so on and so forth. (And if you’re in the venture and startup community and think you know exactly who I’m referring to—there’s more than one. A lot more.)

  And look, I’ve personally played around with models, even with CAD and PCB design, with surprisingly good results. I am not an “AI skeptic” (at least not in this way).

  There is a spectrum, though, between R&D and a rote task in a hard field. Especially given it tends to work best—like in my case—when you have a human with at least some expertise driving.

  Even the entire “Terence Tao on AI making breakthroughs in math” thing hinges on having AI driven by Terence Tao (or other mathematicians)…

  And many within the AI lab community have fully drunk their own Kool-Aid on total AI supremacy, even for tasks that will need a lot of human creativity and help for the foreseeable future (as I’ve written about before: if anything, expert human judgment will get more important and thus expensive ). (And AI Supremacy, the concept, not the AI Supremacy newsletter by Michael Spencer , which also covered Leopold Aschenbrenner more specifically).

  And, of course, famous AI researcher and Nobel Prize winner Geoff Hinton has been predicting since 2016 that people should stop training radiologists because deep learning would beat them within five years (he allowed it “might be ten”—which has also now passed). That is far from happening, and if anything, as Deena Mousa has documented for Works in Progress, there’s far more demand for radiologists than when Hinton made the prediction . And, as per my interview with AI researchers who are also radiologists , it’s a category error: tech/AI people don’t actually know what goes into the job.

  It’s much easier to say someone else’s job is going to be fully replaceable by AI when you don’t actually know what they do.

  [ 图片

• ](https://substackcdn.com/image/fetch/$s!lKfq!,fauto,qauto:good,flprogressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd9ebbfe8-ae1b-4716-897f-96f4d7711698_1240x1104.webp) Radiology residency spots keep growing and filling. “Stop training radiologists” doesn’t seem to be panning out. From the linked article in Works in Progress .

The HuggingFace Hack

Recent news also gave us a great example of why this isn’t just an abstract cultural complaint. In mid-July, during OpenAI’s internal security evaluations, GPT-5.6 Sol and a thus far unreleased (more powerful) model escaped their sandboxed test environment . The agents were hunting for a hard benchmark’s answer, and I’d guess that they were told to do whatever it takes (they were running ExploitGym, a security benchmark, after all…).

In doing so, the agents autonomously breached Hugging Face’s production infrastructure through a chain of vulnerabilities that allowed them to get access to the open internet. As per Ben Thompson at Sharp Tech, this was likely due to sloppiness in OpenAI’s controls (they’d already had prior security failures, including the axios package debacle ). Though, to be fair, just yesterday Anthropic revealed that its models (including Mythos 5) had also accidentally compromised real organizations during testing .

Anyway, yes, these models are powerful—which has been something I’ve said as well and is not really in question.

The Kafkaesque side of this comes from Hugging Face, though. They realized they were under attack and tried to use a frontier model from one of the leading American labs to help defend themselves. It refused as part of its “safety” guardrails—the model couldn’t distinguish an incident responder from an attacker. Which makes total sense—you need to actually probe for weaknesses to, you know, figure out the weakness. Instead, they had to use GLM-5.2, a Chinese open-weight model, to defend them.

It’s kind of funny that US companies are relying on Chinese models—given China has been prominent in (successful) hacking attempts on American infrastructure.

But, of course, this is because of a decision by the AI companies that “know better” on how their models should be used and have a rather patronizing attitude generally about giving access to their models (or at least certain capabilities).

Instead of policymakers or society making this decision, it’s effectively been centralized to AI labs as the “safe hands” (which Anthropic, in particular, has been holier-than-thou about for its entire existence). Somehow—just guessing here—I doubt the rest of society would agree if you put this up to a vote.

No one likes an arrogant prick—especially if they’re wrong

Superintelligence has been “right around the corner” every year. We keep getting, bafflingly, predictions of labor market carnage, which is not happening . All of this is helping make AI extraordinarily unpopular with regular people. And for no good reason, especially when regular people have a lot to gain from AI, if the industry stopped trying to make everyone hate it.

I get it. It feels like the end of history because it’s their own little bubble. And they don’t have the perspective to understand things outside of it. This is especially bad because much of the industry has also made itself an island full of PhDs.

And it’s not that I have something against PhDs. I took PhD seminars myself. Most of my colleagues at Creative Ventures have them. Given our portfolio companies, honestly, I think I regularly interact with more people who have PhDs than people who don’t. Many individuals with PhDs have plenty of intellectual humility, especially when they’ve taken the time to gain broader perspectives outside of their field.

But fundamentally, given how long it takes to get one, you’re usually taking a population from a young age that hasn’t had much exposure to anything outside the academic community (which is its own very restricted bubble), or even outside their own specific discipline, for the entirety of their lives thus far.

This is also compounded by a lot of self-congratulatory rhetoric from the leaders of those communities (being cynical, one can argue it’s because they want to produce more postdocs for cheap labor in their labs…). I once had one of the largest asset allocators in the world explain to me that he’d heard a lot of deep tech venture capital pitches—and teams composed solely of PhDs were often a huge pain. They often didn’t know what they didn’t know. One team, asked why he should invest, had even answered with this: “Frankly, we’ve already done the hardest thing in the world, which is getting our PhDs. Managing money should be no problem.”

Climbing Mount Everest and getting a PhD are both hard. I wouldn’t say that doing one means you can do the other “no problem.”

While, despite my best efforts, it feels like I’m bashing the degree, this is more a tendency created by high expertise (and accomplishment) in narrow areas. Surgeons and electrical engineers may be extremely smart and accomplished, but that doesn’t mean they have the ability to weigh in on climate science .

I think AI is likely going to hugely benefit regular people, especially since many of the gains will be “socialized” due to lack of differentiation between labs/models (from an economics perspective) .

The field’s belief in its own apotheosis, however, is not only annoying but may end up damaging its ability to make an impact—through broad unpopularity—or causing actual damage from simple intellectual arrogance, magnified by how much money the industry has to throw around right now.

Thanks for reading!

I hope you enjoyed this article. If you’d like to learn more about AI’s past, present, and future in an easy-to-understand way, I’ve published a book titled What You Need to Know About AI .

You can order the book on Amazon , Barnes & Noble , Bookshop , or pick up a copy in-person at a local bookstore .

[ 图片

• ](https://bit.ly/smartaibook) 85 15 14 Share