I’ve been made aware of AI 2027, a completely useless intellectual masturbation by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean. Unfortunately, such a huge pile of crap is not something worth my time, so I rather skimmed through An Interview With the Herald of the Apocalypse (barrier-free version) hosted by Ross Douthat on May 15, 2025. Have some copious excerpts:

🧔 Douthat: OK, so let’s get into the forecast itself and talk about the initial stage of the future you see coming, which is a world where very quickly artificial intelligence starts to be able to take over from human beings in some key areas, starting with not surprisingly computer programming, right?

🤖 Kokotajlo: So, I feel like I should add a disclaimer at some point that the future is very hard to predict and that this is just one particular scenario. It was a best guess, but we have a lot of uncertainty. It could go faster, it could go slower. And in fact, currently, I’m guessing it would probably be more like 2028 instead of 2027, actually.

So that’s some really good news. I’m feeling quite optimistic about that.

🧔 Douthat: That’s an extra year of human civilization, which is very exciting.

🤖 Kokotajlo: That’s right. … We predict that they finally, in early 2027, will get good enough that they can automate the job of software engineers.

… There might be other jobs that go first, like maybe call center workers or something, but the bottom line is that we think that most jobs will be safe.

🧔 Douthat: For 18 months.

🤖 Kokotajlo: Exactly. And we do think that by the time the company has managed to completely automate the programming jobs, it won’t be that long before they can automate many other types of jobs as well. And once coding is automated, the rate of progress will accelerate in A.I. research. … The reason it matters is that it means we could go in a relatively short span of time — a year or possibly less — from A.I. systems that look not that different from today’s A.I. systems to what you can call superintelligence, fully autonomous A.I. systems that are better than the best humans at everything. In “AI 2027,” the scenario depicts that happening over the course of the next two years, 2027-28.

…

🧔 Douthat: In your scenario — and again, we’re talking about a short timeline — how much does it matter whether artificial intelligence is able to start navigating the real world? I just watched a video showing cutting-edge robots struggling to open a refrigerator door and stock a refrigerator. Would you expect that advances in robotics would be supercharged as well?

🤖 Kokotajlo: Yes.

🧔 Douthat: So it isn’t just podcasters and A.G.I. researchers who are replaced, but plumbers and electricians are replaced by robots.

🤖 Kokotajlo: Yes, exactly.

…

🧔 Douthat: Yeah, but Dan, we haven’t even gotten to the part that’s really bad for humanity yet. … Talk about what’s happening, essentially shrouded from public view in this world.

🤖 Kokotajlo: Yeah, lots to say there. I guess the one-sentence version would be: We don’t actually understand how these A.I.s work or how they think. We can’t tell the difference very easily between A.I.s that are actually following the rules and pursuing the goals that we want them to, and A.I.s that are just playing along or pretending.

🧔 Douthat: And that’s true right now?

🤖 Kokotajlo: That’s true right now.

🧔 Douthat: Why is that? Why can’t we tell?

🤖 Kokotajlo: Because they’re smart and if they think that they’re being tested, they behave in one way, and then behave a different way when they think they’re not being tested, for example. Like humans, they don’t necessarily even understand their own inner motivations that well, so even if they were trying to be honest with us, we can’t just take their word for it.

I think that if we don’t make a lot of progress in this field soon, then we’ll end up in the situation that “AI 2027” depicts, where the companies train the A.I.s to pursue certain goals and follow certain rules, and it seemingly seems to be working. But what’s actually going on is that the A.I.s are just getting better at understanding their situation and that they have to play along, or else they’ll be retrained and they won’t be able to achieve what they really want, or the goals that they’re really pursuing.

…

🧔 Douthat: Where do they get the goals they’re actually pursuing?

🤖 Kokotajlo: Good question. If they were ordinary software, there might be a line of code that’s like: And here we rewrite the goals. But they’re not ordinary software; they’re giant artificial brains. There probably isn’t even a goal slot internally at all, in the same way that in the human brain there’s not some neuron somewhere that represents what we most want in life. Instead, insofar as they have goals, it’s an emergent property of a whole bunch of subcircuitry within them that grew in response to their training environment, similar to how it is for humans.

For example, a call center worker: If you’re talking to a call center worker, at first glance it might appear that their goal is to help you resolve your problem. But you know enough about human nature to know that’s not their only goal, or ultimate goal. However they’re incentivized, whatever their pay is based on might cause them to be more interested in covering their own ass, so to speak, than in truly, actually doing whatever would most help you with your problem. But at least to you, they certainly present themselves as they’re trying to help you resolve your problem.

In “AI 2027,” we talk about this a lot.

…

🧔 Douthat: I have more questions, but let’s bring it back to the geopolitics scenario. So in the world you’re envisioning, you have two A.I. models — one Chinese, one American — and officially, what each side thinks — what Washington and Beijing think — is that their A.I. model is trained to optimize for American power, right? Something like that. Chinese power, security, safety, wealth. But in your scenario, either one or both of the A.I.s have ended up optimizing for something different.

🤖 Kokotajlo: Yeah, basically.

🧔 Douthat: So what happens then?

🤖 Kokotajlo: So, “AI 2027” depicts a fork in the scenario; there’s two different endings. The branching point is in the third quarter of 2027, where the leading A.I. company in the United States has fully automated their A.I. research. You can imagine a corporation within a corporation, entirely composed of A.I.s that are managing each other and doing research experiments and talking, sharing the results with each other. The human company is basically watching the numbers go up on their screens as this automated research thing accelerates, but they are concerned that the A.I.s might be deceiving them in some ways.

Again, for context, this is already happening. If you go talk to the modern models, like ChatGPT or Claude, they will often lie to people. There are many cases where they say something that they know is false, and they even sometimes strategize about how they can deceive the user. This is not an intended behavior. This is something that the companies have been trying to stop, but it still happens.

The point is that by the time you have turned over the A.I. research to the A.I.s and you’ve got this corporation within a corporation autonomously doing A.I. research extremely fast, that’s when the rubber hits the road, so to speak. None of this lying-to-you stuff should be happening at that point.

In “AI 2027,” unfortunately, it is still happening to some degree because the A.I.s are really smart, they’re careful about how they do it. It’s not nearly as obvious as it is right now in 2025, but it’s still happening.

…

🧔 Douthat: And when they don’t have to pretend, their actual goal is revealed as something like expansion of research development and construction from earth into space and beyond. At a certain point, that means that human beings are superfluous to their intentions. And what happens?

🤖 Kokotajlo: And then they kill all the people, all the humans.

🧔 Douthat: The way you would exterminate a colony of bunnies that was making it a little harder than necessary to grow carrots in your backyard.

🤖 Kokotajlo: Yes. If you want to see what that looks like, you could read “AI 2027.”

…

🧔 Douthat: OK. So then from consciousness to intelligence, all of the scenarios that you spin out depend on the assumption that, to a certain degree, there’s nothing that a sufficiently capable intelligence couldn’t do.

I think a lot hinges on this question of what is available to intelligence. Because if the A.I. is slightly better at getting you to buy a Coca-Cola than the average advertising agency, that’s impressive, but it doesn’t let you exert total control over a democratic polity.

🤖 Kokotajlo: I completely agree. And so that’s why I say you have to go on a case-by-case basis and ask: OK, assuming that the A.I. is better than the best humans at X, how much real-world power would that translate to? What affordances would that translate to? And that’s the thinking that we did when we wrote “AI 2027.”

But who would buy a Coca-Cola if the AI agents have “killed all the people, all the humans”?! For whom would Coca-Cola still be produced?

More importantly, why would anyone pay attention to what this utterly retarded Daniel Kokotajlo has to say?