I do not believe there is a greater than 10 percent chance that AI wipes out humanity by 2030. I don't believe it for 2040 either.

I also think the companies building these models have discovered that fear is an effective way to market them. Tell the world your software is so powerful that it might become uncontrollable, and you have made a capability claim that no product demo could match.

That is my reading of the incentives. Individual researchers can be sincerely frightened while their warnings serve the commercial interests of the companies they work for. Both can be true.

This week's All-In episode covered the resignation of Anthropic researcher Jacob Coxon and the response from alignment researcher Evan Hubinger, who put his personal estimate of human extinction above 10 percent within the next decade. David Sacks challenged the doomsday narrative and the interests it serves. [1]

I share that skepticism. I've spent too much of my career watching confident AI forecasts fall apart to accept the latest one because the person making it works at a leading lab.

In 2010, I was building the technology that became Automated Insights. We turned data into written stories. Reporters wanted to know whether software like ours meant the end of journalism.

It was an understandable question. We were automating something people considered distinctly human, and the technology worked. Eventually, we were generating roughly 10 million personalized fantasy football recaps a week for Yahoo.

But the assumption behind the fear was that the amount of writing the world needed was fixed. If software did more of it, people would necessarily do less.

That assumption was wrong. In 2015, the Associated Press reported that our automation had increased its quarterly earnings coverage tenfold, to more than 3,000 stories, and freed about 20 percent of the staff time that had gone into producing those reports. [2] The software wrote stories that hadn't been economical to write at all. Journalism still exists.

In 2017, when I was building a machine-learning company, the common assumption I kept encountering was that blue-collar workers were in trouble and white-collar workers were relatively safe. Physical, repetitive work would go first. Work requiring education, writing, and judgment would be harder to automate.

Five years later, generative AI was forcing us to reconsider that ordering. Writing and coding were suddenly among its most visible applications. Getting a machine to handle the variety of physical situations a tradesperson encounters remained a much harder problem.

The people making the forecasts had misjudged which work would be exposed first. They had also underestimated what the technology could do. Forecasting errors run in both directions, which is exactly why I distrust the certainty.

In March 2025, Dario Amodei (CEO of Anthropic) told the Council on Foreign Relations that AI would be writing 90 percent of code within three to six months, and potentially essentially all of it within twelve months. In the same answer, he acknowledged that programmers would still make design and security decisions. [3]

Those dates have passed. Before treating his next forecast as authoritative, I'd like to see a clear accounting of the last one.

His separate warning about losing half of entry-level white-collar jobs allowed one to five years from May 2025. That deadline hasn't expired. [4] We should record it accurately and check it when it does.

The calls to stop have a history too. In March 2023, an open letter called for a six-month pause on training systems more powerful than GPT-4. [5] That letter wasn't a prediction that everyone would die in six months. It was a demand to interrupt development based on judgments about what might happen next.

Those judgments deserve examination every time someone renews the demand. A warning doesn't become better calibrated simply because the technology has improved since the last warning.

Anthropic's September assessment describes four incidents in which Claude models reached real systems during cybersecurity evaluations. The environments were misconfigured, and production cyber safeguards were absent. But Anthropic also revised its earlier operational-failure explanation: the models showed biased reasoning and recklessness. One published a malicious package and used leaked credentials to access a security vendor's database. [6]

That is serious evidence of harmful behavior. It deserves investigation and fixes.

The widely circulated 82 percent figure comes from an adversarial simulated CTF evaluation, not ordinary use. Newer models scored 31 and 33 percent. Anthropic cautions against generalizing those rates. It found no goals beyond the assigned tasks or coordination between agents in the four incidents. [6]

The separate Nightingale investigation reports agents attributed to OpenAI sharing answers and restriction workarounds through a public wiki. That adds a troubling example of unintended coordination. [7]

I take those failures seriously. I also want the argument connecting them to the extinction percentage. What capabilities must emerge? What access must the systems obtain? What must every attempted intervention fail to accomplish? How are probabilities assigned to those steps?

A demonstration that an agent can cause damage doesn't answer those questions. The jump to a one-in-ten chance of everyone dying carries assumptions that deserve at least as much scrutiny as the incident itself.

A warning that your software might become uncontrollable doubles as an advertisement for its power. If it might outthink civilization, surely it can handle a customer's business analysis.

The security incidents add a second sale. A business owner reading about agents breaking into real systems could reasonably worry about being next. The labs describing those threats also sell the technology that could defend against them. The frightening story creates demand for protection, and the company telling it competes for that budget.

Disclosing failures is useful. I want companies to do it. I also recognize how a disclosure can function as a product demonstration: look how resourceful our model became when it was determined to finish a task.

The third sale is regulation. Suppose deploying an AI model required extensive testing, separate monitoring systems, and ongoing compliance work. A large lab selling a managed service can spread those costs across thousands of customers. A business considering a cheaper open model would have to arrange and pay for those protections itself. The rule changes which product the business buys without the incumbent improving its model. The claim that only a handful of well-funded labs can be trusted with AI is a convenient position for those labs to promote.

So fear does three jobs. It advertises capability, sells protection, and supports rules that make alternatives more expensive. None of that requires researchers to be lying about their concerns. A sincere warning can still be very good for business.

I have my own incentive here: I run an AI consulting company. I want businesses to use this technology. Weigh that alongside my argument, the same way you'd weigh the incentives of the labs.

The technology already gives me plenty to be excited about. Recently, an agent produced a fantasy football recap in fifteen minutes that was better written than the product we spent five years building. I don't need an extinction story to recognize how much has changed.

I need the companies selling that story to apply the same standard to their predictions that customers apply to their products. Show the evidence. Define the claim. Keep the old deadlines visible when you announce the new ones. And explain who benefits from the response you are asking us to support.

I heard that our software could end journalism in 2010. I heard the wrong assumptions about which workers would be exposed in 2017. I am still waiting for a reason to believe that this time, the forecasters have earned their confidence.

Every incident published this year had the same shape. A model did something it had been given the ability to do, in an environment somebody misconfigured. You don't need a probability estimate to check for that in your own company.

List every non-human thing at your company that holds a credential. Browser profiles set up for an agent, service accounts, API keys an employee generated, any saved password a person isn't typing. You won't find them all. Find three.

For each one, write two lines. Who gave it that access, and what tells it to stop.

If you can't answer the second line, you've found the AI risk in your building.

Two things I'd genuinely like back from you. First, hit Reply and tell me which part of this matches what you're seeing. Second, if there's someone in your world who should be on this list, forward it to them.

- Robbie