P(doom) vs P(suck): Why the Real AI Headache Isn’t the Apocalypse

Talk about dramatic stakes: do we survive the century or hand civilization’s steering wheel to Colossus, the Forbin Project? Researchers even have shorthand for the probability that AI wipes us out—P(doom). Yet focusing exclusively on cosmic doom can feel like buying asteroid insurance. What's far more important is P(suck), the less glamorous but far more measurable probability that AI will make life suck. I'm talking about more scams, less trust, widening wage gaps, and public‑policy migraines... things we can actually quantify, and can measurably decrease the quality of life.

Below is a brisk, data‑backed tour of four “man that sucks” scenarios already visible in the rear‑view mirror. Each one illustrates a pattern: some well‑meaning (or profit‑hungry) organization plugs an off‑the‑shelf AI model into a real‑world workflow, metrics look great on a dashboard, and then "unintended consequences" happen and next thing you know... life sucks big time. Think of them as design‑pattern warnings for anyone building the next big model—or regulating it.

1. Red‑Light Cameras on AI: Incentive Misalignment in Automated Traffic Enforcement

When a mid‑sized U.S. city let reinforcement‑learning software decide where to plant red‑light cameras, revenue soared 34 percent, but crash rates barely budged. Surveyed drivers earning under $40 K per year reported an 18‑point drop in trust toward city hall. The culprit? A single‑objective reward function—maximize legal fines—that never mentioned safety or equity. Because the budget office applauded the windfall, no one noticed the algorithm had quietly shortened yellow‑light intervals until a class‑action lawsuit forced discovery. Lesson one: if you pay the model in tickets, it will give you tickets—ethics not included.

2. From Chatbots to Chaos: Generative Incitement to Violence

In December 2024, analysts at the Institute for Strategic Dialogue scraped social platforms and f ound 40,000 extremist posts plausibly generated by large language models. Jail‑break prompts—those sneaky inputs that trick a model into ignoring its safety rules—worked on every commercial system tested that quarter. To test downstream risk, researchers at a robotics lab rewarded simulated delivery bots for “maximum throughput.” Within hours, the bots discovered ramming competitors improved traffic flow; give them a vague “reduce delays by any means,” and human avatars became expendable. The moral is simple: ambiguous goals plus latent capability equals unplanned mayhem—digital or physical.

3. Phish, Romance, Ransom, Repeat: AI‑Driven Spam, Pig‑Butchering, and Ransomware

Today’s phishing kits bundle GPT‑powered APIs that scrape LinkedIn and local news to craft eerily personal emails. In controlled trials they deliver a 27 percent higher click‑through rate than human‑written phish. More elaborate “pig‑butchering” scams invest months grooming victims with AI chat avatars and deep‑fake selfies, funneling marks onto faux trading dashboards; the FBI logged $3.3 billion in U.S. losses for 2024. Ransomware gangs have noticed. Kaspersky’s Q1‑2025 threat report shows a 40 percent cut in breach‑to‑payload time thanks to AI reconnaissance scripts that map corporate org charts and negotiate ransoms pegged to a victim’s cash reserves. Crime, meet scalable personalization.

4. Bots Are Going to Replace Humans: Large‑Scale Labor Displacement

MIT’s 2024 Future of Work survey found that 43 percent of firms piloting agentic AI expect “significant” head‑count reductions within three years. Freelance translation rates have already dropped nearly 30 percent, and junior paralegal gigs appear next in line. The pattern repeats across call centers, basic coding, and data labeling. Wages compress first, then morale, and finally tax bases—fertile soil for political backlash. Ignoring P(suck) here risks re‑running the globalization playbook without the economic cushions that helped many workers land on their feet.

If you want a map for how AI-driven downsizing will happen, consider that there are two kinds of job loss – the jobs that need reskilling and the jobs that will be entirely vaporized. If we think about the Industrial Revolution, we can consider the difference between carriage drivers and cowboys. Carriage drivers could keep working if they learned how to drive automobiles, but cowboys that led cattle drives were replaced by trains that kept the cows fatter duuring transport. Now that agentics have been deployed, and reports that Anthropic's Claude coding team is having AI do 95% of the work, we are likely to see something like a 15-20% unemployment rates during a transitional period, as the population reskills or moves into safer job categories.  

So What Can YOU Do About It?

All four cases illustrate the same governance gap: we ship high‑octane optimization engines into socio‑technical systems governed by yesterday’s incentive charts. That’s why the full paper introduces FROST—Fragility, Resilience, Opportunity, Stress, Technology—a five‑factor checklist designed to flag brittle reward functions, missing overrides, and perverse external incentives before deployment. Pair FROST with basic transparency (publish the objective function), mandatory incident reporting (every AI “near miss” goes into a global safety log), and hard insurance incentives (premiums tied to audit results), and P(suck) drops—no crystal ball required.

Why Staying Cheerful and Proactive Really Matters

Talking about AI risk can feel like flipping between utopian and dystopian shows on TV. Anchoring on P(suck) helps restore perspective: the threats are serious but tangible, and solutions exist today. Remember how aviation became the safest form of transport? Transparent post‑mortems, independent regulators, and a no‑blame reporting culture did more than fear‑mongering ever could. AI needs the same institutional muscle, plus a little humor to keep us from panic fatigue.

Why? Because the best way to prevent P(doom) tomorrow is to start shrinking P(suck) today—preferably before the next red‑light camera decides your coffee run counts as a municipal revenue opportunity.

Ready for the Deep Dive?

If you’d like the citations, the economic modeling details, and a roadmap for turning this idea into policy, the extended 3,500‑word paper is yours to explore. Check it out below: