What AI Actually Does (and Doesn't Do): Strengths, Weaknesses, Reality
There's a moment most people have had with AI — usually early on — where it does something that feels almost magical. It summarizes a dense 40-page report in 30 seconds. It rewrites a clunky paragraph...
What AI Actually Does (and Doesn't Do): Strengths, Weaknesses, Reality
Part 2 of the "Build with AI" series
There's a moment most people have had with AI — usually early on — where it does something that feels almost magical. It summarizes a dense 40-page report in 30 seconds. It rewrites a clunky paragraph into something crisp and clear. It explains a complex legal clause in plain English.
And then, sometimes in the very next session, it confidently tells you that a company was founded in 1987 when it was actually founded in 2003. Or it generates code that looks completely correct and breaks immediately when you run it. Or it gives you a list of academic citations that don't exist.
Both of these are the same tool. Neither is a glitch.
This is what makes AI genuinely confusing to work with: it operates at two extremes simultaneously — breathtakingly capable in some areas, surprisingly brittle in others — with no obvious signal telling you which mode you're in.
Most of the frustration people experience with AI comes from not knowing where the line is. They either trust it too much (and get burned by confident errors) or trust it too little (and leave enormous value on the table). Understanding where AI actually excels and where it reliably fails isn't just interesting — it's the foundational skill for getting real results.
What AI is exceptionally good at
1. Working with language at scale
This is AI's home territory. Summarizing, rewriting, translating, reformatting, extracting key points, changing tone, adapting content for different audiences — anything that involves transforming text from one form to another. AI does this faster and more consistently than almost any human, at any volume.
If you have 50 customer support emails and need to identify the top five themes, AI can do that in seconds. If you need the same piece of content rewritten for a technical audience and a non-technical one, AI handles both simultaneously. If you need a rough draft turned into polished copy, AI is an extraordinary first-pass editor.
Stop writing from scratch. Use AI to generate the first draft, then edit. Even a mediocre first draft is faster to fix than a blank page.
2. Pattern recognition and synthesis
AI has processed more text than any human could read in thousands of lifetimes. That gives it an unusual ability to spot patterns, make connections across domains, and synthesize information from multiple sources into a coherent whole.
Ask it to compare five different approaches to a problem. Ask it to identify what successful examples of something have in common. Ask it to find the structural tension in an argument. These synthesis tasks — where the value is in connecting dots across a large body of knowledge — are where AI genuinely earns its place.
Use AI for research synthesis. Give it multiple sources or perspectives and ask it to find the through-line, the contradictions, or the gaps.
3. Generating structured starting points
Blank page paralysis is real. AI is a remarkable antidote. Whether it's a project plan, a meeting agenda, a proposal outline, a set of interview questions, a framework for thinking about a decision — AI can produce a solid structured starting point in seconds that would take you 30 minutes to build from scratch.
The key word is "starting point." AI's first output on complex tasks is rarely the final answer. But it's almost always a useful scaffold.
When you're stuck on where to begin, ask AI for a structure first. "Give me an outline for X" before "write me X."
4. Explaining complex things simply
AI is a remarkably patient teacher. Ask it to explain a concept at different levels — "explain this like I'm a curious 12-year-old" versus "explain this assuming I have an MBA" — and it will calibrate accurately. Ask it to explain a technical concept using an analogy from cooking. Ask it to walk through a process step by step until you understand it. It doesn't get frustrated. It doesn't judge the question.
Use AI to get up to speed on unfamiliar domains before important meetings, decisions, or conversations. It's a better pre-briefing tool than most.
5. Processing data with clear, defined rules
Computers have always been good at following rules precisely and at scale — and AI inherits that strength, now with the ability to understand the context around the data too. Tasks with clear, defined logic are where AI is genuinely reliable: calculating tax scenarios, analyzing financial data, applying consistent formatting rules across thousands of rows, flagging entries that violate specific criteria, or running structured comparisons.
The key phrase is "clear rules." When the logic is unambiguous — if X then Y, always — AI handles it consistently and without fatigue. A tax calculation that would take an accountant an hour to verify across 500 line items takes AI seconds. A data quality check that requires applying the same rule to 10,000 records? Same story.
Any task where you can write down the rule clearly enough that a precise robot could follow it — that's a task AI can handle at scale. Think: data validation, formula application, structured classification, compliance checks.
6. Finding things in vast amounts of information
Humans are bad at searching through large bodies of text. We skim, miss things, get fatigued, and anchor on what we found first. AI doesn't have these limitations. Given a large document, a database of records, a long email thread, or a collection of reports, AI can locate specific information, surface relevant passages, identify what's missing, and spot inconsistencies — across the entire body of content, not just what catches the eye.
This is different from summarizing. Summarizing compresses. Finding surfaces specifics. "Does this 200-page contract contain any clause that limits liability in ways that conflict with our standard terms?" is a finding task — and AI handles it extraordinarily well.
Use AI as a search and retrieval layer over your own documents. Feed it a long contract, a research report, a transcript, or a collection of notes — and ask it to find, not just summarize.
Where AI still falls short — and what's changed
Before we go through the limitations, an important caveat: the boundary of what AI can't do is shifting fast.
Modern AI systems can now connect to external tools through frameworks like MCP (Model Context Protocol) — which lets AI call web search, query live databases, pull real-time financial data, read your files, and interact with external services. This isn't science fiction; it's how tools like Claude, ChatGPT, and others work today when properly configured.
What that means: several limitations that felt absolute a year ago are now soft limits or solvable with the right setup. We'll flag which ones have changed — and which ones haven't.
1. Real-time and specific factual information — significantly improved with tools
The classic AI limitation: it generates responses from training data, not live lookup. That meant wrong dates, fabricated statistics, outdated information — all delivered with full confidence.
This is still true of a base AI model used on its own. But connected to web search or live data sources via tools, AI can now retrieve accurate, current information before responding. Ask a tool-connected AI for today's exchange rate, a company's latest earnings, or a recent news event — and it can fetch the actual answer rather than guess.
Even with tools, AI can still hallucinate when it doesn't realize it needs to look something up, or when the information isn't available in the sources it can reach. The confidence problem hasn't fully disappeared.
For specific, verifiable facts — especially recent ones — use AI systems that have search or tool access enabled. And still verify anything consequential against a primary source.
2. Reasoning through novel, multi-step logic — improved, but still different from human reasoning
This one needs an honest update — because reasoning has been one of the fastest-moving areas in AI.
Modern "reasoning models" like OpenAI's o1/o3, Google's Gemini, and Claude's extended thinking mode use a technique called chain of thought (CoT) — they generate intermediate steps before arriving at an answer, essentially thinking out loud. On math, logic, coding, and structured multi-step problems, this has been a genuine leap. These models handle complexity that would have stumped their predecessors entirely.
So the blanket statement "AI can't reason" is no longer accurate. For problems with recognizable patterns and defined structure, AI reasoning is now impressively reliable.
But several important gaps remain, and they're worth understanding because they're structural, not just a matter of more training data.
AI reasons in one direction. Humans backtrack. Human reasoning is recursive — we form a hypothesis, test it, feel something is off, backtrack, revise our assumptions, and try again, often without consciously deciding to. AI chain of thought is largely a one-pass forward generation. Each step is built on what came before. Even when it appears to "reconsider," it's generating the next most plausible token, not genuinely revisiting a prior belief the way a human does. This is why AI can go confidently down a wrong path and never course-correct.
AI also has no stakes in being right. Human reasoning is shaped by consequences. We reason more carefully when we know we'll be held accountable, when being wrong has real costs. AI has none of that friction — no fear of error, no ego investment. This makes it consistent, but it also means it can walk confidently into a wrong conclusion without any of the internal resistance a human would feel along the way.
Then there's confidence, which doesn't signal correctness. A reasoning model can produce a long, detailed chain of thought and arrive at a completely wrong conclusion — with the same confident tone it uses when it's entirely right. The length and apparent thoroughness of the reasoning doesn't reliably indicate whether the answer is correct. With a human expert, deep reasoning and high confidence are at least somewhat correlated. With AI, they're not.
Finally, genuinely novel situations without prior patterns still trip it up. AI reasoning shines on problems that resemble things it has seen before. Unusual edge cases, decisions that require reasoning from physical intuition, social dynamics with no clear precedent — these are where it still struggles. Human reasoning is embodied and adaptive. AI reasoning is pattern-completion, and when there's no pattern to complete from, the scaffolding gets shaky.
For structured, well-defined problems, use reasoning models freely — they're genuinely good. For novel, high-stakes, or consequential reasoning, treat AI as a thought partner that surfaces considerations, not a final authority. Always sanity-check the logic yourself. Pay special attention when AI sounds most confident: that's precisely when the "confidently wrong" failure mode is hardest to catch.
3. Knowing what it doesn't know — still the most dangerous limitation
AI doesn't have a reliable internal signal for uncertainty. A human expert asked something outside their knowledge will typically hesitate or say "I'm not sure." AI often doesn't. It fills the gap with plausible-sounding content — sometimes entirely fabricated — delivered in the same confident tone it uses when it's completely correct.
This is what the AI community calls "hallucination." Tool access reduces it on factual questions because AI can look things up. But it doesn't eliminate the underlying tendency, especially on niche topics, complex specifics, or questions at the edge of what any source can answer.
The more specific and niche the question, the more skeptical you should be of a confident answer. Treat AI output in unfamiliar territory as a starting hypothesis, not a conclusion.
4. Judgment that requires real-world stakes — tools don't help here
This one hasn't changed, and tools don't change it. AI has no skin in the game. It hasn't run a company, navigated a difficult client relationship, or felt the weight of a decision where being wrong has real consequences. It can surface frameworks, list considerations, and play devil's advocate — but it can't replace the judgment of someone who has actually been in the room.
"Should I fire this person?" "Is this partnership worth pursuing?" "Should I take this investor's money?" These aren't information problems. They're judgment calls that require lived context, relationship history, and real accountability.
Use AI to surface considerations you might have missed, not to make the call. "What are the arguments for and against?" is a better question than "what should I do?"
The practical framework: trust tiers
Once you understand these patterns, you can develop a working intuition for when to trust AI's output and when to verify it.
For language transformation — summarizing, rewriting, translating — brainstorming, generating structures and outlines, explaining concepts, drafting communication: trust freely. The cost of a small error here is low, and the efficiency gain is high.
For research synthesis, strategic frameworks, technical explanations, and code generation: trust but verify. AI is genuinely useful here, but always review the output before acting on it.
For specific facts, statistics, citations, legal or medical details, precise historical information: use with caution. Treat AI output in these areas as a starting point for research, not a conclusion.
For final judgment on consequential decisions, complex novel reasoning where every step matters, or anything where being wrong has serious consequences: don't rely on it.
A quick test to run right now
Pick something you've used AI for recently, and ask yourself: which tier did that task fall into?
If you were asking AI to summarize or rewrite something — you were in safe territory.
If you were asking AI for specific facts or data — did you verify them? If not, go back and check.
If you were asking AI to help you make a consequential decision — did you treat its output as one input among many, or as the answer? If the latter, that's worth revisiting.
Most people, once they think about it, realize they've been either over-trusting AI in tier 3 areas or under-using it in tier 1 areas. The awareness alone shifts how you work with it.
What the limitations actually tell you
Here's the thing about AI's limitations: they don't diminish what it's capable of. They just define the game.
A calculator is extraordinarily useful — and you wouldn't trust it to write your strategy memo. A GPS is invaluable — and you still need to know when the route it suggests doesn't make sense. AI is the same. Understanding what it does well and what it doesn't isn't pessimism. It's the prerequisite for using it well.
The people getting extraordinary results from AI aren't ignoring its weaknesses. They've learned to route the right tasks to it and keep judgment where it belongs — with them.
What to remember from this post
AI operates at two extremes with no obvious warning signal. Knowing which mode you're in is the skill worth developing.
AI's home territory is language transformation: summarizing, rewriting, reformatting, explaining, synthesizing — anything that turns text from one form into another. AI doesn't know what it doesn't know, and it produces wrong facts with the same confident tone as correct ones. The more specific the question, the more skeptical you should be.
Use the trust tier framework as your default: trust freely for language work, verify for synthesis and code, stay cautious with specific facts, and keep consequential judgment to yourself. The best AI users aren't the most optimistic ones — they're the ones who've learned exactly where to deploy it and where to stay in the driver's seat.
Want the full framework?
This post covers how AI actually works — the honest version. The AI Development Guide by Jaehee Song builds on this foundation to show you how to apply these principles across the full arc of building with AI: from prompting and data to agents, vibe coding, and production deployment.
📱 Apple Books ▶️ Google Play Books 🌐 All Platforms (Books2Read)
Next in the series: "The AI Lego Stack" — how modern AI tools fit together, and how to pick the right combination for what you're trying to build.