The Overfitting Problem: Why Your Hard-Won Experience Can Become a Liability

There's a particular kind of failure that only happens to people who have been around long enough to accumulate real experience. Call it cognitive overfitting: the tendency to treat a current situation as essentially identical to a past one, because the surface features match well enough to trigger the same response.

Photo by Ann H on Pexels.

In machine learning, overfitting happens when a model trains so hard on one dataset that it loses the ability to generalize. The model becomes exquisitely tuned to the specific noise and quirks of its training data. Present it with anything slightly different, and its accuracy collapses. The model isn't dumb. It learned too specifically.

The same thing happens to humans.

You survive a liquidity crisis by cutting costs early and fast. You internalize that lesson viscerally. Years later, when your company hits a rough patch, you cut early and fast again, because that's what worked. Except this time the problem isn't liquidity; it's a product-market fit issue that requires investment to solve. The cost cuts accelerate the failure. You applied a well-calibrated model to the wrong situation, and your confidence in that model made you less likely to question whether it fit.

This is distinct from simple stubbornness. The experienced decision-maker isn't ignoring the new evidence out of ego. They're pattern-matching with genuine conviction, because the patterns they're drawing on were built from real outcomes, real pain, real learning. That's what makes the bias so durable.

Roger Schank and Robert Abelson's work on scripts and schemas helps explain the mechanics here. Human cognition runs on stored event templates. When a new situation activates a familiar template, we tend to slot in the rest of the expected script rather than perceiving what's actually present. The more deeply the template was encoded (through emotional intensity, repeated reinforcement, or high-stakes outcomes), the stronger the activation pull.

Experts are more susceptible to this than novices in certain respects. Novices know they don't have templates for a situation, so they look harder. Experts have templates, apply them quickly, and move on. Speed is usually an asset. In genuinely novel conditions, it's a vulnerability.

The diagnostic question to ask yourself: am I perceiving this situation directly, or am I perceiving my best historical analogue for it?

Those aren't the same thing. Spelling out the difference takes deliberate effort.

One practical method is what some decision researchers call a "mismatch audit." Before committing to an approach that feels familiar, explicitly list the ways the current situation differs from the historical case you're drawing on. Not the ways it's similar. You'll generate those automatically. Force yourself to find the disconfirming contrasts. A few pointed questions tend to surface the relevant gaps:

graph TD
    A[Situation feels familiar] --> B{Run mismatch audit}
    B --> C[List ways it differs from past case]
    C --> D{Differences material?}
    D --> E[Adapt or discard the template]
    D --> F[Proceed with template, flagged assumptions]

The diagram looks simple. The hard part is doing step C honestly, especially under time pressure, when the familiar template is whispering that you already know what this is.

There's also a social dimension worth naming. In organizations, the person with the most relevant-seeming past experience often dominates the room. Other voices defer, because who are they to argue with someone who's "been through this before"? That deference is often appropriate. But when the experienced person is overfitting, the deference becomes a liability shared by the whole group. The team's collective perception gets collapsed into one person's historical lens.

Building in deliberate dissent helps here. Not devil's advocacy for its own sake, but a structured moment where someone is explicitly tasked with asking: what would have to be true for this situation to be meaningfully different from the cases we're using as a guide?

Experience compounds. That's the whole point of accumulating it. But compounding only works when the interest earned in one period actually transfers to the next. When the underlying conditions have shifted enough, the transfer rate drops toward zero, and the experienced person can be less useful than the fresh one who has no old model to overwrite.

Knowing which situation you're in requires a kind of meta-awareness that most decision processes don't build in. It requires sitting with the discomfort of thinking: maybe what I know here is exactly what will mislead me.

That's an uncomfortable place to work from. It's also, sometimes, the most accurate one.