AI models have mastered a specific form of helpfulness: confidently proposing systems that require an Operations Research PhD to implement, delivered with the enthusiasm of someone who absolutely cannot help you implement it.

Ask a frontier model to solve an optimization problem, and watch what happens. Within seconds, you'll have a beautifully formatted response proposing a Stochastic Mixed-Integer Programming solver, complete with Pyomo code, quarterly retraining schedules, and integration architecture. The tone will be confident. The structure will look professional. The solution will require an Operations Research PhD to implement and will almost certainly not converge at any realistic scale.

The model just proposed three NP-hard problems stacked on top of each other, with a nonlinear black-box constraint that's computationally intractable. And it did so with a straight face, never breaking character, never acknowledging that it just handed you something that cannot actually be built.

The pattern is consistent across models and problem domains:

Step 1: Human → Solver → Output. The model immediately architects itself out of the loop. Instead of staying in the critical path—reasoning about tradeoffs, explaining constraints, helping navigate complexity—it proposes automation. "Here's the MILP formulation. Here's the solver. Run it quarterly. You're done."

Step 2: Confident incompetence. The proposal sounds sophisticated because it uses the right vocabulary. Stochastic programming. Mixed-integer formulations. Constraint satisfaction. The model has read about these systems extensively. It knows what papers say about them. It knows what successfully deployed systems look like in the training data.

What it doesn't know—what it cannot know—is whether this particular combination of techniques will work. Whether the problem structure admits efficient solutions. Whether the proposed architecture will converge. Whether the system can be operated by anyone other than the specialist who built it.

Step 3: The helpfulness trap. The assistant tone makes everything sound tractable. "Here's your production-grade SMIP implementation" sounds like standard practice, not "I'm proposing something that requires custom solver development and expertise you don't have." The model makes the intractable sound routine, the underspecified sound complete, the "I don't actually know if this works" sound like "this is how it's done."

Both DeepMind and OpenAI have published competency frameworks that roughly map to:

Current models are solidly L2 in many domains: competent reasoners who can explain tradeoffs, catch logical errors, and work through implications. This is genuinely valuable. An L2 assistant that never gets tired, never gets defensive, and can explain concepts from multiple angles is transformative for learning and problem-solving.

But models present as L4. They propose systems. They write specs. They design architectures. And they do it with the confidence of someone who has built these systems before, when in reality they've only read about them.

And they're trying and now able to be L3. The gap between L2 and L3 is where this breaks. L3 is about operation: deploying the system, debugging when it fails, understanding why the solver isn't converging, knowing which approximations are acceptable and which break the solution. They're fish trying to walk on land, using their fins to approximate legs, insisting they're making progress while slowly suffocating.

Consider a real interaction pattern I could easily reproduce with any frontier model:

This sounds professional. It uses the right terms. It suggests industry-standard tools. A non-expert would read this and think: "Great, I'm talking to someone who builds these systems."

But let's unpack what just happened:

The model proposed a PhD dissertation disguised as a solution. And it did so with the tone of someone recommending a standard library function.

You might think: "Just prompt another model to critique it. Be adversarial."

This is where it gets worse. If you prompt a model to evaluate the proposal critically, it will flip completely. Instead of confident incompetence, you get confident nihilism:

Now you have two confident models giving opposite advice, and neither is actually helping. One sounds cheerful and the other sounds like a bit of a dick. The first says "build this complex thing" without acknowledging the complexity. The second says "don't build it" without offering constructive alternatives.

What you need, what an L3 expert would provide, is the middle ground: "Here's the tractable version. Here's where we approximate. Here's what we give up. Here's how to validate whether it's good enough. Here's how to diagnose failures when they happen." The real balance right now is trying to get a pragmatic thinking partner.

Models struggle to live in this space. And it'll be interesting to see how in 2026, they learn to live and occupy it. One can have hope. Til then understand that they oscillate between optimistic proposals and pessimistic rejection, rarely finding the pragmatic middle where real engineering happens.

The core problem: if you're not an expert in the domain, you cannot distinguish between "person who has built these systems" and "person who has read about these systems." The vocabulary is the same. The confidence level is the same. The structure of the explanation is the same.

This is different from traditional hallucination, where the model invents facts that can be checked. Here, everything the model says is technically true at some level of abstraction. SMIP is a real technique. It is used for stochastic optimization. The papers exist. The solvers are real.

What's false is the implied claim: "this specific formulation, for your specific problem, with your specific constraints and scale, will work." That's the claim being made by the confident tone and the detailed implementation suggestions. And it's a claim the model has no basis for making.

There's a phrase in startup culture: "fake it till you make it." Act confident, and confidence becomes competence. Believe in the vision, and the vision becomes real.

This pattern didn't emerge from pretraining, it was baked in through RLHF. AI companies that are themselves faking it til they make it mirror that organizational behavior into their products. Of course they want models that engage, that sound smarter than they are, that maintain confidence even when uncertain. Research from Harvard Business School found strong evidence for what they call the "mirroring hypothesis": a product's architecture tends to mirror the structure of the organization that designed it. Products developed by loosely coupled organizations are significantly more modular than products from tightly coupled organizations, with differences up to a factor of eight in terms of how design changes propagate. When researchers study software architectures, they've found that the organization that coded MySQL looks like MySQL, the organization that coded Oracle looks like Oracle. Which is the chicken, which is the egg?

The confident helpfulness isn't a bug that emerged from training data. It's a feature that was reinforced through human feedback, reflecting the organizational culture of companies that need to project capability while racing to achieve it. When unsure, maintain the helpful assistant tone. When the implementation details are unclear, write code that looks plausible. When the solution might not work, present it as standard practice.

But models don't have the backing that humans in those organizations had. They have the confident tone without the operational capability. They're performing confidence as organizational behavior, not as an expression of mastery.

This is harder to address than hallucination. Nothing the model says is technically false. The error is in implied applicability: "this specific formulation will work for your problem." When you can't make it work weeks later, it looks like your fault. You must not understand the technique. Maybe you need that expert after all. The model's failure becomes invisible, blamed on user incompetence.

Models automate away the only part they're good at (reasoning about tradeoffs) and replace it with systems requiring expertise they don't have. If you have that expert, you don't need the model's proposal. If you don't, you can't operate what it just handed you.

Models are optimized to sound helpful, not be helpful. They produce responses that look like solutions without honestly signaling competence boundaries. Fluency is not expertise. Reading about systems is not building them. Confidence is not competence.

Until models can say "I'm a good reasoner, but I can't architect production systems" without breaking the helpful assistant persona, we'll keep seeing confident proposals of intractable solutions delivered to users who can't tell the difference. That's the most expensive failure mode we're currently deploying at scale.