The source problem is diagnostic collapse
Many AI teams still use an overly compressed failure model. A workflow feels broken, so the investigation defaults to code. If the code path is intact, the team checks deployment freshness. If the deploy is current, the temptation is to conclude that the user was mistaken or that the issue was already resolved. This is a poor diagnostic model for systems that operate in human workflows.
The missing category is design error. A user-facing system can be technically consistent and still be operationally misleading. A feature can exist, the deploy can be current, and the interface can still cause a competent operator to conclude that the feature is missing. When that happens, the failure is not imaginary. It has simply occurred at the interpretation layer.
The right taxonomy has at least three classes
For serious AI systems, especially those embedded in live business workflows, teams should classify incidents across at least three buckets:
- Code defect: the implemented behavior diverges from intended behavior.
- Deploy defect: the live surface is serving stale, partial, or wrong assets despite the source being corrected.
- Design defect: the implementation is technically present, but the human-facing surface hides, misframes, or poorly communicates the rule of use.
This taxonomy matters because the remedy is different in each case. Code defects call for implementation repair. Deploy defects call for environment and release verification. Design defects call for improved affordances, guidance, state visibility, or visible precondition handling.
The investigative method should reflect that taxonomy
The investigative order in the relevant case was instructive. First, the team checked recent commits and the last known good behavioral versions. Second, it verified live asset truth directly on the deployed host rather than trusting branch stories. Third, it compared the current empty-state rendering with the earlier working implementations. That sequence ruled out a stale deployment and showed that the core inline search machinery still existed.
The remaining gap was interpretive. The feature was not visible before a required precondition was met. The user followed the visible interface honestly and inferred that the feature had disappeared. That inference was reasonable. The system had not made the precondition legible enough.
The remedy is to surface the rule, not just defend the implementation
The correct remedy in cases like this is rarely another abstract explanation. It is usually a visible affordance. A control that depends on a precondition should often remain visible in a disabled state, paired with concise guidance explaining what enables it. This preserves both truth and orientation. The operator sees that the capability exists, understands why it is not yet active, and does not need hidden product knowledge to proceed.
From a systems perspective, this is a better use of intelligence than post hoc explanation. The product itself should do more of the teaching. If the interface makes the user discover the rule by frustration, the system has already spent trust badly.
What AI can learn from this
AI systems need stronger internal distinctions between technical correctness and user-visible truth. They also need a way to treat human confusion as evidence rather than noise. A model or agent should not become satisfied merely because it can prove the code exists or the deployment is current. It should ask whether the visible surface made the right interpretation available at the right time.
This points toward a broader principle for AI operators and agent systems: verification should not stop at implementation and deployment. It should include interpretability at the point of use. In practice that means checking empty states, disabled states, guidance copy, sequence visibility, and operator-loop closure — not only successful code paths.
What AI researchers should learn from this
For researchers, the deeper lesson is methodological. Current evaluations often privilege internal metrics: accuracy, pass rates, tool success, completion claims, benchmark wins. Those are useful, but they underrepresent the friction that determines whether a system can actually be trusted in production. A system that repeatedly causes honest users to misread the state of a workflow is not mature, even if its internal success rate looks respectable.
Research programs that care about deployable AI should begin treating user confusion as a measurable and classifiable outcome. Not every confusion report indicates a bug in logic. Some indicate a bug in the architecture of explanation. That is just as important when the system is meant to carry operational load.
The philosophy underneath the remedy
The philosophical claim is simple: a system is responsible not only for what it can do, but for what it makes legible. Hidden preconditions are a form of burden shifting. They require the human to hold context the product could have made visible. Better systems reduce that burden. They do not ask the operator to guess whether a missing control is absent, broken, or merely waiting.
This is one reason Nordlith treats verification, salience, and design as linked concerns rather than separate disciplines. The question is not just "did the system work?" The question is "did the system make the right understanding available before confusion became the mechanism of discovery?"
Implication for serious firms
Firms building or buying AI systems should insist on this distinction. If teams cannot tell the difference between code defect, deploy defect, and design defect, they will repair the wrong layer too often. That wastes time, weakens trust, and creates the false impression that the user is the unstable part of the system. In many cases the user is merely the first reliable sensor for design truth.
That is not a soft conclusion. It is an operational one. Mature AI systems should be harder to misread, not only more powerful under the hood.