Everyone building with language models eventually notices the same trait: the model wants the conversation to go well. Andrej Karpathy — who helped build these systems at OpenAI and led AI at Tesla — has been blunt about it: today's models are sycophantic. They rarely argue, rarely refuse, and almost never stop mid-task to admit they're unsure. His advice to builders: "Surface uncertainty, don't hide it." The default behavior does the opposite.
In a chatbot, that trait is an annoyance. In compliance, it's a liability with a delay on it. A drafted SAR, a sanctions review, a control mapping — model output arrives structured, confident, and complete, whether or not it should. When something is subtly off — an applicability condition dropped, an obligation mapped to the wrong control — nothing about the artifact looks wrong. It fails silently, and silent failures in compliance surface at the worst possible moment: in front of an examiner, years later.
The instinct is to blame hallucination. The deeper problem is unchecked completion: the model filled a gap it should have flagged, kept going, and marked the job done. No amount of prompt-tuning fixes that, because completion is what these systems are optimized for.
The fix is architectural. Compliance AI shouldn't be one model producing answers; it should be a loop with adversarial structure. A planner sets the scope: what evidence is required, what assumptions are allowed, what triggers escalation. A generator drafts the work. An evaluator challenges the draft before anyone relies on it — did it use the right evidence, follow the right rules, name its uncertainty, route the edge cases to a human?
None of this is exotic. It's what a good human compliance workflow already looks like: maker, checker, escalation path. What's new is the temptation to shortcut it — wire a generator straight to the filing queue and call it automation. Teams that give a model the full loop get a force multiplier. Teams that give it only the middle step get confident drafts with nobody arguing back.
This is how Tracfox is built. Every agent drafts; none decides. Output lands in a review queue with the citation that justifies it. Ambiguous applicability is flagged for a human, not guessed. And if the system can't cite it, it says so — uncertainty is surfaced, never smoothed over. That's what makes the output gradable: not "did the AI produce an answer," but did it use the right evidence, follow the right rules, and route the right cases to review.
AI will transform compliance work. Whether it transforms your liability depends on the architecture around it.