On June 24 we did something I had long promised the team: we handed Documentolog’s own registry office to our own agent. Not a demo stand, not a sandbox with hand-picked examples, but the live stream of incoming documents that people had handled for nineteen years — with all its odd letters, strange attachments and senders the system had never heard of. In the first month the agent processed 220 documents, and the median for a full pass — read the document, check legal risks, pick the assignee — came to 33 seconds, where the same step used to take a person up to thirty minutes.
Below I’ll walk through how it works from the inside, what the backtest showed before launch — and where the agent gets things wrong, because without that last part this story would be advertising, not experience.
Why the registry office
Every incoming letter in any organization has to be read, understood and routed to an assignee, and in most companies this is done by one to three clerks — manually and from memory. Over nineteen years of working with clients I have learned by heart the four systemic defects of this arrangement, and they are the same in a ministry and in a three-hundred-person private company.
The first defect is a bottleneck built on people: the moment a clerk goes on vacation or sick leave, documents wait hours for distribution. The second is irreplaceability instead of process: the knowledge that “letters like this go to Ivanov” lives in the head of one experienced employee and leaves the company on the day they resign. The third is routing errors: a letter goes to the wrong person, comes back marked “not mine,” goes on a second loop, and every loop costs days. And the fourth, the most expensive one, is regulatory: citizen appeals under administrative procedure law carry a hard 15–30 working-day deadline, a missed deadline costs a real fine per case, and a manual register has no alarm bell until it is too late.
We chose the registry office as the first process precisely because all four defects converge there in a single point — which also means the effect of removing them is visible immediately.
What global practice says
Before building our own, I spent quite some time looking at how the world measures this task. The most studied document pipeline turned out to be invoice processing: by industry benchmarks from Ardent Partners and APQC, manual processing of a single document costs companies $12–19, while automation leaders bring the same document below three dollars. That is a four-to-six-fold difference on every touch — and organizations that push “touchless” processing above 80% show three-year ROI beyond 300%. Our own measurement — roughly 27 tenge (about 5 US cents) per machine pass over a document — lands in the same logic: the cost of a touch drops by an order of magnitude, and judging by the global data this is not an anomaly but a pattern.
The second pillar I looked at is regulatory. Article 14 of the European AI Act requires meaningful human oversight for high-risk AI systems: a person must understand what the system is doing and be able to intervene before something irreversible happens. Out of that requirement the world has grown a very practical consensus for document agents: supervise the irreversible rather than everything, explain every decision, and keep an immutable audit trail. We designed the Registry Clerk on these principles before they became mainstream, and below you can see what they look like not in a declaration but in a living architecture.
The LLM reads — the cascade decides
The question I get asked most often goes roughly like this: how do you fight hallucinations if the model assigns the executors? And my favorite answer is — our model does not assign executors at all.
The language model does what it does best: it reads and understands the document — topic, substance, urgency, deadline, amount, legal risks. The assignment decision is made by a deterministic cascade in which the assignee is computed from verifiable signals in order of priority. First the cascade looks at human rules — a curated playbook where every rule has passed approval. Then at sender history, that is, where this sender’s letters actually went before; in the backtest this step alone gives 97% accuracy at the department level. The third step is the client’s structure: the counterparty’s division and its head. And only at the very end, if nothing else fired, does the model choose from the department catalog, while everything genuinely unclear goes to a live clerk. The lower the cascade descends, the safer the default — hallucinations simply have nowhere to live in this construction.
There is one more detail I consider fundamental: the route is bound to the org structure, not to a person. If an employee resigns, the letter goes to the acting head of their department, not “into the void” at a defunct address.
The assignee, meanwhile, gets a notification in the corporate chat that says who the letter is from, what it is about, why it was assigned to them and with what confidence. If they disagree, they press “not mine,” and the cascade proposes the next candidate. In the first month 85% of the ratings on these explanations were positive — though in fairness, that is 17 “yes” against 3 “no,” a first-month sample, and it is too early to call it statistics.
We measured accuracy before launch, not after
We did not release the agent onto the live stream blind. Before launch we ran a walk-forward backtest on six months of real correspondence — about a thousand letters, 58% of them from new senders; in this kind of test the model cannot see the future and starts with an empty rule base, which makes conditions harsher than production.
The results formed a clear picture. Sender precedent gave those 97% at department level and 88% at person level. Total department accuracy from scratch, with no rules at all, was 71%, and 77.5% with production directories included. The most interesting result came from seeding: just three human rules immediately added 2.4 percentage points. That is where our trajectory toward 80%+ comes from — it is built not on ML infrastructure but on a growing rule base fed by a live clerk’s corrections.
The system learns in a mundane, transparent way: a person corrects an assignment — a precedent appears; three similar corrections accumulate — a draft rule is born; a rule starts to apply — only after approval. Nothing self-applies, and that is the point. The knowledge of “who is responsible for what” stops being one person’s private memory and becomes a company asset — one you can see, verify and edit.
The economics, unvarnished
We calculated conservatively, using the market salary of a records clerk at 300 thousand tenge per month (hh.kz, July 2026; the national median is even higher at 331.5 thousand). A three-person registry office costs an organization 10.8 million tenge a year; the configuration of “one supervisor clerk plus the agent” — about 5.1 million. The license pays for itself in roughly two and a half months against the two freed-up positions, and the LLM pass over one document costs about 27 tenge — the entire pilot month cost us less than an hour and a half of human work.
And here I am obliged to say it plainly: “three clerks become one” is a pricing model, not what happened to us. In our pilot the live clerk went nowhere — the agent took the routine conveyor off their shoulders and left them the role of exception supervisor. Disputed amounts, legal risks, low confidence, unfamiliar counterparties — all of that is still decided by a human, and the agent signs nothing, approves nothing and changes no settings. The economics survive intact: they are built on removed routine, not on layoffs.
Key takeaways
- Incoming mail is the ideal first process for an AI agent: high volume, obvious pain, measurable result.
- Separate reading from deciding: the LLM understands the document, while a deterministic cascade computes the assignee — hallucinations have nowhere to live.
- Measure accuracy before launch, with a backtest on your own correspondence — not after launch, on complaints.
- Human corrections are the system’s fuel: each becomes a precedent or a rule, and company knowledge stops resigning along with employees.
- The clerk stays — their job changes: from the conveyor to supervising exceptions.
- The global human-in-the-loop principles — supervise the irreversible, explain decisions, keep an immutable trail — work. Build on them from day one, not after an incident.
All of this is from our pilot data, May–July 2026, on Documentolog’s own registry. You can watch the agent process an incoming document in 33 seconds in the live demo — the “Incoming → task in 8 seconds” scenario plays itself.
