Last week I noticed that new leads from our website had stopped arriving in my inbox. No errors, no alerts, no complaints. Just silence. Three days of silence.
I’m continuing the series where I show how we are rebuilding Documentolog around AI — on our own processes, with real pain and real fixes. Today’s case is an uncomfortable one, which makes it especially useful: how we built a beautiful AI funnel and, with our own hands, put it between our customers and ourselves.
How it was. For d8n.ai we built Lead Engine — a system where a website request is saved to the CRM, then an AI agent qualifies the lead: determines the segment, drafts a reply, scores readiness for a conversation. And only after qualification it notifies the team and me. The logic looked elegant: first the machine sorts things out, then people step in.
Here’s a detail from our kitchen. We use different models for different jobs: Anthropic models work in customer-facing dialogues, while internal tasks — scoring, analysis, feedback — we are moving to our own LLM deployed inside our perimeter. Lead qualification at the time ran on an Anthropic model. On July 16 the Anthropic account ran out of balance: in three days our agents had burned through 2.4 million input tokens — the economics of AI deserves a separate post, there is a lesson there too. The account went 14 cents into the negative, access was suspended. AI qualification stopped. And with it — the notifications about new customers.
The website kept accepting requests. The CRM kept saving them. Not a single request was lost. But nobody in the company knew they were coming. 14 leads sat in the database for three days, and nobody came back to them. It wasn’t monitoring that caught it. It was me — through a feeling that things were «too quiet». For a founder, that is the worst possible way to learn about a problem.
Now, honestly, about the mistake. It is not that we ran out of tokens — that happens, and we have set limits since. The mistake was architectural, and we made it ourselves: we made a critical business process — «the company learns about a new customer» — depend on an auxiliary one — «AI nicely parses the request». While the AI worked, everything looked great. The moment it went down, the part of the process that must always work went down with it.
What we rebuilt. First — the new-lead notification now goes out immediately on save, before and independently of any AI. Qualification runs separately and asynchronously: if the model is unavailable, the lead still reaches people instantly, just without a drafted reply. Second — we added a watchdog: if the site has traffic but no leads for longer than usual, the system raises the alarm itself. Silence no longer counts as good news — it counts as a signal. Third — the website forms now talk to people honestly: they confirm the request was received, and when something goes wrong they say so directly and keep the entered data. Fourth — we pulled all site, demo and CRM logs into one section with severity levels, and moved token spend into its own tab with daily limits and alerts at 80%. None of this took months: the whole rebuild took days.
The rule I now apply to every AI process we run: every agent must have an answer to the question «what happens when you go down?». AI in a process is an accelerator, not a load-bearing wall. If a model failure stops the business, the process is designed wrong — no matter how smart the model is. Proper degradation looks boring: without AI everything works, just slower and simpler. That boredom is exactly what you should design for.
Short takeaways — for anyone bringing AI agents into their processes:
- The critical path must not depend on AI. Acceleration — yes, dependency — no.
- Silence is not a health indicator. Absence of errors and absence of events look identical until you build observability.
- If the founder found the problem before the system did — that is not luck, that is a hole in the system.
- AI economics is part of the architecture: limits, alerts and token accounting are designed together with the agent, not after the first surprise on the invoice.
Lessons like these don’t make it into press releases. But real AI transformation is built out of them — not out of pretty demos, but out of processes that survive the failure of any of their components. We are walking this path on our own company, and we will keep sharing the lessons — without the gloss.
If you are deploying AI agents yourself — write to me and tell me where you stumbled. I will take the most interesting cases apart in future posts.
