Avoid LLM agents when run costs erode your margins

Signs this is happening in your business

What this is costing you a year

When to avoid LLM agents for cost‑sensitive automations

I avoid LLM agents for processes that run at scale, produce predictable outputs, or touch compliance unless run‑cost math and an error‑cost model show lower total expense. In projects I’ve seen demos look clever and then become a recurring billing and remediation problem when the team failed to prove the economics.

• Choose an agent only after you can show lower total cost (run cost + remediation + monitoring) than deterministic alternatives.
• If the process touches customer money or compliance checks, require an error‑cost model before piloting.

A ten‑minute decision checklist to avoid bad bets

Use this quick yes/no checklist to decide whether to avoid, pilot, or adopt an agent. Answer each question with your process numbers; if the checklist points to avoid, stop and pick a deterministic or hybrid pattern. The decision should take about ten minutes per process.

• Is the task high volume (hundreds+ runs per day)? If yes, avoid unless proven.
• Are outputs required to be identical or strictly formatted? If yes, avoid.
• Would an error cause financial loss, regulatory exposure, or churn? If yes, avoid unless you model remediation cost.
• Can the task be implemented with queries, rules, or parsing? If yes, choose deterministic.
• Do you have a measurable baseline (hours, error rate, SLA breaches)? If no, gather it before piloting.
• Can you cap monthly model spend and enforce it automatically? If no, avoid live pilots.

What drives run cost and how to convert it into $/month

Run cost equals model billing per call plus context storage and retrieval, retries, moderation, latency premiums, and the people cost to monitor and remediate. Convert each component to monthly dollars by multiplying per‑call or per‑GB rates by expected volume, then add headcount and monitoring overhead.

• Per‑call model billing: multiply average text size per call by vendor rate to get cost per call.
• Context storage and retrieval: include per‑query and per‑GB storage costs per call.
• Retries: count extra invocations caused by failures or low confidence.
• Moderation and safety checks: add a fixed per‑call cost for moderation.
• Latency tiers: faster response tiers have higher per‑call prices.
• Prompt engineering and experimentation: budget a month of higher runs during stabilisation.

Per‑process cost breakdown you can do in your spreadsheet

For one process collect volume, average text per run, model price per unit, fetch calls per run, failure rate, and human review time. Multiply to get cost per run and scale to monthly and annual totals. That spreadsheet is the finance proof you need to decide.

• Volume: runs per day × days per month.
• Average text processed per run (words or chunks).
• Model price per unit of text (vendor rate).
• Average fetch/search calls per run and per‑call cost.
• Failure rate and average retries per failure.
• Human review time per flagged item and hourly cost.

A pilot without a hard monthly cap becomes a recurring unbudgeted line item.

Reliability and operational risks unique to agents

Agents add nondeterminism, hallucinations, drift, and a monitoring burden that grows with volume. Those failures translate into repeat human checks, SLA misses, and unpredictable monthly bills unless you instrument output quality, cost per run, and prompt changes and treat prompts like versioned code.

• Non‑determinism: same input can yield different outputs, forcing review.
• Hallucinations: confident but incorrect outputs that need detection and rollback.
• Drift: behaviour changes over time or after prompt tweaks; catch this with monitoring.
• Monitoring burden: instrumentation of quality and cost adds headcount.
• Error remediation: fixing one bad outcome often costs several times the model call.
• Single‑person dependencies: prompt and guardrail changes often fall to one engineer.

Deterministic alternatives and when to pick them

Pick deterministic patterns for high‑volume, predictable, or audited work. They run cheaper, fail predictably, and provide an auditable trail. Map your task to a concrete pattern — queries, batch jobs, rules engines, template parsing, or small classifiers — and plan short maintenance commitments.

• Database queries and batch jobs — for structured data and reporting.
• Screen‑driven automation — when you must keep existing systems and avoid integration.
• Business rules engines — when decisions are fixed if/then rules.
• Template parsing and regular expressions — for consistent documents.
• Rule‑based parsers and small classifiers — for predictable text categorisation at scale.

Hybrid patterns that limit agent exposure and cost

Use hybrids to keep routine traffic deterministic and send only exceptions to an agent. Common patterns include classify‑then‑route, cached responses, a local small model with agent fallback, microservice wrappers that enforce caps, and human‑in‑the‑loop fallbacks for low‑confidence outputs.

• Classify‑then‑route: cheap classifier handles bulk; agent only for rare cases. Pro: most runs cheap. Con: two systems to maintain.
• Cached responses: reuse recent agent answers for identical inputs. Pro: cuts repeated calls. Con: cache invalidation complexity.
• Local small model + agent: on‑device or cheap classifier first pass; agent for complex cases. Pro: low average cost. Con: more orchestration.
• Microservice wrappers: enforce caps and fallbacks around agent calls. Pro: prevents bill spikes. Con: extra development.
• Human‑in‑the‑loop: route low‑confidence outputs to humans. Pro: prevents bad downstream effects. Con: headcount for exceptions.

Worked example 1 — invoice line‑item extraction

Invoice extraction often feels like a headcount win. Run the arithmetic: total payroll consumed by cleanup, deterministic build and maintenance cost, and agent run and monitoring spend. Compare saved payroll against annual agent cost to see which approach lowers total cost.

Example: four people each six hours/week at $45/hour equals $56,160 annually across the team. If a deterministic build reduces time to 2 hours/week, payroll falls to $18,720 and saved payroll is $37,440. Compare that saving to the agent's yearly run and monitoring cost.

• Inputs (example): team = 4 people, each 6 hours/week on invoice cleanup; hourly cost = $45.
• Deterministic build: template parsing + rules, plus small ongoing analyst time.
• Agent build: agent extracts items, with per‑invoice call costs plus human review on low confidence.
• Decision mechanics: compute annual payroll saved, subtract yearly agent run + search + retries + monitoring, add maintenance.

If a task is repetitive and predictable, rules‑based automation will almost always cost less over a year than an always‑on agent.

Worked example 2 — customer support triage

Support triage looks like a perfect agent case when volume is high, but templates plus a small classifier often win on cost. Compute current monthly triage payroll, savings from templates, and the agent's monthly run cost plus remediation and experimentation to decide.

• Inputs (example): 500 tickets/day, 22 working days/month, triage time per ticket = 2 minutes, hourly cost = $30.
• Deterministic approach: reply templates + simple classifier reduce time per ticket and payroll substantially.
• Agent approach: low per‑ticket model cost can look cheap, but add review for edge cases and prompt experimentation.
• At high volume and templateable content, classifier + templates usually wins; use agents for the rare complex tickets.

How these projects fail in practice and why

Projects derail when a pilot moves to live without caps, a baseline, or monitoring. Costs spike, prompt tweaks multiply reviews, alert fatigue sets in, and responsibility falls to one engineer. Prevent this by setting caps, tracking per‑run quality, versioning prompts, and staffing monitoring before production.

• Pilot runs live without a hard monthly cap; volume shifts and cost spikes.
• No per‑run quality measurement; teams react to complaints instead of catching drift.
• Ad hoc prompt changes alter output distribution and increase human review.
• Alert fatigue: many low‑value alerts drown out real issues.
• Operational debt: monitoring, fallbacks, and cost controls left for 'later'.

Operational handover and skills checklist

Before you switch an agent to live traffic, verify the team can run it day to day. You need dashboards for cost per workflow, sample quality reviews, automated caps and fallbacks, prompt version control, incident playbooks, and a named owner with time for monitoring and vendor liaison.

• Run‑cost monitoring: dashboards showing cost per workflow and monthly forecast.
• Quality monitoring: sample outputs reviewed daily until stable and then weekly.
• Cost caps and automated fallbacks: stop agent calls and route to deterministic logic when spend nears cap.
• Prompt and version control: track prompts like code with rollback.
• Incident playbooks and a named owner with allocated time.
• If you lack this capability consider our services or a short advisory engagement via fractional CTO.

What to do next — one concrete step tomorrow

Pick one live process that consumes people time, run the ten‑minute checklist, and plug your numbers into the per‑process cost breakdown. That single exercise will tell you whether to avoid, pilot, or adopt an agent. If you want help, book a 30‑minute scoping call at how to work together.

• If you need a build rather than advice, see custom builds.
• To read pattern tradeoffs and outcomes, see portfolio and case studies.
• If you want my background and rationale, read about me.

Where businesses waste the most time

1. Re‑typing invoice fieldsHours each week copying the same fields and fixing OCR mistakesAdd template parsing that reads invoice layouts and writes back into existing screensA person still checks: An analyst spot‑checks flagged invoices and resolves layout mismatches
2. Manual ticket categorisationLarge queues of tickets delayed by manual taggingA simple classifier assigns tags and uses existing reply templates in the support interfaceA person still checks: Support leads review low‑confidence tickets and edge cases
3. Month‑end spreadsheet fixesAnalysts spend days reconciling reports before closeReplace manual merges with scheduled queries and validation rules while keeping source systemsA person still checks: A senior analyst approves flagged exceptions

How this compares with what you already run

Operational area	The manual way	The automated way	Annual business impact
Data entry	People copy fields between systems	Database queries and batch jobs fill fields	Saves recurring payroll and reduces rework hours
Invoice processing	Analyst reads invoices and types line items	Template parsing and rules extract items	Lowers monthly staffing cost and backlog
Support triage	Agents read and route tickets manually	Small classifier + templates handle bulk, agent for exceptions	Cuts triage hours while keeping complex cases correct
Compliance checks	Manual reviewers inspect records one by one	Business rules engine enforces checks with an audit trail	Reduces regulatory exposure and recurring review hours
Month‑end reporting	Analysts merge spreadsheets and fix mismatches	Queries and scheduled loads produce reports consistently	Lowers overtime and speeds the close
Customer communications	Staff craft custom replies for common questions	Templates with small classifiers deliver standard replies	Saves headcount while keeping response times low

What the end-to-end workflow looks like

The metrics that tell you it worked

Metric	What it tells you
Hours saved per week	How many staff hours the automation removes; it directly reduces payroll.
Cost per automated run	The dollar cost to process one unit (invoice, ticket); it shows marginal scaling expense.
False positive rate	Share of automated outputs that require human correction; it measures remediation burden.
Monthly automation spend vs cap	Total monthly run cost compared to the enforced cap; it prevents billing surprises.
Time to detect drift	Days between a change in output quality and the first human noticing it.
Manual reviews per 1,000 runs	How many items need human intervention per 1,000 attempts; helps forecast headcount.
Average remediation time	Minutes or hours to fix a bad automated outcome; multiplies into total remediation cost.

When this is the wrong move

The mistakes that cost the most

• Pilot to production without a capA promising pilot moves to live traffic with no automated cap; volume shifts and the monthly bill spikes while engineering scrambles to add throttles.
• No baseline or measurementTeams deploy an agent without measuring current hours or error rates; with no baseline they cannot prove savings and keep paying running costs.
• Treating prompts as configurationPrompt changes are made ad hoc by different people; outputs diverge, human review increases, and no one can trace which change caused the problem.
• Automating a broken processAn exception‑filled process is automated end‑to‑end; exceptions multiply, maintenance increases, and staff time does not fall.

Frequently asked questions