No. I do not grant AI agents unsandboxed read/write access to core systems by default. I start with read-only pilots, move to guarded proxy actions and limited canary writes, and only widen access after sustained reliability, proven rollback, a named owner, and clear net financial benefit.
- Direct unsandboxed write access creates governance failures you cannot fix with code alone.
- Follow a staged path: read-only, proxy actions, canary writes, then limited direct access.
- Require proven rollback, sustained low error rates, and a named owner before widening access.
- Ongoing monitoring, incident response and permission audits are recurring costs that often exceed build costs.
- A pilot that only saves operator hours is insufficient; measure remediation cost and business impact first.
Signs this is happening in your business
- Three operators spend six hours a week each fixing the same CRM and invoicing mismatches.
- Staff correct addresses, merge duplicates and retag customers daily.
- Paging alerts fire for failed payments that take hours to diagnose and reverse.
- Approval queues clog because someone must manually confirm every agent suggestion.
- Audit trails are incomplete: nobody can say who or what changed a sensitive record last month.
- Automation requests begin with 'just give it access' as the first technical proposal.
- Teams disable monitoring alerts because they produce too much noise after automations start.
- A vendor asks for write access and the product team says 'it’s easier' instead of explaining the risk.
If several of these are true at the same time, the problem is usually the process rather than the people running it.
What this is costing you a year
(Team hours per week x hourly cost of employees x 52) = annual payroll loss
- People doing this work4
- Hours each per week6
- Hourly cost of employees$45
- Weeks52
Annual payroll cost of repetitive manual reconciliation across four operators.
Find the number before you spend anything
Run the formula above on your three most repetitive processes. If any of them clears a year of a salary, that is the one to bring to a call.
Should I give AI agents direct access to our systems?
You see operators spending hours fixing the same CRM, invoicing and payroll mismatches. That saving tempts leadership, but granting blanket read/write control multiplies failure modes and costs. I run staged access: read-only pilots, guarded proxy actions, then limited canary writes only after rollback, ownership and ROI are proven.
- Start with measurable, time-boxed experiments.
- Prove fast recovery and name who is accountable.
What can go wrong if an agent has direct read/write access?
Write privileges expose three concrete loss vectors: data leakage from readable sensitive fields, incorrect transactions like duplicate invoices or misdirected payments, and cascading failures when downstream automation reprocesses bad data. Each consumes payroll hours, triggers supplier or customer remediation, and damages trust; you must control every vector before authorising writes.
- Sensitive data can be packaged and leaked by routine reports.
- Incorrect transactions require manual reversals and supplier remediation.
- One bad update can trigger downstream automated actions.
- Errors compound quickly when multiple systems assume canonical data.
How to quantify the business impact of those risks
Measure three inputs and multiply: value at risk per wrong action, expected frequency of wrong actions, and remediation cost per incident. That annual exposure is what you must compare to the automation benefit before approving write access.
- Value at risk: average invoice or payroll batch affected.
- Frequency: baseline of how often humans correct errors today.
- Remediation cost: detection time, fix time, external fees and reputational cost.
- Sensitivity list: which fields amplify downstream damage.
Automating a broken process multiplies errors and repair costs; it does not remove them.
How do AI agents fail, and how do you test them?
Agents fail by misunderstanding intent, hallucinating plausible values, and making correlated mistakes across many records. Test these modes by replaying historical interactions against a data copy, injecting malformed or ambiguous records, and running live canaries on low-risk accounts while instrumenting whether actions were suggested, executed or blocked.
- Replay historical tasks against a snapshot and compare decisions.
- Inject malformed or ambiguous records to force failure paths.
- Measure human fallback rate: how often the agent asks for help.
- Run canaries on low-dollar accounts and monitor every action.
What operational measures reduce failure impact?
Design for idempotency, retries with backoff, and staged approvals. Require confirmation on high-risk actions, log intent before execution, and prefer reversible or soft changes so you can back out quickly. These mechanics stop duplicates, limit repetition from transient errors, and keep humans focused on exceptions.
- Idempotent writes: repeated attempts create one change.
- Retries with exponential backoff to avoid duplicates on transient errors.
- Staged approvals for non-routine or high-value actions.
- Soft-change defaults—flags or status updates instead of hard edits.
What audit, logging and observability do you need?
Every agent action must produce an immutable record tying the change to an approver, the agent's view, and the exact mutation performed. Keep before-and-after snapshots, correlation IDs for multi-step flows, and alerts for abnormal write patterns in a separate log store that the agent cannot modify.
- Action record: agent version, decision rationale, approval actor.
- Before/after snapshots for every changed field.
- Correlation IDs for multi-step actions.
- External log store and alerts for unusual patterns.
A project without a kill switch is not automation; it is a timed liability.
Who owns agent behaviour and incidents?
Assign a process owner for decision outcomes, an engineering custodian for deployment and rollback, and a security owner for permissions and audit evidence. Name a single incident commander who can suspend the agent and coordinate external communications during outages.
- Process owner: accountable for decision outcomes.
- Engineering custodian: controls deployment and rollback.
- Security owner: manages permissions and audit trails.
- Incident commander: suspends agent and coordinates response.
How to rollback and stop an agent that is misbehaving
Provide a kill switch that halts writes and returns the agent to read-only within a minute. Make writes reversible using soft deletes, status flags or an undo table. Drill a rollback runbook: pause the agent, snapshot affected data, execute scripted reversions and measure time-to-safe-state as a KPI.
- Kill switch that halts writes quickly.
- Soft-write mode where changes are queued for approval.
- Exportable undo scripts tied to each write action.
- Regular rollback drills that measure time to safe state.
Identify your biggest operational bottleneck
Most owners already know which process is the problem. What they do not have is the annual figure attached to it. That is a 15-minute conversation, not a project.
What does direct access cost day-to-day beyond build time?
Direct access creates recurring operational roles: reviewers for flagged actions, incident responders, permission auditors and model retrainers. Expect daily review time, monthly permission audits and a continuous testing cadence. These ongoing tasks can exceed the initial engineering build if you skip staging and controls.
- Daily review time for flagged actions.
- Monthly audits of permissions and roles.
- Continuous testing and model updates.
- Staff rotations to cover incident response.
If no one owns agent decisions, the first major outage will cost more than the expected savings.
A staged implementation plan you can follow
Use four stages: discovery, read-only pilots, guarded proxy actions, and phased direct writes. Define exit criteria at each stage—measured reliability, rollback proof and owner sign-off—and require written approval from the process owner, engineering custodian and security before progressing.
- Stage 1: discover and measure the manual process.
- Stage 2: read-only pilot with intent logging only.
- Stage 3: guarded proxy actions with human approvals.
- Stage 4: limited canary writes, then phased expansion.
Proxy actions and canary writes — the technical playbook
A proxy action lets the agent propose a change while an intermediary validates inputs, enforces business rules and performs the write under your controls. Canary writes are real writes limited to a tiny subset so you can observe production behaviour and rollback before wider rollout.
- Proxy validates and enforces rules, not the agent directly.
- Canaries run on low-risk accounts and small thresholds.
- Record both the agent rationale and the proxy decision.
- Run canaries for a full business cycle before expanding.
When not to give direct access
Do not grant direct writes when you cannot reverse a change quickly, when the cost of a wrong action exceeds the automation benefit, or when no one is accountable for incidents. Also avoid it if your data is messy—automation amplifies broken processes rather than fixes them.
- Irreversible financial actions with no quick undo.
- Expected remediation cost higher than projected savings.
- No accountable owner or incident runbook in place.
- Poor data quality that forces frequent clarifications.
How these projects fail in practice and why
I see the same sequence: leadership wants quick savings, an engineer connects an agent with blanket write rights, noisy alerts get muted, and a bad change spreads before anyone halts it. The chain is weak testing, skipped canaries, disabled alerts and missing rollback scripts.
- Rapid deployment with weak testing.
- Early noisy alerts are muted.
- A mistake multiplies through automated rules.
- Slow or absent rollback, costly remediation.
Next actions you can take tomorrow
Run a 30-minute risk-to-value audit on one candidate process: identify sensitive fields, estimate value at risk, and name the accountable owner. If you want help mapping the audit into a staged plan, book a short advisory session through How to work together or see Services.
- Map one process and its owners today.
- Run a two-week read-only pilot next week.
Implementation options: build, buy, or bring in leadership
Build when you need tight integration and a custom undo path. Buy when a vendor provides proxy controls and faster time-to-value. Hire temporary CTO support if you lack technical leadership who will own day-to-day decisions. Choose based on whether you need custom rollback or can operate within vendor guardrails.
- Build for full control over rollback and audit.
- Buy for faster time-to-value with vendor guardrails.
- Hire leadership to own day-to-day technical decisions.
- Use advisory support for a staged rollout and strategy.
Where businesses waste the most time
- Re-keying customer dataRepeated typing across systems eats hours and introduces typosAdd a read-only sync that highlights missing fields for humans to confirm, keeping existing systems in placeA person still checks: Operator confirms suggested fixes before they are applied
- Manual invoice reconciliationStaff spend days matching payments to invoices and chasing suppliersAgent proposes matches; a proxy stages reconciliations into a queue without replacing current systemsA person still checks: Finance lead approves reconciliation batches
- Post-hoc payroll correctionsLate payments and corrections create fees and upset employeesAgent flags anomalies in read-only mode for HR review, leaving payroll software unchangedA person still checks: HR reviews and approves any pay changes
How this compares with what you already run
| Operational area | The manual way | The automated way | Annual business impact |
|---|---|---|---|
| Customer data fixes | Operators edit records one-by-one | Agent suggests edits; proxy applies after validation | Fewer annual support hours |
| Invoice corrections | People correct invoices after payments fail | Canary writes for low-value invoices with approvals | Lower annual reconciliation cost |
| Payroll adjustments | HR makes manual changes and runs checks | Read-only analysis; agent flags items for HR | Reduced annual payroll risk |
| Permission changes | Admins manually grant and audit access | Proxy enforces approvals before updates | Faster annual audits and fewer incidents |
What the end-to-end workflow looks like
- Select candidatePick one process with clear owners, measurable cost and limited exposureStays human
- Map fieldsList sensitive fields, who can change them and where they flowAutomation records suggestions only
- Read-only pilotRun the agent in read-only mode for two business cycles and collect metricsAgent analyses and suggests, does not act
- Proxy stageIntroduce a guardian that validates and applies low-risk changes under rulesAgent suggests, proxy enforces
- Canary writesAllow the agent limited writes on a tiny subset and monitor closelyLimited automation with strict scope
- Review and expandMeasure error rates, run rollback drills and collect owner sign-offs before expandingGradual expansion; humans approve thresholds
- Full access guardrailsIf approved, broaden writes with strong logging, alerts and independent auditsAutomated with continuous oversight
The metrics that tell you it worked
| Metric | What it tells you |
|---|---|
| Agent error rate | Share of agent actions that required human correction; measures reliability |
| Time to safe state | Minutes from detecting a bad action to halting writes; measures rollback preparedness |
| Remediation hours per incident | Staff hours needed to investigate and fix an incorrect action; measures failure cost |
| False-positive alert rate | Share of alerts that do not indicate a real issue; measures noise and review burden |
| Days of error-free canary | Consecutive days a canary runs without incident; measures readiness to expand |
| Audit completeness | Share of actions with full before/after snapshots and approval evidence; measures investigability |
| Human review hours saved | Weekly hours freed from manual tasks by the agent; measures productivity gains |
When this is the wrong move
- When you cannot reverse a change within a defined SLA.
- When the expected remediation cost of a wrong action exceeds the projected savings.
- When no clear owner exists who can be held accountable for agent actions.
- When data quality is poor and the agent frequently requests clarifications.
- When you cannot produce reliable audit logs stored separately from production.
The mistakes that cost the most
- Blanket write privilegesEngineers grant broad access to speed delivery; the agent makes a bad change that triggers downstream automation; alerts overwhelm the team and no one halts the agent fast enough.
- Skipping canariesAn agent moves from read-only to full production without restricted scope; a hidden edge case causes repeated incorrect writes and costly reversals.
- Unclear ownershipA data change causes customer churn; product, ops and security each assume another team will fix it, so response stalls and remediation costs rise.
Frequently asked questions
Can I start with read-only access and move to writes later?
Yes. Begin with read-only pilots that log suggestions and measure error rates. After sustained low error rates, proven rollback procedures and owner sign-off, introduce proxy-enforced writes and then small canary writes before any broader direct access.
What is a guardrail I can add right away?
Add an approval threshold: require a named person to confirm any change above a dollar limit or affecting key fields. This immediately reduces exposure while you measure the agent's baseline performance.
How long should a canary run before I expand?
Run a canary for at least one full business cycle relevant to the process. For invoicing that might be a month; for payroll, a full pay period plus one reconciliation cycle. Measure error rate, remediation time and human workload during that period.
Who should I name as the owner?
Name a process owner accountable for outcomes, an engineering custodian who can pause and roll back the agent, and a security owner who manages permissions. All three must sign off before changing stages.
What evidence do auditors want?
Auditors need immutable action logs, before/after snapshots, approval records and an incident history showing drill results. Keep these records separate from the systems the agent writes to so they cannot be altered by a compromise.
Is it cheaper to buy a product with write capabilities or build our own?
Buying gets you guardrails faster but may limit control over rollback behaviour; building offers full control but requires staffing ongoing monitoring and incident response. Choose based on whether you need a custom undo path or can operate within a vendor's controls.
What is the first practical step I should take?
Run a 30-minute risk-to-value audit on one process: identify sensitive fields, estimate remediation cost, and name the owner. That single exercise will tell you whether to pilot read-only or stop.
Stop paying salary for robotic work
Bring your messiest process to a 15-minute audit and we will map where the hours actually go, live on screen. You leave with the number whether or not you work with me.