Tough Questions to Ask Before Buying Process Automation

AI Automation Consultant··16 min read
The short answer

I require vendors to define 'done' on your data: signed acceptance runs using your sanitized invoices, exact ledger outputs, payment tied to passing tests, runnable exports, credential transfer, complete runbooks, a named operational owner with SLAs, and a contract for who pays manual catch-up work after failures.

Key takeaways

  • A concrete acceptance test on your data converts a demo into an operable system or stops payment until the vendor proves it.
  • Runnable exports, test datasets and runbooks are the best predictors of a system your team can operate without the vendor.
  • If you do not price failure modes into the contract, you will carry manual catch-up work and pay for it repeatedly.
  • Operational ownership must be explicit: a named owner, incident contact, SLA and escalation path.
  • Exit terms and a tested transfer exercise determine whether you can recover when the vendor leaves or systems change.

Signs this is happening in your business

Does this look familiar?

  • A polished demo handled ideal invoices but failed on your suppliers' real formats.
  • Finance still opens emails and re-types amounts into the ledger after go-live.
  • Vendors promise 'training' and hand over a single slide deck with no runnable artifacts.
  • Incidents are fixed by the pilot developer, not the operations team.
  • You have no agreed test data, so you and the vendor argue about whether acceptance was met.
  • An upstream change stopped the automation and nobody noticed until month-end reconciliations failed.
  • Vendors refuse to hand over credentials, saying they 'manage' the connection as a service.

If several of these are true at the same time, the problem is usually the process rather than the people running it.

What this is costing you a year

Your team-wide payroll bleed

(Team hours per week x hourly cost of employees x 52) = annual payroll loss

  • People doing this work4
  • Hours each per week6
  • Hourly cost of employees$45
  • Weeks52
Lost to manual work every year$56,160

The annual payroll cost of four people spending six hours a week each on manual invoicing and checks.

Find the number before you spend anything

Run the formula above on your three most repetitive processes. If any of them clears a year of a salary, that is the one to bring to a call.

Discuss your Problem

Questions to ask before buying process automation that prove handover works

I focus contracts on handover because the transfer determines whether automation saves payroll or becomes another vendor dependency. Skip demos and force transfer proof: sample supplier inputs and outputs, an acceptance run on your data, runnable exports, a scripted credential handoff and a signed checklist proving your operators can run the system.

  • Demand sample input/output for your real suppliers, not sanitised examples.
  • Require an acceptance run using your test data before final payment.
  • Insist on a complete export of configuration, credentials, and runnable scripts at handover.

What acceptance criteria must look like

Acceptance criteria must be precise, measurable checks tied to payment and signed by your operations lead. Require exact test datasets, the expected ledger outputs, automated tests that run those datasets, clear operational performance targets, and payment milestones that withhold final funds until those tests pass. Vendors that resist are selling demos, not operations.

  • Sample input: a representative set of supplier invoices, including edge and error cases.
  • Expected output: ledger entries and reconciliation lines matching your chart of accounts format.
  • Automated tests: scripts or procedures that run the sample inputs and compare actual outputs to expected outputs.
  • Performance target: a high processing rate with a low manual correction load.
  • Payment milestones: split payments with final tranche paid only after the acceptance run passes.

What a usable handover deliverable contains

A usable handover is a folder your operations lead and IT can use to run and test the system without the vendor. It must include runnable code and configuration exports, runbooks, credentials transfer procedures, test datasets and the automated tests, plus recorded training. Missing any item means you received a pilot, not an operable product.

  • Runbooks and playbooks for normal operation and incident handling.
  • A runnable export of code and configuration that your team can start in a test environment.
  • Credential handoff procedure and a timeline for rotating credentials.
  • A set of test datasets and the automated tests that exercise them.
  • Training sessions, recorded and scheduled for the day-to-day operators.

Handover training and what documentation actually helps

Training must be a sequence: a transfer rehearsal, live sessions, recordings, role-specific checklists and shadowed runs until operators can run the system without vendor help. Documentation must be step-by-step procedures that let a new hire monitor, reconcile and perform rollbacks; conceptual slides and a single deck do not meet this bar.

  • Operator checklist: daily, weekly and month-end routines.
  • Incident playbook: how to diagnose failures and perform manual workarounds.
  • Reconciliation guide: how to compare system outputs to your ledger and correct mismatches.
  • Training schedule: at least two live sessions and recordings accessible to your team.

A polished demo is not a deliverable; an acceptance run on your data is.

Operational ownership and SLAs you must demand

Operational ownership must name who watches the system, who fixes it and how quickly they respond. Vagueness transfers the cost of every break to you. Insist on a named owner, measurable acknowledgement and resolution targets for priorities, a clear escalation path, and a published cost model for ongoing support and change requests.

  • Named operational owner and contact details for incident reporting.
  • Response SLA: acknowledgement and resolution targets for high, medium and low priorities.
  • Escalation path: who is notified if SLA misses occur.
  • Ongoing costs: hourly rates for support, retainer options, and change request pricing.

Failure modes and rollback plans the vendor must document

Every automation fails; the issue is whether you keep control when it does. Require the vendor to list explicit failure cases, automated retry behaviour, notification thresholds and manual fallback steps tied to business impact. Include reconciliation scripts and a contract clause that specifies who pays manual catch-up when third-party changes break the flow.

  • Explicit failure cases with severity levels, such as supplier format change or downstream unavailability.
  • Automated retry logic and limits, with clear notification thresholds.
  • Manual fallback steps so staff can continue processing without the automation.
  • Reconciliation procedures and scripts for replaying or correcting missed items.
  • Contract clause defining who pays for manual catch-up when third-party changes break the flow.

Data, security and compliance questions that can't be waved off

You must know what the vendor can see, where data is stored, how long it is kept and who is liable if it is lost. Make access levels, encryption practices, audit logs, credential handling, retention and insurance contractual items so finance and legal can accept the work without surprises.

  • Access: least-privilege lists for every account the vendor uses.
  • Data at rest and in transit: how data is protected and where it is stored.
  • Audit logs: evidence of who touched what and when, available to your team.
  • Credential handling: transfer process and rotation schedule.
  • Liability: contractual language on data loss, breach responsibility and required insurance.

Change management and exit terms you must have written down

Scope creep and upstream changes kill projects. Put change processes, pricing bands, ownership of work product, source transfer and a tested transfer exercise in writing. Require the vendor to help move operations to another provider on a defined timeline so you do not continue to pay fees and carry manual work after termination.

  • Change request process with pricing bands and a maximum time to quote.
  • IP and source ownership: you own exported configuration, scripts and runbooks at handover.
  • Exit transfer plan: vendor must perform a transfer run with your staff and hand over credentials, documentation and a runnable export.
  • Penalty or holdback: final payment retained until a successful transfer run is complete.

If failure economics are not in the contract, you will pay repeatedly for the vendor's mistakes.

How this usually fails in practice and why

I see the same sequence: teams admire a shiny demo, release an early payment, accept skimpy documentation, and then find the vendor remains the essential fixer. When systems change or the vendor vanishes you are left with undocumented behaviour, no runbooks and a growing manual backlog because the acceptance and transfer runs never happened.

  • Shiny demo -> early payment -> provisional documentation -> vendor remains key fixer -> systems drift -> vendor becomes unavailable.
  • No acceptance run -> payment released -> disagreements about delivery -> buyer carries manual backlog.
  • Failure to define who pays for third-party API changes -> vendor blames provider -> you pay overtime for staff to keep processes running.

Identify your biggest operational bottleneck

Most owners already know which process is the problem. What they do not have is the annual figure attached to it. That is a 15-minute conversation, not a project.

Discuss your Problem

Where the time goes: a per-process breakdown

Map the task into capture, validation, posting, reconciliation and reporting to see where your team's hours disappear. Each area delivers different automation value and leaves different residual checks. I advise mapping one complete cycle with your team and extracting representative samples for the vendor to use in acceptance testing.

  • Capture: receiving invoices and entering data.
  • Validation: checking totals, duplicates and exception flags.
  • Posting: mapping to accounts and creating ledger entries.
  • Reconciliation: matching payments and resolving mismatches.
  • Reporting: month-end numbers and variance explanations.

How to make the vendor prove it on your data

Nothing forces realism like your actual invoices. Provide a sanitized sample set that preserves structure and edge cases and require the vendor to run acceptance against that set and produce the exact outputs you expect. Keep signed logs of the run and require the vendor to reproduce and fix any discrepancies found.

  • Sanitise data but preserve structure and edge cases.
  • Run a signed acceptance test and keep logs of the run.
  • Require that the vendor reproduce any discrepancies found during the run and update automated tests accordingly.

When buying fixed‑scope automation, insist on build‑to‑operate obligations

Fixed‑scope vendors sell a finished thing; most stop short of making it operable by your staff. Require a build‑to‑operate clause: the vendor must train operators until they can run the system and must remain accountable under SLA for a defined handover period. Make operator competency the acceptance condition.

  • Build‑to‑operate period: a defined time after handover where the vendor remains responsible for incidents at the agreed SLA.
  • Transfer rehearsal: vendor and your operators perform a full run with the vendor stepping back progressively.
  • Acceptance signoff requires operator competency validation, not just a vendor checklist.

Documentation that lives in someone's head is a liability, not a handover.

What to measure to know the automation is actually saving money

Measure the inputs and outputs that touch payroll: weekly invoice volume, hours spent per invoice, exceptions and time to close month-end. Capture baseline measurements before work begins and use a short KPI list to gate final payment and ongoing support. Those numbers tell you if the automation reduced labour costs.

  • Baseline: average invoice processing time and number of exceptions per week.
  • Post-go-live: average processing time, exceptions, operator hours on manual catch-up, and time to close month-end.
  • Use these to validate whether the system meets the performance targets in the acceptance criteria.

How to run a six-question vendor scoping call

Use a 30-minute script that forces the vendor to reveal whether they understand handover and operations. Ask six direct questions covering acceptance, handover artifacts, operational ownership, top failure modes and manual fallbacks, credential handling and exit terms. If they cannot answer concisely, they are not ready for fixed-scope delivery.

  • What exact samples will you run during acceptance and who signs off?
  • What handover artifacts do you provide and how will the transfer rehearsal work?
  • Who will monitor and fix incidents for the first months and under what SLA?
  • List the top three failure modes and the manual fallback for each.
  • How do you handle credential transfer and data retention?
  • What are the exit terms and what will you deliver on termination?

The next step you should take tomorrow

Schedule a 30-minute scoping call with each shortlisted vendor, run them through the checklist and secure written commitments you can insert into contract clauses and payment milestones. If you want help running the call and converting answers into binding acceptance criteria, see my services.

  • Schedule a 30-minute call with any shortlisted vendor.
  • Use the six core questions and insist on written deliverables.
  • Hold back final payment until the acceptance run passes and the transfer rehearsal is complete.

Why I care and how I've seen this go wrong

I work on procurement and handover because I have sat in meetings where a paid automation was abandoned when it broke. The recurring failures are the same: no runbooks, no acceptance on real data and no priced failure modes. If you want help, read about me, my portfolio case studies, or how to work together.

  • I expect vendors to provide runnable exports and to sit with your operators for transfer runs.
  • If they say 'we manage it for you' get the terms in writing: duration, SLA and cost for continued ownership.

Where businesses waste the most time

  1. Re-keying invoice dataWastes staff time and introduces transcription errorsHave the routine read invoice fields and populate existing systems without replacing themA person still checks: Operator spot-checks a sample of processed invoices
  2. Chasing suppliers for missing detailsDelays payments and creates exceptions at month-endAutomate exception detection and generate a task list your staff action in existing email toolsA person still checks: Staff approve and send supplier queries
  3. Manual reconciliationExtends month-end close and requires overtimeMatch payments to ledger lines and surface only mismatches for human reviewA person still checks: Finance reviews and resolves flagged mismatches

How this compares with what you already run

Operational area The manual way The automated way Annual business impact
Data entry Team retypes invoice fields from emails System reads invoices and posts entries automatically Saves operator hours, reducing payroll spend
Validation and exceptions Staff triage mismatches and chase suppliers Exceptions routed to a queue with clear instructions Fewer missed invoices and lower overtime costs
Reconciliation Manual match of payments and ledger lines Automated matching and flagged mismatches Faster month-end close and lower consultancy fees
Reporting Finance compiles spreadsheets and updates reports Reports generated from processed data with variance notes Reduced month-end labour and shorter close cycle
Incident handling Ad hoc fixes by the most senior engineer Named owner with SLA and runbook Predictable cost of fixes and less downtime
Training new staff On-the-job shadowing with inconsistent handover Recorded playbooks and checklists Lower ramp time and fewer errors
System changes Breaks cause firefighting and unpaid overtime Change process with pricing and tested transfer Less operational drag and controlled change costs
Comparing manual and automated approaches for invoice and month-end work

What the end-to-end workflow looks like

Invoice capture to month-end reporting

  1. Receive invoiceInvoices arrive by email or supplier portal and are collected into a queueAutomation reads and classifies invoices
  2. Validate fieldsAmounts, supplier codes and dates are checked against expected patternsAutomation flags exceptions and auto-corrects common formatting
  3. Map to accountsEach invoice is assigned the correct ledger code and departmentAutomation maps routine invoices; edge cases go to a human
  4. Post entryEntries are posted into the accounting systemAutomation posts routine items; operator approves exceptions
  5. Reconcile paymentsPayments are matched to invoices and differences investigatedAutomation matches and creates a mismatch list for review
  6. Produce month-end reportFinance aggregates numbers and explains variancesAutomation generates draft reports and highlights variances for human commentary
  7. Incident resolutionIf an error occurs, the team follows the runbook to recoverStays human

The metrics that tell you it worked

Metric What it tells you
Average invoice processing time Measures minutes from receipt to posted ledger entry; lower means less payroll spent
Exceptions per 100 invoices Counts items requiring human intervention; shows automation quality
Operator hours per week on invoice work Tracks staff time spent; translates directly to payroll cost
Time to acknowledge incidents Measures how quickly someone starts work on a failure; reduces downtime cost when low
Time to resolve critical incidents Measures time until normal operation is restored; ties to business continuity
Month-end close days Counts calendar days to final financials; shorter close frees leadership time
Manual backlog items after a break Number of items requiring manual work after an automation failure; shows business exposure

When this is the wrong move

Do not automate this yet

  • When you lack a stable upstream system: automation amplifies instability and increases firefighting.
  • When you cannot provide real test data: vendors will tune to ideal inputs and miss real edge cases.
  • When the process owner is not committed to operating the system: automation without ownership fails.
  • When the volume is so low that building and maintaining automation costs more than payroll savings.

The mistakes that cost the most

  • Paying on demoVendor shows a polished demo, you pay an upfront tranche, documentation is thin, vendor stays essential, and when the system breaks you pay for fixes and carry manual work.
  • Accepting vague SLAsContract lists vague response promises, incident response is slow, escalation is unclear, and the buyer absorbs downtime and overtime costs during month-end.
  • Not testing handoverVendor hands over documents but not a runnable export, your team can't perform a transfer rehearsal, and when the vendor leaves you cannot run or transfer the system.

Frequently asked questions

What single question reveals whether a vendor understands handover?

Ask: 'Can you run an acceptance test using our sanitized invoices and produce the exact ledger outputs we expect, and will you sign off only after our operators complete a transfer rehearsal?' If they hesitate, they are not prepared to hand over an operable system. A vendor who accepts this can also provide the runnable tests and a transfer schedule.

How much of final payment should be held back until after acceptance?

Hold back a meaningful final tranche large enough to ensure vendor commitment to remediation and transfer. Link the holdback to passing the acceptance run on your data and a successful transfer rehearsal with your operators; this converts vendor incentives into deliverable outcomes.

What counts as acceptable documentation at handover?

Acceptable documentation is step-by-step playbooks, runnable test datasets with expected outputs, recorded training sessions, and a transfer checklist signed by your operator. Conceptual diagrams or a single slide deck do not count. If you cannot run the system from the provided artifacts, the handover is incomplete.

Who should own day-to-day monitoring after handover?

Your operations team should be the named owner for day-to-day monitoring, even if the vendor offers a managed service. The contract should name the owner, list their responsibilities, and include a build-to-operate period where the vendor is accountable under SLA until your staff are competent.

If an upstream system changes and breaks the automation, who pays?

You must define this in the contract: either the vendor absorbs the cost for a defined period, or you accept that changes outside the vendor's control are chargeable work. An explicit clause prevents post-delivery disputes and forces vendors to design observability and graceful failure paths.

Stop paying salary for robotic work

Bring your messiest process to a 15-minute audit and we will map where the hours actually go, live on screen. You leave with the number whether or not you work with me.

Discuss your Problem

Leave a Comment

Your email address will not be published. Required fields are marked *