Direct answer: choose an AI automation company by its ability to prove six connected capabilities in a real workflow: Outcome, Process, Access, Autonomy, Reliability and Transfer. Do not buy from a demonstration that only completes the happy path. The partner must show what happens when inputs are incomplete, a rule changes, a system is unavailable, an action exceeds its limit or a person needs to take over.
Conventional automation executes relatively predictable rules; AI automation can also interpret content, select tools and choose next steps. That expands opportunity and changes risk. NIST recommends governing, mapping, measuring and managing risk throughout the lifecycle. OWASP highlights agent-specific threats such as goal hijacking, tool misuse and identity or privilege abuse. A provider should translate those principles into testable controls.
The MAKINAI Workflow Control-6
Score each proof from zero to four: zero means absent; one, a promise; two, a documented method; three, partial evidence; four, reproducible execution. Require at least three points in Process, Access and Reliability. An automation that saves time while operating without privilege limits or safe recovery should not proceed. The score compares proposals; it does not conceal disqualifying criteria.
1. Outcome Proof: which economic decision improves?
Start with the unit of work rather than the technology. Define input, output, volume, cycle time, cost, quality, risk, owner and the value of correct completion. Ask every bidder to reconstruct the baseline from the same data. “Automate finance” is too broad; “classify invoices, validate fields and route exceptions before posting” can be tested.
- Required evidence: volume and variation map; current manual time; rework rate; exception cost; SLA; error impact; adoption target; quality measure; stop rule. Red flag: ROI assumes every saved hour becomes a cash reduction.
2. Process Proof: does the vendor understand the real work?
Require discovery with operators, process owners, security, data and system teams. The map should cover standard paths, variants, dependencies, queues, approvals and exceptions. Request a before-and-after design marking work eliminated, automated, assisted and retained under human judgment. AI cannot repair a process without a clear rule or owner.
The vendor should separate deterministic steps suited to rules and APIs from probabilistic steps such as interpretation and recommendation. It should also say where AI should not be used. A hybrid architecture is often more controllable than handing everything to one agent. Ask how rules, prompts, models and reference data are versioned and tested together.
3. Access Proof: does the agent receive only what it needs?
Operational automation touches email, documents, ERP, CRM, finance, HR and internal tools. Require a distinct identity for each agent or service, least privilege, temporary credentials, environment separation, approval for sensitive actions, secret management and audit. OWASP treats identity and privilege abuse as a central agentic-application risk.
Ask the provider to demonstrate what the agent cannot do. Test a malicious instruction in a document, an out-of-scope request and an attempt to reach another client or department. The system should block, record and route the event. Access control cannot rely on a prompt that merely tells the model not to act.
4. Autonomy Proof: who decides, acts and answers?
Define progressive levels: observe; recommend; prepare an action; execute with approval; execute within limits; operate autonomously. For each level, record minimum confidence, maximum impact, financial threshold, permitted data and escalation reason. Autonomy should be earned through evidence rather than switched on completely on day one.
Request a responsibility matrix covering the process owner, approver, operator, security team and vendor. A person must be able to pause the workflow, review context and correct state without creating duplicates. The human exception experience is part of the product; if it is slow or confusing, teams will invent workarounds and lose trust.
5. Reliability Proof: does the system fail safely?
The partner should evaluate interpretation accuracy, tool selection, parameters, final action and business outcome. Averages are insufficient: track false positives, false negatives, unknown cases, drift, cost and latency. NIST SP 800-218A strengthens secure practices for developing and acquiring AI systems; require traceability across versions, tests, changes and components.
Run the MAKINAI exception-replay test. Assemble twenty difficult real cases: incomplete document, conflicting data, unavailable API, expired permission, value above limit, injected instruction and repeated action. Replay the same set before and after every change. The vendor should preserve state, prevent duplication, compensate when needed and create audit evidence.
- Minimum outputs: evaluation set; end-to-end logs; idempotency; exception queues; alerts; runbooks; rollback; recovery objectives; post-incident review; change approval; cost and quality monitoring. “95% accuracy” without the error distribution and impact is not sufficient evidence.
6. Transfer Proof: can operations continue without blind dependence?
Clarify ownership of workflows, connectors, code, prompts, evaluations, derived data and documentation. Require log access, configuration export, model and subprocessor inventory, retention policy, exit plan and training. The provider should show how a model or integration can be replaced without rebuilding the whole process.
A recommended 8-to-12-week pilot
Select one process with meaningful volume, known rules, limited impact and available data. Record the baseline and exception set first. Implement in observation and recommendation modes, then allow bounded actions with approval and reversal. Compare quality, time, cost and risk with the baseline. Scale only after the team operates incidents and the score clears the gates.
Red flags and next step
- Synthetic-data demo; total-automation promise; no operator in discovery; broad administrative access; prompt-only security; no evaluation set; hours-only metric; no exception queue; context-free logs; price excluding model and support costs; undefined ownership; vendor unable to demonstrate a failure.
Structure procurement with https://makinai.co/insights/en/how-to-write-rfp-ai-services, compare agent capability at https://makinai.co/insights/en/how-to-evaluate-ai-agent-development-company and assess delivery models at https://makinai.co/insights/en/in-house-ai-team-or-ai-consulting-firm. MAKINAI designs agents, automation and operations at https://makinai.co/services/en/ai-agents-automation-development.