Hire the consultancy that can trace a product or CX recommendation back to research with real, appropriately recruited people. Ask to see the recruitment plan, data permissions, raw-to-finding audit trail, contradictory cases and a named human reviewer. AI can speed transcription, clustering and hypothesis generation; a synthetic persona is not a substitute for an observed customer, and an elegant summary is not a validation study.
Start with the decision, then choose the method
Exploring why users abandon a flow may require interviews and observation; estimating how common a problem is calls for a suitable sampling and measurement design; comparing two interactions calls for an explicit task and success criterion. Ask each bidder which decision changes after the research, whose behavior matters, who could be absent from recruitment and what result would overturn the team's favorite hypothesis. An all-purpose AI interview dashboard should not answer every question the same way.
The MAKINAI Research Evidence Chain-6
- 1. Decision and method — business question, target users, observable behavior, method and the limits of inference.
- 2. Recruitment and coverage — participant source, screening, incentives, exclusions, accessibility needs and gaps between customers and prospective users.
- 3. Permission and handling — notice, recording permission, transcript access, vendor tools, subcontractors, retention and deletion.
- 4. Traceable synthesis — every finding links to authorized observations and distinguishes participant statements from researcher interpretations and AI-generated suggestions.
- 5. Contradictions and uncertainty — rival explanations, dissenting observations, limits on generalization and a plan to validate with other evidence.
- 6. Decision handoff — prioritized opportunities, a testable next step, research repository and ownership that lets the buyer revisit the underlying evidence.
For each proof record absent, claimed, demonstrated or independently reviewed by your team. Treat missing appropriate participants, missing permissions or no observation-to-finding link as blockers. AAPOR's disclosure standards emphasize sufficient methodological detail for independent review; do not apply survey-style representativeness claims to a handful of qualitative sessions. Evidence Chain-6 is a MAKINAI editorial buying tool, not an external certification.
Give finalists one paid contradiction test
Use the same small set of invented, clearly labeled notes for a hypothetical product, with a dominant pattern, a dissenting case and an ambiguous transcript line. Ask each finalist to produce a finding ledger with supporting excerpts, counterexamples, confidence limits and the next real-user study. Separately request a costed recruitment and consent plan. The fictional notes test analytical judgment only; never let a vendor pass them off as completed customer research.
Compare speed to a defensible decision, how omissions are exposed, feasibility of recruiting real users, cost per useful research round and whether your team can reproduce the finding. UK Government Digital Service guidance separates raw observations, findings and actions. NIST's generative AI profile adds a risk-management lens; neither source certifies a private provider. AI-assisted synthesis may save analyst time, but it cannot rescue an irrelevant sample.
Contract for privacy and repeatability
Specify participants and exclusions, outputs and review rights, approved tools, confidentiality, access to recordings and redacted notes, retention, deletion, third-party access and editable deliverables. In the United States, have counsel review applicable state, sector and company policies; UK privacy guidance is a useful operational reference, not US law. In procurement, connect https://makinai.co/insights/en/assess-data-readiness-before-hiring-ai-company, https://makinai.co/insights/en/how-to-evaluate-ai-consulting-proposals-scorecard and https://makinai.co/insights/en/evaluate-ai-consultancy-knowledge-transfer-before-hiring. MAKINAI can help frame the product decision and a comparable research-partner challenge: https://makinai.co/services/en/branding-creative-digital-experience-agency.