How to Evaluate Mutual AI Resilience
Design a civilian tabletop exercise or pilot that tests whether AI-assisted cooperation adds value beyond a structured human-led process.
What should a MAR evaluation ask?
Start with one decision an operator can make and one measurable civilian consequence. For a fictional freight exercise, ask whether a standardized notice and AI-assisted reconciliation help a team confirm shipment records before a deadline while preserving independent review and local fallback.
Name the hypothesis, proposed mechanism, plausible failure, responsible institution, and population or service affected. Define a unit of analysis such as a consignment, audited notice, service interval, or independent team exercise. State which outcome would change the decision and how much change would matter.
Review the 20 original, untested hypothesesCompare three arrangements under the same conditions
| Arrangement | How the team works | What the contrast can examine |
|---|---|---|
| Current fragmented process | Teams use their usual separate records and contact procedures. | Establishes the documented starting process. |
| Structured human-led cooperation | Teams use a common notice format, named contacts, provenance checks and independent human authorization. | Compared with the current process, examines the bundled change in coordination. |
| MAR AI-assisted cooperation | Teams retain the structured process and use a fixed AI tool to flag inconsistencies for human verification. | Compared with structured human-led cooperation, examines the contribution of the specified AI step under the tested conditions. |
Use the same eligibility rules, comparable case workloads, deadlines, source facts, release times, permissions, staffing and physical resources across the arms. Hold minimum safety and review requirements fixed. Log the model version, prompts or rules, tool settings, data available at each decision, and any rejected suggestions or human overrides. Count training, setup, review and rework as costs. If only one arm receives extra staffing or better underlying data, describe that arm as a different package and disclose the difference.
A separate sensitivity exercise can vary stock, staff or demand. Keep that exercise distinct from a comparison designed to isolate a process change.
Set up the three arrangements in Scenario StudioWrite the protocol before the first run
- Specify the cases, eligibility window, primary outcome, adverse outcomes, minimum useful change, and decision thresholds. Choose the primary analysis and how missing, disputed and late records will be handled.
- Choose the assignment unit. If team members share information, assign entire teams or sessions and plan analysis at that level. Use different but matched case sets and counterbalanced order when the same team runs multiple arms.
- Prepare a fixed timeline of fictional notices and independent answer keys. Include routine cases, delayed messages, common-source duplicates, disagreement and a loss of the shared service. Keep scoring materials separate from the people running the exercise.
- Assign an independent evaluator to reconcile timestamps and score outputs with the same rubric. Record deviations, training effects, workarounds, case exclusions, operational approvals and disagreements as they occur.
- Retain original inputs and versioned corrections. Publish aggregate counts, evaluation scope, the analysis plan, deviations, results by arm, and the review decision. Remove restricted and personal material from public files.
Use a written evaluation guide to align each observation with the exercise objective and carry findings into an after-action report. FEMA Exercise Evaluation Guides
For the AI-assisted arm, specify independent reviewers and document the evaluation data, tools, metrics and operating conditions. NIST AI RMF Playbook, Measure
Check bias, confounding and spillover
| Threat | Example | What to record or change |
|---|---|---|
| Selection bias | Experienced operators volunteer for the AI arm. | Assign comparable teams before the exercise where feasible. Report recruitment, exclusions and skill differences. |
| Confounding | One arm gets an extra analyst or an earlier source release. | Match staffing and release times or report an inseparable package comparison. |
| Learning and carryover | Teams remember the answer from the first run. | Rotate the order and use matched alternate case sets. Report prior exposure. |
| Spillover | A team in one arm shares corrected records with another. | Separate sessions where feasible. Record cross-arm information sharing and treat contamination as an interpretation limit. |
| Observer bias | Evaluators know which answer came from the AI tool. | Mask arm labels on outputs where feasible and use a common adjudication rubric. |
| Attrition or selective recording | Failed cases disappear from the final export. | Account for every assigned case. Report missing outcomes and reasons by arm. |
If random assignment is infeasible, record how teams selected their workflows and why the comparison group can represent what would have happened without the change. A pre/post comparison is descriptive on its own. HM Treasury guidance on analytical methods
Check program evaluation readinessDecide what would warrant further testing
Set a safety floor and stop conditions before anyone sees results. Examples include an unauthorized disclosure, a missed human authorization, or failure of an essential local process when the shared channel is disconnected. A failed gate should trigger a review, even if the average time improves.
Compare time, errors, service delivery, cost and fallback separately. A feasible tabletop result can justify a narrower follow-up exercise. An operational pilot requires permission, domain review, monitoring and a tested way to stop. Record the conditions under which you would reverse the decision.
Document the authority and consent gateRecord the reviewed decision and later changes