How to Test Interactions Between MAR Hypotheses

What is an interaction between hypotheses?

A pair of hypotheses may propose related mechanisms. An interaction is a narrower measurement question. Does changing one defined procedure make a different amount of difference depending on whether a second procedure is in place?

In the Lab, choosing several hypotheses lets you inspect their mechanisms and trade-offs side by side. A combined estimate needs a separate study with measured outcomes. To test an interaction, translate two ideas into procedures that can be applied separately to the same measurable problem.

Build a four-condition exercise

In a fictional coordination exercise, A introduces standardized notices with source and timestamp fields. B introduces a pre-rehearsed independent review procedure when the shared service is unavailable. Both changes are tested on the same case eligibility, workload and staffing. All four cells retain the minimum local safety procedures. Each operator records whether an eligible notice is correctly reconciled before the decision deadline.

Test neither change, A alone, B alone, and both. Randomize or counterbalance order where feasible and replicate within each condition. A one-run-per-cell worksheet illustrates the contrast but cannot estimate between-run variation. NIST Engineering Statistics Handbook, full factorial designs

Calculate the interaction on one stated scale

On the percentage-point scale, the standardized notice change is worth 69 − 62 = 7 points under usual fallback practice and 85 − 70 = 15 points with rehearsed review. The interaction contrast is the difference between those two changes. It is 85 − 69 − 70 + 62 = 8 percentage points for the fictional data.

The sign depends on the outcome. Positive is favorable only when a higher outcome is preferable. A numerical interaction can also change when expressed as a ratio instead of a difference. Choose a scale in advance and show all four cell results. A missing, ineligible or materially different cell leaves the four-cell contrast unsupported.

The equation has the same arithmetic form as a four-cell difference-in-differences contrast. Here the two dimensions are procedure settings, so the estimand and design are different. The Workbench program evaluation calculator expects treated and comparison groups with aligned pre/post periods. Keep simultaneous factorial cell results in the evaluation worksheet.

Check dependence, costs and contradictory results

• A shared provider, identity system or evidence origin can affect both procedures at once. Document common dependencies and separately test loss of the shared channel with the authorized local fallback.
• A standardized process may make reconciliation faster and still raise false acceptance of copied or incorrectly timestamped records. Track those errors independently of timeliness.
• Two changes may both require the same reviewer or available stock. Hold those resources constant and count the additional workload rather than assuming capacity will grow.
• If one procedure can only run when the other is enabled, a separate estimate for each procedure is unsupported. Test the feasible package against the baseline and explain the dependency.

Plan for several hypotheses and outcomes

Choose the smallest set needed for the decision. Six independent two-level factors already require 64 settings for a complete factorial design before repeating any runs. Select a primary interaction and primary outcome before looking at the data. Treat additional comparisons as exploratory and report all comparisons attempted.

If a smaller design is necessary, document which effects it cannot distinguish. Validate promising findings in new sessions with independent teams, different cases, and the same declared outcomes. A finding under those conditions remains bounded by the participants, time period, permissions and tested mechanism.

A measured interaction in a fictional civilian workflow does not establish a combined effect for the source hypotheses, a diplomatic outcome, or a change in war probability.