Forecast Error and Baseline Comparison
How accurate were the supplied forecasts against a baseline on the same observations?
Worked example
A planning team can check whether archived demand forecasts beat a fair benchmark on the same observations. Confirm forecasts were available before the period they predict.
Three fictional observations give a forecast mean absolute error of about 1.33 requests. The supplied baseline averages 4 requests of absolute error on those same observations.
Try it yourself
The calculator opens with the fictional example already filled in. Change one input, choose Calculate locally, and compare the result to the original values below. Use Reset fictional fixture to start over.
An unknown result means the available inputs do not support a value. Check the calculator’s Warnings tab for what is missing or uncertain.
- Matched observations 3 records
- Records missing a required value 0 records
- Forecast mean absolute error 1.333333 requests
- Forecast root mean square error 1.632993 requests
- Forecast signed bias 0 requests
- Baseline mean absolute error 4 requests
Import local JSON or CSV
Files stay in this browser. Maximum 1 MiB. JSON imports are recalculated. CSV supports flat observation tables, with column types taken from the displayed fixture. Empty cells become null.
Compare the arrangements
Keep physical resources and evaluation conditions equal. With identical inputs, all three approaches produce identical arithmetic. The comparison refuses material differences in declared conditions; it does not verify those declarations.
Privacy: comparison inputs may contain sensitive notes or operational details. Storage is off by default. Do not enter personal records or sensitive infrastructure details. Reproduction exports include supplied inputs and notes, including raw small-cell inputs suppressed from CORE-009 result tables. Suppression of displayed results does not make a reproduction export safe to publish.
Executive brief and reproducible export
Review for personal, confidential or sensitive material before saving or sharing. No automatic submission occurs.
Read the generated brief
Method, inputs and limits
Compare two already issued forecasts on the same observed targets using absolute error, squared error and signed bias. Method version 0.2.0. Produce new forecasts, confirm that information was available at the declared cutoff, or infer future accuracy.
| Input | Unit / type | Required interpretation |
|---|---|---|
| periods — Forecast and observation records | ISO dates or timestamps and matching target units / number | Each row has cutoffAt, targetAt, actual, forecast and baseline. Optional baselineCutoffAt must match cutoffAt. Optional unit must match the declared target unit. Null means a value is unavailable. |
Visible mathematics / decision rules
Error = forecast − actual. Mean absolute error is the mean of absolute errors.
Root mean square error is the square root of mean squared error. Signed bias is the mean error.
Baseline comparison uses only rows with all three values present.
What this result cannot establish
- Target dates must be unique and strictly after their declared cutoffs.
- The same cutoff, target definition and units apply to both forecasts. Inspect the reported range of forecast lead times before interpreting a pooled result. The matched sample can be biased if missingness is selective.
What would falsify the assumptions?
- Audit dated forecast records and confirm both forecasts used only information available at the declared cutoff.
- Assumptions fail if the supplied definitions, units, period, scope or source records are wrong or incomplete.
- An independent source audit or replication may contradict an input or the recorded dependency.
- This method creates no resources. Physical conservation is checked in the inventory and transfer ledgers; descriptive graphs and checklists are not resource-flow models.
- The same equations and review checklist can be reproduced with a spreadsheet or structured human-only process.
- This is deterministic arithmetic, a descriptive comparison or a structured governance check, not a fitted causal effect or forecast. Fictional inputs remain an exercise.
- Equal or worse measured errors, net review effort, disclosure burden or fallback under MAR would show no added benefit or added risk; no improvement is assumed.
Reproducible fictional fixture and independently calculated expectations
This fixture is synthetic. It does not describe a country, government, company or event.
{
"observations": {
"unit": "requests",
"origin": "synthetic",
"sources": [
{
"id": "SYN-023",
"kind": "synthetic",
"label": "Fictional planning exercise",
"url": null
}
],
"periods": [
{
"cutoffAt": "2026-01-01",
"targetAt": "2026-01-02",
"actual": 12,
"forecast": 14,
"baseline": 8
},
{
"cutoffAt": "2026-01-02",
"targetAt": "2026-01-03",
"actual": 10,
"forecast": 8,
"baseline": 6
},
{
"cutoffAt": "2026-01-03",
"targetAt": "2026-01-04",
"actual": 8,
"forecast": 8,
"baseline": 4
}
]
},
"parameters": {}
}Expected checks
{
"matched": 3,
"mae": 1.333333,
"rmse": 1.632993,
"bias": 0,
"baselineMae": 4
}Download input fixtureAuthoritative background references
- NIST: AI Risk Management Framework — Voluntary governance background; not validation of MAR. Reviewed 2026-11-09. Reference, not endorsement or validation.
These sources inform definitions or review practices. They do not validate this implementation or endorse MAR.