compare_runs

Compare two persisted model-run outputs side-by-side. Resolves run IDs using full UUID or UUID prefix. Both runs must be successful and belong to the same accessible project. Uses base_run_id as the anchor and comparison_run_id as the changed side in returned labels and deltas. Reuses the same diffing engine as compare_branches so result semantics stay aligned. Optionally pass output_name to focus the returned diff on one top-level output key. Returns: base_run_id (str); comparison_run_id (str); project_name (str); base_status (str); comparison_status (str); base_commit_sha (str|null); comparison_commit_sha (str|null); base_branch (str|null); comparison_branch (str|null); base_run_number (int|null); comparison_run_number (int|null); base_started_at (str|null); comparison_started_at (str|null); base_finished_at (str|null); comparison_finished_at (str|null); base_data_snapshot_ref (str|null); comparison_data_snapshot_ref (str|null); diff (list[{metric, base_value, comparison_value, absolute_delta, pct_delta, significant}]) — per-metric comparison; summary (dict) — aggregated delta statistics plus changed_outputs/table_summaries; output_name (str, optional) — focused output when provided; finance_summary (dict, optional) — finance-aware view of list-of-dict outputs: per-column deltas with trend direction and sign-flip flag, row-keyed pairwise deltas when a period/month/date key is found, or shape_diverged warning when column schemas or row counts are incompatible; errors (list[str], optional) — comparison errors if any.

read-only

Parameters

Name	Type	Required	Default	Description
`base_run_id`	`str`	Yes	—
`comparison_run_id`	`str`	Yes	—
`output_name`	`str	None`	No	`None`

Returns

Returns a structured object. Top-level fields:

base_run_id (string)
comparison_run_id (string)
project_name (string)
base_status (string)
comparison_status (string)
base_commit_sha (string)
comparison_commit_sha (string)
base_branch (string)
comparison_branch (string)
base_run_number (integer)
comparison_run_number (integer)
base_started_at (string)
comparison_started_at (string)
base_finished_at (string)
comparison_finished_at (string)
base_data_snapshot_ref (null)
comparison_data_snapshot_ref (null)
diff (array<object>)
summary (object)
output_name (string)

See the response example below for the full payload shape.

Example

Tool Call

{
  "name": "compare_runs",
  "arguments": {
    "base_run_id": "66666666-6666-6666-6666-666666666666",
    "comparison_run_id": "eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee",
    "output_name": "revenue"
  }
}

Response

{
  "base_run_id": "66666666-6666-6666-6666-666666666666",
  "comparison_run_id": "eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee",
  "project_name": "forecasts",
  "base_status": "success",
  "comparison_status": "success",
  "base_commit_sha": "0123456789abcdef0123456789abcdef01234567",
  "comparison_commit_sha": "abc1234def5678901234567890abcdef12345678",
  "base_branch": "main",
  "comparison_branch": "scenario/aggressive-growth",
  "base_run_number": 17,
  "comparison_run_number": 18,
  "base_started_at": "2026-04-28T10:00:00+00:00",
  "comparison_started_at": "2026-04-28T10:05:00+00:00",
  "base_finished_at": "2026-04-28T10:00:01.420000+00:00",
  "comparison_finished_at": "2026-04-28T10:05:01.590000+00:00",
  "base_data_snapshot_ref": null,
  "comparison_data_snapshot_ref": null,
  "diff": [
    {
      "metric": "revenue",
      "base_value": 100,
      "comparison_value": 120,
      "absolute_delta": 20,
      "pct_delta": 0.2,
      "significant": true
    }
  ],
  "summary": {
    "total_diff": 20,
    "rows_compared": 1
  },
  "output_name": "revenue"
}

Notes

Module: Scenario Comparison
Requires authentication via API token or Auth0 JWT.