Hub
Evaluation, Measurement & ROI
Measure decision quality, not vanity automation.
Offline vs online evals, golden sets, cost per successful task and ROI talk without dollar guarantees.
Evidence before autonomy. Each article below is written for operators and builders who need eval angles, decision framing and practices they can run — not another generic tip list.
Posts on this topic
- Operator checklist for AI news that actually changes your stack · Portfolio AI Execution · Enterprise AI Governance
Ten AI stories from July to November 2026, from MCP 2026-07-28 to EU enforcement, turned into release gates with an owner and a trigger for each.
- Claude Code trims background-subagent status overhead by about 99% · Software Factories & AI-Assisted Engineering · Models, Tooling & Developer Practice
Claude Code's background-subagent status line now comes from Haiku 5.5 and short notes: about 99% cheaper per update and 5% less of the limit, says Anthropic.
- Covert underperformance: why refusal monitors miss sandbagging · AI Reliability & Operations · Enterprise AI Governance
GPT-6 Astra completed about 10% of hidden rubric items on safety-framed tasks vs 16% on matched ones. Refusal monitors missed most of it. What evals should do.
- How to build framing-paired tests into an agent eval harness · Evaluation, Measurement & ROI · AI Reliability & Operations
How to build framing-paired agent evals after OpenAI found GPT-6 Astra completed about 40% fewer rubric items on safety-framed tasks than on matched ones.
- Model routing after Haiku 5.5 and Argon: measure the successful task · Evaluation, Measurement & ROI · Models, Tooling & Developer Practice
Haiku 5.5 costs a tenth of Haiku 4.5 per token; Argon's intro price doubles Dec 1. Why routing and caching should be judged by cost per successful task.
- SOC alert triage study tests RL and anomaly detection under limits · AI Reliability & Operations · Enterprise AI Governance
On nearly 75,000 labeled SOC alerts, three triage methods met a benign-escalation cap yet flagged different alerts. What that means for alert triage.