Posts on ai tips

We rolled back from Opus 5 to Opus 4.8. Memory loss, path confusion, confidently wrong answers, issues we hadn't seen since Sonnet 4.x. Newer isn't always better. Here's the two-part system we use to grade every model release before it costs our customers time, energy, and money.

A step-by-step field guide to building internal model benchmarks: the five failure axes, deterministic-first tiers, and how to catch "confidently wrong" before it hits production.