Amazon built an internal leaderboard called KiroRank that ranked engineers by how much they used Kiro, its AI coding tool. Meta built a similar one, nicknamed “Claudeonomics” internally, that ranked roughly 85,000 workers by tokens burned through Anthropic’s Claude over a 30-day window. Within months, both companies quietly shut the boards down. The reason wasn’t that AI coding stopped working. It was that a raw usage count, dressed up as a productivity signal, did exactly what any incentive built on the wrong proxy eventually does: it got optimized for its own sake.
Tokenmaxxing turned usage into the job itself
Once engineers understood that rank was tied to volume, some began what CIO called “tokenmaxxing”: spinning up AI agents to perform unnecessary or trivial tasks purely to inflate their score. An Amazon senior vice president told CIO the leaderboard had been built “with good intentions,” but the compute bills it generated were unsustainable. The pattern wasn’t confined to Amazon. Uber’s chief technology officer, Praveen Neppalli Naga, told Fortune that his company burned through its entire 2026 AI coding budget in four months after leaning on internal leaderboards that ranked teams by usage, and Uber’s chief operating officer, Andrew Macdonald, later told TechCrunch that the spending “hadn’t led to a measurable increase in projects or productivity.” Nvidia vice president Bryan Catanzaro summarized the arithmetic that was catching up with all of them:
“For my team, the cost of compute is far beyond the costs of the employees.”
The economics were made worse by what the extra activity actually produced. TechCrunch cites CodeRabbit’s analysis finding that AI-generated code produced 1.7 times more problems than human-written code, and Entelligence AI’s Aiswarya Sankar estimated that companies were spending roughly 44 percent of their tokens simply fixing bugs in code AI had just written. As developer James Shore put it, quoted in the same piece, writing code twice as fast is only a win if you’ve also halved your maintenance costs — otherwise the leaderboard was rewarding people for generating work that would resurface as cost later, just under a different line item.
Amazon’s fix names the actual thing worth counting
Amazon’s response, after deleting KiroRank, was to replace raw token counts with what it calls a “normalized deployments” metric, tied to code that actually ships as successful commits rather than to activity on the tool. That’s a narrower, less flattering number than “tokens consumed,” and it’s also a more honest one, because it measures the thing engineering managers were presumably trying to measure all along: output that merges, works, and stays in production, not effort expended in its general direction. D.A. Davidson analyst Gil Luria has framed the whole episode through Goodhart’s Law, the observation that once a measure becomes a target, it stops being a good measure — which is close to the cleanest possible description of what happened to two leaderboards built around a number that was always meant to stand in for value rather than be it. It’s a dynamic the METR study’s failed control group gestures at from a different angle: when a workforce has already fused its sense of productivity to AI activity itself, measuring the activity and measuring the value stop being the same exercise.
They failed because they measured the wrong noun, activity instead of output.
A metric correction, not necessarily a retreat
It would be too clean to read the leaderboard collapse as pure proof that companies are cooling on AI coding, and the counter-evidence is real. Microsoft’s decision to cancel most Claude Code licenses, pushing engineers toward its own GitHub Copilot CLI by the end of June, lands suspiciously close to its fiscal year-end and a strategic preference for in-house tooling, which muddies any claim that this was purely about the metric being flawed rather than about budget cycles and vendor competition. Meta’s chief technology officer, Andrew Bosworth, has publicly defended heavy token spending as a genuine multiplier, pointing to a top engineer effectively “spending their salary” in tokens as evidence of real returns, not waste. And Amazon, even after scrapping KiroRank, is still pursuing more than 80 percent weekly AI adoption company-wide alongside roughly $200 billion in 2026 capital expenditure, hardly the posture of a company backing away from the technology itself.
What that leaves is narrower than either the hype or the backlash narrative: not a verdict on whether AI coding tools work, but a correction to how their value gets counted. The leaderboards didn’t fail because engineers used AI too much. They failed because they measured the wrong noun, activity instead of output, and any organization that repeats the mistake with a differently named metric will eventually rediscover the same lesson, probably at a similarly expensive price.
