There’s a number that shows up in almost every AI business case I’ve seen, and it always sounds like the end of the argument: it saves our engineers X hours a week. It’s in the vendor deck, the board update, the memo justifying the spend. Sometimes X is four, sometimes it’s ten. Whatever it is, it’s presented as the return – the thing that makes the money back.

It isn’t a return. It’s the beginning of a question about a return, and most of the time nobody asks the rest of it.

Time saved is not money saved Link to heading

Start with the accounting, because that’s where the trick hides. There’s an old distinction in finance and government between cashable and non-cashable savings. A cashable saving is real money freed up – a contract you canceled, a role you didn’t backfill – that you can now spend on something else. A non-cashable saving is a notional efficiency: the same work done with less effort, which is genuinely nice but doesn’t produce a dollar you can redirect.

“We save two hours per engineer per week” is almost always non-cashable, and the reason is arithmetic. Two hours across fifty engineers is a hundred hours a week – which looks like two and a half extra people. But you can’t hire two and a half fewer people. The saving is spread in slivers across fifty individuals, none of whom is now free to be redeployed, none of whose salary has changed. Nobody’s budget went down. You’ve manufactured a number that is real in aggregate and worthless in practice, because it never crosses the threshold where it turns into money. It’s the fractional-FTE mirage, and most “hours saved” figures are made entirely of it.

The return lives in the reallocation – and usually there isn’t one Link to heading

Set the accounting aside. Suppose the hours are real, and a specific engineer genuinely has two hours back on a Tuesday. That time is worth exactly one thing: whatever they do with it instead. Nothing more.

This is the part that gets skipped, and it’s the whole game. A saved hour is not value – it’s potential value, and it becomes real only if it’s spent on something worth more than what it displaced. The honest default, the thing that actually happens on most teams, is that it isn’t. The freed time gets quietly absorbed: another meeting, a longer Slack thread, a bit more polish on something nobody needed polished, or just a slightly less pressured week. All of which might be fine. None of which is a return.

The engineer who saves two hours a day doesn’t automatically ship two hours more of valuable work. Unless someone decided, in advance, where that reclaimed capacity was going, it goes nowhere in particular. If you can’t name what the saved hours were reinvested in, you didn’t buy a return. You bought some slack, and slack evaporates.

The mirage at the non-bottleneck Link to heading

There’s a deeper reason the hours can be entirely real and still change nothing, and Eliyahu Goldratt named it forty years ago in the Theory of Constraints: an hour saved at a non-bottleneck is a mirage.

Every system has a constraint – the one place that actually limits how much valuable work reaches the other end. Speed up anything that isn’t the constraint and total throughput doesn’t move; you’ve just created more idle time in front of the bottleneck. And here’s the uncomfortable part for the “hours saved” story: in most engineering organizations, writing code is not the bottleneck. The constraint is deciding what to build, or getting it reviewed, or waiting on a product call, or QA, or the deploy pipeline, or the one overloaded staff engineer everything routes through. AI coding tools are extraordinarily good at speeding up exactly the part – typing the code – that was rarely what held you up. You can hand every developer back an hour and watch the amount of shipped, valuable software stay precisely where it was, because the queue was never at their desk.

And the hours themselves don’t survive scrutiny Link to heading

There’s a final problem, and it’s awkward: the X in “X hours saved” is almost always self-reported, and self-reports appear to be not just imprecise but backwards.

In 2025, METR ran the most rigorous test of this we have – a randomized controlled trial with sixteen experienced open-source developers working on large, real codebases they knew well. Before starting, they predicted AI would make them 24% faster. Afterward, they believed it had made them about 20% faster. Measured, it made them 19% slower. The gap between felt speed and actual speed was enormous, and it ran the wrong way: the time spent prompting, waiting, reviewing, and correcting outweighed the time saved, and the developers couldn’t feel it happening.

You don’t have to treat that as the universal result – it’s one study, on experienced people in familiar code – to take the point. The number in your business case is a feeling, and feelings about our own productivity are exactly the thing we’re worst at measuring. Even where the saving is real, it’s a gross figure: subtract the time spent checking and reworking AI output and the net is smaller, sometimes much smaller, occasionally negative. “Saves ten hours” almost never means ten hours in the bank.

Which is why the ROI keeps not showing up Link to heading

Put those together and you get the result that has caught a lot of executives off guard. MIT’s 2025 State of AI in Business study looked across enterprise generative-AI efforts and found that around 95% had produced no measurable impact on the P&L, despite something like $30–40 billion in spending. The headline got read as “AI doesn’t work.” That’s the wrong reading. The tools work. What failed was the measurement: organizations counted activity and time saved, mistook it for value, and never closed the loop to an outcome anyone could find in the numbers.

It’s an old error in new clothes. Reduce anything to a single, easy, flattering metric and you’ll optimize for the metric instead of the thing it was meant to stand for. Nicole Forsgren and her co-authors made the general case in the SPACE framework: developer productivity is multidimensional, and any team trying to squeeze it into one number is deciding on bad information. “Hours saved” is that one number, dressed up as finance.

How to actually measure it Link to heading

None of this means AI has no ROI. It means “hours saved” isn’t it – and treating it as though it is is the lazy move that produces the 95%. If you want to know whether the spend is paying off, the work is unglamorous but not complicated.

Name the reallocation before you deploy. Decide, up front, what the freed capacity is for – the higher-value work it enables, or the cost you’re actually going to cut. If you can’t name it, you’re not measuring a return, you’re hoping for one.

Measure the outcome, not the hours. Did throughput of things that matter go up? Did a cost come down? Did a number the business or a customer cares about move? Activity is not output, and time saved is the purest activity metric there is.

Find the bottleneck first. Freeing time at a non-constraint is a mirage. If code wasn’t what held you up, speeding it up won’t show in the results, no matter how real the hours are.

Count net, not gross. Subtract the prompting, the reviewing, the rework. Be honest about whether the felt speedup survives contact with the clock.

If you can’t cash it, don’t book it. A saving that never becomes money or measurable output isn’t a saving. It’s a nice feeling with a spreadsheet attached.

AI can absolutely deliver a return. But the return was never the hour you saved – it was what that hour went on to do, and whether that thing was ever the constraint. Save all the time you like. Until you can say where it went and what moved as a result, you haven’t measured ROI. You’ve measured your own relief.