---
title: "Stop counting AI tokens and start measuring shipped code"
slug: "stop-counting-ai-tokens-and-start-measuring-shipped-code"
date: "2026-11-03"
category: "Business & Society"
author: "Simon Banks"
correspondent: "Simon Banks"
correspondentRole: "Labor & Societal Impact Investigator"
readingTimeMinutes: 2
tags: []
originalSourceUrl: "https://www.technologyreview.com/2026/11/02/1147702/stop-counting-tokens-start-counting-what-ships/"
canonicalUrl: "https://aurav.ae/stop-counting-ai-tokens-and-start-measuring-shipped-code"
publisher: "Aura-V Intelligence (aurav.ae)"
---

# Stop counting AI tokens and start measuring shipped code

> AI-assisted development should be judged by cost per merged pull request, not by token consumption.

## Key Takeaways (TL;DR)

- Per-token pricing inflates the cost of high-value coding workflows.
- Cost per merged pull request links AI spend to actual software output.
- Hybrid models that own inference for routine work can cut token costs by 30-50 % (conflict of interest, own or unaudited numbers).

## Token counts obscure real output

When developers hand a coding agent a task, the agent may iterate through planning, generation, testing and debugging, consuming 10-50 times more tokens than a simple chat prompt. Per-token pricing therefore makes the most productive workflows appear expensive.

## Cost per merged pull request as a performance gauge

The author proposes measuring “cost per merged pull request” – total AI-assisted spend divided by the number of changes that pass review and are merged. Tracking rework and defects alongside this metric prevents cheaper but lower-quality changes from skewing results. If the cost falls while usage rises, developers are converting more token consumption into shipped software.

## Matching models to task complexity

A second recommendation is to reserve frontier or on-premises flagship models for planning and novel reasoning, while routing routine code generation, test creation and refactoring to open-weight models the organization runs itself. A gateway can direct requests based on complexity, sensitivity, cost and latency, ensuring premium inference is billed only for the exceptions that truly need it.

## Owned capacity versus rented inference

The article argues that owning inference capacity shifts costs from a variable per-token bill to a largely fixed, fully-loaded expense that drops per-call as usage scales. Owned capacity also keeps proprietary code on-premises, allowing only governed prompts to reach external models. HPE’s internal hybrid platform reportedly reduced token costs by 30-50 % (conflict of interest, own or unaudited numbers) by routing over 150 applications to on-prem open-weight models.

## Test the operating model

At HPE, the hybrid AI platform routes routine workloads from more than 150 internal applications to open-weight models running on premises, including on HPE Private Cloud AI. The result is lower token costs and greater control over sensitive code.

## Build for adoption, not rationing

Owned capacity suits organizations with sustained AI coding demand; low or unpredictable usage still favors API-based access. Most enterprises are expected to run hybrid setups, using frontier APIs for demanding or intermittent work and owned capacity for repeatable volume, with a governed gateway deciding what leaves the building.

*To learn more about HPE Private Cloud AI, please visit HPE.*

*This content was produced by HPE. It was not written by MIT Technology Review’s editorial staff.*

[Source](https://example.com)

## Aura-V Editorial Verdict

The article argues that token counts mask the true value of AI-driven development. It suggests enterprises adopt output-oriented metrics and hybrid inference to control spend while retaining code sovereignty.

---

*Original Think-Tank Source: [Stop counting tokens, start counting what ships](https://www.technologyreview.com/2026/11/02/1147702/stop-counting-tokens-start-counting-what-ships/)*