Nvidia Blackwell Systems Expected to Drive Down AI Token Costs

The introduction of Nvidia's Blackwell GPU systems is poised to lead to a substantial reduction in the cost of AI tokens, the fundamental units models use to process information and price AI usage. These advanced systems, which function more like supercomputers than individual chips, are currently being deployed and are projected to operate at scale by the second half of this year.

Concerns regarding AI token costs have been a significant topic in the industry. An AI infrastructure company CEO recently suggested that new, more efficient AI models emerging later this year, supported by this technology, would likely make AI tokens more abundant and considerably cheaper. OpenAI CEO Sam Altman has also acknowledged AI costs as a major issue, indicating the company's efforts to help users gain more value for less expenditure.

Early signs of this trend may be evident in market data. A token spending index tracked by Silicon Data peaked at approximately 2.06 in late May before falling to 1.75 by June 10. Carmen Li, CEO of Silicon Data, indicated that this drop could signify decreasing token prices across numerous AI models.

The primary force driving this potential price reduction is the new wave of technology embodied by Nvidia's Blackwell GPUs. These systems require specialized installations, including water-cooling and new data center configurations. Once fully operational, their impact on efficiency is expected to be profound.

SemiAnalysis, an AI research firm, compared Nvidia's top Blackwell system, the GB 300 NVL72, with its predecessor, the Hopper HGX 200. The Hopper system generated 90 tokens per second per GPU, whereas the new Blackwell system can generate 6,000 tokens per second, representing a 65-fold increase.

Considering energy consumption, SemiAnalysis also evaluated tokens generated per megawatt. The Hopper system produced 54,000 tokens per second per megawatt, while Blackwell generated 2.8 million tokens per second, a 50-fold improvement. In terms of cost efficiency, the Hopper system cost $4.20 per million tokens, compared to just $0.12 for the Blackwell system, making it 35 times cheaper. As new AI models are increasingly trained and run on these Blackwell systems as 2026 progresses, the volume of cheaply generated tokens is expected to surge, likely prompting AI model providers to reduce their token prices.

According to Business Insider, these developments are expected to lead to a significant increase in the availability of cost-effective AI tokens. (Source: Business Insider)