By Connie · Last reviewed: April 2026 — pricing & tools verified · AI-assisted, human-edited · This article contains affiliate links. We may earn a commission at no extra cost to you if you sign up through our links.

April 17, 2026 · Happycapy Team · 11 min read

HBM Now Eats Two-Thirds of AI Chip Costs — What Epoch AI's Data Means for Microsoft, Meta, and Your AI Bill (2026)

1. What Epoch AI's Data Shows — The Headline Numbers

Epoch AI, the nonprofit AI research organization known for tracking compute trends and training cost trajectories, published data on AI chip component cost shares that has significant implications for every company building or consuming frontier AI. According to their analysis, high-bandwidth memory (HBM) now constitutes approximately 63% of total AI chip component costs — a figure that has risen sharply from roughly 52% just 18 months earlier in Q1 2024.

The absolute numbers are even more striking. Based on Epoch AI's data insights, total annual industry HBM spending roughly doubled from approximately $12 billion per year in early 2024 to around $32 billion per year by Q4 2025. That is a 167% increase in absolute dollars — a rate of growth that substantially outpaces even the most aggressive projections from memory industry analysts at the start of the AI infrastructure buildout.

To put these numbers in context: a single Nvidia H100 GPU costs roughly $25,000–$35,000 to manufacture and procure. Of that figure, according to industry estimates consistent with Epoch AI's broader cost analysis, the HBM component alone represents approximately $15,000–$20,000 per unit — more than the GPU die itself in many configurations. This is not a footnote in the AI hardware story. It is the main event.

The data also reveals a structural shift in where value accrues in the AI supply chain. Traditional semiconductor economics assumed logic — the compute die — would be the premium-priced component. In the AI era, memory has inverted that assumption. The memory companies, long treated as commodity suppliers, now sit at the chokepoint of the most consequential technology buildout since the mobile internet.

HBM Cost Share Over Time

Source: Epoch AI data insights on AI chip component cost shares. 2026 projections are analyst estimates based on HBM4 ramp timelines and announced hyperscaler capex budgets. Data hedged as approximate figures based on published Epoch AI analysis.

2. What HBM Is and Why AI Needs It

High-bandwidth memory is not simply faster RAM. It is a fundamentally different architecture that emerged from the realization that feeding data to modern processors fast enough had become the hardest problem in computing — not the compute itself.

Standard DRAM (like the DDR5 in your laptop) connects to the processor via a relatively narrow bus — typically 64 bits wide, sometimes 128. HBM stacks multiple DRAM dies vertically, one on top of another, connecting them through thousands of microscopic holes called through-silicon vias (TSVs). This stack is then placed directly beside the GPU die on a silicon interposer using a manufacturing technique called 2.5D packaging. The result is a memory interface that is 1,024 bits wide — sixteen times wider than typical DRAM. HBM3e, the generation currently used in Nvidia's H200 and B200, delivers over 3.35 terabytes per second of memory bandwidth per stack.

Why does bandwidth matter so much for AI? Because training and running large language models is not fundamentally a math problem — it is a data movement problem. A GPT-4-scale model contains hundreds of billions of parameters. During each forward pass, the GPU must read the weights for every layer from memory, perform matrix multiplications, and write activations back. During backpropagation, it repeats this in reverse. The arithmetic happens in picoseconds. The memory reads and writes take nanoseconds. The memory bottleneck is the binding constraint.

Without HBM's extreme bandwidth, GPU utilization on large models collapses. The compute units sit idle, waiting for data. This is why you cannot simply use cheaper consumer graphics cards at scale — an RTX 4090 with GDDR6X runs at roughly 1 TB/s memory bandwidth. An H100 with HBM3 runs at 3.35 TB/s. The H100 is not just faster at math; it feeds the math engine data at more than three times the rate. For frontier model training, HBM is not a luxury. It is a prerequisite.

3. The Shift from Logic to Memory — What Changed in 2024–2025

For most of semiconductor history, the processor die — the logic chip — was the premium component. Memory was relatively cheap. Intel sold $500 CPUs with $30 of DRAM beside them. Nvidia's early GPU era followed similar economics. The GPU die was the value-add; the GDDR memory was an afterthought on the bill of materials.

The AI infrastructure supercycle inverted this relationship. Several forces converged simultaneously in 2024. First, the scale of frontier model training crossed thresholds that required more HBM capacity per chip — GPT-4, Llama 3, and Claude 3 generation models each required cluster-scale deployments where memory bandwidth became the limiting variable. Second, Nvidia's Hopper and Blackwell architectures deliberately optimized for memory bandwidth rather than raw FLOPS, signaling where the bottleneck lay. Third, the hyperscalers — Microsoft, Google, Meta, Amazon, Oracle — simultaneously accelerated procurement, competing for a supply chain that had been dimensioned for a lower-demand world.

The result was a supply shock. SK Hynix, Samsung, and Micron found themselves unable to build HBM fast enough regardless of price. Lead times stretched to 12–18 months. Spot prices for H100s, HBM3e-equipped variants, reached $40,000 and above on secondary markets in 2023 and 2024. The economics of the AI chip had fundamentally reoriented: the value — and the cost — had migrated from logic to memory.

By 2025, according to Epoch AI's data, this shift had fully consolidated. HBM was no longer merely the most expensive component — it was the defining cost variable. Two-thirds of what you pay when you buy an AI chip is paying for memory. One-third is paying for everything else.

4. Who Makes HBM — Supply Chain Analysis

The HBM market is an oligopoly defined by extreme barriers to entry. Three companies manufacture virtually all of the world's high-bandwidth memory: SK Hynix (South Korea), Samsung Electronics (South Korea), and Micron Technology (United States). No other company currently produces HBM at commercial scale.

SK Hynixis the current market leader, estimated to hold approximately 50–55% of global HBM supply by units as of 2025. The company secured a first-mover advantage by being the initial supplier of HBM3 and HBM3e for Nvidia's H100 and H200 programs. SK Hynix's relationship with Nvidia is widely described as the most strategically important customer relationship in the semiconductor industry. The company has been investing aggressively in new HBM capacity, including a major fab expansion in South Korea and an advanced packaging facility in Indiana, United States.

Samsung Electronicsholds the second-largest share, estimated at 35–40% of global HBM supply. Samsung made headlines in early 2026 when it became the first company to announce mass production of HBM4, the next-generation standard targeting 50% higher bandwidth than HBM3e. As reported in this site's coverage of Samsung's Q1 2026 record profits, HBM revenue surged over 300% year-over-year, contributing to a six-fold earnings increase. Samsung has historically lagged SK Hynix in HBM qualification timelines but is aggressively investing to close the gap.

Micron Technologyis the smallest of the three at approximately 10–15% market share but is growing fastest in percentage terms. Micron has been the beneficiary of U.S. policy support for domestic chip supply chains, including CHIPS Act funding. Micron's HBM3e is qualified for use in Nvidia's H200 and B200 chips, and the company is ramping capacity at its Boise, Idaho and Singapore fabs. Micron also benefits from being a strategic supply option for U.S. government customers who prefer American-sourced memory, a consideration relevant to the $9 billion AI chip procurement for U.S. spy agencies.

HBM Manufacturer Comparison (2025–2026)

Source: SK Hynix, Samsung, and Micron earnings reports; industry analyst estimates. Market share figures are approximate and vary by quarter. No other company currently produces HBM at commercial scale.

5. Why Memory Dictates Training Economics

The relationship between memory bandwidth and AI training economics is precise and quantifiable. In the GPU cluster configurations used to train frontier models, memory bandwidth determines how efficiently the compute units can be utilized. This metric — MFU, or model FLOP utilization — is the ratio of actual compute throughput to theoretical peak FLOPS.

A cluster running at 50% MFU is effectively wasting half its silicon. The primary reason MFU drops below 100% is not communication overhead (though that matters at extreme scale), nor is it power throttling. It is memory bandwidth. When the GPU cannot get weights from memory fast enough, the tensor cores — the chips that do the matrix math — stall. Every stalled tensor core is a wasted dollar of capital expenditure and operating cost.

HBM solves this by placing memory physically adjacent to the compute die and providing a wide-enough interface that the tensor cores can be kept fed. Higher HBM bandwidth allows higher MFU, which means the same number of GPUs can train a model faster, or a smaller cluster can achieve the same training time. Either way, the economics improve in direct proportion to memory bandwidth.

This is why every new GPU generation leads with memory bandwidth improvements. The H100 (HBM3: 3.35 TB/s) outperformed the A100 (HBM2e: 2 TB/s) by far more than the raw FLOPS difference suggested. The H200 (HBM3e: 4.8 TB/s) improved inference throughput substantially over the H100 despite modest compute die changes. The B200 (HBM3e: 8 TB/s on a NVL configuration) continues this pattern. The trend is consistent: bandwidth is the performance variable, compute is secondary.

For model pricing — the rates OpenAI, Anthropic, Google, and others charge per token — this creates a hard floor. Training cost is dominated by the amortized cost of HBM-equipped GPUs. Inference cost is dominated by serving those models on HBM-equipped clusters. Until HBM prices fall meaningfully, API costs cannot fall proportionally, regardless of software efficiency improvements. This is the structural reason why price cuts like DeepSeek's permanent 75% price reduction are more about distressed pricing and strategic positioning than about underlying cost reductions at the infrastructure level.

6. Microsoft's $190B Capex — How Much Goes to HBM?

Microsoft has committed to approximately $190 billion in AI infrastructure investment over the next five years, according to company disclosures and executive statements tied to its Azure AI and OpenAI partnership build-out. This is the largest single capital commitment in corporate history for AI infrastructure and represents a fundamental reorientation of Microsoft's capital allocation strategy toward compute.

Microsoft has not publicly itemized HBM within this figure. However, working from Epoch AI's data — HBM at approximately 63% of AI chip component costs — and applying industry estimates for what fraction of Microsoft's total capex goes to AI chips specifically versus data center civil construction, networking, power infrastructure, and cooling, analysts suggest that HBM procurement could represent $20–$40 billion of the five-year total.

Microsoft is a confirmed multiyear customer of SK Hynix, Samsung, and Micron. Its Azure AI infrastructure runs primarily on Nvidia H100, H200, and increasingly B200 chips — all of which use HBM3e. Microsoft has also been investing in custom AI silicon through its Maia chip program, but Maia also requires HBM for competitive performance, meaning Microsoft cannot escape the memory premium regardless of which chip vendor it uses.

Beyond direct chip procurement, Microsoft's investment in OpenAI compounds this exposure. OpenAI trains and serves models on Microsoft Azure compute, meaning that every dollar of OpenAI's model training runs — GPT-4o, GPT-5 series, and future generations — consumes HBM that Microsoft has procured and is amortizing. The OpenAI partnership is in effect a bet not just on model quality but on the ability to secure sufficient HBM supply at scale.

7. Meta, Google, Amazon, Oracle — Hyperscaler Spending Compared

Microsoft is not alone. Every major hyperscaler has dramatically increased AI infrastructure spending, and across all of them, HBM is the dominant cost variable in the hardware stack. The scale of commitments announced in 2025 and early 2026 represents a synchronized surge in demand that the memory supply chain is struggling to absorb.

Metaincreased its 2025 capex guidance by more than $10 billion to a range of approximately $60–$65 billion, driven explicitly by AI infrastructure investment across its data centers. Meta's AI training clusters — used to train Llama 4 and future Llama generations — are among the largest in the world, with multi-thousand-chip configurations requiring enormous HBM allocations. Meta is also deploying AI inference at a scale matched only by Microsoft and Google, serving billions of users across Facebook, Instagram, and WhatsApp.

Googlebenefits from a partially differentiated position through its custom TPU (Tensor Processing Unit) program. However, Google still operates massive Nvidia GPU clusters alongside TPUs, particularly for training runs where Nvidia software ecosystem compatibility matters. Google's total AI capex has been estimated at $75 billion for 2025 across Alphabet, with HBM costs embedded in both the GPU procurement and the HBM used in its custom accelerator program.

Amazon Web Services has committed to over $100 billion in total infrastructure investment over multiple years. AWS operates large Nvidia GPU clusters for Amazon Bedrock and its SageMaker AI training services, while simultaneously developing custom AI chips (Trainium and Inferentia). Like Google, Amazon cannot fully escape HBM costs — both Trainium2 and Nvidia chips in its fleet use HBM.

Oraclehas emerged as a significant player through large-scale GPU cluster commitments to AI startups, including a reported 100,000-chip cluster deployment. Oracle's AI infrastructure strategy centers on providing large training clusters to model companies, with HBM costs embedded in its cluster pricing.

Hyperscaler AI Capex Comparison

Sources: Microsoft 10-K and executive disclosures; Meta Q4 2025 earnings and capex guidance increase announcement; Alphabet/Google investor presentations; AWS earnings commentary; Oracle investor relations. Figures based on publicly reported commitments and initial analyst estimates — actual allocations vary.

8. What This Means for AI Model Pricing

The conventional wisdom in AI has been that model pricing will follow a Moore's Law-style trajectory downward — that as training becomes more efficient and inference hardware improves, API costs will compress toward zero. This is true in the long run and has been true in the short run for specific model tiers. But the HBM constraint introduces a hard floor that the conventional narrative underestimates.

API pricing for frontier models is set by three primary cost components: amortized training cost, inference infrastructure operating cost, and margin. The first two are both dominated by HBM-equipped GPU costs. A frontier model API call requires the model weights to be loaded into GPU memory and the inference computation to run. The GPU doing this work costs $25,000–$40,000 to acquire, must be replaced every 3–5 years, and the HBM component of that cost is approximately 63%.

Software efficiency improvements — better quantization, faster kernels, speculative decoding, reduced memory footprint through architectural changes — can reduce the amount of HBM required per token. DeepSeek's architectural choices, for instance, use a mixture-of-experts approach that reduces active parameter count per inference pass, lowering effective HBM requirements. These are real savings. But they operate on the margins of a cost structure where the underlying memory hardware price is still set by a three-company oligopoly in constrained supply.

The practical implication is that frontier API prices — GPT-4o-level and above — are unlikely to drop dramatically in 2026. Prices in the tier below frontier (GPT-4o-mini class, Claude Haiku class, Gemini Flash class) will continue to compress as efficiency gains outpace hardware cost increases for those workloads. But for the most capable models, HBM constraints provide structural pricing support. Users and businesses planning AI budgets around the assumption of sharp price declines in frontier models in 2026 should revisit those assumptions.

9. Implications for Startups — Rent vs. Buy and Cloud GPU Pricing

For AI startups, the HBM cost story has concrete operational implications that affect every stage of the development lifecycle from prototyping to production.

The case for renting is stronger than ever. Building a private GPU cluster requires negotiating multiyear supply agreements with hyperscalers or hardware vendors, securing HBM-based chips on long lead times, and absorbing the full capital cost upfront. For most startups, this is prohibitive. The major cloud providers — AWS, Google Cloud, Azure — can amortize HBM costs across thousands of customers and negotiate better supply terms than any individual company. Even at cloud GPU rental rates of $2–$8 per GPU-hour for H100 class hardware, the economics typically favor renting versus owning for workloads below 5,000 GPU-hours per week.

Cloud GPU prices will not fall quickly.The hyperscalers' pricing for GPU compute reflects their HBM procurement costs plus data center and networking overhead plus margin. As HBM costs have risen — from $12B to $32B per year industry-wide — cloud providers have limited room to cut GPU rental rates substantially. Spot instance prices for H100s have moderated somewhat from the 2023 peak as more supply came online, but forward contracts and reserved instance pricing for Blackwell-class hardware remains elevated.

Startups using API access rather than raw compute benefit from the aggregation economics of the large model providers. When OpenAI, Anthropic, or Google serves inference across millions of users, they can amortize HBM hardware costs more efficiently than a startup running its own cluster. This makes API-based development cheaper per unit of compute for most startup workloads than owning infrastructure — though it surrenders control over model behavior and costs can be unpredictable at scale.

The most efficient option for many product teams is not raw API access either, but multi-model platforms that aggregate access to the best frontier models without requiring per-model subscriptions. Rather than managing separate API keys and billing relationships with Anthropic, OpenAI, and Google, a platform subscription consolidates costs and routes intelligently based on task requirements — much as the Claude Opus 4.7 release showed: new model access that would cost $200/mo direct is available through Happycapy at $17/mo.

10. The 2026–2028 Outlook — HBM4 Ramp, Supply Constraints, and Alternatives

The memory supply chain is not static. Both Samsung and SK Hynix have announced aggressive HBM4 production ramps targeting 2026 and 2027. Micron is expanding HBM capacity at its U.S. and international fabs. The question is whether this supply growth will outpace demand growth — and the current evidence suggests it will not, at least not in 2026.

HBM4 is the immediate next generation, targeting approximately 1.5 TB/s of bandwidth per stack — a roughly 50% improvement over HBM3e. It introduces a base die architecture that improves signal integrity and enables higher capacities per stack. Samsung announced first HBM4 mass production in Q1 2026. SK Hynix has HBM4 in qualification with Nvidia for the Rubin architecture (expected H3 2026). The transition from HBM3e to HBM4 involves retooling fabs, new packaging processes, and customer qualification cycles — supply is likely to be tight through 2026 regardless of production announcements.

HBM4E, the enhanced variant expected in 2027–2028, pushes bandwidth further toward 2+ TB/s per stack and is designed for Nvidia's next-generation architecture beyond Rubin. Samsung and SK Hynix have both disclosed development roadmaps. HBM4E will likely be constrained at introduction for similar reasons — leading-edge memory packaging remains a bottleneck even when prior-generation supply improves.

Alternatives to HBMare being explored but face fundamental physics constraints. GDDR7, Samsung's newest consumer memory generation, is faster than GDDR6 but still delivers bandwidth in the 1–1.5 TB/s range per module — well below HBM. LPDDR5X, used in Apple Silicon and mobile devices, offers excellent power efficiency but modest absolute bandwidth. Processing-in-memory (PIM) architectures, where computation happens inside the memory itself, are a longer-term research direction that could disrupt the HBM economics but are not deployable at scale in the 2026–2028 window.

Custom silicon programs — Google TPUs, Amazon Trainium, Meta MTIA, Microsoft Maia — offer partial insulation from the Nvidia-HBM complex by optimizing memory architecture for specific inference workloads. But all major custom AI chips in deployment today still use HBM or HBM-equivalent memory. The custom silicon hedge reduces Nvidia dependency; it does not eliminate HBM dependency.

The realistic outlook for 2026–2028 is gradual supply improvementoutpacing demand growth by 2027 at the earliest, with HBM4 ramp providing some relief in late 2026 for cloud GPU pricing. The structural oligopoly of three suppliers limits how fast prices can fall even as capacity grows — cartel pricing behavior is not required when the supply constraint is physical. Meaningful HBM price relief that flows through to API pricing is more likely a 2027–2028 story than 2026.

Frequently Asked Questions

Sources and Further Reading

• Epoch AI, “AI Chip Component Cost Shares,” Epoch AI Data Insights — epoch.ai/data-insights/ai-chip-component-cost-shares (primary source for HBM cost share figures cited throughout this article)
• Microsoft Corporation, Annual Report (10-K) and AI infrastructure investment announcements — microsoft.com/investor (source for $190B capex commitment)
• Meta Platforms, Q4 2025 Earnings Call and 2025 Capex Guidance Revision — investor.fb.com (source for $10B+ capex increase)
• SK Hynix Investor Relations, HBM3e and HBM4 supply disclosures — investor.skhynix.com
• Samsung Electronics Semiconductor Division, Q1 2026 Earnings and HBM4 Mass Production Announcement — news.samsung.com/semiconductor
• Micron Technology, HBM3e Qualification and CHIPS Act Capacity Announcements — micron.com/investor
• Related reading: Samsung Q1 2026 Record Profit: AI Chip Demand Drove a 6-Fold Earnings Surge
• Related reading: White House Clears $9 Billion for U.S. Spy Agencies to Buy AI Chips
• Related reading: DeepSeek Makes 75% Price Cut Permanent: What It Means for Every AI Developer
• Related reading: Claude Opus 4.7 Just Released: What's New and How to Access It