HBM4: How Samsung, SK Hynix and Micron Are Fighting Over AI's Memory Bottleneck

Everyone talks about the GPU. Almost nobody talks about the eight stacks of DRAM bolted to its side, and yet those stacks decide how many AI accelerators actually ship. High Bandwidth Memory, or HBM, has quietly become the most contested component in the AI supply chain, and HBM4 is the generation where Samsung, SK Hynix and Micron have to prove they can keep up.

Before you read: Our NVL72 overheating breakdown covered what happens when a rack runs past its thermal budget. CXL and the End of the Motherboard explained why memory placement matters, and this piece is about the memory sitting millimetres from the GPU die.

Table of contents

• The GPU Is Waiting on Memory
• Stacking DRAM Like a Skyscraper
• The Three-Way Fight
• HBM4 Changes the Rules
• Fun facts
• The Catch
• Sources

The GPU Is Waiting on Memory

A modern accelerator spends a lot of its life idle. Nvidia’s B200 can churn through enormous amounts of matrix math, but only as fast as data arrives. That is why it ships with 192GB of HBM3E delivering roughly 8 TB/s of bandwidth. For comparison, the RTX 5090, the fastest consumer card Nvidia sells, tops out around 1.8 TB/s on GDDR7. The compute is rarely the constraint. The memory is.

Bandwidth decides how quickly a large model can stream its weights during inference, which is why a chip with modest math and fat memory pipes can beat a faster chip on real workloads. Nobody puts memory bandwidth on the front page of a keynote, but it is the number that determines how many tokens you get per dollar.

Stacking DRAM Like a Skyscraper

HBM works by stacking DRAM dies vertically, usually 8 or 12 layers today and 16 on the roadmap. Thousands of vertical copper channels, called through-silicon vias or TSVs, punch straight through each die. A base logic die handles the interface. The finished stack then sits on a silicon interposer next to the GPU, wired together through TSMC’s CoWoS packaging. Each die is thinned to a few dozen micrometres, less than half the width of a human hair.

Thinness is where the pain starts. Every additional layer multiplies the chance of a defect, and a single bad die can kill the whole stack. When that stack is already bonded to a GPU that can cost more than a new car, the loss is not just the memory.

The economics are brutal. Industry estimates put HBM at roughly three times the wafer area per bit of conventional DRAM, because the stack carries logic dies, TSVs and interface circuitry that a DDR5 stick never needs. Every wafer that goes to HBM is a wafer that does not go to the DDR5 in your PC, which is one reason consumer memory prices have been so jumpy.

The Three-Way Fight

SK Hynix got there first. HBM was co-developed with AMD, and the standard debuted in 2015 on the Radeon R9 Fury X. SK Hynix then became Nvidia’s main HBM3E supplier and announced mass production in March 2024. TrendForce analysts estimate it held roughly half the HBM market through 2024, which is a lot of leverage for one memory maker to hold.

Samsung is the comeback story, and it spent much of 2024 stuck in Nvidia’s qualification process for HBM3E. That stumble made for brutal headlines in the Korean financial press. The bet now is HBM4, where Samsung plans to build the logic base die in its own foundry rather than outsource it, which would give it more control over the whole stack. Control is the word to watch: whoever owns more of the stack owns more of the margin.

Micron is the outsider with the cleanest pitch. It is the only one of the three headquartered outside Asia, and it has pushed HBM3E on power efficiency, an angle that matters when a single rack already draws 120kW.

Jensen Huang gets the leather jacket and the keynote stage. SK Hynix gets a footnote in the spec sheet.

HBM4 Changes the Rules

JEDEC finalized the HBM4 standard in April 2025, and the headline change is the interface. HBM4 doubles the bus width from 1,024 bits per stack to 2,048 bits, so bandwidth can climb without pushing pin speeds to uncomfortable levels. The base logic die also moves to a more advanced process node, which means some memory controller work can live inside the stack instead of on the GPU.

That shifts the design problem. More layers and wider interfaces mean more heat buried between dies that have nowhere to shed it, which brings us straight back to the NVL72. A memory stack that runs hot throttles the GPU beside it, and no amount of coolant plumbing fixes a hotspot in the middle of a stack.

The packaging story gets harder too. Current stacks rely on microbumps and filler materials that differ between suppliers. Taller 16-high stacks squeeze the height budget, so the industry is eyeing hybrid bonding, where copper pads fuse directly with no solder between them. Whoever masters that first gets a serious head start on the generation after HBM4.

Fun facts

• Did you know the first GPU to ship with HBM was AMD’s Radeon R9 Fury X in 2015, and Nvidia followed with the P100 in 2016?
• Did you know a single 12-high HBM3E stack can hold 36GB, which is more memory than many laptops shipped a decade ago?

The Catch

Here is my problem with the race: most of its numbers come from press releases, analyst estimates and anonymous supply chain sources. Yields are guarded like state secrets, and the qualification drama gets told in scraps. Samsung’s comeback should be judged on shipped volume that clears Nvidia’s bar, not on a booth demo at a trade show.

The concentration risk is real too. One company holding roughly half the market for the most valuable chip designer on Earth means a single fab disruption in South Korea ripples through AI buildouts everywhere. HBM also sells at a steep premium per bit over standard DRAM, and that margin is exactly why all three companies are fighting so hard for capacity.

The next AI shortage may not be a chip that is missing. It may be a stack of memory that nobody could make fast enough.

Sources

• JEDEC: JESD270-4 High Bandwidth Memory (HBM4) standard, April 2025
• Nvidia: B200 and GeForce RTX 5090 product specifications
• TrendForce: HBM market share estimates, 2024 to 2025
• SK hynix Newsroom: HBM3E mass production announcement, March 2024
• Reuters: coverage of Samsung HBM3E qualification with Nvidia, 2024
• AnandTech: AMD Radeon R9 Fury X launch coverage, 2015