Skip to content
Artwork for EUV The Focal Point
EUV The Focal Point · November 9 · 25 min

[072] Deep Dive Topic - Memory: How AI Reshapes DRAM

This post was created using AI. Please check the information if you want to use it as a basis for decision-making. Memory: How AI Reshapes DRAM Why can an AI rack use as much working memory as thousands of smartphones, yet remain a completely different manufacturing problem? This episode connects DRAM cell physics, LPDDR, DDR, GDDR and HBM to fab specialization, shared capacity and customer-specific engineering. It examines DDR4 migration and agentic AI without assuming that older memory is always available or that more CPU work means less RAM. KEY TAKEAWAYS - DRAM and logic fabs share tool families but optimize different devices and process flows; repetitive patterns still need rigorous control. - LPDDR's mobile energy-efficiency heritage also serves data-center designs, where deliberately parallel channels can provide high aggregate bandwidth. - Capacity, bandwidth and latency are separate requirements; adding packages alone does not guarantee more bandwidth. - The named GB200 NVL72 specification totals about 30 TB of CPU LPDDR5X plus GPU HBM3E. At an assumed 8 GB RAM per phone, this is about 3,800 phones in capacity only, not an equivalence in storage, bandwidth or manufacturing effort. - HBM competes for DRAM resources and adds its own stacking and test constraints; front-end and backend capacity must both be available. - DDR5 improves bandwidth and concurrency, but AI has no universal DDR5 requirement. Declining DDR4 demand can coexist with scarcity if supply contracts faster. - Agentic AI adds CPU execution and orchestration while potentially raising memory and bandwidth demand; it does not establish a financial rescue for Intel. - Consolidation reflects capital intensity, technology costs and cycles. Custom HBM and system co-design offer differentiation, with customer-concentration and flexibility risks. GLOSSARY - DRAM: Dynamic random-access memory; working memory storing charge that needs refresh. - DDR: Double data rate; a family of system-memory interfaces, including DDR4 and DDR5. - LPDDR: Low-power double data rate; energy-efficient DRAM interfaces rooted in mobile applications. - GDDR: Graphics double data rate; a separate graphics-memory interface family, including GDDR7. - HBM: High-bandwidth memory; stacked DRAM with a wide interface close to a processor. - EUV: Extreme ultraviolet lithography; patterning used for selected critical advanced-DRAM and logic layers. - TSV: Through-silicon via; a vertical electrical connection through a silicon die. - KV cache: Key-value cache; saved intermediate attention state used during model inference. - Base die: The control and interface die beneath an HBM stack; advanced logic enables additional functionality and customization. - Commodity: A product bought largely against comparable specifications and price, subject to qualification; a tendency rather than a description of all DRAM.

0:00-25:56

transcript

No transcript — this publisher did not publish one.

show notes

This post was created using AI. Please check the information if you want to use it as a basis for decision-making.


Memory: How AI Reshapes DRAM


Why can an AI rack use as much working memory as thousands of smartphones, yet remain a completely different manufacturing problem? This episode connects DRAM cell physics, LPDDR, DDR, GDDR and HBM to fab specialization, shared capacity and customer-specific engineering. It examines DDR4 migration and agentic AI without assuming that older memory is always available or that more CPU work means less RAM.


KEY TAKEAWAYS

- DRAM and logic fabs share tool families but optimize different devices and process flows; repetitive patterns still need rigorous control.

- LPDDR's mobile energy-efficiency heritage also serves data-center designs, where deliberately parallel channels can provide high aggregate bandwidth.

- Capacity, bandwidth and latency are separate requirements; adding packages alone does not guarantee more bandwidth.

- The named GB200 NVL72 specification totals about 30 TB of CPU LPDDR5X plus GPU HBM3E. At an assumed 8 GB RAM per phone, this is about 3,800 phones in capacity only, not an equivalence in storage, bandwidth or manufacturing effort.

- HBM competes for DRAM resources and adds its own stacking and test constraints; front-end and backend capacity must both be available.

- DDR5 improves bandwidth and concurrency, but AI has no universal DDR5 requirement. Declining DDR4 demand can coexist with scarcity if supply contracts faster.

- Agentic AI adds CPU execution and orchestration while potentially raising memory and bandwidth demand; it does not establish a financial rescue for Intel.

- Consolidation reflects capital intensity, technology costs and cycles. Custom HBM and system co-design offer differentiation, with customer-concentration and flexibility risks.


GLOSSARY

- DRAM: Dynamic random-access memory; working memory storing charge that needs refresh.

- DDR: Double data rate; a family of system-memory interfaces, including DDR4 and DDR5.

- LPDDR: Low-power double data rate; energy-efficient DRAM interfaces rooted in mobile applications.

- GDDR: Graphics double data rate; a separate graphics-memory interface family, including GDDR7.

- HBM: High-bandwidth memory; stacked DRAM with a wide interface close to a processor.

- EUV: Extreme ultraviolet lithography; patterning used for selected critical advanced-DRAM and logic layers.

- TSV: Through-silicon via; a vertical electrical connection through a silicon die.

- KV cache: Key-value cache; saved intermediate attention state used during model inference.

- Base die: The control and interface die beneath an HBM stack; advanced logic enables additional functionality and customization.

- Commodity: A product bought largely against comparable specifications and price, subject to qualification; a tendency rather than a description of all DRAM.