How DeepSeek Shrinks Memory and OpenAI Scales Thinking

How DeepSeek Shrinks Memory and OpenAI Scales Thinking The Transformer Is Evolving | AI MECHANICS Every token your AI remembers costs GPU memory. At short context that looks harmless. At long context, the KV cache becomes one of inference’s biggest constraints. DeepSeek’s answer is not “the Transformer is dead. ” It is compress what you store, then spend compute where thinking helps.

This episode walks both moves: • Why key and value states accumulate into a KV cache • How Grouped-Query Attention shares K and V across query heads • How Multi-Head Latent Attention stores history in a smaller latent • The published 93. 3% KV-cache reduction vs DeepSeek-67B - not total GPU memory • Why a smaller cache also eases bandwidth during decoding • DeepSeek-V3: still a Transformer, combining MLA with Mixture-of-Experts • 671B total parameters, 37B activated per token • OpenAI’s second scaling curve: more inference compute, better hard-task performance • Remember cheaper.

Think longer. Complementary, not rivals. SOURCES DeepSeek-V2 (MLA, 93. 3% KV cache vs DeepSeek-67B): https://arxiv. org/abs/2405. 04434 DeepSeek-V3 (671B total / 37B active, MLA + MoE): https://arxiv. org/abs/2412. 19437 Grouped-Query Attention: https://arxiv. org/abs/2305. 13245 OpenAI - Learning to Reason with LLMs: https://openai.

com/index/learning-to-reason-with-llms/ MORE AI MECHANICS How LLMs work: https://www. youtube. com/playlist? list=PLK3S1GR94Fzg AI agents: https://www. youtube. com/playlist? list=PLNeIEtc2-vsw LLM Inference & Performance: https://www. youtube. com/playlist? list=PLICV_vVYRlDk AI Mechanics. Understand the system, not the hype.

Watch the video for the full walkthrough. Use this page when you want the argument in writing without scrubbing the timeline.