Systems of intelligence

AI Mechanics

@DrIbrarAhmedAI

How modern AI works when you strip away the demos: next-token prediction, attention, embeddings, vector search, RAG, KV cache, speculative decoding, and agents with MCP. Each article pairs with the full lecture.

Open channel on YouTube

Articles

Kimi K3: 2.8T Parameters, But Only 104B Active Per Token
September 3, 2026
Kimi K3 has 2. 8 trillion parameters-but only about 104 billion are activated for each token. This visual deep dive explains how 16 of 896 routed experts, 2 shared experts, Stable LatentMoE, and Kimi Delta Attention make that possible. Kimi K3 is a Mixture-of-Experts model with a 1M-token context window. We follow one token through expert routing, distributed communication, the 7168 → 3584 latent projection, Quantile Balancing, KDA, Gated MLA, and Block Attention Residuals.
How ChatGPT Actually Writes: Next Token Prediction Explained
August 31, 2026
What is an LLM? ChatGPT generates answers by predicting the next token. Beginner visual guide-no math required. Tokens, weights, context, and hallucinations. What you will learn • What LLM means • How next-token prediction builds an answer • Where probabilities come from • Context vs memory • LLM vs the full ChatGPT-style app Playlists Beginner AI: https://www.
Why AI Models Think Longer Before Answering | Test-Time Compute Explained
August 30, 2026
Why do reasoning models spend more compute before answering? Test-time compute. Best-of-N, backtracking, verification, rewards, DeepSeek-R1, and tool use-visually explained. What you will learn • Training-time vs test-time scaling • Reasoning budgets and candidate paths • Best-of-N, backtracking, and verification • Outcome vs process rewards • When thinking longer helps-and when it fails Playlists DEEP AI: https://www.
Speculative Decoding: Faster LLM Token Generation
August 30, 2026
What is speculative decoding? A small draft model proposes tokens; the large LLM verifies them. Same output distribution as the big model-faster when the draft is often right. What you will learn • Why one token per forward pass wastes GPU parallelism • Draft model vs target model • One-pass verification and rejection • Why draft quality controls the speedup • How it sits next to GQA, quantization, and PagedAttention Playlists DEEP AI: https://www.
Why LLM Prediction Is Compression (Entropy Explained)
August 28, 2026
What is entropy? Next-token prediction and compression are the same math. From Shannon’s information theory to why language models train with cross-entropy. Watch next: Transformer Attention Explained - https://www. youtube. com/watch? v=rbrSteyXx_0 What you will learn • Variable-length and prefix-free codes • Self-information I = -log₂ p • Why prediction equals compression • Entropy rate of language • How this leads to LLM training loss Playlists LLM Fundamentals: https://www.
How AI Agents Work: Tools, Memory, MCP, Guardrails
August 27, 2026
How AI agents work beyond the LLM: prompt → tool call → policy → runtime → observation. Follow one agent from a user request to a controlled real-world action-plus memory, MCP, and guardrails. What you will learn • Why writing a plan is not taking an action • Structured tool calls and the agent loop • Session context vs durable memory • MCP discovery is not permission • Policy gates, human approval, budgets, and tracing Playlists AI Agents: https://www.
How Vector Databases Work: Embeddings, HNSW & Semantic Search
August 25, 2026
One query. A million stored vectors. Do we compare against every one? That is exact nearest neighbor, and it is correct, and it is linear. At that scale, latency is the product. A point in space is not a search. This lecture is the store those numbers live in, and the hop that finds the nearest neighbor without reading every row.
RAG Explained: Why LLM Guesses Without Retrieval
August 24, 2026
What is RAG? Retrieval-augmented generation fetches document chunks with embeddings before the language model writes an answer. This visual explanation shows query, embed, retrieve, stuff, generate, and the failures when the index misses. Watch next: Why AI Hallucinates - https://www. youtube. com/watch? v=PVFRCS6pt84 Then: AI Agent Architecture - https://www.
AI Embeddings Explained: Vectors Behind RAG Search
August 21, 2026
What are embeddings? They turn text into vectors so semantic search can find nearest neighbors by meaning, not spelling. This visual explanation shows cosine similarity, chunking, HNSW, and why RAG retrieval depends on that vector space. Watch next: RAG - https://www. youtube. com/watch? v=qxN9X9nMPmE Vector Databases is the next episode in this chain and is in production.

AI Mechanics

Articles

Kimi K3: 2.8T Parameters, But Only 104B Active Per Token

How ChatGPT Actually Writes: Next Token Prediction Explained

Why AI Models Think Longer Before Answering | Test-Time Compute Explained

Speculative Decoding: Faster LLM Token Generation

Why LLM Prediction Is Compression (Entropy Explained)

How AI Agents Work: Tools, Memory, MCP, Guardrails

How Vector Databases Work: Embeddings, HNSW & Semantic Search

RAG Explained: Why LLM Guesses Without Retrieval

AI Embeddings Explained: Vectors Behind RAG Search