StockDuty ← Dashboard Nov 09, 2026 02:51 PM ET
SignalTech/AI

6 Hours of YouTube and 85 handtyped pages of notes on LLM Inference – Start here

High-priority alert based on source significance and keyword relevance.

What happened: A developer published a from-scratch LLM inference engine in C++17 and CUDA, with a video series and handwritten PDF notes covering the path from prompt to generated token. Key numbers: 85 hand-typed pages of notes, 80+ pages in the PDF, Llama 3.2 1B Instruct with 16 layers, 2,048-token context, 128 MiB default KV pool, 16-token KV pages, CMake 3.24+. Why it matters: The project documents prefill, paged decode attention, and static or continuous batching as teaching material, and notes that BF16 rounding can make greedy outputs differ across batch sizes.
A developer published a from-scratch LLM inference engine in C++17 and CUDA, with a video series and handwritten PDF notes covering the path from prompt to generated token.
The project documents prefill, paged decode attention, and static or continuous batching as teaching material, and notes that BF16 rounding can make greedy outputs differ across batch sizes.
Sources