Step by Step
Start with the roadmap to build a global picture, then dive into individual page interactions. This way, when you learn about Attention, RAG, and Agent, they won't seem like isolated concepts.
SimLabs LLM Visualization
This is a series of interactive courses for understanding large model technology from scratch. Its goal is not to pile up jargon, but to make Token, vectors, Tokenizer, Attention, QKV, Transformer and decoding strategies observable, operable, and progressively learnable HTML pages.
Roadmap first, no getting lost
Intuition first, then formulas, then engineering
Perfect for teaching, self-learning and demos
How to Learn with This Series
One Topic per Page
Each page focuses on one core question, avoiding cramming multiple new concepts into a single page.
Interactive Exploration
Use heatmaps, step-by-step playback, parameter sliders, matrix highlighting, and comparison cards to let users observe changes themselves, rather than just reading text.
Current Progress: Series homepage, roadmap data, and common components have been placed in the `ai` directory. Technology map, Token/vector, matrix fundamentals, tensor Tensor, neural network basics, Embedding semantic space, language model objective, Tokenizer, Attention intuition, QKV, causal Mask, positional encoding, multi-head attention, Residual/LayerNorm/FFN, Transformer Block, decoding strategies, training & alignment, RAG, Tool Use, Agent, safety, evaluation, deployment & cost are all live.
Currently Available Content
If this is your first time exploring this series, we recommend starting with these key pages. After the page script loads, all available courses will be automatically expanded here.
Live
AI, Machine Learning & LLM Map
Build a panoramic view first to understand where AI, machine learning, deep learning, and large models fit in the technology landscape.
Starting Page
Technology Map
Live
Characters, Tokens & Vectors
Understand why language needs to be tokenized and why tokens are mapped to vectors.
Text Representation
Live
Vectors, Matrices & Linear Transformations
Build visual mathematical foundation for Embedding, linear layers, and QK dot product.
Math Foundation
Live
Tensor: From Matrix to Multidimensional Array
Use rank, shape, slicing and common operations to turn tensors from abstract concepts into observable and manipulable objects.
Tensor
Shape
Math Foundation
Live
Tokenizer & BPE
Understand why models see token sequences instead of words, and why tokens are more stable.
Tokenizer
Live
Self-Attention Intuition
Understand why a token needs to look at other tokens, not just itself.
Attention
Legacy Experiment
QKV & Attention Scoring
Directly experience Q, K, V, scoring matrix and final output. Quickly turn attention mechanism into an operable process.
QKV
Attention
Live
Multi-Head Attention: Why Multiple Heads
Compare single-head mixed views vs multi-head specialization, then see how Concat + W^O fuses results from different heads.
Multi-Head Attention
Transformer
Live
Residual, LayerNorm & FFN: The Other Half Beyond Attention
Break down "preserve backbone, normalize scale, process features" into actionable experiments to complete the intuition beyond attention in Transformer Block.
Residual
LayerNorm
FFN
Live
RAG: Retrieval-Augmented Generation
Understand from a system perspective why models often need to search before answering, and how retrieval enhances generation quality.
RAG
System
Live
Agent: Planning, Execution, Observation & Memory
Evolve from "can answer" to "can continuously complete tasks". Understand why Agent is a cyclic system.
Agent
System
Recommended First Pass Order: Start with the map page to build global structure, then cover Token/vector, matrix, tensor, neural network basics, Embedding and language model objective. Next explore Tokenizer, Attention, QKV, causal Mask, positional encoding, multi-head attention, Residual/LayerNorm/FFN and Transformer Block. Finally return to system topics like RAG, Agent and safety.
Complete Roadmap
Clickable cards represent currently accessible pages; gray cards indicate planned content still under development. After the page script loads, this will automatically switch to the complete phase-based course list.
Phase 1 · Foundation
Build the overall map first, then understand how text is transformed into model-computable representations step by step.
Core Questions
Where do AI and large models fit in the technology spectrum? Why does text need to become Tokens, vectors, matrices and tensors?
Map Awareness
Token
Matrix Basics
Tensor
Phase 2 · Representation Learning
This phase bridges "basic representation" to "how models learn semantic structure", avoiding gaps when jumping into Transformer later.
Core Questions
How do neural networks recombine raw features into hidden representations? Why can Embedding bring similar concepts closer? What exactly are language models optimizing during training?
Neural Network
Embedding
LM Objective
Phase 3 · Transformer Core
This is the technical heart of the entire series, focusing on Tokenizer, Attention, QKV, causal Mask, positional encoding, multi-head attention, Residual/LayerNorm/FFN and Transformer Block.
Core Questions
How does the model view relationships between tokens? Why can't decoder peek into the future? How is sequential information explicitly injected? Why are multiple heads more complete? How does the other half beyond attention preserve backbone and continue processing representations?
Tokenizer
Attention
Mask
Positional Encoding
Multi-Head Attention
Residual/LayerNorm/FFN
Transformer Block
Phase 4 · Training & Inference
Understand how models are trained and why the same model can behave very differently with different decoding parameters.
Core Questions
How do model capabilities gradually emerge from pre-training to alignment? How do temperature, Top-k and Top-p affect output style?
Training
Alignment
Decoding Strategies
Phase 5 · Application Systems
Place individual model capabilities into real product systems. Understand RAG, Tool Use, Agent, safety, evaluation and deployment.
Core Questions
A "usable LLM system" is more than just a conversation - it requires retrieval, tools, evaluation, safety boundaries and engineering trade-offs.
RAG
Tool Use
Agent
Evals
Recommended Starting Sequence
1
Technology Map
Understand the layers you need to learn before diving into specific mechanisms.
2
Text Representation
Understand how text becomes tokens, vectors, matrices and tensors.
3
Representation Learning
Build intuition on neural networks, Embedding and LM objectives before entering Transformer.
4
Transformer Core
Dive into Attention, QKV, causal Mask, positional encoding, multi-head attention, Residual/LayerNorm/FFN and Block structure.
5
Inference & Systems
Finally explore sampling, RAG, tool use, safety and deployment.
One-sentence summary of this series: Not just copying papers to web pages, but transforming abstract concepts into interactive learning experiences you can click, explore and verify yourself.