List

Music
Album
Videos

Life is not a race, but a journey

List

Music
Album
Videos

tags

Natural Language Processing CMU Study Notes 11711 AI Evaluation Benchmarks Multimodal Vision Diffusion Distributed Training Parallelism Scaling Quantization Inference Efficiency Retrieval RAG LLM Systems GPU Programming 11868 ML Systems Deep Learning Auto Differentiation 15642 Decoding Speculative Decoding DDP NCCL AllReduce Pipeline Parallelism Tensor Parallelism Model Parallelism GPipe Megatron-LM Gradient Checkpointing Mixture of Experts MoE Expert Parallelism Switch Transformer DeepSeek GShard DeepSpeed-MoE Tokenization NLP 15618 Parallel Programming Assignment SIMD Pthreads Systems Performance Optimization Data Parallelism ZeRO Transformer Attention FlashAttention GPU Optimization Database Systems Database 15645 Hash Tables Concurrency Control Latches B+Tree Hash Table Index Filter Bloom Filter Join Hash Join Sort-Merge Join Query Execution Query Optimization Cost Model Cardinality Estimation Sorting Aggregation Storage B-Tree AI Agent LLM Engineering Design Claude Code OpenAI Codex Memory System Context Management Technical Reading Blog Writing Tips TinyKV Distributed Systems KV Storage Raft Consensus Learning CS336 PyTorch Stanford Architectures Hyperparameters C++ Programming C++ Language UI Programming CNN CMU 11-785 Course Experiments Operating System Technical Blog Code Agent Engineering Experience 个人反思生活面经 Computer Networks - Campus Network - Routing Study Computer Networks git CS Tools Makefile Basics

Xiang Li

Xiang Li's Blog

Announcement

Talk is cheap. Show me the code.

Recent Posts

11711 Advanced NLP: Parallelism and Scaling

11711 Advanced NLP: Parallelism and Scaling2026-04-06

11711 Advanced NLP: Quantization

11711 Advanced NLP: Quantization2026-04-06

11868 LLM Sys: Systems for Mixture-of-Experts Models

11868 LLM Sys: Systems for Mixture-of-Experts Models2026-03-21

15645 Database systems: Query Optimization

15645 Database systems: Query Optimization2026-03-20

15645 Database systems: Query Execution

15645 Database systems: Query Execution2026-03-15

Categories

AI Agent1
Blog Setup1
C/C++ Programming10
CMU 11-711 Advanced NLP8
CMU 11-868 LLM Systems6
CMU 15-618 Parallel Programming2
CMU 15-642 ML Systems2
CMU 15-645 Database Systems9

Tags

11868 Attention CNN Speculative Decoding LLM Systems Aggregation Claude Code MoE Evaluation TinyKV Assignment Data Parallelism Course Experiments Distributed Training Quantization Index Storage Vision GPU Programming KV Storage Query Optimization Bloom Filter Tokenization 11711 Blog Writing Tips Hash Join Engineering Experience 15645 CS Tools CS336 AI Agent Gradient Checkpointing C++ Programming 15642 Pipeline Parallelism PyTorch Learning OpenAI Codex GPU Optimization Switch Transformer

Archives

April 2026 2
March 2026 10
February 2026 9
January 2026 10
December 2025 5
October 2025 2
September 2024 3
December 2023 1

Website Info

Article Count :

62

Unique Visitors :

Page Views :

Last Update :

© 2025 - 2026 By Xiang LiFramework Hexo 8.1.1|Theme Butterfly 5.4.3