Life is not a race, but a journey

Some writing techniques for the butterfly theme in markdown format

Notes on parallelism and scaling from CMU 11-711 Advanced NLP, covering single-GPU training basics, data/tensor/pipeline parallelism, ZeRO memory optimization, and strategy selection for large-scale LLM training.

Notes on quantization from CMU 11-711 Advanced NLP, covering number representation, 8-bit quantization (absmax, zero-point), LLM.int8(), GGML/GGUF, and quantization-aware training with QLoRA.

Notes on CMU 11-868 LLM Systems Lec16, covering Mixture-of-Experts architecture, Switch Transformer, shared vs. routed experts, expert parallelism with GShard, load balancing, DeepSeek V3 MoE, and DeepSpeed-MoE inference optimization.

Notes and summaries for CMU 15-645 Database Systems.

Notes and summaries for CMU 15-645 Database Systems.

Notes on CMU 11-868 LLM Systems Lec13-15, covering distributed training, NCCL collectives, Ring AllReduce, PyTorch DDP, pipeline parallelism, tensor parallelism, and their combination.

A curated collection of essential readings on AI Agent engineering design, covering Claude Code, OpenAI Codex, context management, memory mechanisms, and the OpenClaw architecture. Each article includes key excerpts and core takeaways.

Notes and summaries for CMU 15-645 Database Systems.

Notes and summaries for CMU 15-645 Database Systems.