11868 LLM Sys: Tokenization and Embedding
Comprehensive notes on tokenization techniques and vocabulary learning for Large Language Models, covering BPE, VOLT, and practical considerations in multilingual LLMs.
15642 Machine Learning Systems: Transformer, Attention, and Optimizations
Notes and summaries for CMU 15-642 Machine Learning Systems on Transformer architecture, attention mechanisms, and GPU optimizations including FlashAttention.
15645 Database systems: Index & Filter
Notes and summaries for CMU 15-645 Database Systems.
TinyKV: A Raft-Based KV Storage System
Implementation and architecture of TinyKV, a Raft-based distributed key-value storage system covering Multi-Raft, MVCC transactions, and the scheduler.
CS336 Assignment 1: Building a Transformer Language Model from Scratch
A comprehensive reflection on implementing a complete Transformer language model pipeline from scratch — including BPE tokenizer with parallel pre-tokenization, decoder-only Transformer with RMSNorm/RoPE/SwiGLU, AdamW optimizer, and autoregressive text generation. Trained on TinyStories and OpenWebText with extensive experiments on learning rate sweeps, batch size studies, and architectural ablations.
15645 Database systems: Hash Tables
Notes and summaries for CMU 15-645 Database Systems.
15645 Database systems: Storage
Notes and summaries for CMU 15-645 Database Systems.
11711 Advanced NLP: Learning & Inference
Notes on learning algorithms and inference methods from CMU 11-711 Advanced NLP.
11711 Advanced NLP: Architectures
Notes on RNN architectures, encoder-decoder models, and attention mechanisms from CMU 11-711 Advanced NLP.
11868 LLM Sys & 15642 ML Sys: DL Frameworks and Auto Differentiation
Comprehensive notes on deep learning frameworks and automatic differentiation from CMU 11-868 LLM Systems and 15-642 ML Systems courses.
