Life is not a race, but a journey

A comprehensive reflection on implementing a complete Transformer language model pipeline from scratch — including BPE tokenizer with parallel pre-tokenization, decoder-only Transformer with RMSNorm/RoPE/SwiGLU, AdamW optimizer, and autoregressive text generation. Trained on TinyStories and OpenWebText with extensive experiments on learning rate sweeps, batch size studies, and architectural ablations.

Notes and summaries for CMU 15-645 Database Systems.

Notes and summaries for CMU 15-645 Database Systems.

Notes on learning algorithms and inference methods from CMU 11-711 Advanced NLP.

Notes on RNN architectures, encoder-decoder models, and attention mechanisms from CMU 11-711 Advanced NLP.

Comprehensive notes on deep learning frameworks and automatic differentiation from CMU 11-868 LLM Systems and 15-642 ML Systems courses.

Notes and summaries for CMU 11-868 LLM Systems: GPU Programming & Acceleration.

Notes and summaries for CMU 11-711 Advanced NLP.

Assignment 1 report for CMU 15-618, covering pthreads speedup analysis and SIMD vectorization.

Notes and summaries for CMU 15-645 Database Systems.