My notes:
BitNet: https://drive.google.com/file/d/1iA2t...
Era of 1-bit LLMs: https://drive.google.com/file/d/1iNy9...

BitNet: Scaling 1-bit Transformers for Large Language Models: https://arxiv.org/abs/2310.11453
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits: https://arxiv.org/abs/2402.17764


00:00 Intro
03:10 BitLinear Intuition
08:05 Weight Quantization
10:35 Activation Quantization
16:30 Matrix Multiplication and Dequantizing
23:08 Model Parallelism with Group Quantization and Normalization
32:36 Other Training Stuff
37:11 BitNet Results
39:11 The Era of 1-Bit LLMs