Blog

Technical deep dives.

Articles on transformer architectures, model quantization, inference optimization, and the systems behind production AI.

Attention

Self-Attention Mechanism

Visual walkthrough of single-headed and multi-headed self-attention — understanding the matrix dimensions and operations.

#attention #transformers #deep-learning

Frameworks

Framework FastTrack

Quick refresher guides for data-related frameworks — from TensorFlow to PyTorch, simplified for rapid learning.

#frameworks #tensorflow #pytorch

Fundamentals

Cross Entropy Loss

A comprehensive guide to understanding and implementing cross entropy in machine learning — from information theory to PyTorch.

#loss-functions #neural-networks #fundamentals

Elements of Neural Nets: Building Blocks of AI

An essential guide to the foundational components of modern neural networks — activations, cost functions, optimizers, and regularization.

#neural-networks #fundamentals #deep-learning

Quantization

Quantization: What, Why, and How

A deep dive into model quantization — reducing model size while preserving accuracy. Covers symmetric and asymmetric approaches with code.

#quantization #optimization #inference #pytorch

Quick Books

Quick Books: Key Learnings from Recent Reads

Distilled insights from books on technology, AI, and engineering — the golden takeaways worth remembering.

#books #learning #reading

Transformers

Building the Transformer from Scratch

An object-oriented implementation of the complete Transformer architecture in PyTorch — from input embeddings to the full encoder-decoder model.

#transformers #pytorch #deep-learning #implementation

Implementing Transformers

A practical guide to coding the Transformer architecture from scratch — turning the 'Attention is All You Need' paper into working code.

#transformers #pytorch #deep-learning