Gradient Descent Explained: How Models Actually Learn (and Where RAG Fits)
Summary
Every trained model, from a two-parameter line to a trillion-parameter LLM, learns the same way: measure the error, follow the slope downhill, repeat. This visual walkthrough builds gradient descent from a loss landscape, shows learning rates and local minima, and connects it to information retrieval: ranking functions and rerankers are trained by the same descent on relevance labels and click signals.
About this video
Every trained model, from a two-parameter line to a trillion-parameter LLM, learns the same way: measure the error, follow the slope downhill, repeat. This visual walkthrough builds gradient descent from a loss landscape, shows learning rates and local minima, and connects it to information retrieval: ranking functions and rerankers are trained by the same descent on relevance labels and click signals.