IISc Bangalore · Computer Science
Stochastic Approximation: Theory and Applications
Prof. Gugan Chandrashekhar Mallika Thoppe · 46 free video lectures
Stochastic approximation refers to a class of iterative algorithms that can locate zeroes or optimal points of functions in scenarios where the function evaluations are compromised by noise. These algorithms are prominently utilized in regression, system identification, adaptive control, and…
▶ Play on Open Lecture HallCourse page ↗
Lectures
- Overview of Stochastic Approximation
- Estimating the Mean of a Random Variable
- Estimating change rates of webpages
- An Introduction to Reinforcement Learning.
- Policy Evaluation in Reinforcement Learning
- Probability Spaces- A Measure Theoretic Perspective
- Random Variables as Measurable Maps
- Expectation of Random Variable as Lebesgue Integration
- Conditional Expectation: A Formal Introduction
- Properties of Conditional Expectation.
- Martingales: Definition and Examples
- Doob’s Uncrossing Lemma
- Doob’s Forward Convergence Theorem for Supermartingales
- $L^2$ Martingales
- Revisiting the Martingale Convergence Theorem via Stopped Processes
- Existence and Uniqueness of Solutions to Ordinary Differential Equations
- Continuity of ODE Solutions with Respect to Initial Conditions
- Asymptotic Behaviour of Solutions to ODEs
- Stability of ODEs and Lyapunov Methods
- Foundations of Stochastic Approximation – Assumptions and Key Definitions
- Almost Sure Convergence of Stochastic Approximation Iterates
- Concluding the Convergence Proof: Internal Chain Transitivity, Connectedness, and Invariance
- How Stochastic Approximation Iterates Track an ODE: The Key Lemma
- Proof of the Key Lemma - Part I
- Proof of the Key Lemma - Part II
- Extensions, Variants, and Applications of Stochastic Approximation
- Convergence Rate for Linear Stochastic Approximation - Part 1
- Convergence Rate for Linear Stochastic Approximation - Part 2
- Lower Bounds and the Minimax Risk in Estimation
- Stability Requirements in Stochastic Approximation
- Almost Sure Boundedness of Iterates: Theorem and Example
- Distributed Estimation of the Mean of a Random Vector
- Robust Distributed Learning under Adversaries
- Motivation for Stochastic Recursive Inclusion in Distributed Mean Estimation
- Well-Posedness of Differential Inclusions
- Convergence of Distributed Robust Mean Estimation Iterates through the Lens of Differential Inclusions
- Almost Sure Convergence via Robbins–Siegmund Theorem – Part 1
- Almost Sure Convergence via Robbins–Siegmund Theorem – Part 2
- Introduction to Reinforcement Learning and Value Function Approximation
- Temporal Difference Algorithm Through the Lens of Stochastic Approximation
- How Good Is the TD Solution? Fixed Point Analysis in Linear Approximation
- Almost Sure Convergence Analysis of Temporal Difference Learning via the ODE Method
- Best Policy Algorithm for Q-Value Functions: A Stochastic Approximation Formulation
- Asymptotic Analysis of Q-Learning Algorithm
- Asymptotic Behaviour of the Q-Learning Limit ODE — A Switching Systems Perspective
- Concluding the Asymptotic Analysis of Q-Learning
From NPTEL (IITs and IISc) · CC BY-NC-SA. More from IISc Bangalore.
Open Lecture Hall is free, with no ads and no sign-up. It is not run by or connected to any university; the lectures are the universities' own free recordings, played from where they host them. About & rights