IISc Bangalore · Computer Science

Stochastic Approximation: Theory and Applications

Prof. Gugan Chandrashekhar Mallika Thoppe · 46 free video lectures

Stochastic approximation refers to a class of iterative algorithms that can locate zeroes or optimal points of functions in scenarios where the function evaluations are compromised by noise. These algorithms are prominently utilized in regression, system identification, adaptive control, and…

▶ Play on Open Lecture HallCourse page ↗

Lectures

  1. Overview of Stochastic Approximation
  2. Estimating the Mean of a Random Variable
  3. Estimating change rates of webpages
  4. An Introduction to Reinforcement Learning.
  5. Policy Evaluation in Reinforcement Learning
  6. Probability Spaces- A Measure Theoretic Perspective
  7. Random Variables as Measurable Maps
  8. Expectation of Random Variable as Lebesgue Integration
  9. Conditional Expectation: A Formal Introduction
  10. Properties of Conditional Expectation.
  11. Martingales: Definition and Examples
  12. Doob’s Uncrossing Lemma
  13. Doob’s Forward Convergence Theorem for Supermartingales
  14. $L^2$ Martingales
  15. Revisiting the Martingale Convergence Theorem via Stopped Processes
  16. Existence and Uniqueness of Solutions to Ordinary Differential Equations
  17. Continuity of ODE Solutions with Respect to Initial Conditions
  18. Asymptotic Behaviour of Solutions to ODEs
  19. Stability of ODEs and Lyapunov Methods
  20. Foundations of Stochastic Approximation – Assumptions and Key Definitions
  21. Almost Sure Convergence of Stochastic Approximation Iterates
  22. Concluding the Convergence Proof: Internal Chain Transitivity, Connectedness, and Invariance
  23. How Stochastic Approximation Iterates Track an ODE: The Key Lemma
  24. Proof of the Key Lemma - Part I
  25. Proof of the Key Lemma - Part II
  26. Extensions, Variants, and Applications of Stochastic Approximation
  27. Convergence Rate for Linear Stochastic Approximation - Part 1
  28. Convergence Rate for Linear Stochastic Approximation - Part 2
  29. Lower Bounds and the Minimax Risk in Estimation
  30. Stability Requirements in Stochastic Approximation
  31. Almost Sure Boundedness of Iterates: Theorem and Example
  32. Distributed Estimation of the Mean of a Random Vector
  33. Robust Distributed Learning under Adversaries
  34. Motivation for Stochastic Recursive Inclusion in Distributed Mean Estimation
  35. Well-Posedness of Differential Inclusions
  36. Convergence of Distributed Robust Mean Estimation Iterates through the Lens of Differential Inclusions
  37. Almost Sure Convergence via Robbins–Siegmund Theorem – Part 1
  38. Almost Sure Convergence via Robbins–Siegmund Theorem – Part 2
  39. Introduction to Reinforcement Learning and Value Function Approximation
  40. Temporal Difference Algorithm Through the Lens of Stochastic Approximation
  41. How Good Is the TD Solution? Fixed Point Analysis in Linear Approximation
  42. Almost Sure Convergence Analysis of Temporal Difference Learning via the ODE Method
  43. Best Policy Algorithm for Q-Value Functions: A Stochastic Approximation Formulation
  44. Asymptotic Analysis of Q-Learning Algorithm
  45. Asymptotic Behaviour of the Q-Learning Limit ODE — A Switching Systems Perspective
  46. Concluding the Asymptotic Analysis of Q-Learning

From NPTEL (IITs and IISc) · CC BY-NC-SA. More from IISc Bangalore.

Open Lecture Hall is free, with no ads and no sign-up. It is not run by or connected to any university; the lectures are the universities' own free recordings, played from where they host them. About & rights