IIT Bombay · Computer Science

Bandit Algorithm (Online Machine Learning)

Prof. Manjesh Hanawal · 2023 · 59 free video lectures

In many scenarios one faces uncertain environments where a-priori the best action to play is unknown. How to obtain best possible reward/utility in such scenarios. One natural way is to first explore the environment and to identify the `best’ actions and exploit them. However, this give raise to an…

▶ Play on Open Lecture HallCourse page ↗

Lectures

  1. Lecture 1 : Introduction to Online Learning –I
  2. Lecture 2 : Introduction to Online Learning –II
  3. Lecture 3 : Basics of Statistical Learning
  4. Lecture 4 : Empirical risk minimization
  5. Lecture 5 : Consistency Halving algorithm
  6. Lecture 6 : Online Learnability
  7. Lecture 7 : Standard Optimal Algorithm
  8. Lecture 8 : Classification in unrealizability case
  9. Lecture 9 : Covers Impossibility Result
  10. Lecture 10 : Weighted Majority
  11. Lecture 11 : Proof Weighted Majority
  12. Lecture 12 : Full Information vs Bandit Setting
  13. Lecture 13 : Adversarial Bandit Setting
  14. Lecture 14 : Exponential Weights for Exploration and Exploitation Algorithm
  15. Lecture 15 : Regret Bound of Exp3
  16. Lecture 16 : Regret Bound of Exp3(Contd.)
  17. Lecture 17 : Exp3.P and Exp3.IX
  18. Lecture 18 : Online Convex Optimisation
  19. Lecture 19 : Follow the Leader (FTL) Algorithm
  20. Lecture 20 : Follow the Regularized Leader
  21. Lecture 21 : Online Gradient Descent
  22. Lecture 22 : Strongly Convex Function
  23. Lecture 23 : FoReL with Strongly Convex Regulariser
  24. Lecture 24 : FoReL with Strongly Convex Regulariser (Contd.)
  25. Lecture 25 : Euclidean and Entropy Regularizer
  26. Lecture 26 : Introduction to Stochastic Bandits
  27. Lecture 27 : Concentration Inequalities
  28. Lecture 28 : Subgaussian Random Variable
  29. Lecture 29 : Regret Definition and Regret Decomposition
  30. Lecture 30 : Explore and Commit (ETC) Algorithm
  31. Lecture 31 : Regret Analysis and ETC
  32. Lecture 32 : Optimism in the Face of Uncertainty
  33. Lecture 33 : Upper Confidence Bound Algorithm
  34. Lecture 34 : Regret Analysis of UCB
  35. Lecture 35 : Problem Dependent and Independent Bounds of UCB
  36. Lecture 36 : KL-UCB Algorithm
  37. Lecture 37 : Thompson Sampling - Brief Discussion
  38. Lecture 38 : Proof Idea of Lower Bounds - 1
  39. Lecture 39 : Proof Idea of Lower Bounds - 2
  40. Lecture 40 : Proof of Lower Bound-1
  41. Lecture 41 : Proof of Lower Bound-2
  42. Lecture 42 : Stochastic Contextual Bandits
  43. Lecture 43 : Introduction to Stochastic Linear Bandits
  44. Lecture 44 : Stochastic Linear Bandits
  45. Lecture 45 : Regret Analysis of SLB-I
  46. Lecture 46 : Regret Analysis of SLB - II
  47. Lecture 47 : Regret Analysis of SLB-III
  48. Lecture 48 : Construction of Confidence Ellipsoid - I
  49. Lecture 49 : Construction of Confidence Ellipsoids - II
  50. Lecture 50 : Adversarial Contextual Bandits - I
  51. Lecture 51 : Adversarial Contextual Bandits II
  52. Lecture 52 : Exp4 Algorithm
  53. Lecture 53 : Regret of Exp4
  54. Lecture 54 : Adversarial Linear Bandits
  55. Lecture 55 : Exp3 for Adversarial Linear Bandits
  56. Lecture 56 : Introduction to Pure Exploration and its lower bounds
  57. Lecture 57 : Uniform Exploration
  58. Lecture 58 : KL-LUCB
  59. Lecture 59 : Lil’ UCB

From NPTEL (IITs and IISc) · CC BY-NC-SA. More from IIT Bombay.

Open Lecture Hall is free, with no ads and no sign-up. It is not run by or connected to any university; the lectures are the universities' own free recordings, played from where they host them. About & rights