MIT · Computer Science

RES.6-012 Introduction to Probability

John Tsitsiklis, Patrick Jaillet · 2018 · 265 free video lectures

The tools of probability theory, and of the related field of statistical inference, are the keys for being able to analyze and make sense of data. These tools underlie important advances in many fields, from the basic sciences to engineering and management. This resource is a companion site to…

▶ Play on Open Lecture HallCourse page ↗

Lectures

  1. 01.1 Lecture Overview
  2. 01.2 Sample Space
  3. 01.3 Sample Space Examples
  4. 01.4 Probability Axioms
  5. 01.5 Simple Properties of Probabilities
  6. 01.6 More Properties of Probabilities
  7. 01.7 A Discrete Example
  8. 01.8 A Continuous Example
  9. 01.9 Countable Additivity
  10. 01.10 Interpretations & Uses of Probabilities
  11. S01.0 Mathematical Background Overview
  12. S01.1 Sets
  13. S01.2 De Morgan's Laws
  14. S01.3 Sequences and their Limits
  15. S01.4 When Does a Sequence Converge?
  16. S01.5 Infinite Series
  17. S01.6 The Geometric Series
  18. S01.7 About the Order of Summation in Series with Multiple Indices
  19. S01.8 Countable and Uncountable Sets
  20. S01.9 Proof that a Set of Real Numbers is Uncountable
  21. S01.10 Bonferroni's Inequality
  22. 02.1 Lecture Overview
  23. 02.2 Conditional Probabilities
  24. 02.3 A Die Roll Example
  25. 02.4 Conditional Probabilities Obey the Same Axioms
  26. 02.5 A Radar Example and Three Basic Tools
  27. 02.6 The Multiplication Rule
  28. 02.7 Total Probability Theorem
  29. 02.8 Bayes' Rule
  30. 03.1 Lecture Overview
  31. 03.2 A Coin Tossing Example
  32. 03.3 Independence of Two Events
  33. 03.4 Independence of Event Complements
  34. 03.5 Conditional Independence
  35. 03.6 Independence Versus Conditional Independence
  36. 03.7 Independence of a Collection of Events
  37. 03.8 Independence Versus Pairwise Independence
  38. 03.9 Reliability
  39. 03.10 The King's Sibling
  40. 04.1 Lecture Overview
  41. 04.2 The Counting Principle
  42. 04.3 Die Roll Example
  43. 04.4 Combinations
  44. 04.5 Binomial Probabilities
  45. 04.6 A Coin Tossing Example
  46. 04.7 Partitions
  47. 04.8 Each Person Gets An Ace
  48. 04.9 Multinomial Probabilities
  49. 05.1 Lecture Overview
  50. 05.2 Definition of Random Variables
  51. 05.3 Probability Mass Functions
  52. 05.4 Bernoulli & Indicator Random Variables
  53. 05.5 Uniform Random Variables
  54. 05.6 Binomial Random Variables
  55. 05.7 Geometric Random Variables
  56. 05.8 Expectation
  57. 05.9 Elementary Properties of Expectation
  58. 05.10 The Expected Value Rule
  59. 05.11 Linearity of Expectations
  60. S05.1 Supplement: Functions
  61. 06.1 Lecture Overview
  62. 06.2 Variance
  63. 06.3 The Variance of the Bernoulli & the Uniform
  64. 06.4 Conditional PMFs & Expectations Given an Event
  65. 06.5 Total Expectation Theorem
  66. 06.6 Geometric PMF Memorylessness & Expectation
  67. 06.7 Joint PMFs and the Expected Value Rule
  68. 06.8 Linearity of Expectations & the Mean of the Binomial
  69. 07.1 Lecture Overview
  70. 07.2 Conditional PMFs
  71. 07.3 Conditional Expectation & the Total Expectation Theorem
  72. 07.4 Independence of Random Variables
  73. 07.5 Example
  74. 07.6 Independence & Expectations
  75. 07.7 Independence, Variances & the Binomial Variance
  76. 07.8 The Hat Problem
  77. S07.1 The Inclusion-Exclusion Formula
  78. 07.2 The Variance of the Geometric
  79. 07.3 Independence of Random Variables Versus Independence of Events
  80. 08.1 Lecture Overview
  81. 08.2 Probability Density Functions
  82. 08.3 Uniform & Piecewise Constant PDFs
  83. 08.4 Means & Variances
  84. 08.5 Mean & Variance of the Uniform
  85. 08.6 Exponential Random Variables
  86. 08.7 Cumulative Distribution Functions
  87. 08.8 Normal Random Variables
  88. 08.9 Calculation of Normal Probabilities
  89. 09.1 Lecture Overview
  90. 09.2 Conditioning A Continuous Random Variable on an Event
  91. 09.3 Conditioning Example
  92. 09.4 Memorylessness of the Exponential PDF
  93. 09.5 Total Probability & Expectation Theorems
  94. 09.6 Mixed Random Variables
  95. 09.7 Joint PDFs
  96. 09.8 From The Joint to the Marginal
  97. 09.9 Continuous Analogs of Various Properties
  98. 09.10 Joint CDFs
  99. S09.1 Buffon's Needle & Monte Carlo Simulation
  100. 10.1 Lecture Overview
  101. 10.2 Conditional PDFs
  102. 10.3 Comments on Conditional PDFs
  103. 18.10 Total Probability & Total Expectation Theorems
  104. 10.5 Independence
  105. 10.6 Stick-Breaking Example
  106. 10.7 Independent Normals
  107. 10.8 Bayes Rule Variations
  108. 10.9 Mixed Bayes Rule
  109. 10.10 Detection of a Binary Signal
  110. 10.11 Inference of the Bias of a Coin
  111. 11.1 Lecture Overview
  112. 11.2 The PMF of a Function of a Discrete Random Variable
  113. 11.3 A Linear Function of a Continuous Random Variable
  114. 11.4 A Linear Function of a Normal Random Variable
  115. 11.5 The PDF of a General Function
  116. 11.6 The Monotonic Case
  117. 11.7 The Intuition for the Monotonic Case
  118. 11.8 A Nonmonotonic Example
  119. 11.9 The PDF of a Function of Multiple Random Variables
  120. S11.1 Simulation
  121. 12.1 Lecture Overview
  122. 12.2 The Sum of Independent Discrete Random Variables
  123. 12.3 The Sum of Independent Continuous Random Variables
  124. 12.4 The Sum of Independent Normal Random Variables
  125. 12.5 Covariance
  126. 12.6 Covariance Properties
  127. 12.7 The Variance of the Sum of Random Variables
  128. 12.8 The Correlation Coefficient
  129. 12.9 Proof of Key Properties of the Correlation Coefficient
  130. 12.10 Interpreting the Correlation Coefficient
  131. 12.11 Correlations Matter
  132. 13.1 Lecture Overview
  133. 13.2 Conditional Expectation as a Random Variable
  134. 13.3 The Law of Iterated Expectations
  135. 13.4 Stick-Breaking Revisited
  136. 13.5 Forecast Revisions
  137. 13.6 The Conditional Variance
  138. 13.7 Derivation of the Law of Total Variance
  139. 13.8 A Simple Example
  140. 13.9 Section Means and Variances
  141. 13.10 Mean of the Sum of a Random Number of Random Variables
  142. 13.11 Variance of the Sum of a Random Number of Random Variables
  143. 13.1 Conditional Expectation Properties
  144. 14.1 Lecture Overview
  145. 14.2 Overview of some Application Domains
  146. 14.3 Types of Inference Problems
  147. 14.4 The Bayesian Inference Framework
  148. 14.5 Discrete Parameter, Discrete Observation
  149. 14.6 Discrete Parameter, Continuous Observation
  150. 14.7 Continuous Parameter, Continuous Observation
  151. 14.8 Inferring the Unknown Bias of a Coin and the Beta Distribution
  152. 14.9 Inferring the Unknown Bias of a Coin—Point Estimates
  153. 14.10 Summary
  154. S14.1 The Beta Formula
  155. 15.1 Lecture Overview
  156. 15.2 Recognizing Normal PDFs
  157. 15.3 Estimating a Normal Random Variable in the Presence of Additive Noise
  158. 15.4 The Case of Multiple Observations
  159. 15.5 The Mean Squared Error
  160. 15.6 Multiple Parameters; Trajectory Estimation
  161. 15.7 Linear Normal Models
  162. 15.8 Trajectory Estimation Illustration
  163. 16.1 Lecture Overview
  164. 16.2 LMS Estimation in the Absence of Observations
  165. 16.3 LMS Estimation of One Random Variable Based on Another
  166. 16.4 LMS Performance Evaluation
  167. 16.5 Example: The LMS Estimate
  168. 16.6 Example Continued: LMS Performance Evaluation
  169. 16.7 LMS Estimation with Multiple Observations or Unknowns
  170. 16.8 Properties of the LMS Estimation Error
  171. 17.1 Lecture Overview
  172. 18.2 LLMS Formulation
  173. 17.3 Solution to the LLMS Problem
  174. 17.4 Remarks on the LLMS Solution and on the Error Variance
  175. 17.5 LLMS Example
  176. 17.6 LLMS for Inferring the Parameter of a Coin
  177. 17.7 LLMS with Multiple Observations
  178. 17.8 The Simplest LLMS Example with Multiple Observations
  179. 17.9 The Representation of the Data Matters in LLMS
  180. 18.1 Lecture Overview
  181. 18.2 The Markov Inequality
  182. 18.3 The Chebyshev Inequality
  183. 18.4 The Weak Law of Large Numbers
  184. 18.5 Polling
  185. 18.6 Convergence in Probability
  186. 18.7 Convergence in Probability Examples
  187. 18.8 Related Topics
  188. 18.1 Convergence in Probability of the Sum of Two Random Variables
  189. S18.2 Jensen's Inequality
  190. S18.3 Hoeffding's Inequality
  191. 19.1 Lecture Overview
  192. 19.2 The Central Limit Theorem
  193. 19.3 Discussion of the CLT
  194. 19.4 Illustration of the CLT
  195. 19.5 CLT Examples
  196. 19.6 Normal Approximation to the Binomial
  197. 19.7 Polling Revisited
  198. 20.1 Lecture Overview
  199. 20.2 Overview of the Classical Statistical Framework
  200. 20.3 The Sample Mean and Some Terminology
  201. 20.4 On the Mean Squared Error of an Estimator
  202. 20.5 Confidence Intervals
  203. 20.6 Confidence Intervals for the Estimation of the Mean
  204. 20.7 Confidence Intervals for the Mean, When the Variance is Unknown
  205. 20.9 Other Natural Estimators
  206. 20.9 Maximum Likelihood Estimation
  207. 20.10 Maximum Likelihood Estimation Examples
  208. 21.1 Lecture Overview
  209. 21.2 The Bernoulli Process
  210. 21.3 Stochastic Processes
  211. 21.4 Review of Known Properties of the Bernoulli Process
  212. 21.5 The Fresh Start Property
  213. 21.6 Example: The Distribution of a Busy Period
  214. 21.7 The Time of the K-th Arrival
  215. 21.8 Merging of Bernoulli Processes
  216. 21.9 Splitting a Bernoulli Process
  217. 21.10 The Poisson Approximation to the Binomial
  218. 22.1 Lecture Overview
  219. 22.2 Definition of the Poisson Process
  220. 22.3 Applications of the Poisson Process
  221. 22.4 The Poisson PMF for the Number of Arrivals
  222. 22.5 The Mean and Variance of the Number of Arrivals
  223. 22.6 A Simple Example
  224. 22.7 Time of the K–th Arrival
  225. 22.8 The Fresh Start Property and Its Implications
  226. 22.9 Summary of Results
  227. 22.10 An Example
  228. 23.1 Lecture Overview
  229. 23.2 The Sum of Independent Poisson Random Variables
  230. 23.3 Merging Independent Poisson Processes
  231. 23.4 Where is an Arrival of the Merged Process Coming From?
  232. 23.5 The Time Until the First (or Last) Lightbulb Burns Out
  233. 23.6 Splitting a Poisson Process
  234. 23.7 Random Incidence in the Poisson Process
  235. 23.8 Random Incidence in a Non–Poisson Process
  236. 23.9 Different Sampling Methods Can Give Different Results
  237. S23.1 Poisson Versus Normal Approximations to the Binomial
  238. S23.2 Poisson Arrivals During an Exponential Interval
  239. 24.1 Lecture Overview
  240. 24.2 Introduction to Markov Processes
  241. 24.3 Checkout Counter Example
  242. 24.4 Discrete-Time Finite-State Markov Chains
  243. 24.5 N–Step Transition Probabilities
  244. 24.6 A Numerical Example — Part I
  245. 24.7 Generic Convergence Questions
  246. 24.8 Recurrent and Transient States
  247. Brief Introduction
  248. 25.2 Lecture Overview
  249. 25.3 Markov Chain Review
  250. 25.4 The Probability of a Path
  251. 25.5 Recurrent and Transient States: Review
  252. 25.6 Periodic States
  253. 25.7 Steady-State Probabilities and Convergence
  254. 25.8 A Numerical Example — Part II
  255. 25.9 Visit Frequency Interpretation of Steady-State Probabilities
  256. 25.10 Birth-Death Processes — Part I
  257. 25.11 Birth-Death Processes — Part II
  258. 26.2 Lecture Overview
  259. 26.3 Review of Steady-State Behavior
  260. 24.6 A Numerical Example — Part III
  261. 26.6 Design of a Phone System
  262. 26.6 Absorption Probabilities
  263. 26.7 Expected Time to Absorption
  264. 26.8 Mean First Passage Time
  265. 26.9 Gambler's Ruin

From MIT OpenCourseWare · CC BY-NC-SA 4.0. More from MIT.

Open Lecture Hall is free, with no ads and no sign-up. It is not run by or connected to any university; the lectures are the universities' own free recordings, played from where they host them. About & rights