IISc Bangalore · Engineering

Information Theory

Prof. Himanshu Tyagi · 70 free video lectures

70 video lectures. Topics: What is information?; How to model uncertainty?; Basic concepts of probability; Estimates of random variables; Limit theorems; Review; Source model; Motivating examples; A compression problem; Shannon entropy; Random hash; Review 2; Uncertainty and randomness; Total…

▶ Play on Open Lecture HallCourse page ↗

Lectures

  1. What is information?
  2. How to model uncertainty?
  3. Basic concepts of probability
  4. Estimates of random variables
  5. Limit theorems
  6. Review
  7. Source model
  8. Motivating examples
  9. A compression problem
  10. Shannon entropy
  11. Random hash
  12. Review 2
  13. Uncertainty and randomness
  14. Total variation distance
  15. Generating almost random bits
  16. Generating samples from a distribution using uniform randomness
  17. Typical sets and entropy
  18. Review 3
  19. Hypothesis testing and estimation
  20. Examples.
  21. The log-likelihood ratio test
  22. Kullback-Leibler divergence and Stein's lemma
  23. Properties of KL divergence
  24. Review 4
  25. Information per coin-toss
  26. Multiple hypothesis testing
  27. Error analysis of multiple hypothesis testing
  28. Mutual information
  29. Fano's inequality
  30. Measures of information
  31. Chain rules
  32. Chain rules
  33. Shape of measures of information
  34. Data processing inequality
  35. Midyear Review
  36. Proof of Fano's inequality
  37. Capacity as information radius
  38. Proof of Pinsker's inequality
  39. Continuity of entropy
  40. Lower bound for compression
  41. Lower bound for hypothesis testing
  42. Review 7
  43. Lower bound for random number generation
  44. Strong converse
  45. Lower bound for minmax statistical estimation
  46. Variable length source codes
  47. Review 8
  48. Kraft's inequality
  49. Shannon code
  50. Huffman code
  51. Minmax Redundancy
  52. Type based universal compression
  53. Review 9
  54. Arithmetic code
  55. Online probability assignment
  56. Compression of databases: A scheme
  57. Compression of databases: A lower bound
  58. Repetition code
  59. Channel capacity
  60. Sphere packing bound for BSC
  61. Random coding bound for BSC
  62. Random coding bound for general channel
  63. Review 11
  64. Converse proof for channel coding theorem
  65. Additive Gaussian Noise channel
  66. Mutual information and differential entropy
  67. Channel coding theorem for Gaussan channel
  68. Parallel channels and water-filling
  69. Live Session 23-09-2020
  70. Live Session 07-10-2020

From NPTEL (IITs and IISc) · CC BY-NC-SA. More from IISc Bangalore.

Open Lecture Hall is free, with no ads and no sign-up. It is not run by or connected to any university; the lectures are the universities' own free recordings, played from where they host them. About & rights