IISc Bangalore · Engineering
Information Theory
Prof. Himanshu Tyagi · 70 free video lectures
70 video lectures. Topics: What is information?; How to model uncertainty?; Basic concepts of probability; Estimates of random variables; Limit theorems; Review; Source model; Motivating examples; A compression problem; Shannon entropy; Random hash; Review 2; Uncertainty and randomness; Total…
▶ Play on Open Lecture HallCourse page ↗
Lectures
- What is information?
- How to model uncertainty?
- Basic concepts of probability
- Estimates of random variables
- Limit theorems
- Review
- Source model
- Motivating examples
- A compression problem
- Shannon entropy
- Random hash
- Review 2
- Uncertainty and randomness
- Total variation distance
- Generating almost random bits
- Generating samples from a distribution using uniform randomness
- Typical sets and entropy
- Review 3
- Hypothesis testing and estimation
- Examples.
- The log-likelihood ratio test
- Kullback-Leibler divergence and Stein's lemma
- Properties of KL divergence
- Review 4
- Information per coin-toss
- Multiple hypothesis testing
- Error analysis of multiple hypothesis testing
- Mutual information
- Fano's inequality
- Measures of information
- Chain rules
- Chain rules
- Shape of measures of information
- Data processing inequality
- Midyear Review
- Proof of Fano's inequality
- Capacity as information radius
- Proof of Pinsker's inequality
- Continuity of entropy
- Lower bound for compression
- Lower bound for hypothesis testing
- Review 7
- Lower bound for random number generation
- Strong converse
- Lower bound for minmax statistical estimation
- Variable length source codes
- Review 8
- Kraft's inequality
- Shannon code
- Huffman code
- Minmax Redundancy
- Type based universal compression
- Review 9
- Arithmetic code
- Online probability assignment
- Compression of databases: A scheme
- Compression of databases: A lower bound
- Repetition code
- Channel capacity
- Sphere packing bound for BSC
- Random coding bound for BSC
- Random coding bound for general channel
- Review 11
- Converse proof for channel coding theorem
- Additive Gaussian Noise channel
- Mutual information and differential entropy
- Channel coding theorem for Gaussan channel
- Parallel channels and water-filling
- Live Session 23-09-2020
- Live Session 07-10-2020
From NPTEL (IITs and IISc) · CC BY-NC-SA. More from IISc Bangalore.
Open Lecture Hall is free, with no ads and no sign-up. It is not run by or connected to any university; the lectures are the universities' own free recordings, played from where they host them. About & rights