1. Basic Laws of Probability and Random Variables

• Events and Sets
• Joint and Conditional Probabilities, Independence and Bayes Theorem
• Continuous and Discrete Random Variables
• Key Parameters of Random Variables: Mean, Variance, Moments
• Jointly Distributed Random Variables

2. Entropy and Information Content

• Information Content
• Entropy and Redundancy

3. Source Coding

• The Source Coding Theorem
• Shannon-Fano Coding
• Huffman Coding

4. Conditional Entropy and Mutual Information

• Conditional and Joint Entropy
• Mutual Information
• Chain Rules and the Data Processing Theorem

5. Channel Capacity

• The Channel Coding Theorem
• The Binary Symmetric and the Binary Erasure Channel
• Entropy and Mutual Information for Continuous Random Variables
• The AWGN Channel

6. Channel Coding

• Coding in Digital Communications
• Error Detection and Error Correction
• Binary Linear Block Codes
• Decoding of Short Binary Linear Block Codes

Stefan. M. Moser, Po-Ning Chen, A Student’s Guide to Coding and Information Theory, Cambridge University Press, 2012.
Benedetto, S., Biglieri, E., Principles of Digital Transmission, Kluwer Academic, Plenum Publishers, 1999.
Robert McEliece: The Theory of Information and Coding, Student Edition, Cambridge University Press, 2004.
David MacKay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.
Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, Wiley, 2006.Alan V. Oppenheim, Alan S. Willsky: Signals & Systems. Pearson, 2013.

1. Analog and Digital Signals in Time Domain

• Definition of Signals
• Elementary Signals: step, rectangle, triangle, sinusoidal signals, complex exponential
• Dirac Impulse
• Signal Properties and Operations
• Orthogonality of Signals

2. Description of Systems in Time Domain

• Definition and Basic Properties
• Memoryless and Dynamic Systems
• Linear Time-Invariant (LTI) Systems
• Impulse Response and Convolution Integral
• Unit Step Response
• Eigenfunctions

3. Fourier Series and Fourier Transform

• Orthogonal Periodic Functions
• Fourier Series
• Fourier Transform: definition, properties, transforms of periodic functions, the Dirac impulse train, application to LTI systems
• A/D Conversion and the Sampling Theorem

Stefan. M. Moser, Po-Ning Chen, A Student’s Guide to Coding and Information Theory, Cambridge University Press, 2012.
Benedetto, S., Biglieri, E., Principles of Digital Transmission, Kluwer Academic, Plenum Publishers, 1999.
Robert McEliece: The Theory of Information and Coding, Student Edition, Cambridge University Press, 2004.
David MacKay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.
Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, Wiley, 2006.Alan V. Oppenheim, Alan S. Willsky: Signals & Systems. Pearson, 2013.