|
Lernziele / Kompetenzen
|
After successful completion of this course, students will be able to apply the basic laws of probability in order to quantify information. They will be able to find the optimum source code for a memoryless source and will be proficient in the computation of joint, conditional and marginal entropies, as well as the mutual information. Furthermore, students will be competent in evaluating the channel capacity of fundamental channel models, including the binary symmetric channel and various forms of the AWGN channel. Additionally, they will have a comprehensive understanding of the properties of binary linear block codes and their associated decoding algorithms.
Students will be able to mathematically describe signals and linear systems in time and frequency domain and calculate the interaction of signals in linear systems. They are proficient in the use of the Fourier series and the Fourier transform to describe signals and systems in the frequency domain.
|
|
Veranstaltungen
|
Information Theory and Coding
| Art |
Vorlesung |
| Nr. |
EMI405 |
| SWS |
2.0 |
| Lerninhalt |
1. Basic Laws of Probability and Random Variables
- Events and Sets
- Joint and Conditional Probabilities, Independence and Bayes Theorem
- Continuous and Discrete Random Variables
- Key Parameters of Random Variables: Mean, Variance, Moments
- Jointly Distributed Random Variables
2. Entropy and Information Content
- Information Content
- Entropy and Redundancy
3. Source Coding
- The Source Coding Theorem
- Shannon-Fano Coding
- Huffman Coding
4. Conditional Entropy and Mutual Information
- Conditional and Joint Entropy
- Mutual Information
- Chain Rules and the Data Processing Theorem
5. Channel Capacity
- The Channel Coding Theorem
- The Binary Symmetric and the Binary Erasure Channel
- Entropy and Mutual Information for Continuous Random Variables
- The AWGN Channel
6. Channel Coding
- Coding in Digital Communications
- Error Detection and Error Correction
- Binary Linear Block Codes
- Decoding of Short Binary Linear Block Codes
|
| Literatur |
Stefan. M. Moser, Po-Ning Chen, A Student’s Guide to Coding and Information Theory, Cambridge University Press, 2012.
Benedetto, S., Biglieri, E., Principles of Digital Transmission, Kluwer Academic, Plenum Publishers, 1999.
Robert McEliece: The Theory of Information and Coding, Student Edition, Cambridge University Press, 2004.
David MacKay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.
Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, Wiley, 2006.Alan V. Oppenheim, Alan S. Willsky: Signals & Systems. Pearson, 2013. |
Signals and Systems
| Art |
Vorlesung |
| Nr. |
EMI403 |
| SWS |
2.0 |
| Lerninhalt |
1. Analog and Digital Signals in Time Domain
- Definition of Signals
- Elementary Signals: step, rectangle, triangle, sinusoidal signals, complex exponential
- Dirac Impulse
- Signal Properties and Operations
- Orthogonality of Signals
2. Description of Systems in Time Domain
- Definition and Basic Properties
- Memoryless and Dynamic Systems
- Linear Time-Invariant (LTI) Systems
- Impulse Response and Convolution Integral
- Unit Step Response
- Eigenfunctions
3. Fourier Series and Fourier Transform
- Orthogonal Periodic Functions
- Fourier Series
- Fourier Transform: definition, properties, transforms of periodic functions, the Dirac impulse train, application to LTI systems
- A/D Conversion and the Sampling Theorem
|
| Literatur |
Stefan. M. Moser, Po-Ning Chen, A Student’s Guide to Coding and Information Theory, Cambridge University Press, 2012.
Benedetto, S., Biglieri, E., Principles of Digital Transmission, Kluwer Academic, Plenum Publishers, 1999.
Robert McEliece: The Theory of Information and Coding, Student Edition, Cambridge University Press, 2004.
David MacKay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.
Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, Wiley, 2006.Alan V. Oppenheim, Alan S. Willsky: Signals & Systems. Pearson, 2013. |
|