Course Information Package
|Course Unit Title||INFORMATION THEORY|
|Course Unit Code||AEEE515|
|Course Unit Details||MSc Electrical Engineering (Technical Electives) -|
|Number of ECTS credits allocated||7|
|Learning Outcomes of the course unit||By the end of the course, the students should be able to:|
|Mode of Delivery||Face-to-face|
|Recommended optional program components||NONE|
Entropy, Relative Entropy, and Mutual Information
Entropy. Joint entropy and conditional entropy. Relative entropy and mutual information. Relationship between entropy and mutual information. Chain rules for entropy, relative entropy, and mutual Information. Jensen’s inequality and its consequences. Log sum inequality and its applications. Sufficient statistics. Fano’s inequality. Asymptotic equipartition property theorem.
Examples of codes. Kraft inequality. Optimal codes. Bounds on the optimal code length. Kraft inequality for uniquely decodable codes. Huffman codes. Optimality of Huffman codes. Shannon–Fano–Elias coding.
Examples of channel capacity. Noiseless binary channel. Noisy channel with non-overlapping outputs. Binary symmetric channel. Binary erasure channel. Symmetric channels. Properties of channel capacity. Channel coding theorem. Zero-error codes. Hamming codes.
Gaussian channel. Converse to the coding theorem for Gaussian channels. Band-limited channels. Parallel Gaussian channels.
Network Information TheoryGaussian multiple-user channels. Gaussian multiple-access channel with m users. Gaussian broadcast channel. Gaussian relay channel. Gaussian interference channel. Uncertainty on
|Recommended and/or required reading:|
|Planned learning activities and teaching methods||The teaching of the course is lecture-based (3 hours per week) in a classroom. A combination of traditional teaching with written notes on a white board and slide presentations using a projector for the presentation of the more complicated diagrams and graphs is utilized. Students are assessed continuously and their knowledge is checked through tests, assignments and the final exam. Lectures include the solution and discussion of example problems regarding the material presented. Relevant homework and assignments are given to the students for further study at their own. Due to the level and type of the course students are urged to participate in discussing the various topics and provide their opinion during problem-solving sessions. Lecture notes are compiled by students which serve to cover the main issues under consideration and serve as a guide for further reading. Students are also required to seriously use the textbook assigned to the course, in addition to other sources found either in the library or elsewhere in order to broaden their perspective on the various subjects presented in class and in the textbook.|
|Assessment methods and criteria|
|Language of instruction||English|