Information and Coding Theory

Course Title: Information and Coding Theory
Full Marks: 45 + 30
Course No: C.Sc. 667
Pass Marks: 22.5 + 15
Nature of the Course: Theory
Credit Hrs: 3

Course Description:

This course deals with the basic concepts of information theory and coding. Students will get idea about entropy, data compression, channel capacity and error control coding

1.Entropy 12Hrs

1.1 entropy,
1.2 relative entropy,
1.3 mutual information,
1.4 chain rules,
1.5 data processing inequality,
1.6 the asymptotic equi-partition property,
1.7 entropy rates for stochastic processes.

2.Data Compression 11hrs

2.1 source coding theorem,
2.2 Kraft inequality,
2.3 Shannon‐Fano codes,
2.4 Huffman codes,
2.5 universal source codes.

3.Channel Capacity 11hrs

3.1 discrete channels,
3.2 random coding bound and converse,
3.3 Gaussian channels,
3.4 parallel Gaussian channels and “water‐pouring”,
3.5 bandlimited channels.

4.Error Control Coding 11hrs

4.1 linear block codes and their properties,
4.2 hard‐decision decoding,
4.3 convolutional codes,
4.4 Viterbi decoding algorithm,
4.5 iterative decoding.

Reference:

1. T.M. Cover and J.A. Thomas, Elements of Information Theory, John Wiley India Limited, New Delhi, 2010.