Information Theory & Coding

What does the concept of entropy measure in information theory? CO1

• Data transmission speed

• Information content and uncertainty

• Signal strength

• Error rate

Which theorem establishes the fundamental limits of lossless data compression?CO1

• Shannon's source coding theorem

• Huffman coding theorem

• Kraft's inequality

• Mutual information theorem

What property of source codes ensures that no codeword is a prefix of another codeword?CO1

• Uniquely decodable

• Prefix-free

• Redundant

• Variable-length

Mutual information quantifies the amount of information that two random variables share about each other. It is: CO1

• Always negative

• Symmetric

• Additive

• Dependent on the alphabet size

Which coding technique aims to minimize the average length of a source code?CO1

• Huffman coding

• Shannon-Fano coding

• Arithmetic coding

• Run-length encoding

What is the channel capacity of a noiseless channel?CO2

• Zero

• Infinite

• Variable

• Constant

In a binary symmetric channel, if the channel transition probability is 0.1, what is the probability of error for transmitting a single bit?CO2

• 0.1

• 0.2

• 0.5

• 0.9

What does the mutual information measure in the context of communication channels?CO2

• The total information transmitted

• he information lost during transmission

• The information shared between input and output

• The channel capacity

Which theorem defines the maximum rate at which information can be reliably transmitted over a noisy channel?CO2

• Shannon's source coding theorem

• Shannon's channel coding theorem

• Huffman coding theorem

• Nyquist theorem

For a binary erasure channel, what type of errors does it primarily encounter?CO2

• Bit errors

• Burst errors

• Erasure errors

• Phase errors