information theory 52-104

Code rate R (k information bits and n total bits) is defined as

• k = n/R

• R = k * n

• R = k/n

• n = R * k

Conditional entropy H(Y|X) lies between

• - H(Y) and 0

• 0 and H(Y)

• - H(Y) and H(Y)

• 0 and 1

Convert the message into a signal suitable for transmission over the channel of communication, referred to as …

• Encoding

• Decoding

• Entropy

• Redundancy

Determine the Hamming distance for code that can detect 3 errors and correct 2 errors.

• 6

• 5

• 7

• 9

Determine the Hamming distance for code that can detect 3 errors and correct 1 errors.

• 5

• 4

• 6

• 8

Determine the Hamming distance for code that can detect 5 errors and correct 3 errors.

• 9

• 8

• 10

• 14

Encode a string "0000" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)

• 0000001

• 0000111

• 0000000

• 0000101

Encode a string "0001" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)

• 0001010

• 0001001

• 0001011

• 0001111

Encode a string "0010" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)

• 0010010

• 0010111

• 0010110

• 0010100

Encode a string "0011" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)

• 0011100

• 0011001

• 0011101

• 0011111