Font size
WorksheetsInf theory base
Total questions: 152
Worksheet time: 1hrs 16mins
______ indicate(s) an error in a received combination.
Parity bits
Data bits
None of the given
Error syndrome
... is a measure of uncertainty
Encoding
Entropy
Information
Redundancy
A code has two allowable combinations 101 and 010. What is the allowable combination for the error combination 001?
101
010
001
None
A code has two allowable combinations 101 and 010. What is the allowable combination for the error combination 100?
101
010
100
None
A code has two allowable combinations 101 and 010. What is the allowable combination for the error combination 000?
101
000
None
010
A code has two allowable combinations 101 and 010. What is the allowable combination for the error combination 111?
101
010
111
None
A code has two allowable combinations 101 and 010. What is the allowable combination for the error combination 011?
010
101
011
None
A code has two allowable combinations 101 and 010. What is the allowable combination for the error combination 110?
010
101
110
None
A codeword of the Hamming code consists of ________ and ________ bits.
data; parity
with errors; without errors
allowable; not allowable
none of the given
A Huffman code is a = 1, b = 000, c = 001, d = 01. Probabilities are p(a) = 0.4, p(b) = 0.1, p(c) = 0.2, p(d) = 0.3. The average length of codewords q is
2.1 bit
1.9 bit
2.0 bit
8.0 bit
A redundancy of a code S = ...
1 - Iavr/Imax
Iavr/Imax
1 + Iavr/Imax
Imax/Iavr
An average length of codewords qavr = ...
∑ (pi*qi)
∑ (pi/qi)
∑pi / n
∑qi / n
An efficiency of a code E = ...
Iavr/Imax
Imax/Iavr
Iavr/100
Imax - Iavr
ASCII code is a
Variable length code
Fixed length code
Error-correction code
None of the given
By the Bayes' rule for conditional entropy H(Y|X) = ...
H(X|Y) - H(X) + H(Y)
[P(A)] /P(B)
H(X|Y) - H(X)
H(X|Y)+ H(Y)
By the Bayes' theorem ...
P(B|A) = P(A and B)/P(A)
P(A|B) = [P(B|A)][P(A)] /P(B)
P(B|A) = P(A and B)*P(A)
P(A|B) = [P(B|A)][P(A)] * P(B)
By the Chain rule H(X,Y) = H(Y|X) + ...
H(X)
H(Y)
H(Y|X)
H(X|Y)
By the Hartley's formula the amount of information I = ...
I = n*log m
I = m*n
I = log (m/n)
I = log (m*n)
By the Hartley's formula the entropy H = ...
H = - ∑(pi*log pi)
H = - ∑ (log pi)
H = log m
H = - ∑ (pi/log pi)
By the property of joint entropy H(X,Y) <= ...
H(X)
H(Y)
H(X) + H(Y)
None of the given
By the property of joint entropy H(X,Y) ...
H(X,Y) >= H(X) and H(X,Y) <= H(Y)
H(X,Y) <= H(X) and H(X,Y) >= H(Y)
H(X,Y) >= H(X) and H(X,Y) >= H(Y)
H(X,Y) >= H(X) + H(Y)
By the Shannon's formula the amount of information I = ...
H = - n * ∑(pi * log pi)
H = - n * ∑ (log pi)
H = - n * ∑ pi
H = - n * ∑ (pi/log pi)
By the Shannon's formula the entropy H = ...
H = - ∑(pi*log pi)
H = - ∑ (log pi)
H = - ∑ pi
H = - ∑ (pi/log pi)
alculate the code rate for Hamming (15,11) code
1
0.733
0.571
0.839
Calculate the code rate for Hamming (31,26) code
1
0.839
0.733
0.571
Calculate the code rate for Hamming (7,4) code
1
0.571
0.733
0.839
Calculate the efficiency of the language if it has 32 letters and its I average is 1 bit.
0.8
0.2
5
1
Calculate the redundancy of the language if it has 32 letters and its I average is 1 bit.
0.8
0.2
5
1
Choose an example of block code
Shannon-Fano code
Huffman code
Hamming code
None of the given
Choose conditions of an optimal coding (p – probability, l – length of a code word)
pi < pj and li<=lj
pi > pj and li<=lj
pi > pj and li>=lj
none of the given
Choose the formula to create the Hamming code
(n, k) = (2r - 1, 2r - 1 - r)
(n, k) = (2r, 2r - 1 - r)
(n, k) = (2r - 1, 2r - r)
(n, k) = (2r - 1, 2r - 1 + r)
Choose the formula to determine the number N of possible messages with length n if the message source alphabet consists of m characters, each of which can be an element of the message.
N = mn
N = nm
N = m*n
N = log m
N = m^n
Code has dmin = 1. How many errors can be corrected by this code?
2
3
0
1
Code has dmin = 1. How many errors can be detected by this code?
2
3
0
1
Code has dmin = 10. How many errors can be detected by this code?
4
8
9
10
Code has dmin = 11. How many errors can be corrected by this code?
11
7
5
10
Code has dmin = 11. How many errors can be detected by this code?
5
9
10
11
Code has dmin = 12. How many errors can be detected by this code?
5
10
11
12
Code has dmin = 2. How many errors can be corrected by this code?
2
3
0
1
Code has dmin = 2. How many errors can be detected by this code?
2
3
1
0
Code has dmin = 3. How many errors can be corrected by this code?
2
3
1
4
Code has dmin = 3. How many errors can be detected by this code?
1
3
2
4
Code has dmin = 4. How many errors can be detected by this code?
5
1
3
4
Code has dmin = 5. How many errors can be corrected by this code?
5
3
2
4
ode has dmin = 5. How many errors can be detected by this code?
6
2
4
5
Code has dmin = 6. How many errors can be detected by this code?
6
2
5
4
Code has dmin = 7. How many errors can be corrected by this code?
5
6
3
4
Code has dmin = 7. How many errors can be detected by this code?
7
3
6
5
Code has dmin = 8. How many errors can be detected by this code?
8
6
7
3
Code has dmin = 9. How many errors can be corrected by this code?
5
7
4
8
Code has dmin = 9. How many errors can be detected by this code?
7
9
8
4
Code rate R (k information bits and n total bits) is defined as
k = n/R
R = k * n
R = k/n
n = R * k
Conditional entropy H(Y|X) lies between
- H(Y) and 0
0 and H(Y)
- H(Y) and H(Y)
0 and 1
Convert the message into a signal suitable for transmission over the channel of communication, referred to as …
Encoding
Decoding
Entropy
Redundancy
Determine the Hamming distance for code that can detect 3 errors and correct 2 errors.
6
7
5
9
Determine the Hamming distance for code that can detect 3 errors and correct 1 errors.
5
4
6
8
Determine the Hamming distance for code that can detect 5 errors and correct 3 errors.
9
8
10
14
Encode a string "0000" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
0000001
0000111
0000000
0000101
Encode a string "0001" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
0001010
0001001
0001011
0001111
Encode a string "0010" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
0010010
0010111
0010110
0010100
Encode a string "0011" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
0011100
0011001
0011101
0011111
Encode a string "0100" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
0100011
0100110
0100111
0100101
Encode a string "0110" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
0110101
0110011
0110001
0110000
Encode a string "0101" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
0101101
0101000
0101100
0101110
Encode a string "0111" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
0111110
0111000
0111010
0111011
Encode a string "1000" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
1000111
1000100
1000101
1000001
Encode a string "1001" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
1001111
1001010
1001110
1001100
Encode a string "1010" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
1010111
1010001
1010011
1010010
Encode a string "1011" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
1011100
1011010
1011000
1011001
Encode a string "1100" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
1100110
1100000
1100010
1100011
Encode a string "1101" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
1101101
1101011
1101001
1101000
Encode a string "1110" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
1110000
1110101
1110100
1110110
Encode a string "1111" with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
1111110
1111011
1111111
1111101
The length of the message is 16 symbols and the message’s alphabet consists of 4 symbols. Find the amount of information in this message.
16
80
64
32
The efficiency of the language is 0,5 and its I average is equal to 1 bit. Calculate the number of letters in this language’s alphabet?
64
32
4
16
Elements of alphabets X and Y are statistically related. It is known that H(X)=4 bits and H(Y)=10 bits. What are a range of variation for a conditional entropy H(Y|X) when H(X|Y) changes from its min to max?
(from 7 to 11)
(from 4 to 12)
(from 6 to 10)
(from 6 to 11)
For Hamming distance dmin and r errors in the received word, the condition to be able to detect the errors is
dmin>= r+1
dmin>= 2r+1
dmin>= 2r+2
dmin>= r+2
Hamming (7,4) code can correct ___ error(s)
2
3
1
0
For Hamming distance dmin and s errors in the received word, the condition to be able to correct the errors is
dmin>= s+1
dmin>= 2s+1
dmin>= 2s+2
dmin>= s+2
Hamming distance can easily be found with ...
XNOR operation
XOR operation
OR operation
AND operation
How does a noise affect the data?
change only the 0 to 1
change only the 1 to 0
change the 0 to 1 and the 1 to 0
None of the above
How many data bits are in the (15, 11) Hamming code?
11
4
15
5
How many data bits are in the (31, 26) Hamming code?
26
31
5
4
How many data bits are in the (7, 4) Hamming code?
4
3
7
10
How many parity bits are in the (15, 11) Hamming code?
4
15
11
5
How many parity bits are in the (31, 26) Hamming code?
26
31
5
4
How many parity bits are in the (7, 4) Hamming code?
3
4
7
11
A Huffman code is a = 0, b = 10, c = 110, d = 1110, e = 1111. Probabilities are p(a) = 0.50, p(b) = 0.30, p(c) = 0.15, p(d) = 0.03, p(e) = 0.02. The average length of a code words is
1.75 bit
2.0 bit
1.3 bit
1.7 bit
Which letter will get the shortest codeword after Huffman coding of the word «bbaacccabaac»?
a
b
c
none
An alphabet consist of the letters a, b, c, d, e and f. The probability of occurrence is p(a) = 0.06, p(b) = 0.15, p(c) = 0.4 and p(d) = 0.18, p(e)=0.17, p(f)=0.04. The Huffman code is
c=1,d=000,e=001,b=010,a=0110,f=0111
c=0,d=111,e=110,b=101,a=1001,f=1000
c=1,d=01,e=001,b=0000,a=00010,f=00011
c=1,d=01,e=001,b=000,a=0010,f=00011
c=0,d=101,e=110,b=101,a=1000,f=1001
If k - number of bits before Hamming encoding and n - number of bits after Hamming encoding then
k > n
k < n
k = n
k = 1/2 n
In digital communication system, smaller the code rate, ... are the redundant bits.
less
equal
more
unpredictable
Main idea of error control codes is
To add some redundancy
To delete some redundancy
To double all bits
None of the given
Noise affects ...
information source
receiver
channel
transmitter
Shannon-Fano and Huffman codes are an encoding algorithms used for
lossy data compression
lossless data compression
error correction
error detection
Specify the case when entropy is maximum
p1=0,5 and p2=0,5
p1=1 and p2=0
p1=0 and p2=1
p1=0,9 and p2=0,1
Specify the error position in the string "0001110", if the initial string was encoded with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
i4
i1
i2
r2
Specify the error position in the string "1000110", if the initial string was encoded with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
i1
i4
i2
i3
Specify the error position in the string "1001010", if the initial string was encoded with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
r2
r1
r3
i3
Specify the error position in the string "1001100", if the initial string was encoded with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
r3
r2
r1
no error
Specify the error position in the string "1001110", if the initial string was encoded with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
r1
no error
r2
i4
Specify the error position in the string "1001111", if the initial string was encoded with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
r1
r3
r2
i4
Specify the error position in the string "1011110", if the initial string was encoded with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
i1
i3
i2
i4
Specify the error position in the string "1101110", if the initial string was encoded with Hamming (7,4) code using the following structure (i1, i2, i3, i4, r1, r2, r3)
i1
i2
i3
i4
Specify the formula to find the amount of information if events have different probabilities.
Hartley's formula
Shannon's formula
Fano's formula
Bayes' formula
Specify the formula to find the amount of information if events have the same probabilities.
Shannon's formula
Hartley's formula
Fano's formula
Bayes' formula
Specify the right formula if dmin is Hamming distance, s - number of correctable errors and r - number of detecteable errors.
dmin>= s+r+1
dmin>= 2s+r+1
dmin>= s+2r+1
dmin>= s+r+2
Suppose the letters a, b, c, d, e, f have probabilities 1/2, 1/4, 1/8, 1/16, 1/32, 1/32 respectively. Which of the following is the Huffman code for the letter a, b, c, d, e, f?
11, 10, 011, 010, 001, 000
0, 10, 110, 1110, 11110, 11111
11, 10, 01, 001, 0001, 0000
110, 100, 010, 000, 001, 111
Suppose the letters a, b, c, d, e, f have probabilities 1/2, 1/4, 1/8, 1/16, 1/32, 1/32 respectively. What is the average length q of the Huffman code?
3.0
1.9
2.7
4.3
The amount of information in the message is 120 bits. Calculate the length of this message, which is written by characters of 16-character alphabet.
30
480
120
130
The amount of information in the message is 60 bits. Calculate the length of this message, which is written by characters of 4-character alphabet.
30
60
15
510
The basic idea behind Shannon-Fano coding is to
compress data by using more bits to encode more frequently occuring characters
compress data by using fewer bits to encode more frequently occuring characters
compress data by using fewer bits to encode fewer frequently occuring characters
expand data by using fewer bits to encode more frequently occuring characters
The efficiency of the language is 0,25 and its I average is 1 bit. Calculate the number of letters in this language's alphabet?
32
16
8
64
The first code combination is 0000 and the Hamming distance of this code equals 4. Choose the second combination.
1111
1011
0011
0000
The Hamming code is a method of _______.
Error control coding
Optimal coding
None of the above
The Hamming distance between "client" and "server" is
0
1
6
impossible to detect
The Hamming distance between "make" and "made" is
4
3
1
impossible to detect
The Hamming distance between "push" and "pull" is
0
4
2
impossible to detect
The Hamming distance between "starting" and "finishing" is
4
3
impossible to detect
5
The Hamming distance between 001111 and 010011 is
1
2
3
4
The Hamming distance between 010111 and 010011 is
2
3
1
4
The Hamming distance between 011111 and 010011 is
1
3
2
4
The Hamming distance between 101001 and 010011 is
1
2
4
3
The length of the message is 16 symbols and the message's alphabet consists of 32 symbols. Find the amount of information in this message.
80
16
64
32
The length of the message is 6 symbols and the message's alphabet consists of 32 symbols. Find the amount of information in this message.
30
6
32
24
The redundancy of the language is 0,75 and its I average is 1 bit. Calculate the number of letters in this language's alphabet?
32
16
8
64
The string was encoded with Hamming (7,4) code using the structure (i1, i2, i3, i4, r1, r2, r3). After a channel the error syndrome is 000. Specify the position of the error.
i1
r1
no error
r3
The string was encoded with Hamming (7,4) code using the structure (i1, i2, i3, i4, r1, r2, r3). After a channel the error syndrome is 001. Specify the position of the error.
i1
r1
r3
no error
he string was encoded with Hamming (7,4) code using the structure (i1, i2, i3, i4, r1, r2, r3). After a channel the error syndrome is 010. Specify the position of the error.
r3
r1
r2
i2
The string was encoded with Hamming (7,4) code using the structure (i1, i2, i3, i4, r1, r2, r3). After a channel the error syndrome is 011. Specify the position of the error.
r4
i1
i4
r1
The string was encoded with Hamming (7,4) code using the structure (i1, i2, i3, i4, r1, r2, r3). After a channel the error syndrome is 100. Specify the position of the error.
r3
i1
r1
r2
The string was encoded with Hamming (7,4) code using the structure (i1, i2, i3, i4, r1, r2, r3). After a channel the error syndrome is 101. Specify the position of the error.
no error
r1
i1
i2
The string was encoded with Hamming (7,4) code using the structure (i1, i2, i3, i4, r1, r2, r3). After a channel the error syndrome is 110. Specify the position of the error.
i4
r3
i3
i1
The string was encoded with Hamming (7,4) code using the structure (i1, i2, i3, i4, r1, r2, r3). After a channel the error syndrome is 111. Specify the position of the error.
i4
r2
i2
no error
This is the method for data processing for reducing errors during transmission via channel with noise.
Error Correction code
Uniform code
Non-uniform code
Optimal code
We can divide coding schemes into two broad categories: ________ and ______coding.
block; linear
linear; nonlinear
block; convolution
none of the given
What is the first step of Shannon-Fano algorithm?
Characters of the original alphabet are setted in descending order of probability.
Letters divided to the two subsets so that the overall probability of these subsets were about equal.
For all characters (letters) of the top subset assign a code element 1, and for characters of the lower subset the 0 code.
For all characters (letters) of the top subset assign a code element 0, and for characters of the lower subset the 1 code.
What is the Hamming distance between two strings of equal length?
the number of positions at which the corresponding symbols are different
the number of positions at which the corresponding symbols are equal
the number of identical symbols in the first string
the number of identical symbols in the second string
What is the meaning of number “2” in the formula I = n*log2m?
Binary number system
Message length equals to 2
It hasn’t any meaning
Information is measured in nits
What is the sample space of one dice roll?
{1,2,3,4,5,6}
{1,3,5}
{2,4,6}
{1,2,3,4,5,6,7,8,9,10,11,12}
When the base of the logarithm is 10, then the unit of measure of information is
bytes
dits
nits
bits
When the base of the logarithm is 2, then the unit of measure of information is
bytes
bits
nits
dits
When the base of the logarithm is e, then the unit of measure of information is
bytes
nits
dits
bits
Which letter will get the shortest codeword after Huffman coding of the word "abracadabra"?
c
r
d
a
Which of the following codes can be the Huffman code for the letters a,b,c,d,e?
10,011,11,001,010
0,10,110,1110,1111
10,01,0001,100,1010
100,110,001,000,010
Which of the following codes has the highest code rate?
code rate is constant for all of the Hamming codes
Hamming (31,26)
Hamming (15,11)
Hamming (7,4)
Which of the following codes has the highest redundancy?
redundancy is constant for all of the Hamming codes
Hamming (7,4)
Hamming (15,11)
Hamming (31,26)
Which of the following codes is prefix?
0, 111, 11
0, 111, 10
0, 101, 10
00, 10, 101
Which of the following codes is prefix?
0, 01, 11
0, 10, 11
0, 10, 1
0, 01, 001
Which of the following codes is uniform?
ASCII
Shannon-Fano
Huffman
None of the given
Which of the following codes is uniform?
10,011,11,001,010
0,10,110,1110,1111
10,01,0001,100,1010
100,110,001,000,010
Which of the following symbols will get the shortest codeword after Shannon-Fano coding if probabilities are p(a) = 0.05, p(b) = 0.6, p(c) = 0.2 and p(d) = 0.15?
c
a
d
b
