CS4417/9117/9647 Final Exam Review Game

Suppose you have a 10000 word vocabulary, and are learning 500-dimensional word embeddings. The GloVe model minimizes this objective: True/False: Xij is the number of times word j appears in the context of word i.

• True

• False

Which of these equations do you think should hold for a good word embedding? [Check all that apply)

• 
• 
• 
• 

True/False: Suppose you learn a word embedding for a vocabulary of 60000 words. Then the embedding vectors could be 60000 dimensional, so as to capture the full range of variation and meaning in those words.

• True

• False

In beam search, if you increase the beam width B, which of the following would you expect to be true?

• Beam search will use up more memory.

• Beam search will converge after fewer steps.

• Beam search will generally find better solutions (i.e. do a better job maximizing P(y∣x)).

• Beam search will run more quickly.

Consider using this encoder-decoder model for machine translation.

True/False: This model is a “conditional language model” in the sense that the decoder portion (shown in purple) is modeling the probability of the output sentence y given the input sentence x.

• True

• False

In machine translation, if we carry out beam search without using sentence normalization, the algorithm will tend to output overly short translations.

• True

• False

A Transformer Network, unlike its predecessors RNNs, GRUs and LSTMs, can process entire sentences all at the same time. (Parallel architecture).

• True

• False

What letter does the "??question mark" represent in the following representation of Attention?

• Q

• K

• V

• t

Which of the following statements represents Key (K) as used in the self-attention calculation?

• Represent the actual information or content associated with each entity.

• The order of words in a sentence

• Represent the entities in the dataset that can be matched against the query.

• Represent the entity that is seeking information.

Following is the architecture within a Transformer Network. (without displaying positional encoding and output layers(s))

What is the output layer(s) of the Decoder ? (Marked YYY, pointed by the independent arrow)

• Linear layer

• SoftMax layer followed by a linear layer

• Linear layer followed by a SoftMax layer

• SoftMax layer