FNLP_W8

LSTM is a specialized recurrent neural network

• TRUE

• FALSE

Compared to vanilla RNNs, LSTM is different due to

• memory gate

• length of the network

• parallel computation

• 2D Convolutional Layers

Seq2Seq stands for

• Sequence-To-Sequence

• Sequential-To-Sequential

• Square-to-Square

• Sequel-To-Sequel

What was the main issue of Seq2Seq networks

• Bottleneck of information

• Vanishing Gradients

• Only long length of documents were allowed

• Too many UNK tokens

Which one was not a gate in the memory gate of LSTM

• Forget Gate

• Candidate Gate

• Output Gate

• Attention Gate

Training a LSTM network also updates the weights in the memory gate

• True

• False

What were the common concepts in both Seq2Seq and Transformers

• Encoder-Decoder

• Normalisation

• Memory Gates

• Residual Connections

This feature of Transformer gives unique positional information to the tokens

• Positional Encoding

• Feed Forward Networks

• Normalisation

• Addition

The addition layer of Transformers help preserve information

• True

• False

Which sentence is not true about "vanilla" Transformers?

• Positional Encoding helps give positional information for each token

• Add + Norm layer allows residual information to be preserved

• Attention = Query x Key

• Feed Forward Layer adds averaging of the attention