Use recurrent features from dataset to find the best answers.

Use a loop between inputs and outputs in order to achieve the better prediction.

Use loops between the most important features to predict next output.

Use previous inputs to find the next output according to the training set.

the model is prone to ignoring the hidden states with the highest gradients.

the model is prone to mixing the content of distant positions it with following positions’ content.

the model is prone to overfitting easily as exploding gradients hinder efficient back-propagation.

the model is prone to forgetting the content of distant positions in sequence.

V and K are outputted from the input embedding and Q from the output embedding.

Q and K are outputted from the input embedding and V from the output embedding.

V is outputted from the input embedding and K and Q from the output embedding.

Q is outputted from the input embedding and K and V from the output embedding.

$$Attention\left(Q,\ K,\ V\right)\ =\ soft\max\left(\frac{VK^T}{\sqrt[]{d_k}}\right)\ Q$$  

$$Attention\left(Q,\ K,\ V\right)\ =\ soft\max\left(\frac{QK^T}{\sqrt[]{d_k}}\right)\ V$$  

$$Attention\left(Q,\ K,\ V\right)\ =\ soft\max\left(\frac{QV^T}{\sqrt[]{d_k}}\right)\ K$$  

True

False

Every position should have the same identifier irrespectively of the sequence length.

Positional encodings have the same dimension as the embeddings.

There are many choices of positional encodings.

Using sinusoidal functions for positional embeddings allow large displacements of positional similarity/dissimilarity.

True

False

True

False

Soft Attention is the global Attention where all image patches are given some weight.

Hard Attention is the global Attention where all image patches are given some weight.

In Soft Attention, only one image patch is considered at a time.

In Hard Attention, only one image patch is considered at a time.