Rappel: sequence models

What are the limitations of CNNs?

• Fixed input size

• Lack of memory

• Order invariance

• Parallel processing capability

What is a recurrent neural network (RNN)?

• Designed to handle sequences of variable length

• Does not have memory

• Captures temporal dependencies

• Used only for image processing

What is backpropagation through time (BPTT)?

• Learning method for RNN

• Dimensionality reduction technique

• Approach for image processing

• Classification algorithm

What are the two common problems that occur during backpropagation?

• Vanishing gradients

• Exploding gradients

• Both of the above

• None of the above

What are the steps that an LSTM network performs during each pass?

• Detection of past information in the cell state via the forget gate

• Selection of relevant long-term information through the input gate

• Adding the selected information to the cell state

• Detection of important short-term information in the cell state

• Generation of the new hidden state through the output gate

What is the function of the input gate in an LSTM?

• Decide which new information will be stored.

• Eliminate irrelevant information.

• Regulate the network with the tanh function.

• Produce short-term memory.

How does the forget gate work in an LSTM?

• It decides which information should be retained.

• It multiplies the long-term memory by a forget vector.

• It uses the sigmoid function.

• It produces a new version of the long-term memory.

What is the role of the output gate in an LSTM?

• Produce short-term memory.

• Take the current input and the previous short-term memory.

• Pass long-term memory through an activation function.

• Decide which information should be retained.

What is a Gated Recurrent Unit (GRU)?

• A variant of the RNN architecture.

• A type of long-term memory.

• A mechanism for controlling the flow of information.

• A method for network regulation.