sem6_ai_week_9

In a simple RNN cell, the hidden‑state update is

• h_t = W_x x_t

• h_t = x_t + h_{t-1}

• h_t = tanh(x_t)

• h_t = tanh(W_x x_t + W_h h_{t-1} + b)

The LSTM input‑gate activation is computed as

• sigmoid(W_c c_{t-1})

• softmax(W_i x_t)

• tanh(W_i h_{t-1})

• sigmoid(W_i x_t + U_i h_{t-1} + b_i)

In Keras, a bidirectional GRU layer is created with

• GRU(units, bidirectional=True)

• BiGRU(units)

• GRU(units, go_backwards=True)

• Bidirectional(GRU(units))

Teacher‑forcing ratio controls

• dropout on embeddings

• learning‑rate warmup

• hidden‑state size

• probability of feeding ground‑truth token

The GRU reset‑gate formula is

• z_t * h_{t-1}

• sigmoid(W_r x_t)

• tanh(W_r h_{t-1})

• sigmoid(W_r x_t + U_r h_{t-1} + b_r)

Masking padded steps in Keras uses

• layers.Dropout()

• layers.Cropping1D()

• layers.ZeroPadding1D()

• layers.Masking()

Sequence‑to‑sequence models share parameters through

• max‑pool layers

• CNN kernels

• independent decoders

• encoder hidden‑state context vector

The cell‑state update in an LSTM is

• c_t = f_t

• c_t = i_t

• c_t = o_t

• c_t = f_t ⊙ c_{t-1} + i_t ⊙ ĝ_t

To reverse input sequences you call

• np.flip(seq,1)

• seq[::-1]

• seq.sort()

• tf.reverse(seq, [-1])

In seq‑to‑seq inference, the loop stops when

• batch size is zero

• gradient = 0

• loss < 0.1

• EOS token is generated