mi1_03_gradient

The pace so far is....

• too fast, I can't follow!

• ok.

• too slow, stop rambling!

Did you attempt the last exercise sheet?

• Yes

• No

Which is NOT one of the Robbins-Monro conditions for convergence?

• The error function is strictly convex

• A solution w* exists

• the gradients are bounded over time

• The components in the gradient are always > 0

The mini-batch cost with batch size K becomes less noisy with...

• larger K

• smaller K

stochastic optimization (online learning) is....

• slower at convergence than batch or mini-batch training

• very noisy in the beginning only

• very noisy, always

• mini-batch with batch size = 1

impulse terms and line search are all about...

• selecting a better constant learning rate.

• avoiding overfitting

• computing gradients at each time step faster

• reaching the optimum faster

What are possible pitfalls of the analytical solution for regression?

• It overfits to the training data.

• infeasible with very large datasets or large dimensions

• It only works with very large datasets

• It does not have any pitfalls