C5M2

Suppose you learn a word embedding for a vocabulary of 10000 words. Then the embedding vectors should be 10000 dimensional, so as to capture the full range of variation and meaning in those words.

• TRUE

• FALSE

What is t-SNE?

• A linear transformation that allows us to solve analogies on word vectors

• A non-linear dimensionality reduction technique

• A supervised learning algorithm for learning word embeddings

• An open-source sequence modeling library

• TRUE

• FALSE

Which of these equations do you think should hold for a good word embedding? (Check all that apply)

• e_boy - e_girl ≈ e_brother - e_sister

• e_boy - e_girl ≈ e_sister - e_brother

• e_boy - e_brother ≈ e_girl - e_sister

• e_boy - e_brother ≈ e_sister - e_girl

Let EE be an embedding matrix, and let o₁₂₃₄ be a one-hot vector corresponding to word 1234. Then to get the embedding of word 1234, why don’t we call E * o₁₂₃₄ in Python?

• It is computationally wasteful.

• The correct formula is E^T* o₁₂₃₄.

• This doesn’t handle unknown words ().

• None of the above: calling the Python snippet as described above is fine.

When learning word embeddings, we create an artificial task of estimating P(target∣context). It is okay if we do poorly on this artificial prediction task; the more important by-product of this task is that we learn a useful set of word embeddings.

• TRUE

• FALSE

In the word2vec algorithm, you estimate P(t∣c), where t is the target word and c is a context word. How are t and c chosen from the training set? Pick the best answer.

• c is a sequence of several words immediately before t.

• c is the one word that comes immediately before t.

• c and t are chosen to be nearby words.

• c is the sequence of all the words in the sentence before t.

• θ_t and e_c are both 500 dimensional vectors.

• θ_t and e_c are both 10000 dimensional vectors.

• θ_t and e_c are both trained with an optimization algorithm such as Adam or gradient descent.

• After training, we should expect θ_t to be very close to e_c when t and c are the same word.

• θ_i and e_j should be initialized to 0 at the beginning of training.

• θ_i and e_j should be initialized randomly at the beginning of training.

• X_ij is the number of times word j appears in the context of word i.

• The weighting function f(.) must satisfy f(0) = 0

You have trained word embeddings using a text dataset of m₁ words. You are considering using these word embeddings for a language task, for which you have a separate labeled dataset of m₂ words. Keeping in mind that using word embeddings is a form of transfer learning, under which of these circumstance would you expect the word embeddings to be helpful?

• m₁ >> m₂

• m₁ << m₂