ML B2 CH4

What is a major bottleneck of the original CBOW model?

• The size of the input vector
• One-hot encoding of the context
• Large matrix calculations with softmax over huge vocabulary
• Too few context words

Answer explanation

The original CBOW model uses softmax across a large vocabulary, which is computationally expensive.

What is the function of the embedding layer in word2vec?

• Compress the entire matrix into a scalar
• Skip one-hot encoding and extract a word's vector directly
• Generate audio features
• Apply dropout to context words

Answer explanation

Embedding layers replace the need for one-hot vectors by selecting specific word vectors directly from the matrix.

What is the main goal of negative sampling in word2vec?

• To sample only frequent words
• To train the model faster using binary classification
• To filter out correct labels
• To convert softmax into sigmoid

Answer explanation

Negative sampling simplifies training by converting the multi-class problem into several binary decisions.

In negative sampling, what kind of output do we expect for negative examples?

• Close to 1
• Exactly 1
• Close to 0
• Negative values

Answer explanation

Negative examples should result in output values close to 0, indicating they are not the correct context.

Why is the sigmoid function used in binary classification for word2vec?

• It simplifies matrix multiplication
• It outputs discrete values only
• It provides probabilities between 0 and 1
• It reduces the training data size

Answer explanation

Sigmoid functions output values between 0 and 1, ideal for interpreting binary classification probabilities.

What does the cross-entropy error measure in binary classification?

• How long training takes
• The difference between output probability and the correct label
• The number of samples per class
• The angle between word vectors

Answer explanation

Cross-entropy measures the loss based on the distance between predicted probabilities and actual labels.

What technique helps select which negative examples to use?

• Random dropout
• Context padding
• Probability-based sampling
• Gradient descent

Answer explanation

Negative examples are sampled based on their frequency using probability-based techniques.

Why is 0.75 applied to probability values during sampling?

• To eliminate rare words
• To balance the selection of frequent and rare words
• To normalize the context
• To convert softmax to sigmoid

Answer explanation

Raising probabilities to 0.75 allows rare words a fairer chance of being sampled.

What analogy problem demonstrates the power of word2vec vectors?

• king - man + woman = queen
• dog + cat = pet
• fast - car + plane = slow
• Paris - France + Spain = France

Answer explanation

This classic example shows how vector arithmetic can capture semantic relationships.

What does word2vec convert words and sentences into?

• Charts and graphs
• Fixed-length vectors
• Syntax trees
• Word clouds

Answer explanation

The goal of word2vec is to represent words and even sentences as vectors in a continuous space.