ML B2 CH3

What is one key problem of statistics-based NLP techniques?

• They require GPU for training
• They use real-time prediction
• They create huge matrices and require full-batch learning
• They ignore text data

Answer explanation

Statistics-based methods create large co-occurrence matrices and need full-batch learning, which is computationally heavy.

How do inference-based techniques differ from statistics-based techniques?

• They predict scores for unseen programming code
• They do not need context words
• They use mini-batch learning and GPU acceleration
• They require a thesaurus to function

Answer explanation

Inference-based techniques learn patterns using mini-batches, allowing GPU-based parallel training.

What does a one-hot vector represent in NLP?

• A gradient value
• A probability distribution
• A vector with one element set to 1 and others to 0
• A random binary pattern

Answer explanation

One-hot vectors represent words using binary vectors with only one active position.

What is the purpose of the CBOW model in word2vec?

• To convert text into audio
• To predict the next sentence
• To predict a target word from context words
• To translate between languages

Answer explanation

CBOW (Continuous Bag of Words) predicts the center (target) word using the surrounding context.

What does the output layer in CBOW represent?

• Loss values
• Weights only
• Scores for each word in the vocabulary
• Sentence embeddings

Answer explanation

The output layer produces scores for each vocabulary word; softmax is applied to get probabilities.

Which function converts output scores into probabilities in CBOW?

• ReLU
• Sigmoid
• Tanh
• Softmax

Answer explanation

Softmax converts the output scores into a probability distribution over all vocabulary words.

Why are only input-side weights often used in word2vec representation?

• They require less memory
• They are easier to normalize
• They provide a meaningful vector representation of word semantics
• They include grammatical rules

Answer explanation

Input-side weights in word2vec reflect word meanings and are commonly used in vector space models.

What does the Skip-gram model do?

• It infers the target word from context words
• It classifies parts of speech
• It infers context words from the target word
• It encodes documents

Answer explanation

Skip-gram does the reverse of CBOW; it predicts the surrounding context from a given word.

Which is a main advantage of the CBOW model over Skip-gram?

• More accurate word predictions
• Faster training speed
• Larger context window
• Handles synonyms better

Answer explanation

CBOW trains faster than Skip-gram because it averages context vectors and uses fewer updates.

What does the cross-entropy error measure in the CBOW training?

• How long training takes
• The accuracy of synonym detection
• The difference between predicted and true word probabilities
• The number of neurons in the model

Answer explanation

Cross-entropy measures the error between predicted probabilities and the correct answer's probability.