Which of the following is NOT a type of word embedding technique?

• a) Word2Vec
• b) GLoVE
• c) Bag of Words
• d) Skip-gram

What is the purpose of the Continuous Bag of Words (CBoW) model?

• To predict the context words given a target word
• b) To predict the target word given a set of context words
• c) To generate new text based on a given set of words
• To predict the next word given pervious words

What is the difference between static embeddings and contextualized embeddings?

• Static embeddings are trained on a specific task, while contextualized embeddings are trained on a general task
• b) Static embeddings are the same as binary bag of words

• c) Static embeddings are fixed for all instances of a word, while contextualized embeddings vary depending on the context of the word
• Static embeddings are generated using CBoW while contextualized embedding are based on skip-grams

What is the purpose of word embeddings?

• To capture the meaning of words in a numeric format
• To create hand-crafted features for machine learning models

• To collect specially-designed data for machine learning models

• to compress high dimensional sparse representation of words into a compact form

What is the advantage of language models over most other machine learning models?

• They need hand-crafted features and specially-collected data
• b) They can be trained on running text in a self-supervised manner
• c) They require a small corpus of text data
• d) They are much smarter

What type of loss function used in both CBOW and Skip-Gram models?

• A) sigmoid
• B) They exclusively use a mean squared error loss function.
• C) The loss functions prioritize syntactic accuracy over semantic meaning.
• D) They employ softmax functions

• E) Loss functions in these models are irrelevant as long as the embeddings are accurate.

What is the dimension of the softmax layer in the CBoW model

• one as this is a binary classification
• Vocabulary size
• equals the embedding dimension used
• the same as the dimension of the weight matrix

What is the role of Cross-Entropy loss in CBOW and Skip-Gram models?

• A) Balancing the weight distribution across different layers of the neural network.
• B) Reducing the impact of less frequent words in the vocabulary.
• C) Maximizing the probability of the target word given its context (or vice versa).
• D) Simplifying the model to work with binary classification tasks.

How does the Skip-Gram model in word embeddings operate?

• A) By predicting surrounding context words from a given target word.
• B) By converting each word into a fixed numerical vector irrespective of context.
• C) By using a sequence of words to predict the next word in a sentence.
• D) By skipping irrelevant words in a sentence to focus on key terms.

What distinguishes the Continuous Bag of Words (CBOW) model in word embeddings?

• A) It predicts a target word based on a set of context words surrounding it.
• B) It transforms words into binary representations for classification tasks.
• C) CBOW is used exclusively for speech recognition.
• D) It predicts context words from a target word.