NLP Module 2.2

What is an N-gram in natural language processing?

• An N-gram is a method for translating languages.

• An N-gram is a random selection of words from a text.

• An N-gram is a contiguous sequence of 'n' items from a text.

• An N-gram is a type of machine learning algorithm.

How do language models utilize N-grams?

• N-grams are used to translate languages directly.

• Language models use N-grams to predict the next word based on the previous 'n' words.

• Language models ignore previous words when generating text.

• N-grams are only used for image recognition tasks.

What is the purpose of a corpus in language modeling?

• To create a list of grammar rules for language learning.

• To provide a source of historical texts for analysis.

• To serve as a dictionary for word definitions.

• The purpose of a corpus in language modeling is to provide a dataset for training models on language patterns and usage.

What is the difference between training and testing in language models?

• Testing is done before training to prepare the model.

• Training is for teaching the model, while testing is for evaluating its performance.

• Training and testing are the same processes in language models.

• Training is for evaluating performance, while testing is for teaching the model.

What are some common applications of language models?

• Video editing

• Image recognition

• Weather forecasting

• Common applications of language models include chatbots, content generation, language translation, sentiment analysis, and code generation.

Describe the role of tokenization in preparing text data.

• Tokenization is used to encrypt sensitive information in text.

• Tokenization combines multiple texts into a single unit.

• Tokenization eliminates punctuation and special characters from text.

• Tokenization breaks text into smaller units (tokens) for analysis in NLP.

What challenges do N-gram models face in language understanding?

• N-gram models excel at understanding context and meaning.

• N-gram models are immune to data sparsity issues.

• N-gram models effectively handle all types of language ambiguity.

• N-gram models struggle with context awareness, long-range dependencies, data sparsity, and polysemy.

What is the significance of stop words in text processing?

• Stop words are essential for understanding the meaning of a text.

• Stop words are the most important words in a sentence.

• Stop words are commonly used words that are often filtered out in text processing.

• Stop words are unique identifiers for each document in a corpus.

How does stemming differ from lemmatization in natural language processing?

• Stemming is more accurate than lemmatization.

• Lemmatization is faster than stemming.

• Stemming and lemmatization are identical processes in NLP.

• Stemming reduces words to their base or root form, while lemmatization considers the context and converts words to their dictionary form.

What is the purpose of part-of-speech tagging in NLP?

• Part-of-speech tagging is used to identify the sentiment of a text.

• Part-of-speech tagging is used to translate text into different languages.

• Part-of-speech tagging is irrelevant in modern NLP applications.

• Part-of-speech tagging assigns grammatical categories to words in a sentence, aiding in understanding their roles.