NLP Basics

What is tokenization in Natural Language Processing?

• Tokenization is the process of translating text into different languages.

• Tokenization refers to the process of summarizing text.

• Tokenization is the process of converting text to images.

• Tokenization is the process of breaking down text into smaller units called tokens.

Explain the concept of Part-of-Speech Tagging.

• Part-of-Speech Tagging is the process of assigning a color to each word in a text.

• Part-of-Speech Tagging is the process of assigning a part of speech to each word in a given text based on its definition and context.

• Part-of-Speech Tagging is the process of translating a text into a different language.

• Part-of-Speech Tagging is the process of counting the number of words in a text.

How is Sentiment Analysis used in NLP?

• Sentiment Analysis is used in NLP to analyze weather patterns.

• Sentiment Analysis is used in NLP to calculate mathematical equations.

• Sentiment Analysis is used in NLP to determine the emotional tone behind text data.

• Sentiment Analysis is used in NLP to identify colors in images.

Explain the concept of cosine similarity in NLP.

• Cosine similarity calculates the Euclidean distance between two documents.

• Cosine similarity is used to compare images based on pixel color values.

• Cosine similarity measures the difference in document length between two texts.

• Cosine similarity is a measure used to determine how similar two documents are based on their word frequencies. It calculates the cosine of the angle between two vectors in a multi-dimensional space.

What is TF-IDF and how is it used in NLP?

• TF-IDF is a type of neural network used in NLP for language translation

• TF-IDF is a numerical statistic used in NLP to determine the importance of a word in a document relative to a collection of documents by multiplying the term frequency (TF) of a word in a document by the inverse document frequency (IDF) of the word across the entire corpus.

• TF-IDF is a software tool used in NLP to generate random text

• TF-IDF is a programming language used in NLP to analyze text data

Which of the following tasks is NOT a typical NLP application?

• Speech recognition

• Image classification

• Machine translation

• Sentiment analysis

Which technique is used to reduce words to their base or root form in NLP?

• Stemming

• Tokenization

• Named Entity Recognition (NER)

• Part-of-Speech Tagging (POS)

Which Python library is commonly used for NLP tasks such as stemming and lemmatization?

• Matplotlib

• Scikit-learn

• NLTK

• Pandas

What is the primary difference between stemming and lemmatization?

• Stemming removes suffixes, while lemmatization considers the context and reduces words to their base or dictionary form

• Stemming is used for translation, while lemmatization is used for sentiment analysis

• Stemming is more accurate than lemmatization

• Lemmatization is faster but less accurate than stemming

What is the main difference between simple counting techniques (like Bag of Words) and TF-IDF in text analysis?

• Counting techniques consider the context of words, while TF-IDF does not.

• Counting techniques only count the frequency of terms in a document, while TF-IDF also considers the importance of terms across the entire corpus.

• TF-IDF is used for machine translation, while counting techniques are used for sentiment analysis.

• Counting techniques are more complex and computationally intensive compared to TF-IDF.