What is the primary purpose of tokenization in NLP?

• To convert text into numerical values for machine learning.

• The primary purpose of tokenization in NLP is to break text into smaller units for analysis.

• To summarize text into a single sentence.

• To translate text from one language to another.

Name one common stemming algorithm and describe its function.

• Porter Stemming Algorithm

• Support Vector Machine Algorithm

• Naive Bayes Classifier

• K-means Clustering Algorithm

How does lemmatization differ from stemming?

• Lemmatization and stemming are the same process.

• Lemmatization considers context and meaning, while stemming focuses on removing affixes.

• Stemming considers context and meaning, while lemmatization does not.

• Lemmatization only works with nouns, while stemming works with all parts of speech.

What is the formula for calculating the edit distance between two strings?

• The edit distance is calculated using a simple subtraction of string lengths.

• Edit distance can be determined by counting the number of common characters.

• The formula for edit distance is based on the Levenshtein algorithm without any matrix.

• The formula for calculating edit distance is based on dynamic programming, using a matrix to track the minimum operations needed.

Explain the role of finite state transducers in NLP.

• Finite state transducers are used in NLP for tasks like morphological analysis, tokenization, and part-of-speech tagging.

• Finite state transducers are primarily used for image processing.

• Finite state transducers are used to generate random text.

• Finite state transducers are only applicable in speech recognition.

What are the two main types of morphological analysis?

• Inflectional morphology and derivational morphology

• Semantic morphology and pragmatic morphology

• Lexical morphology and syntactic morphology

• Syllabic morphology and phonological morphology

Define finite automata and its significance in language processing.

• Finite automata can process any type of language without limitations.

• Finite automata are only used in hardware design.

• Finite automata are models of computation used to recognize patterns and process languages, essential in compiler design and text processing.

• Finite automata are primarily used for numerical calculations.

What is a regular expression and how is it used in text processing?

• A regular expression is a sequence of characters used to define search patterns in text processing.

• A regular expression is used only for data storage.

• A regular expression is a type of programming language.

• A regular expression is a graphical representation of text.

Describe the difference between derivational and inflectional morphology.

• Both derivational and inflectional morphology create new words but serve different functions.

• Derivational morphology is used only in verbs; inflectional morphology applies to nouns.

• Derivational morphology modifies existing words; inflectional morphology creates new words.

• Derivational morphology creates new words; inflectional morphology modifies existing words for grammatical purposes.

What is the Porter Stemmer and how does it work?

• The Porter Stemmer is an algorithm that reduces words to their root form by applying a series of rules to remove suffixes.

• The Porter Stemmer is a graphical tool for visualizing word structures.

• The Porter Stemmer is a type of machine learning model that predicts word meanings.

• The Porter Stemmer is a dictionary that lists all possible word forms.