Text Mining with Python (week 9)

Natural language processing (NLP) is a subfield of ... and ... that uses machine learning to enable computers to understand and communicate with human language.

• corpus linguistics

• computer science

• statistics

• artificial intelligence (AI)

What is the correct order of preprocessing the text for further mining?

• tokenization, lemmatization, and removing stop words

• lemmatization, tokenization, and removing stop words

• lemmatization, removing stop words, and tokenization

• removing stop words, lemmatization, and tokenization

What does this piece of code do?

def most_common(df, top_n=10):
all_words = [word for word in ' '.join(df['processed_comments']).split() if word not in string.punctuation]
word_counts = Counter(all_words)
top_10_words = word_counts.most_common(top_n)
return top_10_words

• identifies the most recurrent themes or issues

• tokenizes the textual data

• categorizes comments

• scrapes the html files to csv

What Python libraries are used for text mining?

• NLTK

• Pandas

• BeutifulSoup

• TextBlob

What can NLTK (natural language toolkit) be used for?

• stemming and lemmatization

• tokenization

• sentiment analysis

• speech recognition

What is Pandas library used for?

• working with MS Office files

• building neural networks

• data manipulation and analysis

• speech recognition

The goal of sentiment analysis is to determine the overall sentiment polarity of a piece of text, which can be ...

• positive, negative, or neutral

• positive, negative, emotional, or neutral

• sentimental, rational

• poetical, prosaic, academic, legal, etc.