WorksheetsPySpark ML Workflow
Total questions: 10
Worksheet time: 7mins
In Pyspark, what does StringIndexer do?
Turns numbers into text
Turns text categories into number
Deletes text columns
Finds missing values
What does OneHotEncoder create?
A single number for each category
A vector of 0s and 1s for categories
A column with only null values
A summary of the data
In PySpark MLlib, which of the following algorithms can be used for clustering?
KMeans
Logistic Regression
Random Forest
Naive Bayes
What does df.describe.show() do?
Shows the schema
Removes missing values
Converts text to numbers
Shows summary statistics of columns
This is the only metric available in the BinaryClassificationEvaluator
Weighted Precision
Accuracy
F1-Score
ROC AUC
True or False: Applying standardization through StandardScaler or MinMaxScaler will immediately improve the model performance.
True
False
What is the function to combine many columns into one feature column?
VectorAssembler
StringIndexer
OneHotEncoder
VectorIndexer
True or false: fillna() can only be used on numeric columns, not text.
True
False
In PySpark MLib, what is the main purpose of a Pipeline?
To train multiple models at once
To automate feature engineering, model training, and evaluation steps in sequence
To parallelize Spark jobs across multiple clusters
To tune hyperparameters automatically
This is the only metric available in the ClusteringEvaluator
Inertia
Recall
Silhouette Score
Accuracy
