Apache Spark 3 for Data Engineering and Analytics with Python - Challenge Part 2 – Brief

What is the first task in the data cleaning challenge?

• Change data types of columns

• Remove bad records from the sales data frame

• Extract city and state information

• Add year and month columns

How can you extract the city from the purchase address?

• By extracting the first element

• By extracting the last element

• By extracting the third element

• By extracting the second element

What data type should the 'quantity ordered' column be changed to?

• String

• Boolean

• Float

• Integer

Which columns should be added to the data frame based on the order date?

• Day and hour

• Year and month

• Week and day

• Minute and second

What should be done with columns containing null values?

• Leave them as they are

• Replace them with zeros

• Remove them

• Fill them with average values

How should the final data frame be saved?

• As a Parquet file partitioned by year and month

• As a JSON file

• As an Excel file

• As a CSV file

What is a recommended approach if you encounter a Spark function not covered in the course?

• Search online and use forums like Stack Overflow

• Ignore the task

• Wait for the next lesson

• Ask a colleague for help