WorksheetsSpark
Total questions: 9
Worksheet time: 5mins
We can have as many spark sessions as we want per application
Yes
No
Spark works best in which file format?
ORC
CSV
Parquet
Avro
Which one of the following is an action?
groupBy()
count()
where
join
Which of the following operation does NOT involve shuffling?
join
count
groupBy
Window Functions
A New Stage is created in a DAG when
A Narrow Transformation is triggered
A Wide Transformation is Triggered
Any transformation
Cannot be determined
A file size is 588 MB. Spark tries to divide this into how many partitions?
5
4
6
8
All 4 Lines
Line 2-4 Only
Line 1-3 Only
Line 1-2 Only
Give the following code, what is the first step that gets executed when df3.show() is called?
df1 = spark.read.csv("file.csv")
df2 = df1.groupBy("book_id").count()
df3 = df2.where("book_id==2")
df3.show()
It will start grouping the data by book_id df1.groupBy("book_id")
It will first fetch all the counts per book id using df1.groupBy("book_id").count()
It will first get the overall file count and then continue
It will first apply the filter df2.where("book_id==2") and then group by is executed
What is the disadvantage of Spark compared to MapReduce?
Spark is slower than MR
Spark is more resource intensive
Spark doesn't work well with all systems as well as MR does
Spark has no disadvantages at all
