Understanding Apache Spark RDDs

What is an RDD in Apache Spark?

• An RDD in Apache Spark is a Resilient Distributed Dataset, which is an immutable distributed collection of objects.

• A Random Data Distribution in Apache

• A Readable Data Document in Spark

• A Resilient Data Distribution in Spark

What is the purpose of RDD transformations?

• To visualize the data contained in RDDs.

• To optimize the performance of RDDs without creating new ones.

• The purpose of RDD transformations is to create new RDDs by applying functions to existing RDDs.

• To delete existing RDDs from memory.

Name three common RDD transformations.

• map, filter, reduceByKey

• distinct

• join

• groupBy

What is the difference between map and flatMap transformations?

• map can return a collection of different sizes, while flatMap returns the same size

• The difference is that 'map' returns a collection of the same size, while 'flatMap' flattens the results into a single collection, potentially of different size.

• flatMap applies a function to each element and returns a collection of the same size

• map flattens the results into a single collection, while flatMap returns a collection of the same size

How do you create an RDD from a collection in Spark?

• Use sc.parallelize(collection) to create an RDD from a collection.

• Use sc.load(collection) to create an RDD from a collection.

• Call RDD.fromCollection(collection) to create an RDD.

• Use sc.createRDD(collection) to create an RDD from a collection.

What method is used to create an RDD from a text file?

• textFile

• loadText

• readFile

• importFile

What are RDD actions?

• RDD actions are only used for data transformation.

• RDD actions are operations that do not return any value.

• RDD actions are operations that return a value or write data to storage.

• RDD actions are exclusively for filtering data.

Give two examples of RDD actions.

• reduce()

• map()

• count(), collect()

• filter()

What is the difference between count() and collect() actions?

• count() is an action that modifies the dataset, while collect() does not.

• count() provides a numeric result, whereas collect() provides the full dataset.

• count() returns the full dataset, while collect() provides a numeric result.

• collect() is used for filtering data, whereas count() aggregates it.

How do RDDs differ from DataFrames in Spark?

• RDDs require a schema to be defined before use.

• RDDs are always faster than DataFrames.

• DataFrames can only handle structured data, while RDDs can handle any type of data.

• RDDs are low-level, unstructured data collections, while DataFrames are high-level, structured data collections with schema support.