Lecture #4.1 - Spark Structured Streaming API II

What is the difference between Event Time and Processing Time in Structured Streaming?

• Event Time is only used for windowing operations, while Processing Time is used for join operations.

• Event Time and Processing Time are the same in Structured Streaming.

• Event Time is the time when the event is processed, while Processing Time is the time when the event occurred.

• Event Time is the time when the event occurred, while Processing Time is the time when the event is processed.

Which type of windows operation in Structured Streaming allows defining windows by a specific time duration?

• Time based windows

• Bucketized windows

• Sliding windows

• Tumbling windows

What is the purpose of watermarking in Structured Streaming?

• To perform join operations between streaming DataFrames

• To define the time duration for windowing operations

• To wait for late arriving events for a limited time

• To remove duplicate events from the stream

Which type of join operation is challenging in Spark Structured Streaming due to the combination of late events from both DataFrames?

• Right Outer Join

• Semi Join

• Outer Join

• Inner Join

What is the purpose of the function dropDuplicates in Structured Streaming?

• To define the time duration for windowing operations

• To perform join operations between streaming DataFrames

• To remove repeated data and enable exactly-once processing

• To enable watermarking for late arriving events

Which type of windowing technique in Structured Streaming is used to define windows by a fixed time duration?

• Time based windows

• Tumbling windows

• Sliding windows

• Bucketized windows

What is the function used to create a new column for Processing Time in Structured Streaming?

• create_timestamp

• processing_timestamp

• event_timestamp

• current_timestamp

What is the purpose of data deduplication in Structured Streaming?

• To remove late arriving events from the stream

• To remove repeated data and enable exactly-once processing

• To define the time duration for windowing operations

• To perform join operations between streaming DataFrames

Which type of join operation is supported in Spark Structured Streaming for joining a Streaming DataFrame with a Static one?

• Right Outer Join

• Inner Join

• Outer Join

• Semi Join

What is the purpose of the function window in Structured Streaming?

• To bucketize rows into one or more windows based on window definition

• To create a new column for Event Time

• To remove repeated data and enable exactly-once processing

• To define the time duration for windowing operations