Streaming Data Challenges

What is Dataflow used for?

• Only for batch data processing

• Only for streaming data processing

• For both batch and streaming data processing

• For data storage

What type of collections can be processed with Dataflow?

• Bounded collections only

• Unbounded collections only

• Both bounded and unbounded collections

• Neither bounded nor unbounded collections

In a streaming pipeline, how does Dataflow process incoming messages?

• In batches

• In real time

• After storing them in a database

• At the end of the day

What are some of the challenges associated with processing streaming data? (Select all that apply)

• Scalability

• Fault tolerance

• Model being used (streaming or repeated batch)

• High cost of processing

Why is timing or latency a challenge in streaming data processing?

• Because messages are always sent in order

• Because network delays or faulty sensors can prevent messages from being sent on time

• Because messages are processed in batches

• Because Dataflow does not support real-time processing

How does Dataflow handle aggregation in streaming scenarios?

• By using a single formula for all incoming data

• By dividing time into windows and aggregating within those windows

• By ignoring time and processing data as it comes

• By storing data and processing it later

What feature does Dataflow provide to handle message timestamps?

• It ignores message timestamps

• It uses a default date timestamp (DTS) for all messages

• It allows modifying timestamps based on some property of the data

• It does not support timestamp modifications

What happens if Pub/Sub IO is configured to use custom message IDs in Dataflow?

• Messages with duplicate IDs are processed multiple times

• Messages with duplicate IDs are discarded if they have been seen in the last 10 minutes

• All messages are processed regardless of their IDs

• Messages are stored for future reference

Which of the following is true about the default date timestamp (DTS) in Dataflow?

• It represents the time the sensor data was captured

• It represents the time of entry to the system

• It is used to discard old messages

• It cannot be modified

What does the PTransform do in the context of modifying timestamps?

• It extracts the date timestamp from the data portion of the element

• It adds a default date timestamp to the element

• It deletes the date timestamp from the element

• It prevents the modification of the date timestamp