Data 251-257 + 309-311

You work for an airline and you need to store weather data in a BigQuery table. Weather data will be used as input to a machine learning model. The model only uses the last 30 days of weather data. You want to avoid storing unnecessary data and minimize costs. What should you do?

• Create a BigQuery table where each record has an ingestion timestamp. Run a scheduled query to delete all the rows with an ingestion timestamp older than 30 days.

• Create a BigQuery table partitioned by datetime value of the weather date. Set up partition expiration to 30 days.

• Create a BigQuery table partitioned by ingestion time. Set up partition expiration to 30 days.

• Create a BigQuery table with a datetime column for the day the weather data refers to. Run a scheduled query to delete rows with a datetime value older than 30 days.

You need to look at BigQuery data from a specific table multiple times a day. The underlying table you are querying is several petabytes in size, but you want to filter your data and provide simple aggregations to downstream users. You want to run queries faster and get up-to-date insights quicker. What should you do?

• Run a scheduled query to pull the necessary data at specific intervals dally.

• Use a cached query to accelerate time to results.

• Limit the query columns being pulled in the final result.

• Create a materialized view based off of the query being run.

Your chemical company needs to manually check documentation for customer order. You use a pull subscription in Pub/Sub so that sales agents get details from the order. You must ensure that you do not process orders twice with different sales agents and that you do not add more complexity to this workflow. What should you do?

• Use a Deduplicate PTransform in Dataflow before sending the messages to the sales agents.

• Create a transactional database that monitors the pending messages.

• Use Pub/Sub exactly-once delivery in your pull subscription.

• Create a new Pub/Sub push subscription to monitor the orders processed in the agent's system.

What should you do?

• Create a Cloud Storage bucket with Autoclass enabled.

• Create a Cloud Storage bucket with an Object Lifecycle Management policy to transition objects from Standard to Coldline storage class if an object age reaches 30 days.

• Create a Cloud Storage bucket with an Object Lifecycle Management policy to transition objects from Standard to Coldline storage class if an object is not live.

• Create two Cloud Storage buckets. Use the Standard storage class for the first bucket, and use the Coldline storage class for the second bucket. Migrate objects from the first bucket to the second bucket after 30 days.

What should you do?

• 
• 
• 
• 

You have an Oracle database deployed in a VM as part of a Virtual Private Cloud (VPC) network. You want to replicate and continuously synchronize 50 tables to BigQuery. You want to minimize the need to manage infrastructure. What should you do?

• Deploy Apache Kafka in the same VPC network, use Kafka Connect Oracle Change Data Capture (CDC), and Dataflow to stream the Kafka topic to BigQuery.

• Create a Pub/Sub subscription to write to BigQuery directly. Deploy the Debezium Oracle connector to capture changes in the Oracle database, and sink to the Pub/Sub topic.

• Deploy Apache Kafka in the same VPC network, use Kafka Connect Oracle change data capture (CDC), and the Kafka Connect Google BigQuery Sink Connector.

• Create a Datastream service from Oracle to BigQuery, use a private connectivity configuration to the same VPC network, and a connection profile to BigQuery.

What should you do to improve performance?

• Enable Vertical Autoscaling to let the pipeline use larger workers.

• Change the pipeline code, and introduce a Reshuffle step to prevent fusion.

• Update the job to increase the maximum number of workers.

• Use Dataflow Prime, and enable Right Fitting to increase the worker resources.

You are deploying a batch pipeline in Dataflow. This pipeline reads data from Cloud Storage, transforms the data, and then writes the data into BigQuery. The security team has enabled an organizational constraint in Google Cloud, requiring all Compute Engine instances to use only internal IP addresses and no external IP addresses. What should you do?

• Ensure that your workers have network tags to access Cloud Storage and BigQuery. Use Dataflow with only internal IP addresses.

• Ensure that the firewall rules allow access to Cloud Storage and BigQuery. Use Dataflow with only internal IPs.

• Create a VPC Service Controls perimeter that contains the VPC network and add Dataflow, Cloud Storage, and BigQuery as allowed services in the perimeter. Use Dataflow with only internal IP addresses.

• Ensure that Private Google Access is enabled in the subnetwork. Use Dataflow with only internal IP addresses.

What should you do?

• Create a normalized model with tables for each entity. Use snapshots before updates to track historical data.

• Create a normalized model with tables for each entity. Keep all input files in a Cloud Storage bucket to track historical data.

• Create a denormalized model with nested and repeated fields. Update the table and use snapshots to track historical data.

• Create a denormalized, append-only model with nested and repeated fields. Use the ingestion timestamp to track historical data.

You have important legal hold documents in a Cloud Storage bucket. You need to ensure that these documents are not deleted or modified. What should you do?

• Set a retention policy. Lock the retention policy.

• Set a retention policy. Set the default storage class to Archive for long-term digital preservation.

• Enable the Object Versioning feature. Add a lifecycle rule.

• Enable the Object Versioning feature. Create a copy in a bucket in a different region.