Data Engineer 288-297

What should you do?

• 
• 
• 
• 

What should you do?

• Setup a Kafka Connect bridge between Kafka and Pub/Sub. Use a Google-provided Dataflow template to read the data from Pub/Sub, and write the data to BigQuery.

• Use a proxy host in the VPC in Google Cloud connecting to Kafka. Write a Dataflow pipeline, read data from the proxy host, and write the data to BigQuery.

• Use Dataflow, write a pipeline that reads the data from Kafka, and writes the data to BigQuery.

• Setup a Kafka Connect bridge between Kafka and Pub/Sub. Write a Dataflow pipeline, read the data from Pub/Sub, and write the data to BigQuery.

You are designing the architecture to process your data from Cloud Storage to BigQuery by using Dataflow. The network team provided you with the Shared VPC network and subnetwork to be used by your pipelines. You need to enable the deployment of the pipeline on the Shared VPC network. What should you do?

• Assign the compute.networkUser role to the Dataflow service agent.

• Assign the compute.networkUser role to the service account that executes the Dataflow pipeline.

• Assign the dataflow.admin role to the Dataflow service agent.

• Assign the dataflow.admin role to the service account that executes the Dataflow pipeline.

What should you do?

• Create a new topic, and publish the last 30 days of data each time a new subscriber connects to an existing topic.

• Set the topic retention policy to 30 days

• Set the subscriber retention policy to 30 days.

• Ask the source system to re-push the data to Pub/Sub, and subscribe to it

Your organization is modernizing their IT services and migrating to Google Cloud. You need to organize the data that will be stored in Cloud Storage and BigQuery. You need to enable a data mesh approach to share the data between sales, product design, and marketing departments. What should you do?

• 
• 
• 
• 

What should you do?

• Create BigQuery connections to both Cloud SQL databases. Use BigQuery federated queries on the two databases and the Google Analytics data on BigQuery to run these queries.

• Create a job on Apache Spark with Dataproc Serverless to query both Cloud SQL databases and the Google Analytics data on BigQuery for these queries

• Create streams in Datastream to replicate the required tables from both Cloud SQL databases to BigQuery for these queries.

• Create a Dataproc cluster with Trino to establish connections to both Cloud SQL databases and BigQuery, to execute the queries

You designed a data warehouse in BigQuery to analyze sales data. You want a self-serving, lowmaintenance, and cost- effective solution to share the sales dataset to other business units in your organization. What should you do?

• Create an Analytics Hub private exchange, and publish the sales dataset.

• Enable the other business units’ projects to access the authorized views of the sales dataset.

• Create and share views with the users in the other business units.

• Use the BigQuery Data Transfer Service to create a schedule that copies the sales dataset to the other business units’ projects.

How should you configure the Pub/Sub subscription?

• Increase the acknowledgement deadline to 10 minutes.

• Use immediate redelivery as the subscription retry policy, and configure dead lettering to a different topic with maximum delivery attempts set to 10.

• Use exponential backoff as the subscription retry policy, and configure dead lettering to the same source topic with maximum delivery attempts set to 10.

• Use exponential backoff as the subscription retry policy, and configure dead lettering to a different topic with maximum delivery attempts set to 10.

What should you do?

• Use Cloud Data Fusion and Wrangler to normalize the data, and set up a recurring job.

• Use Dataflow SQL to create a job that normalizes the data, and that after the first run of the job, schedule the pipeline to execute recurrently.

• Create a Spark job and submit it to Dataproc Serverless.

• Use BigQuery and GoogleSQL to normalize the data, and schedule recurring queries in BigQuery.

How should you build the solution?

• Use BigLake to convert the current solution into a data lake architecture.

• Build a new data discovery tool on Google Kubernetes Engine that helps with new source onboarding and data lineage tracking.

• Use BigQuery to track data lineage, and use Dataprep to manage data and perform data quality validation.

• Use Dataplex to manage data, track data lineage, and perform data quality validation.