Training Data Exam

1. Your company built a TensorFlow neutral-network model with a large number of neurons and layers. The model fits well for the training data. However, when tested against new data, it performs poorly. What method can you employ to address this?

• Threading
• Serialization
• Dropout Methods
• Dimensionality Reduction

2. An external customer provides you with a daily dump of data from their database. The data flows into Google Cloud Storage GCS as comma-separated values (CSV) files. You want to analyze this data in Google BigQuery, but the data could have rows that are formatted incorrectly or corrupted. How should you build this pipeline?

• Use federated data sources, and check data in the SQL query.
• Enable BigQuery monitoring in Google Stackdriver and create an alert.
• Import the data into BigQuery using the gcloud CLI and set max_bad_records to 0.
• Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.

3. Your weather app queries a database every 15 minutes to get the current temperature. The frontend is powered by Google App Engine and server millions of users. How should you design the frontend to respond to a database failure?

• Issue a command to restart the database servers.
• Retry the query with exponential backoff, up to a cap of 15 minutes.
• Retry the query every second until it comes back online to minimize staleness of data.
• Reduce the query frequency to once every hour until the database comes back online.

4. You are building new real-time data warehouse for your company and will use Google BigQuery streaming inserts. There is no guarantee that data will only be sent in once but you do have a unique ID for each row of data and an event timestamp. You want to ensure that duplicates are not included while interactively querying data. Which query type should you use?

• Include ORDER BY DESK on timestamp column and LIMIT to 1.
• Use GROUP BY on the unique ID column and timestamp column and SUM on the values.
• Use the LAG window function with PARTITION by unique ID along with WHERE LAG IS NOT NULL.
• Use the ROW_NUMBER window function with PARTITION by unique ID along with WHERE row equals 1.

5. You are designing a basket abandonment system for an ecommerce company. The system will send a message to a user based on these rules: • No interaction by the user on the site for 1 hour • Has added more than $30 worth of products to the basket • Has not completed a transaction You use Google Cloud Dataflow to process the data and decide if a message should be sent. How should you design the pipeline?

• Use a fixed-time window with a duration of 60 minutes.
• Use a sliding time window with a duration of 60 minutes.
• Use a session window with a gap time duration of 60 minutes.
• Use a global window with a time based trigger with a delay of 60 minutes.

6. Your company is migrating their 30-node Apache Hadoop cluster to the cloud. They want to re-use Hadoop jobs they have already created and minimize the management of the cluster as much as possible. They also want to be able to persist data beyond the life of the cluster.

• Create a Google Cloud Dataflow job to process the data.
• Create a Google Cloud Dataproc cluster that uses persistent disks for HDFS.
• Create a Hadoop cluster on Google Compute Engine that uses persistent disks.
• Create a Cloud Dataproc cluster that uses the Google Cloud Storage connector.
• Create a Hadoop cluster on Google Compute Engine that uses Local SSD disks.

7. Your company's on-premises Apache Hadoop servers are approaching end-of-life, and IT has decided to migrate the cluster to Google Cloud Dataproc. A like-for-like migration of the cluster would require 50 TB of Google Persistent Disk per node. The CIO is concerned about the cost of using that much block storage. You want to minimize the storage cost of the migration.

• Put the data into Google Cloud Storage.
• Use preemptible virtual machines (VMs) for the Cloud Dataproc cluster.
• Tune the Cloud Dataproc cluster so that there is just enough disk for all data.
• Migrate some of the cold data into Google Cloud Storage, and keep only the hot data in Persistent Disk.

8. You are deploying 10,000 new Internet of Things devices to collect temperature data in your warehouses globally. You need to process, store and analyze these very large datasets in real time.

• Send the data to Google Cloud Datastore and then export to BigQuery.
• Send the data to Google Cloud Pub/Sub, stream Cloud Pub/Sub to Google Cloud Dataflow, and store the data in Google BigQuery.
• Send the data to Cloud Storage and then spin up an Apache Hadoop cluster as needed in Google Cloud Dataproc whenever analysis is required.
• Export logs in batch to Google Cloud Storage and then spin up a Google Cloud SQL instance, import the data from Cloud Storage, and run an analysis as needed.

9. You are working on a sensitive project involving private user data. You have set up a project on Google Cloud Platform to house your work internally. An external consultant is going to assist with coding a complex transformation in a Google Cloud Dataflow pipeline for your project.

• Grant the consultant the Viewer role on the project.
• Grant the consultant the Cloud Dataflow Developer role on the project.
• Create a service account and allow the consultant to log on with it.
• Create an anonymized sample of the data for the consultant to work with in a different project.

11. You work for an economic consulting firm that helps companies identify economic trends as they happen. As part of your analysis, you use Google BigQuery to correlate customer data with the average prices of the 100 most common goods sold, including bread, gasoline, milk, and others. The average prices of these goods are updated every 30 minutes. You want to make sure this data stays up to date so you can combine it with other data in BigQuery as cheaply as possible.

• Load the data every 30 minutes into a new partitioned table in BigQuery.
• Store and update the data in a regional Google Cloud Storage bucket and create a federated data source in BigQuery
• Store the data in Google Cloud Datastore. Use Google Cloud Dataflow to query BigQuery and combine the data programmatically with the data stored in Cloud Datastore
• Store the data in a file in a regional Google Cloud Storage bucket. Use Cloud Dataflow to query BigQuery and combine the data programmatically with the data stored in Google Cloud Storage.