Week3-Day2

You are employed by a startup building self-driving cars. You are in charge of detecting road signs (stop sign, pedestrian crossing sign, construction ahead sign) and traffic signals (red and green lights) in images. The goal is to recognize which of these objects appear in each image. As an example, the above image contains a pedestrian crossing sign and red traffic lights

Your 100,000 labeled images are taken using the front-facing camera of your car. This is also the distribution of data you care most about doing well on. You think you might be able to get a much larger dataset off the internet, that could be helpful for training even if the distribution of internet data is not the same.

You are just getting started on this project. What is the first thing you do? Assume each of the steps below would take about an equal amount of time (a few days).

• Spend a few days checking what is human-level performance for these tasks so that you can get an accurate estimate of Bayes error.

• Spend a few days getting the internet data, so that you understand better what data is available.

• Spend a few days collecting more data using the front-facing camera of your car, to better understand how much data per unit time you can collect.

• Spend a few days training a basic model and see what mistakes it makes

Your goal is to detect which road signs (stop sign, pedestrian crossing sign, construction ahead sign) appear in images. You plan to use a deep neural network with ReLU units in the hidden layers. For the output layer, a softmax activation would be a good choice for the output layer because this is a multi-task learning problem. True/False?

• True

• False

You are carrying out error analysis and counting up what errors the algorithm makes. Which of these datasets do you think you should manually go through and carefully examine, one image at a time?

• 500 randomly chosen images

• 10,000 randomly chosen images

• 10,000 images on which the algorithm made a mistake

• 500 images on which the algorithm made a mistake

After working on the data for several weeks, your team ends up with the following data: 100,000 labeled images taken using the front-facing camera of your car. 900,000 labeled images of roads downloaded from the internet. which data distribution is useful for us?

• labeled images of roads downloaded from the internet

• images taken using the front-facing camera of your car

The distribution of data you care about contains images from your car’s front-facing camera; which comes from a different distribution than the images you were able to find and download off the internet. How should you split the dataset into train/dev/test sets?

• Mix all the 100,000 images with the 900,000 images you found online. Shuffle everything. Split the 1,000,000 images dataset into 600,000 for the training set, 200,000 for the dev set and 200,000 for the test set.

• Mix all the 100,000 images with the 900,000 images you found online. Shuffle everything. Split the 1,000,000 images dataset into 980,000 for the training set, 10,000 for the dev set and 10,000 for the test set.

• Choose the training set to be the 900,000 images from the internet along with 20,000 images from your car’s front-facing camera. The 80,000 remaining images will be split equally in dev and test sets.

• Choose the training set to be the 900,000 images from the internet along with 80,000 images from your car’s front-facing camera. The 20,000 remaining images will be split equally in dev and test sets.

You also know that human-level error on the road sign and traffic signals classification task is around 0.5%. Which of the following are True?

• Your algorithm overfits the dev set because the error of the dev and test sets are very close.

• You have a large avoidable-bias problem because your training error is quite a bit higher than the human-level error.

You also know that human-level error on the road sign and traffic signals classification task is around 0.5%. Which of the following are True?

• Your algorithm overfits the dev set because the error of the dev and test sets are very close.

• You have a large avoidable-bias problem because your training error is quite a bit higher than the human-level error.

You decide to focus on the dev set and check by hand what are the errors due to. Here is a table summarizing your discoveries. In this table, 4.1%, 8.0%, etc.are a fraction of the total dev set (not just examples your algorithm mislabeled). I.e. about 8.0/14.3 = 56% of your errors are due to foggy pictures. The results from this analysis implies that the team’s highest priority should be to bring more foggy pictures into the training set so as to address the 8.0% of errors in that category. True/False?

• False because data augmentation (synthesizing foggy images by clean/non-foggy images) is more efficient.

• False because this would depend on how easy it is to add this data and how much you think your team thinks it’ll help

You can buy a specially designed windshield wiper that help wipe off some of the raindrops on the front-facing camera. Based on the table from the previous question, which of the following statements do you agree with?

• 2.2% would be a reasonable estimate of the maximum amount this windshield wiper could improve performance.

• 2.2% would be a reasonable estimate of the minimum amount this windshield wiper could improve performance.