Week3Day1

Which of these techniques are useful for reducing variance (reducing overfitting)?

• Data augmentation

• Gradient Checking

Which of these techniques are useful for reducing variance (reducing overfitting)?

• Dropout

• Gradient Checking

Why do we normalize the inputs x?

• It makes the cost function faster to optimize

• It makes the parameter initialization faster

• It makes it easier to visualize the data

• Normalization is another word for regularization--It helps to reduce variance

This example is adapted from a real production application, but with details disguised to protect confidentiality.

You are a famous researcher in the City of Peacetopia. The people of Peacetopia have a common characteristic: they are afraid of birds. To save them, you have to build an algorithm that will detect any bird flying over Peacetopia and alert the population.

The City Council gives you a dataset of 10,000,000 images of the sky above Peacetopia, taken from the city’s security cameras. They are labelled:

y = 0: There is no bird on the image y = 1: There is a bird on the image Your goal is to build an algorithm able to classify new images taken by security cameras from Peacetopia.

There are a lot of decisions to make:. After further discussions, the city narrows down its criteria to:

"We need an algorithm that can let us know a bird is flying over Peacetopia as accurately as possible."

"We want the trained model to take no more than 10sec to classify a new image.”

“We want the model to fit in 10MB of memory.”

If you had the three following models, which one would you choose?

• Test Accuracy 97% ; Runtime 1 sec ; Memory size 3MB

• Test Accuracy 98% ; Runtime 9 sec ; Memory size 9MB

This example is adapted from a real production application, but with details disguised to protect confidentiality.

You are a famous researcher in the City of Peacetopia. The people of Peacetopia have a common characteristic: they are afraid of birds. To save them, you have to build an algorithm that will detect any bird flying over Peacetopia and alert the population.

The City Council gives you a dataset of 10,000,000 images of the sky above Peacetopia, taken from the city’s security cameras. They are labelled:

y = 0: There is no bird on the image y = 1: There is a bird on the image Your goal is to build an algorithm able to classify new images taken by security cameras from Peacetopia.

There are a lot of decisions to make:

What is the evaluation metric? How do you structure your data into train/dev/test sets? Metric of success The City Council tells you that they want an algorithm that

Has high accuracy Runs quickly and takes only a short time to classify a new image. Can fit in a small amount of memory, so that it can run in a small processor that the city will attach to many different security cameras. Note: Having three evaluation metrics makes it harder for you to quickly choose between two different algorithms, and will slow down the speed with which your team can iterate. True/False?

• True

• False

This example is adapted from a real production application, but with details disguised to protect confidentiality.

You are a famous researcher in the City of Peacetopia. The people of Peacetopia have a common characteristic: they are afraid of birds. To save them, you have to build an algorithm that will detect any bird flying over Peacetopia and alert the population.

The City Council gives you a dataset of 10,000,000 images of the sky above Peacetopia, taken from the city’s security cameras. They are labelled:

y = 0: There is no bird on the image y = 1: There is a bird on the image Your goal is to build an algorithm able to classify new images taken by security cameras from Peacetopia.

There are a lot of decisions to make: One member of the City Council knows a little about machine learning, and thinks you should add the 1,000,000 citizens’ data images to the test set. You object because:

• The 1,000,000 citizens’ data images do not have a consistent x-->y mapping as the rest of the data

• The test set no longer reflects the distribution of data (security cameras) you most care about.

This example is adapted from a real production application, but with details disguised to protect confidentiality.

You are a famous researcher in the City of Peacetopia. The people of Peacetopia have a common characteristic: they are afraid of birds. To save them, you have to build an algorithm that will detect any bird flying over Peacetopia and alert the population.

The City Council gives you a dataset of 10,000,000 images of the sky above Peacetopia, taken from the city’s security cameras. They are labelled:

y = 0: There is no bird on the image y = 1: There is a bird on the image Your goal is to build an algorithm able to classify new images taken by security cameras from Peacetopia.

There are a lot of decisions to make: After setting up your train/dev/test sets, the City Council comes across another 1,000,000 images, called the “citizens’ data”. Apparently the citizens of Peacetopia are so scared of birds that they volunteered to take pictures of the sky and label them, thus contributing these additional 1,000,000 images. These images are different from the distribution of images the City Council had originally given you, but you think it could help your algorithm. You should not add the citizens’ data to the training set, because this will cause the training and dev/test set distributions to become different, thus hurting dev and test set performance. True/False?

• True

• False

You train a system, and its errors are as follows (error = 100%-Accuracy):

Training set error 4.0%

Dev set error 4.5%

This suggests that one good avenue for improving performance is to train a bigger network so as to drive down the 4.0% training error. Do you agree?

• yes, because having 4.0% training error shows you have high bias.

• Yes, because this shows your bias is higher than your variance.

• No, because this shows your variance is higher than your bias.

• No, because there is insufficient information to tell.

This example is adapted from a real production application, but with details disguised to protect confidentiality.

You are a famous researcher in the City of Peacetopia. The people of Peacetopia have a common characteristic: they are afraid of birds. To save them, you have to build an algorithm that will detect any bird flying over Peacetopia and alert the population.

The City Council gives you a dataset of 10,000,000 images of the sky above Peacetopia, taken from the city’s security cameras. They are labelled:

y = 0: There is no bird on the image y = 1: There is a bird on the image Your goal is to build an algorithm able to classify new images taken by security cameras from Peacetopia.

There are a lot of decisions to make: You ask a few people to label the dataset so as to find out what is human-level performance. You find the following levels of accuracy:

Bird watching expert #1 0.3% error

Bird watching expert #2 0.5% error

Normal person #1 (not a bird watching expert) 1.0% error

Normal person #2 (not a bird watching expert) 1.2% error

If your goal is to have “human-level performance” be a proxy (or estimate) for Bayes error, how would you define “human-level performance”?

• 0.0% (because it is impossible to do better than this)

• 0.3% (accuracy of expert #1)

• 0.4% (average of 0.3 and 0.5)

• 0.75% (average of all four numbers above)

Which of the following statements do you agree with?

• A learning algorithm’s performance can be better than human-level performance but it can never be better than Bayes error.

• A learning algorithm’s performance can never be better than human-level performance but it can be better than Bayes error.

• A learning algorithm’s performance can never be better than human-level performance nor better than Bayes error.

• A learning algorithm’s performance can be better than human-level performance and better than Bayes error.