Deep Learning - Crash Course 2023 - Various Activation Functions

The video tutorial covers various activation functions in TensorFlow, including sigmoid, hyperbolic tangent, ReLU, ELU, CELU, Swish, and Softmax. It explains their mathematical foundations, use cases, and implementation in neural networks. The tutorial also highlights the advantages of each function and provides practical examples of their application in deep learning models.

What is the primary purpose of using TF.linspace in the setup?

• To import necessary libraries

• To generate random data points

• To perform linear regression

• To visualize data

Which activation function scales the output to the range of 0 to 1?

• Softmax

• ReLU

• Sigmoid

• Tanh

How does the hyperbolic tangent function differ from the sigmoid function?

• It does not require bias terms

• It scales output to the range of -1 to 1

• It is used only in convolutional networks

• It is faster than sigmoid

What is the main advantage of using the ReLU activation function?

• It scales output to the range of -1 to 1

• It is computationally complex

• It is only used in recurrent networks

• It provides better accuracy and speed

What happens to the output of ReLU when the input is less than zero?

• It returns a constant value

• It returns zero

• It returns the input value

• It returns a negative value

Which activation function is known for preventing neuron dying?

• ReLU

• ELU

• Sigmoid

• Softmax

What is the formula for the Swish activation function?

• X multiplied by sigmoid of X

• X plus sigmoid of X

• X divided by sigmoid of X

• X minus sigmoid of X

In which scenario is the softmax activation function typically used?

• Multiclass classification

• Clustering tasks

• Binary classification

• Regression tasks

What does the softmax function output represent?

• A binary decision

• A vector of real numbers

• A probability distribution

• A single probability value

What is the sum of the probabilities in the output of a softmax function?

• 0

• Infinity

• 1

• Depends on the input