Transformers, the tech behind LLMs | Deep Learning Chapter 5

What do the initials GPT represent in the context of artificial intelligence models?

• General Purpose Technology

• Generative Pre-trained Transformer

• Global Processing Tool

• Graphical Programming Technique

In the context of Generative Pre-trained Transformers, what does the "Pre-trained" component primarily signify?

• The model is designed for specific, pre-defined tasks without further modification.

• The model has undergone initial learning from a vast dataset, allowing for subsequent fine-tuning on specialized tasks.

• The model's architecture is fixed and cannot be altered after its initial development.

• The model is trained exclusively on synthetic data generated prior to deployment.

How are input texts processed into discrete units within a Transformer model?

• They are converted directly into a single, continuous numerical stream.

• They are broken down into "tokens," which can represent words, sub-word units, or common character combinations.

• They are analyzed as complete sentences, with each sentence forming a single processing unit.

• They are transformed into visual representations before any numerical processing occurs.

What is the primary function of an "Attention block" in the Transformer architecture?

• To convert numerical vectors back into human-readable text.

• To allow different word vectors to interact and update their meanings based on contextual relationships.

• To perform parallel, independent computations on each word vector without intercommunication.

• To compress the input data into a smaller, more manageable format.

What is the fundamental difference between a machine learning approach and a traditional programming approach for tasks requiring intuition and pattern recognition?

• Machine learning explicitly defines every step of a procedure in code.

• Traditional programming uses tunable parameters to learn from data.

• Machine learning sets up a flexible structure with tunable parameters that are adjusted based on examples.

• Traditional programming relies on large datasets to determine model behavior.

How is input data typically processed within a deep learning model?

• Input data is directly converted into a single output value without intermediate steps.

• Input data is formatted as an array of real numbers and progressively transformed through multiple distinct layers, each structured as an array of real numbers.

• Input data is processed by explicitly defined procedural code for each task.

• Input data is converted into a single vector and then directly mapped to the output.

In deep learning models, what do "weights" represent and how do they interact with the input data?

• Weights are the specific input data fed into the model for a given run.

• Weights are fixed, non-tunable parameters that define the model's architecture.

• Weights are the tunable parameters that define the model's behavior, interacting with data primarily through weighted sums, often packaged as matrix-vector products.

• Weights are the final output probabilities generated by the model.

What is the primary function of an embedding matrix in a deep learning model processing text?

• It stores the final probability distribution for all possible next tokens.

• It defines the model's architecture and the number of layers.

• It maps each word from a predefined vocabulary to a unique vector of real numbers, which are learned based on data.

• It performs matrix-vector multiplication to combine input data with model parameters.

In deep learning, how are words typically represented, and what is a characteristic feature of their dimensionality in advanced models like GPT-3?

• As scalar values, typically in 3 dimensions.

• As one-hot encoded vectors, with dimensionality equal to the vocabulary size.

• As high-dimensional vectors, often exceeding 10,000 dimensions.

• As binary strings, with variable length depending on word complexity.

Which of the following best illustrates how vector arithmetic in word embedding spaces can capture semantic relationships?

• E(cat) + E(dog) = E(pet)

• E(king) - E(man) + E(woman) ≈ E(queen)

• E(red) * E(apple) = E(fruit)

• E(run) / E(fast) = E(sprint)