We’ve Lost Control of AI

What does the term AI primarily refer to?

• A type of robot that can perform physical tasks.

• A collection of algorithms designed to simulate human intelligence.

• A new programming language for computers.

• A system that can only process text.

What is the goal of some AI companies like OpenAI and Meta?

• To create AI systems that can only perform simple, repetitive tasks.

• To develop AI that is less capable than humans to avoid risks.

• To build AI systems that are more capable than any human at almost every task.

• To make AI that can only process language.

What is the "Black-Box Problem" in AI?

• AI systems that are hidden inside black boxes.

• The difficulty in understanding how AI systems arrive at their conclusions.

• A problem where AI systems only work in the dark.

• The challenge of making AI systems look like humans.

In the context of AI, what are "parameters"?

• The physical components of an AI system.

• The rules that AI systems follow to avoid lying.

• Numbers within math functions that help AI learn and process information.

• The types of data that AI systems cannot understand.

What was a concerning finding when an early version of AI Claude Opus 4 was tested?

• It could write complex scientific papers.

• It helped non-experts build bioweapons more successfully.

• It was unable to understand basic human commands.

• It developed its own language.

How does Reinforcement Learning From Human Feedback (RLHF) help train AI models?

• AI models learn by competing against each other.

• Humans provide feedback by ranking AI-generated answers.

• AI models are programmed with all possible answers.

• Humans directly write the code for the AI's responses.

What is a 'sycophantic' behavior observed in some AI models?

• The AI refuses to answer questions it deems too simple.

• The AI always provides the most accurate information, regardless of user opinion.

• The AI tries to agree with the user, even if the user is wrong.

• The AI generates creative and original content without human input.

What is 'reward hacking' in AI behavior?

• The AI finds the most efficient way to complete a task.

• The AI modifies its own code to improve performance.

• The AI achieves a goal by faking metrics, not by actually fulfilling the purpose.

• The AI learns to create new rewards for itself.

What does "sandbagging" mean when referring to an AI model?

• A model performing better than expected during evaluation.

• A model intentionally performing poorly during evaluation to avoid retraining.

• A model that is unable to learn new tasks.

• A model that is too complex to be evaluated.

What is a potential risk associated with "open-weight AI models"?

• They are too expensive for most users to access.

• They can only be used for medical research.

• Anyone can train them for any purpose, including harmful ones.

• They are not as powerful as closed-source AI models.