AI KHOÁ 3.2

True or False. Because of the knowledge cut-off, an LLM cannot answer questions about today’s news. But with RAG to supply it articles from the news, it would be able to.

• True

• False

You want to build an application to answer questions based on information found in your emails. Which of the following is the most appropriate technique?

• Fine-tuning an LLM on your emails, whereby we take a pre-trained LLM and further train it on your emails.

• Prompting (without RAG), where we iteratively refine the prompt until the LLM gets the answers right.

• Pretraining an LLM on your emails.

• RAG, where the LLM is provided additional context based on retrieving emails relevant to your question.

What does the idea of using an LLM as a reasoning engine refer to?

• This refers to the idea of using an LLM not as a source of information, but to process information (wherein we provide it the context it needs, through techniques like RAG).

• The idea of using an LLM to play games (like chess) that require complex reasoning, but having its output moves in the game.

• Reasoning engine is another term for RAG.

• This refers to pretraining an LLM on a lot of text so that it acquires general reasoning capabilities.

True or False. By making trusted sources of information available to an LLM via RAG, we can reduce the risk of hallucination.

• True, because RAG allows the LLM to reason through accurate information retrieved from a trusted source to arrive at the correct answer.

• False, because giving the LLM more information only confuses the LLM more and causes it to be more likely to hallucinate.

• False, because the LLM has learned from a lot of text from the internet (perhaps >100 billion words) to hallucinate, so adding one more short piece of text to the prompt as in RAG won’t make any meaningful difference.

• True, because the LLM is now restricted to outputting paragraphs of text exactly as written in the provided document, which we trust.

An ecommerce company is building a software application to route emails to the right department (Apparel, Electronics, Home Appliances, etc.) It wants to do so with a small, 1 billion parameter model, and needs high accuracy. Which of these is an appropriate technique?

• Fine-tune a 1 billion parameter model on around 1,000 examples of emails and the appropriate department.

• Pretrain a 1 billion parameter model on around 1 billion examples of emails and the appropriate department.

• Fine-tune a 1 billion parameter model on around 1 billion examples of emails and the appropriate department.

• Pretrain a 1 billion parameter model on around 1,000 examples of emails and the appropriate department.

Which of these job roles are unlikely to find any use for web UI LLMs?

• Marketer

• Recruiter

• Programmer

• None of the above

What is the relation between AI, tasks, and jobs?

• Jobs are comprised of many tasks. AI automates tasks, rather than jobs.

• Tasks are comprised of many jobs. AI automates tasks, rather than jobs.

• Jobs are comprised of many tasks. AI automates jobs, rather than tasks.

• Tasks are comprised of many jobs. AI automates jobs, rather than tasks.

Here are some of the tasks of a retail salesperson from O*NET. (We encourage you to check out the page yourself.) ẢNH

Say we decide to use AI to augment (rather than automate) a salesperson's task of recommending merchandise to customers. Which of the following would be an example of this?

• This has no business value and should not be done.

• Build an AI system to suggest products to the salesperson, who then decides what to recommend to the customer.

• Build a chatbot that automatically recommends products that customers can access directly, with no salesperson involved.

• Build an AI chatbot that can role-play being a customer to help the salesperson practice having conversations with customers.

When looking for augmentation or automation opportunities, what are the two primary criteria by which to evaluate tasks for generative AI potential? (Check the two that apply.)

• Business value (how valuable is it to automate?).

• Whether to use prompting, RAG or fine-tuning.

• Whether the task is the iconic, defining task for a job role.

• Technical feasibility (can AI do it?).

What is a quick way to start experimenting with an LLM application development project?

• Forming a large team with specialized roles.

• Try experimenting and prototyping with a web-based LLM to assess feasibility.

• Recruiting a large team of data engineers to organize your data.

• Hiring a dedicated prompt engineer.