Font size
WorksheetsUnit15
Total questions: 150
Worksheet time: 3hrs 30mins
What is the main goal of AI system monitoring?
To increase app downloads
To ensure accuracy and performance
To design the UI
To reduce developer workload
Which of the following is NOT a reason to monitor AI systems?
Ensuring model accuracy
Tracking costs
Designing interfaces
Detecting anomalies
What are the three pillars of telemetry in AI monitoring?
Logs, Traces, Metrics
Tables, Views, Reports
APIs, Servers, Clients
Files, Directories, Users
In AI systems, "logs" mainly record:
User passwords
System events and errors
Image data
Business profits
"Traces" help developers:
Translate logs
Track request flow through components
Delete unused files
Encrypt data
What do "metrics" measure in AI systems?
Quantitative performance data
Log file length
Network passwords
UI design speed
Which tool is often used for metric collection and visualization?
PowerPoint
Prometheus
Paint
Photoshop
What is predictive maintenance in AI APMs?
Fixing bugs after failure
Predicting and preventing failures early
Cleaning code manually
Restarting servers daily
Which feature of APM tools helps find the source of performance issues?
Root Cause Analysis
Cloud Backup
File Compression
UI Testing
Which of the following is an example of anomaly detection?
Predicting user location
Detecting abnormal CPU spikes
Sending emails
Creating dashboards
What is a major risk of monitoring too many unnecessary metrics?
Missing critical insights
Improved performance
Faster reporting
Reduced cost
Which mistake increases cloud storage costs unnecessarily?
Deleting old logs
Forgetting log storage policies
Using short retention
Compressing files
Why is analyzing logs important?
To identify trends and anomalies
To store more data
To speed up Wi‑Fi
To format the database
What type of AI monitoring tool learns from historical data?
Static monitoring tools
AI‑based APMs
Manual checklists
File explorers
In the Smart Factory case, predictive maintenance detected:
Cooling system error
Motor temperature rise before failure
Network login attempts
Hardware theft
Which performance metric applies to API usage?
Response time
Color scheme
Model accuracy
Design load
What is the purpose of AI system observability?
Real‑time insight into performance and health
Managing finances
Tracking office attendance
Data encryption
AI monitoring ensures the system remains accurate and reliable.
True
False
Logs are mainly used to design user interfaces.
True
False
Metrics provide numerical insights into AI performance.
True
False
Predictive maintenance only occurs after a system failure.
True
False
Anomaly detection can identify unusual resource usage.
True
False
What are the three pillars of telemetry?
Tables, Views, Reports
APIs, Servers, Clients
Files, Directories, Users
Why is AI monitoring important?
To ensure accuracy, detect anomalies, and manage resources efficiently
To delete logs regularly
To design interfaces faster
To replace developers with automation
What is the purpose of APM tools?
To monitor application performance and identify issues proactively
To compress media files
To manage payroll
To design graphic assets
Define "observability."
Real‑time insight into system health and performance
Periodic manual checks of servers
A visual dashboard theme
User ability to access logs
Mention one example of an AI monitoring tool.
Prometheus
Photoshop
WordPad
FTP client
The three pillars of telemetry are ________.
logs, traces, and metrics
tables, views, and reports
servers, clients, and APIs
files, directories, and users
Monitoring helps detect ________ and prevent failures.
anomalies
passwords
file sizes
UI colors
APM stands for ________.
Application Performance Monitoring
Advanced Performance Metrics
Automated Process Management
Application Policy Manual
Continuous monitoring ensures ________ over time.
model accuracy
higher network fees
shorter logs
less storage use
Predictive maintenance uses data to prevent ________.
system breakdowns
UI bugs only
file uploads
password resets
KPIs help organizations:
Design interfaces
Measure success objectively
Hire developers
Delete logs
What does KPI stand for?
Key Performance Indicator
Knowledge Prediction Index
Kernel Process Insight
Key Productivity Information
Which KPI type measures model accuracy?
Model quality
Business impact
Throughput
Adoption
"Coherence" measures:
User login rate
Logical consistency of model output
CPU speed
File size
Which KPI checks how fluent a model’s language output is?
Fluency
Coherence
Throughput
Latency
“Safety” in AI metrics means:
Preventing harmful or misleading outputs
Ensuring faster response time
Saving log files
Encrypting servers
“Uptime” measures:
The percentage of time the system is operational
The number of GPUs used
The training time
Idle time
What does “Model Latency” mean?
Time to generate a model response
Model training duration
Cost to deploy
Delay between updates
“Error Rate” refers to:
Percentage of failed system requests
API latency average
User satisfaction
Storage cost
“Throughput” measures:
Number of requests handled per second
Energy usage
GPU cost
Memory speed
What is “Adoption Rate” used to track?
Active users using a new AI system
Server uptime
Developer productivity
App downloads
Business value KPIs measure:
Financial impact and ROI
Log structure
Server logs
Training epochs
What KPI measures cost efficiency?
Cost Savings
Query Length
Fluency
Verbosity
What is an advantage of proprietary APM tools?
Vendor cloud integration and support
Free licenses
Low maintenance
No updates
Which open-source APM is listed in Unit 15?
Deepchecks
Photoshop
MS Access
Canva
Which KPI measures how often users send prompts to an AI?
Frequency of use
Adoption rate
Session length
Response time
Why are KPIs across model, system, and business levels necessary?
To make data-driven improvements
To slow updates
To increase ad views
To reduce logging
KPIs stand for Key Performance Indicators.
True
False
Model latency measures how long the model takes to process a request.
True
False
Uptime refers to the time the system is unavailable.
True
False
Business KPIs measure how AI affects organizational goals.
True
False
Proprietary APMs have no vendor support.
True
False
What does KPI mean?
Key Performance Indicator
Knowledge Performance Index
Kernel Performance Index
Key Productivity Insight
Name one model quality metric for generative AI.
Coherence
Fluency
Safety
Groundedness
Response time
What does “Error Rate” represent?
Percentage of failed system requests
Average API latency
User satisfaction
Storage cost
What is “Adoption Rate”?
Percentage of active users for a new AI system
GPU utilization
Number of prompts per session
Training epochs
Give one open-source APM example.
Deepchecks
WhyLogs
NannyML
Photoshop
Uptime measures the percentage of system availability.
True
False
Model Latency refers to response generation time.
True
False
Cost Savings KPIs show the financial benefit of AI.
True
False
Proprietary APMs provide cloud integration and vendor support.
True
False
Adoption KPIs measure how frequently users interact with AI systems.
True
False
What is the main goal of user activity monitoring (UAM)?
Detect performance issues and improve UX
Increase storage
Reduce developers
Change UI colors
What are “Audit Trails”?
Records of user actions for accountability
Chat transcripts
System cache logs
Error reports
Which metric shows user engagement frequency?
Login frequency
Server uptime
Model latency
GPU usage
A very short session length usually means:
Frustration or low value
Strong engagement
Excellent system
Long usage
“Feature usage” data helps:
Identify popular features
Increase ad views
Reduce database size
Adjust GPU
Which log records access attempts?
Access log
Error log
System log
Event log
What does an “Error log” show?
Failures during execution
Logins
UI changes
Internet usage
“System logs” mainly track:
CPU, memory, and server health
User clicks
Browser types
Survey data
What format stores logs in key-value pairs?
JSON
CSV
TXT
XML
A “403 Forbidden” entry in logs means:
Unauthorized access attempt
Successful login
File missing
User timeout
Which log format is standard in Linux systems?
Syslog
XML
DOCX
CSV
“ModelInferenceException – Invalid input shape” would appear in:
Error log
Access log
System log
Audit trail
A user staying logged for 2 hours instead of 10 minutes might indicate:
Anomaly
Security breach
Normal behavior
Deleted data
Detecting unusual log patterns can reveal:
Data drift or system errors
Ad performance
Graphic quality
Storage speed
Peak usage analysis helps:
Optimize server load times
Increase prices
Slow apps
Block users
During peak loads, AI systems often face:
Slower responses
Reduced energy use
Better speed
User reduction
Which tool can visualize usage heatmaps?
Grafana
Paint
Excel
PowerPoint
True / False: User Activity Monitoring helps improve system performance.
True
False
True / False: Audit trails are unrelated to security.
True
False
True / False: Short session lengths often indicate user frustration.
True
False
True / False: Error logs store successful transactions.
True
False
True / False: JSON is a structured log format.
True
False
Short Answer: What is the main purpose of User Activity Monitoring (UAM)?
To track user behavior, detect issues, and improve UX
To compress files for storage
To design user interfaces
To generate marketing emails
Short Answer: Define “Audit Trail.”
A record of all user activities for accountability
A tool for drawing charts
A cache of temporary files
A list of favorite features
Short Answer: What are the three common log types?
Access, Error, and System logs
Click, Scroll, and Zoom logs
Backup, Restore, and Archive logs
GPU, Camera, and Audio logs
Short Answer: Why analyze feature usage data?
To identify popular or underused features
To reduce server hardware wear
To comply with tax reporting
To encrypt all user data
Short Answer: Which tool is used to visualize log data?
Grafana or Kibana
Paint or Notepad
FTP or Telnet
Photoshop or Illustrator
Fill-in-the-Blank: Access logs record _____.
who used the system and when
system failures and exceptions
CPU, memory, and performance
backup schedules
Fill-in-the-Blank: Error logs capture _____.
system failures and exceptions
who used the system and when
user demographics
image resolution
Fill-in-the-Blank: System logs monitor _____.
CPU, memory, and performance
user frustration
ad impressions
browser themes
Fill-in-the-Blank: A very short session may suggest _____.
user frustration
successful onboarding
high storage speed
balanced workloads
Fill-in-the-Blank: Anomaly detection finds _____.
unusual activity patterns
file formats for export
color palettes for dashboards
default password lists
What is a database?
Structured collection of data
Image editor
Coding IDE
Presentation app
Why are databases used instead of spreadsheets?
To handle larger, structured data efficiently
For picture editing
For text formatting
For sound storage
What are relational databases based on?
Tables with relationships
XML files
Binary trees
Random files
Example of a relational database:
MySQL
MongoDB
Firebase
JSON
Example of a non-relational database:
MongoDB
SQL Server
Oracle
PostgreSQL
Why is backup important?
To recover data after failure
To increase server speed
To delete files
To reduce cost
Full backup copies:
Entire data set
Only changes
Only metadata
No files
Incremental backup copies:
Only changes since the last backup
Full system each time
Nothing
Cache data
Query optimization helps:
Improve database performance
Change UI layout
Increase memory
Decrease data
SQL stands for:
Structured Query Language
Standard Question Log
Sequential Queue Logic
Server Query Loader
Which clause filters records in SQL?
WHERE
ORDER BY
GROUP BY
SELECT
Database restoration means:
Recovering data from backup
Creating a new DB
Deleting old data
Compressing files
Indexing in databases helps:
Speed up queries
Delete duplicates
Compress images
Track users
Query optimization reduces:
Response time
Users
File size
RAM
Which database type uses key-value pairs?
NoSQL
SQL
Relational
Spreadsheet
The purpose of normalization is to:
Reduce redundancy
Increase duplication
Encrypt passwords
Delete schemas
A database stores structured information.
True
False
Backups are optional in production environments.
True
False
Query optimization improves data retrieval speed.
True
False
NoSQL databases use tables and foreign keys.
True
False
SQL stands for Structured Query Language.
True
False
What is the purpose of a database backup?
To prevent data loss and enable recovery
To increase server CPU usage
To remove outdated tables permanently
To compress media files for storage
Name two types of database backup.
Full and Incremental
Differential and Snapshot
Mirror and Log-only
Cold and Hot standby
What is query optimization?
Techniques to improve query performance
Designing the user interface for queries
Encrypting queries to secure data
Archiving old queries to save space
Define normalization.
Organizing data to reduce redundancy
Aggregating all records into a single table
Encrypting tables to protect data
Replicating tables across servers
Give one example of a relational database.
MySQL
MongoDB
Firebase
Spreadsheet
Choose the term that correctly completes the statement: ensures data recovery after system failure.
Backup
SQL
Normalization
Query optimization
Choose the term that correctly completes the statement: is used for managing relational databases.
SQL
Backup
Incremental backup
Regression testing
Choose the term that correctly completes the statement: reduces data duplication.
Normalization
Query optimization
Regression testing
Backup
Choose the term that correctly completes the statement: increases database efficiency.
Query optimization
Normalization
SQL
Incremental backup
Choose the term that correctly completes the statement: saves only changed data.
Incremental backup
Backup
Normalization
SQL
Why are system updates critical in AI applications?
To ensure continued performance and security
To reduce team size
To save log storage
To disable old features
What should always be done before applying updates?
Backup the system
Delete users
Stop logging
Reboot manually
What is regression testing?
Testing old functions after updates
Building dashboards
Installing drivers
Testing only APIs
Why is version control important?
Tracks changes to system code
Deletes logs
Prevents backups
Slows updates
Patch updates are:
Small fixes for bugs
Complete redesigns
System deletions
User reports
A rollback plan is used when
Updates fail
Updates succeed
Logs are missing
Costs rise
Which tool automates testing after updates
Jenkins
Photoshop
Access
Slack
Continuous integration (CI) means
Frequent merging and testing of updates
Manual deployments
Weekly coding
Random testing
System verification ensures
Updates meet intended goals
Logs deleted
Users removed
UI redesigned
Beta testing involves
Real users testing before public release
Admin-only testing
Internal rollback
No testing
What’s the risk of skipping testing
System bugs in production
Faster updates
Cheaper operation
Better feedback
The main purpose of update documentation is
Record changes for traceability
Promote products
Delete backups
Reduce logs
Critical security updates should be applied
Immediately
Yearly
After one month
Never
What is rollback verification
Confirming system stability after reverting
Updating code faster
Uploading images
Disabling backups
Continuous deployment differs from CI by
Automatically pushing updates to production
Deleting tests
Reducing code
Manual release
Hotfixes are
Emergency fixes for live issues
Optional UI changes
Hardware updates
Beta releases
System updates improve performance and security
True
False
Regression testing verifies old features after updates
True
False
Rollback plans are not necessary for updates
True
False
Continuous integration automates testing
True
False
Beta testing is done after public release
True
False
What is regression testing
Testing previous functions after system updates
Deploying updates directly to production
Creating new features only
Backing up system logs
Why is version control important
It tracks and manages code changes
It speeds up graphics rendering
It stores user passwords
It disables backups
