NEW
Font size
WorksheetsPre-requitse Test Big Data, Hadoop and ML
Total questions: 55
Worksheet time: 28mins
What does the 'Variety' aspect of Big Data refer to?
The amount of data being generated
The speed at which data is produced
The types and formats of data
The correctness of data
Which of the following is used in Hadoop for distributed storage?
Hive
HDFS
YARN
Spark
Which technology enables resource management in a Hadoop cluster?
MapReduce
YARN
HDFS
Pig
What is Apache Spark primarily known for?
Real-time processing using batch jobs
Disk-based computation
Resource management
In-memory computation for fast analytics
What does MapReduce do in the Hadoop ecosystem?
Manages job execution
Provides data security
Splits and processes large data sets in parallel
Stores data
What is the purpose of Apache Zookeeper?
Coordinates and manages distributed applications
Stores massive unstructured data
In-memory computation
Provides SQL support
Why is traditional RDBMS not suitable for Big Data?
It lacks GUI
It cannot support SQL
It fails to handle large volume, variety, and velocity of data
It is open-source
A research lab is storing high-resolution satellite images, videos, and sensor data from different instruments. What Big Data characteristic does this scenario highlight?
Volume
Variety
Veracity
Viscosity
You are developing a healthcare monitoring system using wearable sensors that stream data continuously. Which Big Data technologies should you consider for processing this stream?
Spark Streaming and Kafka
Hive and Pig
HDFS and MapReduce
Cassandra and ZooKeeper
Which of the following is a NoSQL database suitable for handling unstructured data?
Oracle
Hive
Cassandra
MySQL
Which of the following best describes the reason for data locality in HDFS?
To reduce disk I/O latency
To improve CPU utilization
To increase network throughput
To reduce data transfer latency and improve performance
Which feature was introduced in HDFS Federation to overcome scalability issues?
Block-level striping
Multiple NameNodes with independent namespaces
Centralized metadata server
Heterogeneous replication
Why is the HDFS default block size larger than traditional file systems?
It supports better file security
It enables faster metadata lookup
It minimizes disk seek time and maximizes throughput
It reduces replication overhead
What happens when a DataNode fails to send heartbeat to the NameNode in time?
DataNode is upgraded to active mode
NameNode increases replication factor
DataNode is marked dead, and blocks are re-replicated
System restarts the failed DataNode automatically
Which MapReduce feature makes it highly fault tolerant?
Re-execution of failed tasks on other nodes
Heartbeat mechanism
Vertical scaling
Immediate job termination on failure
What would be the outcome if the replication factor is set to 1 in HDFS?
No data access will be allowed
Data will be more robust
There will be no tolerance to node failure
Performance will drastically increase
Which of the following is NOT a component of Hadoop 2.x's YARN architecture?
Node Manager
Application Master
Resource Manager
Task Tracker
In HDFS, the parameter dfs.blocksize is primarily tuned to:
Reduce network latency
Increase memory size of DataNode
Control the number of blocks a file is divided into
Manage CPU allocation per task
Which MapReduce phase is responsible for combining all values with the same key?
Map
Shuffle
Reduce
Partition
Why are many small files problematic in HDFS?
They increase disk seek time
They cause NameNode memory overhead
They slow down the replication
They reduce the number of map tasks
Which CAP property is most likely reduced when a system uses asynchronous replication?
Availability
Consistency
Partition tolerance
Throughput
Which of the following systems is an example of CP under CAP theorem?
DNS
MongoDB (default)
HBase
Cassandra
A stock trading platform must ensure that once a transaction is confirmed, all users see the same updated balance immediately, even if it delays responses during network issues. Which CAP property is being prioritized?
Availability
Consistency
Partition tolerance
Durability
How does CAP theorem impact the design of distributed systems?
It emphasizes data accuracy over system availability
It requires trade-offs between consistency, availability, and partition tolerance
It prioritizes system performance over data security
It eliminates the need for fault tolerance measures
If a distributed system must always be available, which property will it have to sacrifice during partitions?
Consistency
Partition tolerance
Durability
Reliability
If a distributed system chooses Consistency and Partition Tolerance (CP) according to the CAP theorem, what may it sacrifice?
Fault Tolerance
Availability
Scalability
Reliability
Which CAP theorem property ensures the system continues to operate even if some nodes cannot communicate?
Availability
Partition tolerance
Consistency
Fault tolerance
Why is it impossible to achieve all three CAP properties simultaneously in a distributed system?
Due to hardware limitations
Because of network latency
Because network partitions are inevitable in distributed systems
Because data replication is slow
HBase is primarily designed to run on top of:
MySQL
Hadoop Distributed File System (HDFS)
MongoDB
PostgreSQL
HBase prefers ________ over availability in the CAP theorem.
Availability
Consistency
Partition Tolerance
Durability
Zookeeper in HBase is used for:
Data Storage
Distributed Coordination
Caching
Query Execution
A company stores billions of sensor readings in HBase. Each reading is identified by a unique sensor ID and timestamp. The application frequently queries by sensor ID and requires strong consistency. Which HBase feature makes this possible?
Bloom Filters
Row Key
HFile Compression
MemStore
Which is an example of classification?
Predicting next day’s temperature in °C
Identifying whether a tumor is malignant or benign
Estimating sales revenue for next month
Predicting rainfall in millimeters
In regression tasks, the output is:
A discrete label
A probability distribution
A continuous numeric value
A similarity score
In K-means clustering, the elbow method is used for:
Choosing the number of clusters (k)
Reducing dimensionality
Selecting features
Measuring classification accuracy
Which similarity measure is not a proper distance metric but efficient for sparse vectors?
Euclidean distance
Manhattan distance
Cosine similarity
Jaccard distance
Which evaluation metric measures exactness of predictions?
Recall
Precision
Accuracy
F-measure
Which method divides data into k partitions and uses each once for validation?
Holdout method
Leave-one-out validation
K-fold cross-validation
Random subsampling
What technique helps to avoid overfitting in decision trees?
Increasing tree depth indefinitely
Using pre-pruning or post-pruning
Using noisy data intentionally
Removing validation set
An e-commerce company wants to recommend related items when a customer buys something (e.g., “Customers who bought X also bought Y”). Which ML technique should be used?
Classification
Regression
Association analysis
Clustering
A weather department wants to group weather patterns into categories like monsoon, snowy, dry, and humid without prior labels. Which technique is best?
Regression
Classification
Clustering
Precision-recall analysis
A company is using K-means clustering to segment customers. After plotting WSSE for different values of k, they notice the curve bends (forms an elbow) at k=4 . What does this suggest?
The best number of clusters is likely 4
They should always increase k for better results
Clustering is not possible with this data
WSSE does not apply to K-means
Which is generally considered the best predictive method for classification/regression?
Single Decision Tree
Bagging
Gradient Boosted Trees
Random Forest
Why do Random Forests lose interpretability compared to single trees?
They prune nodes
They contain hundreds of trees
They use PCA internally
They are only regression models
Cross-validation in Spark ML is used for:
Building deeper trees
Improving feature scaling
Selecting best model parameters
Reducing overfitting by pruning
Which of the following is true for Gradient Boosting?
Combines weak classifiers iteratively
Uses bagging of decision trees
Selects features randomly
Cannot be used for regression
Random Forest differs from Bagging by:
Selecting random features for each split
Using pruning
Building only shallow trees
Handling missing values
What is Bagging mainly used for?
Reducing bias
Reducing variance
Improving interpretability
Feature selection
What is the purpose of entropy in decision trees?
Measure the number of attributes
Measure dataset purity/impurity
Reduce dimensionality
Increase interpretability
In regression trees, what happens if no split is made?
Model predicts the median of targets
Model predicts the average of targets
Model predicts the mode of targets
Model predicts randomly
A bank wants to predict whether a loan applicant will default or not. They use a decision tree, but the model is overfitting badly. Which strategy should they apply?
Allow the tree to grow deeper
Use pruning or set maxDepth
Remove the validation set
Train on fewer features
If a page has fewer outgoing links, its PageRank contribution to each link is:
Higher
Lower
Zero
Independent of number of links
Why is MapReduce inefficient for PageRank?
It cannot store graphs
It redundantly shuffles graph structure each iteration
It cannot perform joins
It does not support parallelism
A flight network graph represents airports as vertices and flights as edges. A company wants to find the most influential airport (hub) in the network. Which algorithm in GraphX should they use?
K-means
PageRank
Logistic Regression
Linear Regression
Which algorithm can GraphX execute for graph analytics?
PageRank
Triangle Counting
Shortest Path
All of the above
