WorksheetsBig Data 2 - Hadoop Ecosystem
Total questions: 75
Worksheet time: 38mins
Name
Class
Date
1.
Which pairing best reflects Hadoop's primary goal and the main challenge it explicitly targets?
a)
Low-latency OLTP and expensive enterprise storage
b)
Scalable, reliable storage and slow, failure-prone commodity hardware
c)
Interactive query serving and specialized high-end servers
d)
Transactional integrity and distributed ACID across nodes
e)
GPU-accelerated computation and NVMe-only storage
2.
Why does Hadoop emphasize scale-out rather than scale-up for data processing?
a)
To minimize replication overhead in shared-memory systems
b)
Because increasing single-node memory is cheaper than adding nodes
c)
To grow by adding many commodity machines for both storage and compute
d)
To enforce strict global locking across a large SMP server
e)
To support synchronous mirroring between two monolithic frames only
3.
Which statement best captures Hadoop's design philosophy as presented?
a)
High-cost specialized hardware with minimal fault tolerance
b)
Centralized data processing with limited parallelism
c)
Distributed storage and processing on commodity nodes with fault tolerance
d)
Single-node processing with vertical scaling only
e)
GPU-first compute model with optional replication
4.
Hadoop's design was inspired by which external architecture?
a)
Amazon Aurora storage engine internals
b)
Google's distributed data processing architecture
c)
PostgreSQL buffer manager design
d)
Berkeley DB log-structured storage
e)
OpenMP shared-memory runtime
5.
Which set correctly lists Hadoop's core components according to the slides?
a)
HDFS, MapReduce, Hadoop Common, YARN
b)
HDFS, Spark, Zookeeper, Kafka
c)
HBase, Hive, Pig, Oozie
d)
YARN, Mesos, Kubernetes
e)
HDFS, Tez, Flink, Presto
6.
What is the role of Hadoop Common within the ecosystem?
a)
A SQL layer to translate queries into MapReduce jobs
b)
A set of shared utilities supporting other Hadoop modules
c)
A distributed coordination service for leader election
d)
A columnar storage format for HDFS
e)
A stream processing engine for low-latency events
7.
In Hadoop, a 'node' typically participates in which roles by default?
a)
Compute only; storage is centralized
b)
Storage only; compute is centralized
c)
Both storage and compute within the cluster
d)
Neither storage nor compute; nodes act as routers
e)
Only metadata management
8.
Which scalability statement aligns with the slides' claim?
a)
Clusters are capped at dozens of nodes to reduce coordination overhead
b)
Hadoop can scale out to tens of thousands of nodes
c)
Hadoop must scale up a single powerful server for performance
d)
Scalability is achieved via shared-disk architectures
e)
Hadoop discourages adding new nodes after initial deployment
9.
How does HDFS handle a data node failure according to the slides?
a)
It pauses the cluster until the failed node returns
b)
It restores from centralized tape backups
c)
It re-replicates affected blocks from remaining replicas to other nodes
d)
It invalidates all blocks belonging to the failed node
e)
It promotes the failed node to a standby role
10.
MapReduce achieves fault tolerance primarily by:
a)
Maintaining a global transactional log across reducers
b)
Replicating entire job binaries across all nodes
c)
Decomposing jobs into independent tasks that can be rescheduled
d)
Using eager checkpointing after every mapper record
e)
Running all tasks on a single node to avoid network failures
11.
Which access pattern is emphasized by HDFS?
a)
Frequent random writes to arbitrary offsets
b)
Write-once, read-many with append-only semantics
c)
Row-level updates with strict transaction isolation
d)
In-place updates with versioned snapshots per record
e)
Byte-range locking for concurrent in-place edits
12.
HDFS is optimized for files of what size range?
a)
A few kilobytes to a few megabytes
b)
Tens of megabytes only
c)
Hundreds of megabytes to several terabytes
d)
Fixed-size blocks under 1 MB
e)
Sub-record fragments under 4 KB
13.
In HDFS, which component manages the namespace and file-to-block mapping?
a)
Secondary NameNode
b)
DataNode
c)
NameNode (master)
d)
YARN ResourceManager
e)
JobTracker
14.
Which is a direct responsibility of a DataNode?
a)
Maintaining the in-memory namespace tree
b)
Scheduling MapReduce tasks
c)
Performing I/O on local block replicas
d)
Coordinating leader election
e)
Translating SQL to MapReduce jobs
15.
Why does HDFS use large block (chunk) sizes (e.g., 64 MB) by default?
a)
To increase metadata size and reduce throughput
b)
To reduce metadata overhead and lower transfer costs per block
c)
To enforce small-file optimization
d)
To allow random in-place updates
e)
To eliminate the need for replication
16.
What is the typical replication factor per block in HDFS as presented?
a)
1
b)
2
c)
3
d)
4
e)
5
17.
According to the slides, how is the Secondary NameNode described?
a)
A hot-standby that instantly replaces the primary on failure
b)
A component that assists with checkpointing-related tasks at startup
c)
A distributed lock manager for block placement
d)
A transactional log replicator across data nodes
e)
A YARN application master
18.
Which statement about MapReduce in Hadoop is accurate?
a)
It is a programming language focusing on syntax sugar for SQL
b)
It is a processing model that runs via mapper and reducer tasks
c)
It is a scheduler to allocate cluster memory only
d)
It is a file format for storing columnar data
e)
It is limited to single-node execution
19.
Which trio summarizes MapReduce characteristics listed in the slides?
a)
Latency, consistency, partition tolerance
b)
Simplicity, flexibility, scalability
c)
Availability, isolation, durability
d)
Compression, encryption, indexing
e)
Batching, replication, compaction
20.
Why are MapReduce tasks designed to be isolated and independent?
a)
To maximize inter-task communication for synchronization
b)
To reduce coordination needs and enable scalable parallel execution
c)
To enforce strict global ordering of records
d)
To allow transactional updates across mappers
e)
To ensure a single reducer processes all data
21.
What is a side effect of minimizing inter-node communication in MapReduce?
a)
Higher contention on shared disks
b)
More complex mapper code generation
c)
Improved scalability due to less synchronization
d)
Loss of data locality benefits
e)
Mandatory speculative execution
22.
How does Hadoop typically execute Map tasks relative to data?
a)
Moves large input files to where mappers run
b)
Sends program logic to nodes where the data already resides
c)
Aggregates all data into a single node prior to mapping
d)
Runs mappers only on the master for consistency
e)
Executes reducers where the NameNode is located
23.
In the MapReduce programming model, inputs and outputs are viewed as:
a)
Relational tuples with schema constraints
b)
Byte streams without structure
c)
Key-value pairs at each stage
d)
Immutable graphs of operators
e)
Fixed-length records only
24.
Which mapping between components is correct?
a)
Mapper -> groups values by key; Reducer -> emits intermediate pairs
b)
Mapper -> emits intermediate pairs; Reducer -> aggregates by key
c)
Mapper -> performs final aggregation; Reducer -> partitions input
d)
Mapper -> sorts by key; Reducer -> splits input files
e)
Mapper -> assigns replication; Reducer -> manages metadata
25.
What determines the number of Map tasks in a job per the slides?
a)
Number of reducers configured
b)
Size of NameNode metadata
c)
Amount of input data, roughly aligned with input splits/chunks
d)
Cluster CPU core count
e)
Number of HBase regions
26.
During mapping, what can a mapper emit for each input record?
a)
Exactly one intermediate key-value pair
b)
Zero or more intermediate key-value pairs
c)
At most two intermediate key-value pairs
d)
Only the original record as-is
e)
Only fixed-length values
27.
What is the purpose of the shuffle and sort phase?
a)
To compress input files before mapping
b)
To randomly distribute keys across reducers
c)
To group and order mapper outputs into partitions for reducers
d)
To replicate intermediate outputs three times
e)
To checkpoint NameNode state
28.
What defines a reducer's input partition in Hadoop?
a)
A random sample of mapper outputs
b)
All records with keys assigned to that reducer's partition
c)
Only keys local to the reducer's node
d)
Only the largest keys by frequency
e)
Only mapper outputs from a single node
29.
Which invariant holds during the reduce phase?
a)
Each key's values may be split across multiple reducers
b)
A single reducer processes all keys
c)
All values for the same key are processed by exactly one reducer
d)
Reducers can emit new mappers during the same job
e)
Reducers modify HDFS metadata directly
30.
In the sales aggregation example, the reducer primarily performs:
a)
Sorting employees by name
b)
Counting unique employees
c)
Summing sales grouped by employee
d)
Joining employees to departments
e)
Computing median sale per order
31.
In the Word Count example, what is the mapper's typical output for each word?
a)
A key of (document, position) with value 0
b)
Key as the word and value 1
c)
Key as 1 and value as the word
d)
Key as the line number and value 1
e)
Key as the file name and value as the line
32.
What does the reducer compute in Word Count?
a)
Maximum word length
b)
Concatenation of all words
c)
Total occurrences per word
d)
Lexicographic order of words
e)
Average frequency per line
33.
In classic Hadoop MapReduce (v1), which components coordinate jobs and tasks?
a)
ResourceManager and NodeManager
b)
JobTracker and TaskTracker
c)
ApplicationMaster and Container
d)
Zookeeper and JournalNode
e)
NameNode and DataNode
34.
A key role of the JobTracker in Hadoop v1 is to:
a)
Maintain HDFS namespace
b)
Persist Zookeeper znodes
c)
Schedule and monitor tasks across TaskTrackers
d)
Serve as the SQL query planner
e)
Manage block placement in HDFS
35.
Which capability best characterizes Apache Pig per the slides?
a)
High-level data flow language (Pig Latin) compiled into MapReduce jobs
b)
A NoSQL column store layered on HDFS
c)
A cluster resource negotiator for memory and CPU
d)
A streaming platform for low-latency events
e)
A workflow scheduler for DAGs of jobs
36.
Pig is particularly strong at which class of operations?
a)
Distributed transactions with two-phase commit
b)
Complex joins and transformations
c)
Leader election among nodes
d)
Block placement policies
e)
Schema-on-write enforcement
37.
Hive provides which abstraction as described?
a)
Map-only tasks with no SQL
b)
A SQL-like language (HiveQL) translated to MapReduce jobs
c)
A persistent coordination service
d)
A streaming engine for time-series
e)
A graph processing framework
38.
Where does the Hive compiler execute according to the slides?
a)
On the NameNode
b)
On each DataNode
c)
On the client, submitting jobs to the cluster
d)
Within Zookeeper
e)
Inside the ResourceManager
39.
Which description of HBase aligns with the slides?
a)
A distributed row-store optimized for OLTP joins
b)
A distributed, scalable column-oriented database on HDFS
c)
An in-memory SQL engine with ACID transactions
d)
A message queue built on YARN
e)
A batch scheduler replacing MapReduce
40.
Which throughput characteristic is highlighted for HBase?
a)
Hundreds of thousands of inserts per second
b)
Single-digit transactions per second
c)
Only batch writes via MapReduce
d)
Writes are disabled; reads only
e)
Throughput depends on Hive compaction
41.
What is Apache Sqoop primarily used for?
a)
Scheduling DAG workflows
b)
Transferring bulk data between Hadoop and relational databases
c)
Coordinating leader election across services
d)
Column-family compactions in HBase
e)
Streaming event processing
42.
Which job type may Sqoop use to import tables into HDFS?
a)
Reducer-only jobs
b)
Map-only or MapReduce jobs
c)
YARN ApplicationMaster-only jobs
d)
Zookeeper sessions
e)
Spark-only jobs
43.
According to the slides, Kafka helps decouple which parties?
a)
NameNodes and DataNodes
b)
Producers and consumers of data streams
c)
Mappers and reducers
d)
Hive and Pig compilers
e)
Oozie and YARN
44.
What cluster component set is shown for Kafka?
a)
Reducers and combiners
b)
Brokers with Zookeeper tracking offsets
c)
NameNodes and JournalNodes
d)
ResourceManagers and NodeManagers
e)
TaskTrackers and JobTrackers
45.
Which graph abstraction does Oozie use for workflows?
a)
Undirected cyclic graphs
b)
Directed acyclic graphs (DAGs)
c)
Bipartite graphs
d)
Complete graphs
e)
Hypergraphs
46.
Which actions can Oozie orchestrate per the slides?
a)
Only Java MapReduce programs
b)
Pig and Hive scripts, MapReduce jobs, SSH, HDFS operations, email
c)
Only SQL queries
d)
Only Zookeeper leader elections
e)
Only Spark applications
47.
Which responsibility is attributed to Zookeeper in the slides?
a)
Block replication and placement
b)
High-reliability distributed coordination (e.g., leader election)
c)
SQL query planning and execution
d)
Streaming stateful operator management
e)
Client-side caching of HDFS reads
48.
Which is NOT listed as a Zookeeper function in the deck?
a)
Group membership management
b)
Leader election
c)
Dynamic configuration management
d)
Reducer-side aggregation
e)
System state monitoring
49.
Paxos appears in the slides primarily to:
a)
Detail HDFS replication math
b)
Serve as an example of a consensus algorithm related to coordination
c)
Explain MapReduce partitioning logic
d)
Describe HBase compaction scheduling
e)
Define Sqoop import formats
50.
YARN was introduced with which Hadoop generation?
a)
Hadoop 0.x
b)
Hadoop 1.0
c)
Hadoop 2.0
d)
Hadoop 3.5
e)
Not tied to Hadoop
51.
Which resource types does YARN explicitly manage per the slides?
a)
Disk and network only
b)
GPU and TPU only
c)
Memory and CPU cores
d)
NIC queues and file handles
e)
HDFS blocks and replicas
52.
What major capability does YARN add beyond classic MapReduce v1?
a)
Mandatory single reducer per job
b)
Exclusive support for streaming only
c)
Running both MapReduce and non-MapReduce applications on the same cluster
d)
Eliminating the need for replication
e)
Centralizing all compute on the NameNode
53.
Under YARN, which role replaces the v1 JobTracker for a MapReduce job?
a)
NameNode
b)
DataNode
c)
ApplicationMaster (application tracker)
d)
Zookeeper leader
e)
JournalNode
54.
Which mapping between Hadoop versions and roles is correct per the diagram?
a)
Hadoop 1.0: MapReduce handles both resource management and data processing; Hadoop 2.0: YARN handles resource management
b)
Hadoop 1.0: YARN handles resource management; Hadoop 2.0: MapReduce handles both
c)
Both versions: MapReduce is only a SQL engine
d)
Both versions: HDFS is replaced by Kafka
e)
Hadoop 2.0 removes MapReduce entirely
55.
Which best describes the relationship between HDFS and YARN in Hadoop 2.0?
a)
YARN embeds HDFS metadata
b)
They are independent layers: HDFS for storage; YARN for resource management
c)
YARN replaces HDFS replication
d)
HDFS schedules containers
e)
YARN provides the POSIX interface
56.
In YARN's allocation model, applications request containers primarily sized by:
a)
Block count and HDFS throughput
b)
Network bandwidth quotas only
c)
Memory and CPU needs declared to the scheduler
d)
SQL query complexity
e)
Number of HBase regions
57.
Which statement reflects the slides' view of the Hadoop ecosystem?
a)
Only HDFS and MapReduce are part of Hadoop; all others are unrelated closed-source tools
b)
Many open-source systems (e.g., Pig, Hive, HBase, Sqoop, Kafka, Oozie, Zookeeper) complement core Hadoop
c)
Ecosystem tools are all proprietary and require licenses
d)
Ecosystem tools replace HDFS entirely
e)
The ecosystem is limited to stream processing only
58.
The deck's 'platforms for big data management' slide implies what trend?
a)
Consolidation around a single vendor-only stack
b)
Multiple platforms co-exist to manage storage, compute, and orchestration needs
c)
Replacement of YARN by HDFS-level schedulers
d)
End of batch processing in favor of only streaming
e)
Mandatory GPU-based scheduling for all jobs
59.
What concurrency benefit arises from HDFS's append-only write model?
a)
It enables fine-grained byte-range locking
b)
It reduces contention costs by avoiding in-place updates
c)
It guarantees snapshot isolation for OLTP
d)
It allows atomic row updates across files
e)
It eliminates the need for checksums
60.
Increasing HDFS block size primarily affects metadata how?
a)
Increases the number of metadata entries
b)
Reduces the number of metadata entries per file
c)
Eliminates the need for the NameNode
d)
Forces replication factor to increase
e)
Requires smaller input splits
61.
Which factor determines which reducer processes a given key?
a)
A partitioner function applied to the key
b)
The reducer's available disk space
c)
The mapper's hostname
d)
The size of the value list
e)
The NameNode's placement policy
62.
Both Pig and Hive compilers run where, per the slides?
a)
On every DataNode as a daemon
b)
Inside the NameNode process
c)
On the client machine, submitting jobs to the cluster
d)
Within Zookeeper for coordination
e)
As YARN ApplicationMasters
63.
Which limitation is explicitly noted for HBase compared to traditional RDBMS?
a)
Lack of a SQL-like high-level query language
b)
No support for scanning by key range
c)
No horizontal scalability
d)
No write support
e)
No replication of regions
64.
Which naming best aligns with HDFS data units discussed in the slides?
a)
Pages; each 4 KB
b)
Records; variable length
c)
Chunks/blocks; large fixed size (e.g., 64 MB)
d)
Extents; size chosen per directory
e)
Sectors; 512 bytes
65.
Which file system characteristic of HDFS mirrors UNIX semantics?
a)
Hierarchical directory tree and user/group permissions
b)
Mandatory global locks per file
c)
Per-record transactional updates
d)
Case-insensitive file names
e)
Block device passthrough to applications
66.
Which HDFS write pattern is supported according to the slides?
a)
Overwriting bytes at arbitrary offsets
b)
Multiple concurrent writers editing the same region
c)
Single writer with append at the end of the file
d)
Transactional row updates inside a file
e)
In-place deletes with hole punching
67.
Which statement about MapReduce's status as a language is correct per the slides?
a)
MapReduce is a general-purpose language like Java
b)
MapReduce is a processing model proposed by Google, not a language
c)
MapReduce is a SQL dialect tailored to HDFS
d)
MapReduce is a scripting language for Pig
e)
MapReduce is an RPC library for NameNode
68.
During shuffle, mapper outputs are prepared for reducers by:
a)
Random sampling keys to balance reducers only
b)
Encrypting blocks to the NameNode
c)
Sorting and grouping keys into partitions
d)
Converting values to fixed width records
e)
Replacing keys with offsets
69.
Which benefit is directly tied to running code where the data resides?
a)
Elimination of replication
b)
Reduced network transfer, improving scalability
c)
Guaranteed sub-millisecond latency
d)
Removal of reducers
e)
Centralized metadata caching on clients
70.
What output format does a typical Sqoop import create in HDFS per the slides?
a)
A single monolithic binary file
b)
A directory of delimited text files (e.g., comma or tab separated)
c)
A set of ORC tables with indexes
d)
A Hive-managed table with ACID transactions
e)
A Kafka topic with Avro records
71.
Which Kafka metadata is mentioned as being tracked with Zookeeper?
a)
NameNode inode counts
b)
Reducer key ranges
c)
Consumer offsets
d)
YARN container priorities
e)
HBase region boundaries
72.
Which of the following is listed as an Oozie capability?
a)
Sending and receiving email as part of workflows
b)
Replacing NameNode in failover
c)
Acting as a MapReduce combiner
d)
Serving as a Kafka broker
e)
Providing SQL transactions for Hive
73.
Which statement best distinguishes NameNode from DataNode responsibilities?
a)
NameNode handles block I/O; DataNode stores metadata
b)
Both handle the same tasks for redundancy
c)
NameNode manages namespace/metadata; DataNode performs block I/O
d)
DataNode schedules reducers; NameNode runs mappers
e)
NameNode stores user data; DataNode stores only logs
74.
What does 'write-once, read-many' imply about application patterns on HDFS?
a)
Frequent in-place updates are preferred
b)
Sequential write followed by many parallel reads
c)
Random overwrite workloads are optimized
d)
Record-level locking is required
e)
Reads must be serialized after write completes
75.
Which statement about reducers' key ownership is correct?
a)
A given key may be split across reducers for load balancing
b)
All records for the same key are routed to the same reducer partition
c)
Reducers receive records based on mapper host locality only
d)
Reducers only process keys with a single associated value
e)
Key assignment is decided by the NameNode
100 %
