WorksheetsBig Data 3 - HDFS
Total questions: 88
Worksheet time: 44mins
Name
Class
Date
1.
In HDFS write once read many semantics, which operation is valid for modifying a file after creation
a)
Appending at the end of the file
b)
Overwriting bytes in the middle of the file
c)
Deleting a random byte range and shifting contents
d)
In place update of any block
e)
Byte level patching via random writes
2.
HDFS is designed for large files on commodity hardware. Which architectural requirement follows from this assumption
a)
Software level fault tolerance via replication of blocks
b)
Exclusive reliance on high end RAID controllers
c)
Synchronous mirroring across data centers for every write
d)
Mandatory random write support inside blocks
e)
Single node storage to reduce coordination
3.
Why does append only I O reduce synchronization in HDFS
a)
Writers avoid coordination on in place updates so fewer locks are needed
b)
It allows every client to lock the whole file for longer
c)
It forces smaller blocks which reduces contention
d)
It disables replication during writes which removes locks
e)
It moves all locking to the operating system page cache
4.
HDFS splits files into large blocks primarily to achieve which benefit
a)
Reduce metadata size and limit network communication per file
b)
Increase block scheduling frequency for fairness
c)
Enable fine grained random writes within a block
d)
Ensure each block fits in CPU cache
e)
Avoid the need for replication entirely
5.
Which statement about fault tolerance in HDFS is most accurate
a)
A failed DataNode triggers re replication of under replicated blocks
b)
Namenode automatically reconstructs missing bytes from parity only
c)
Replication happens only when files are closed and never after
d)
Clients are responsible for repairing missing replicas
e)
Only rack failures are handled not node failures
6.
Which role maintains the filesystem namespace and block mapping in HDFS
a)
Namenode
b)
Secondary Namenode
c)
JournalNode
d)
Datanode
e)
ResourceManager
7.
How are blocks stored by Datanodes according to HDFS architecture
a)
As local files in the node native file system
b)
As in memory byte arrays only
c)
Inside a relational database
d)
Inside a distributed object store external to the cluster
e)
As raw devices without a file system
8.
Which action is the Namenode responsible for
a)
Monitoring Datanodes via heartbeats and block reports
b)
Serving file data bytes directly to clients
c)
Compressing data blocks on the fly
d)
Computing CRC for every client read
e)
Scheduling MapReduce tasks
9.
When a client opens a file the mapping from file to blocks and blocks to Datanodes is provided by
a)
Namenode namespace and block mapping metadata
b)
Secondary Namenode checkpoint directory
c)
Datanode local metadata only
d)
Client cached list from previous job
e)
Balancer process
10.
Which activity is part of the Namenode replication engine
a)
Choosing target Datanodes for new replicas
b)
Streaming user data to clients during reads
c)
Performing checksum validation on client behalf
d)
Rewriting block contents to change compression
e)
Maintaining time travel snapshots of files
11.
Cluster configuration management in HDFS belongs to which component
a)
Namenode
b)
Datanode
c)
ApplicationMaster
d)
Balancing CLI only
e)
Hive Metastore
12.
Where does the Namenode hold the active metadata structures during operation
a)
In main memory without demand paging
b)
On a local relational database with demand paging
c)
On each Datanode distributed evenly
d)
In a key value store external to Hadoop
e)
In client memory caches only
13.
Which is a direct example of Namenode metadata
a)
List of Datanodes for each block of a file
b)
Data bytes for a block
c)
User code that consumed the file
d)
CPU metrics for each Datanode core
e)
Network packet captures for client reads
14.
What is the role of the transaction log on the Namenode
a)
Record namespace operations such as creates and deletes
b)
Store raw data blocks for recovery
c)
Persist only user quotas
d)
Maintain MapReduce job history
e)
Enforce rack placement rules on the Datanodes
15.
A Datanode acts primarily as which service
a)
Block server storing blocks and their checksums
b)
Namespace manager storing file paths
c)
Distributed scheduler for compute tasks
d)
Front end proxy for all HDFS clients
e)
Checkpointing coordinator
16.
How often do Datanodes send heartbeats to the Namenode by default per the slides
a)
About every 3 seconds
b)
About every 30 milliseconds
c)
Once per minute
d)
Only on block report submission
e)
Only when a client connects
17.
Which item is included in a Datanode block report
a)
The list of all blocks stored on that node
b)
The CPU temperature history
c)
Only a sample of blocks accessed recently
d)
A copy of the FsImage
e)
Detailed file paths for each block
18.
During pipelined writes what does a Datanode do besides persisting data
a)
Forwards received bytes to the next Datanode in the pipeline
b)
Rewrites block size to match local disk geometry
c)
Merges small files into a SequenceFile automatically
d)
Updates the client application checkpoint file
e)
Repartitions the block into columns for analytics
19.
What is the default three replica placement strategy described
a)
Local rack awareness with one local replica one on a remote rack and a third on the same remote rack
b)
All three replicas on the local node for fast reads
c)
Two replicas on the same node and one on a remote node in the same rack
d)
All replicas distributed evenly across all racks by hash only
e)
One replica per data center in three regions
20.
Which statement best describes client read behavior with replicas
a)
Clients read from the nearest replica when possible
b)
Clients always read from the primary replica only
c)
Clients round robin across replicas per packet
d)
Clients must read from all replicas to verify consistency
e)
Clients read from the most remote rack to reduce contention
21.
When a Datanode fails the Namenode first
a)
Detects the failure via missed heartbeats and schedules new replicas
b)
Deletes all metadata for blocks on that node permanently
c)
Rolls back the FsImage to the last checkpoint
d)
Halts the cluster to avoid split brain
e)
Switches the file format to Parquet
22.
Why does the Namenode balance communication traffic to Datanodes
a)
To avoid hotspots when many clients read popular blocks
b)
To guarantee perfect fairness across users
c)
To reduce checksum overhead on clients
d)
To minimize the number of racks used by a file
e)
To allow random writes to a subset of replicas
23.
What is the primary goal of the HDFS rebalancer tool
a)
Keep percent disk full across Datanodes similar
b)
Rebuild the FsImage from edits automatically
c)
Recreate checksums for all blocks
d)
Change replication factor for every file to three
e)
Compact small files into ORC
24.
Which is true about running the rebalancer
a)
It runs while the cluster is online and is throttled to limit network load
b)
It requires stopping all client reads
c)
It only works when no new Datanodes were added
d)
It moves the Namenode process to a less busy host
e)
It deletes under replicated blocks before moving data
25.
Which checksum mechanism is stated for HDFS data validation
a)
CRC32 over fixed byte ranges
b)
MD5 over the whole file only
c)
SHA256 over every packet
d)
Parity bits on each sector only
e)
Adler32 on Datanode only
26.
Who computes checksums when creating a file and at what granularity per the slides
a)
Client computes a checksum per 512 bytes during creation
b)
Datanode computes one checksum per block during creation
c)
Namenode computes a checksum per file during creation
d)
Balancer computes checksums after rebalancing
e)
Client computes a checksum per 8 kilobytes only on close
27.
If checksum validation fails during a read the client should
a)
Retry reading from another replica of the block
b)
Overwrite the corrupted bytes with zeros
c)
Ask the Namenode to recompute the block contents
d)
Abort the job and mark the file deleted
e)
Switch to reading from the furthest rack to avoid cache effects
28.
Which ordering best describes a client write of a block with replication
a)
Client sends to first Datanode which forwards to next then to last until all replicas ack
b)
Client sends to all Datanodes in parallel then commits locally
c)
First Datanode writes and commits then Namenode pushes data to others
d)
Client streams to Namenode which mirrors to Datanodes
e)
Client writes to local disk then a background process uploads later
29.
When does the client start writing the next block of a file
a)
After all replicas of the current block are acknowledged
b)
As soon as the first replica is stored
c)
Immediately after the checksum is computed
d)
Only after rebalancer completes
e)
When the Secondary Namenode checkpoints
30.
What is a direct benefit of the pipeline design
a)
Bandwidth is conserved because each byte is forwarded rather than duplicated by the client to all nodes
b)
Clients can perform random writes within existing blocks
c)
It removes the need for checksums
d)
It keeps all data on the same rack by design
e)
It eliminates latency on the first replica
31.
What problem does the Secondary Namenode address according to the slides
a)
Checkpointing FsImage and transaction log to reduce restart time of the Namenode
b)
Serving reads while the primary Namenode is busy
c)
Acting as a hot standby that takes over automatically on failure
d)
Providing a second copy of every data block
e)
Balancing rack placement in real time
32.
During checkpointing the Secondary Namenode first
a)
Copies FsImage and edits from the Namenode to a temporary directory
b)
Deletes the current FsImage to force a rebuild
c)
Rolls the edits into Parquet files
d)
Stops all Datanode heartbeats
e)
Changes replication factor of system files
33.
After a restart what happens to the Namenode transaction log per the process described
a)
It is purged after merging with FsImage into a new checkpoint
b)
It is retained indefinitely for auditing and never cleared
c)
It is replayed on every client read
d)
It is sent to Datanodes as a block report
e)
It becomes the new FsImage without merge
34.
Which command category checks the health of blocks and under replication in HDFS
a)
fsck operations
b)
put operations
c)
cat operations
d)
text operations
e)
copyFromLocal operations
35.
Which activity belongs to file ownership and validation in HDFS CLI
a)
Using chown and chmod to manage permissions
b)
Using distcp to copy across clusters only
c)
Using yarn application to submit jobs
d)
Using spark shell to read Parquet
e)
Using hive cli to run SQL
36.
An administrator wants to limit the network bandwidth used during HDFS block rebalancing. Which tool or feature should be used?
a)
Use the HDFS Balancer with bandwidth throttling parameters
b)
Exit HDFS safe mode using dfsadmin -safemode leave
c)
Upload files to HDFS using dfs -put with a custom block size
d)
Configure output compression for MapReduce jobs using hive.exec.compress.output
e)
Submit a Spark application with a reduced number of shuffle partitions
37.
Which is a typical upload pattern for large files via CLI
a)
Use hdfs dfs put which splits data into blocks on the client before sending
b)
Write directly into the Namenode memory then flush to disk later
c)
Use copyToLocal to push data into HDFS
d)
Disable checksums to speed up upload
e)
Write to all replicas in parallel from the client
38.
Which administrative UI insight is most aligned with the Namenode web UI
a)
Counts of live and dead Datanodes and replication status
b)
Exact byte content of each block
c)
User passwords in clear text
d)
CPU instruction traces for each task
e)
The full list of SQL queries run by Hive
39.
A practical use of the Namenode UI during incidents is to
a)
Verify under replicated blocks and dead node counts
b)
Change the replication factor by editing a form field
c)
Rewrite the FsImage online
d)
Start and stop Datanodes directly from the page
e)
Execute shell commands on workers
40.
Which information is least likely to appear in the Namenode UI per the slides
a)
Full byte level contents of a specific data block
b)
Number of live Datanodes
c)
Amount of under replicated blocks
d)
Safemode state
e)
Cluster summary
41.
Which list matches formats commonly stored on HDFS as per the slides
a)
Text SequenceFile Avro Parquet ORC
b)
XML only
c)
JPEG only
d)
Only proprietary binary formats
e)
Only row oriented formats
42.
Why is SequenceFile mentioned alongside Avro Parquet and ORC
a)
It was optimized for early Hadoop MapReduce workloads
b)
It is the default format of Hive tables today
c)
It provides ACID transactions for updates
d)
It is a columnar format with predicate pushdown
e)
It is a lossy image compression format
43.
Why does choosing a data format matter for analytics at scale
a)
It affects compression I O reduction and tool compatibility
b)
It only changes how names appear in HDFS UI
c)
It replaces the need for query engines
d)
It guarantees lower latency regardless of query shape
e)
It removes the need for schema design
44.
Which scenario highlights the weakness of plain CSV at scale mentioned in the slides
a)
Querying one column still requires reading the entire file
b)
Appending rows is impossible
c)
CSV cannot store integers
d)
CSV enforces a strict schema by default
e)
CSV is always faster due to simplicity
45.
Which advantage of text based formats is accurate
a)
Human readable and easy to exchange between tools
b)
Built in column level indexes for skipping
c)
Native schema evolution with partition transforms
d)
Automatic compression with no overhead
e)
Block aware predicate pushdown
46.
Which limitation of text based formats is emphasized
a)
Lack of schema and compression leads to full scans
b)
They cannot be stored on HDFS
c)
They forbid appends
d)
They are not splittable on HDFS
e)
They require specialized readers only
47.
What is the primary structure of a SequenceFile
a)
Binary key value pairs in a row based layout
b)
Column oriented pages with statistics
c)
Triples with RDF semantics
d)
Tree of nested records with JSON text
e)
Blocks of vectors for GPU access
48.
Why can a compressed SequenceFile still be splittable
a)
It stores synchronization markers that allow splitting even when compressed
b)
It disables compression headers entirely
c)
It compresses each row separately with no header
d)
It requires bzip2 which is always splittable
e)
It stores a global index in the Namenode
49.
Which property differentiates Avro from plain text formats
a)
Embedded JSON schema enables schema evolution and self describing files
b)
It stores only values and infers schema on read
c)
It is columnar by default for analytics
d)
It cannot be compressed or split
e)
It forbids complex types such as maps
50.
For what workload is Avro positioned per the slides
a)
Data exchange and row oriented storage rather than analytics heavy scans
b)
Interactive SQL with predicate pushdown
c)
Image processing with lossy compression
d)
Transactional updates with ACID
e)
Graph queries with traversal indexes
51.
Which example best reflects Avro data types per the slides
a)
Primitive types like int long and complex types like records arrays maps
b)
Only strings and bytes without numeric types
c)
Only nested structs without arrays
d)
Only floating point and decimal types
e)
Only Boolean and null
52.
What metadata does an Avro file carry according to the example
a)
The schema itself along with the data rows
b)
Only a pointer to an external schema registry
c)
Only column statistics per row group
d)
Only a list of block replicas
e)
Only compression codec with no schema
53.
Which is a core benefit of Parquet for column selective queries
a)
Column oriented layout reduces disk I O when reading a subset of columns
b)
Row based layout improves sequential scan speed only
c)
Inline JSON makes parsing simple for small files
d)
It removes the need for compression entirely
e)
It stores each row as a separate small file
54.
What enables predicate pushdown and data skipping in Parquet
a)
Lightweight statistics such as min max and null counts per column per row group
b)
Global query indexes stored in the Namenode
c)
Replication factor metadata embedded per row
d)
Checksums computed per row only
e)
A separate SQL index file
55.
How does Parquet handle nested data according to the slides
a)
It supports nested columns using Dremel style encoding
b)
It flattens all nested fields into one wide row
c)
It stores nested fields as raw JSON only
d)
It forbids arrays and maps
e)
It requires schema on read only
56.
Which engines were cited as commonly reading Parquet
a)
Spark Hive Impala Presto Trino
b)
Only MapReduce v1
c)
Only Hive and Pig
d)
Only Spark Streaming
e)
Only Snowflake
57.
Which Parquet feature most directly speeds up planning on large datasets
a)
Per row group statistics that allow skipping entire chunks
b)
Row level locks protecting readers
c)
Centralized secondary indexes in Zookeeper
d)
Client side caching of entire files always
e)
Mandatory small row groups for all tables
58.
Which is the likely tradeoff when using Parquet for write heavy workloads
a)
Higher write amplification due to columnar encoding and page organization
b)
Inability to compress data effectively
c)
Loss of schema information at write time
d)
Requirement to read whole files for single column access
e)
Incompatibility with vectorized reads
59.
Which statement compares ORC to RCFile in Hive ecosystem
a)
ORC replaces RCFile with block mode compression and stripes for better performance
b)
RCFile replaces ORC for better compression
c)
Both are row only formats without columnar storage
d)
ORC is only for transactional logs not analytics
e)
RCFile supports more complex types than ORC
60.
What storage layout does ORC use
a)
Stripes containing rows where each column is stored column wise within the stripe
b)
Pure row wise blocks with no column grouping
c)
One file per column for the entire table
d)
JSON text blocks grouped by document
e)
Key value pairs per line
61.
Which ORC feature improves query speed on large datasets
a)
Lightweight indexes and column level aggregates enabling skipping
b)
Mandatory global index stored in Namenode
c)
Replication aware readers choosing racks
d)
Checksums replaced with MD5 to speed reads
e)
Random writes inside stripes
62.
Which performance characteristic is associated with ORC per the slides
a)
Higher compression ratios and faster reads than RCFile
b)
Slower reads but easier appends than Parquet
c)
Best suited for tiny files and many small writes
d)
Faster writes due to no metadata
e)
No benefit for analytical workloads
63.
Which capability is missing if you rely only on Parquet or ORC
a)
ACID transactions for updates and deletes with time travel
b)
Splittable reads in distributed engines
c)
Compression of data pages
d)
Schema definition
e)
Column projection
64.
Why is schema evolution tricky with only file formats
a)
Changing layout across files complicates readers and coordination
b)
Because files cannot be versioned at all
c)
Because compression forbids adding columns
d)
Because engines cannot read JSON
e)
Because replication must be disabled
65.
Which modern table formats were highlighted to add a transaction layer over files
a)
Delta Lake Apache Hudi Apache Iceberg
b)
CSV TSV JSON
c)
RCFile SequenceFile
d)
MyISAM InnoDB
e)
YAML TOML
66.
What do these table formats add beyond files
a)
Metadata and transactions enabling updates deletes and better schema evolution
b)
Lossy compression for faster scans
c)
A requirement to use a single engine only
d)
Removal of partitioning concepts entirely
e)
Built in visualization UI
67.
Which combination of features is core to Apache Iceberg
a)
ACID transactions schema and partition evolution and time travel across engines
b)
Only schema on read with no metadata
c)
Replica aware reads for HDFS only
d)
Row based encoding with CRC per row only
e)
A separate metastore required for all queries
68.
Why is Iceberg positioned as an open standard for lake houses
a)
It works across multiple engines like Spark Flink Trino Hive and others
b)
It requires a single vendor notebook
c)
It mandates a proprietary file format
d)
It cannot interoperate with Parquet
e)
It only supports streaming not batch
69.
Which benefit directly follows from partition evolution in Iceberg
a)
Changing partition strategy over time without rewriting the table
b)
Removing the need for any partitioning ever
c)
Encoding partitions as folders only
d)
Binding queries to static directory listings
e)
Requiring users to specify partition columns in every query
70.
What role does the catalog play in Iceberg architecture
a)
It points to current table metadata and mediates access to snapshots
b)
It stores all data rows centrally
c)
It replaces the object store with a database
d)
It computes checksums on read
e)
It performs compaction only
71.
Why does Iceberg avoid expensive directory listings during planning
a)
Metadata files enumerate data files and statistics for direct pruning
b)
It caches entire tables in driver memory by default
c)
It uses Namenode RPCs to list blocks
d)
It requires Hive Metastore to list partitions always
e)
It hashes file names to avoid reads
72.
Which relationship between metadata and snapshots is highlighted
a)
A metadata file references the current snapshot which in turn references manifests
b)
A snapshot stores the entire data bytes inline
c)
Manifests point to the catalog which points to data files
d)
The catalog holds data files while metadata holds queries
e)
Snapshots directly list every record value
73.
How is a snapshot defined in Iceberg
a)
A consistent view of a table at a point in time created on data or schema changes
b)
A copy of every file stored twice for durability
c)
A best effort listing of files per directory
d)
A partial state that may be visible before commit
e)
A write lock held by the first writer only
74.
Which property of snapshots enforces atomicity
a)
A new snapshot becomes visible only after a successful commit
b)
Readers merge partial writes with last successful writes
c)
Manifests are rewritten in place
d)
The catalog overwrites files while reads continue
e)
Writers update files in place inside row groups
75.
Which chain captures how queries discover data in Iceberg
a)
Catalog to metadata file to manifest list to manifest files to data files
b)
Catalog to directory listing to file read
c)
Metastore to Datanode to Namenode to data files
d)
Driver to HDFS UI to block map
e)
Client to rebalancer to data files
76.
What user facing benefit follows from snapshot immutability
a)
Time travel queries that can access prior table states
b)
Ability to bypass checksums in reads
c)
Zero cost updates without new files
d)
Elimination of compaction needs
e)
Direct random writes inside Parquet pages
77.
What does the manifest list file contain
a)
Pointers to manifest files along with summary information
b)
Raw data rows for fast sampling
c)
User access logs for auditing
d)
A list of NameNode RPC endpoints
e)
Executable code for predicate pushdown
78.
What does each manifest file primarily track
a)
A set of data files with per file statistics used for pruning
b)
Only table level properties like owner
c)
Only partition names without files
d)
All SQL queries ever run
e)
Replication factors per HDFS block
79.
Which sequence occurs on INSERT in Iceberg
a)
Data file is written then added to a manifest then to a manifest list then a new metadata file is committed
b)
Data file is appended to the end of an existing Parquet file in place
c)
Catalog updates first then data files are written later
d)
Manifest list is updated in place without a new snapshot
e)
Only the partition directory is touched
80.
Which statement about MERGE UPSERT in Iceberg is accurate
a)
Existing files are read and new files with updates are written which then form a new snapshot
b)
Rows are updated in place at byte offsets
c)
Only metadata is updated without touching data files
d)
It requires a full rewrite of the entire table always
e)
It cannot be expressed in SQL like syntax
81.
Why can a SELECT query plan skip reading many files in Iceberg
a)
File level statistics and partition specs allow pruning during planning
b)
Because all files are kept in memory
c)
Because the catalog filters rows by value
d)
Because each row has its own index record
e)
Because readers sample only 1 percent of files
82.
What is the effect of PARTITIONED BY HOUR order ts in the sample create table
a)
Rows are logically partitioned by hour of the timestamp using a transform
b)
Rows are stored in a single hourly JSON file
c)
Only the hour column is materialized and minutes are dropped from data
d)
The catalog creates one database per hour
e)
It disables pruning across days
83.
Which engine syntax in the example indicates the table uses Iceberg
a)
USING iceberg clause in the create table
b)
FORMAT ORC on the table
c)
ENGINE MyISAM in create table
d)
CREATE DATABASE ICEBERG first
e)
PARTITION BY RANGE two columns
84.
After an INSERT into an Iceberg table what metadata effect occurs
a)
A new snapshot is produced and the catalog points to it
b)
No snapshot is created until a compaction
c)
The current snapshot is edited in place
d)
Only the manifest list is updated without a commit
e)
Only statistics are recalculated without new files
85.
In the MERGE example what makes an UPSERT possible
a)
Matching on a key and updating or inserting rows leads to new files and a new snapshot
b)
Editing rows inline inside Parquet pages
c)
Overwriting the table directory in place
d)
Relying on the Namenode to mutate blocks
e)
Using SequenceFile random writes
86.
Which is a likely side effect of many small merges in Iceberg
a)
Need for compaction to combine many small data files
b)
Loss of ACID guarantees until a vacuum
c)
Creation of unsplittable monolithic files
d)
Requirement to switch to RCFile
e)
Disabling of partition pruning
87.
Hidden partitioning primarily helps users by
a)
Decoupling physical partition layout from query syntax and still giving good performance
b)
Disabling partitioning and relying on directory listing
c)
Hiding all statistics from the planner
d)
Forcing users to specify partitions in every query
e)
Replacing metadata with heuristics only
88.
Which pair of properties together enable reliable concurrent reads and writes
a)
Snapshot isolation and atomic writes
b)
Global locks in a single coordinator
c)
Client side retries without metadata
d)
Row level locking inside Parquet pages
e)
Directory level rename only
100 %
