Introduction to Hadoop Quiz

What is the primary purpose of the Hadoop Distributed File System (HDFS)?

• To manage database transactions

• To provide a scalable and reliable storage system

• To process large-scale data analytics

• To handle network communications

Answer explanation

• Scalable Storage: HDFS is designed to store vast amounts of data across multiple nodes in a Hadoop cluster. It can handle petabytes of data by distributing it across many servers.

• Reliable Storage: HDFS ensures reliability through data replication. Each data block is replicated across multiple nodes, so if one node fails, the data can still be accessed from another node where it is replicated.

How does HDFS achieve fault tolerance?

• By using RAID configurations

• By storing data in cloud backups

• By replicating data across multiple nodes

• By using a single point of storage

Answer explanation

• Replication: HDFS achieves fault tolerance by replicating each data block across multiple nodes. By default, each block is replicated three times across different nodes. This means that if one node fails, the data is still available from another node that holds a replica of the same block.

• Fault Tolerance: When a node fails, HDFS can continue to operate normally because the data can be retrieved from the remaining nodes that hold the replicas. The system also detects the failure and automatically creates new replicas of the affected blocks on other nodes to maintain the desired level of replication.

What is the default block size in HDFS of Hadoop 2.x or 3.X?

• 16 MB

• 32 MB

• 64 MB

• 128 MB

What are the two main components of a MapReduce job?

• Split and Combine

• Map and Reduce

• Sort and Shuffle

• Input and Output

In a MapReduce job, what does the Mapper do?

• Aggregates data

• Distributes data across nodes

• Filters and transforms input data into intermediate key-value pairs

• Combines intermediate data into final results

What is the purpose of the Reducer in a MapReduce job?

• To distribute tasks to Mappers

• To sort input data

• To process intermediate key-value pairs and produce the final output

• To split input data into smaller chunks

What is the role of the Combiner in MapReduce?

• To split input data into smaller chunks

• To combine output from multiple Reducers

• To perform a local reduce task on the output of Mappers before sending it to the Reducer

• To distribute data across nodes

What is the role of the Namenode in HDFS?

• To store actual data blocks

• To manage metadata and the file system namespace

• To process MapReduce jobs

• To handle network communications

How does Hadoop ensure data locality in processing?

• By sending the data to the computation node

• By sending the computation to the node where the data resides

• By using centralized data processing

• By copying data to all nodes

How does the NameNode know that a DataNode is still active and functioning?

• By periodically checking the disk space

• By receiving heartbeat signals from the DataNode

• By running a scheduled job to ping the DataNode

• By logging into the DataNode manually

Answer explanation

The NameNode relies on heartbeat signals sent by DataNodes at regular intervals. These signals confirm that the DataNode is active and functioning properly. If a DataNode fails to send a heartbeat signal within a certain timeframe, the NameNode considers it inactive or failed and takes appropriate actions, such as replicating the blocks stored on that DataNode to other active DataNodes.