Technical Quiz & Challenge

After changing the default block size and restarting the cluster, to which data does the new size apply?

• all data

• no data

• existing data

• new data

Answer explanation

In Hadoop HDFS, block size is specified in conf. file – hdfs-site.xml. To change the block size, parameter, dfs.block.size can be changed to required value(default 64mb/128mb) in hdfs-site.xml file. Once this is changed, cluster restart is required for the change to effect, for which will be applied only to the new files. Existing files’ block size doesn’t change. In order to change the existing files’ block size, ‘distcp’ utility can be used.

Data node failure is handled by?

• Replication-factor

• Checkpointing

• Block Report

• Secondary NameNode

Answer explanation

If the DataNode fails then the data block which it holds are created on another DataNode. So, the replication factor is always maintained. If the failed DataNode comes back then it's treated as fresh node and used for newer data blocks.

What will happen if the block size in Hadoop cluster is set to 4KB?

• Under utilization of cluster.

• Lesser number of blocks are created.

• Over burdening of NameNode

• Better parallelism will be achived.

Answer explanation

If a block size is set to 4KB then will lead to many data blocks. NameNode stores the metadata and DataNode will always ask NameNode to get next data block address whcih will over burden the NameNode.

Suppose, in Hadoop 2.0 we have a 750 MB of input file and there are 3 nodes in the cluster, with default replication factor, what will be the total number of blocks generated in HDFS for that file?

• 10

• 24

• 12

• 18

Answer explanation

File size/data block size = number of data blocks 750/128=6

Default replication factor 3, 6*3=18

What kind of scaling does HDFS supports primarily?

• Vertical

• Horizontal

• Adaptive

• Diagonal

Answer explanation

Hadoop adds more node to existing cluster which can be done without stopping the existing system.

How NameNode gets to know if a data block is corrupted?

• Heartbeat

• Secondary NameNode sends Notification

• Metadata

• Block Report

What does Apache Spark provide?

• storage + computation

• all things whatever Hadoop core provides

• Only computation

• Only storage.

Command to rename a file

• Hadoop fs -cp

• Hadoop fs -mv

• Hadoop fs -get

• Hadoop fs -rename

What will this command do?

ls -ltr|tail

• It will error out

• display 10 oldest files/folders

• display 10 newest files/folders

• display 20 newest files/folders

Answer explanation

tail

-t option will sort the entries by modification date(with newest first)

-r will reverse the sorting order.

As -t will sort by modification time with newest first, -r will cause the reverse i.e. oldest entries will be shown first now.

How to find out how many lines in file contains the word hello?

• ls file.txt|grep hello| wx -l

• cat file.txt|grep hello| wc -l

• grep hello| wc -l|cat file.txt

• it will error out