During the execution of a MapReduce v2 (MRv2) job on YARN, where does the Mapper place the intermediate data each Map task?

A. The Mapper stores the intermediate data on the mode running the job's ApplicationMaster so that is available to YARN's ShuffleService before the data is presented to the Reducer

B. The Mapper stores the intermediate data in HDFS on the node where the MAP tasks ran in the HDFS / usercache/and[user]sppcache/application_and(appid) directory for the user who ran the job

C. YARN holds the intermediate data in the NodeManager's memory (a container) until it is transferred to the Reducers

D. The Mapper stores the intermediate data on the underlying filesystem of the local disk in the directories yarn.nodemanager.local-dirs

E. The Mapper transfers the intermediate data immediately to the Reducers as it generated by the Map task

You are planning a Hadoop cluster and considering implementing 10 Gigabit Ethernet as the network fabric. Which workloads benefit the most from a faster network fabric?

A. When your workload generates a large amount of output data, significantly larger than amount of intermediate data

B. When your workload generates a large amount of intermediate data, on the order of the input data itself

C. When workload consumers a large amount of input data, relative to the entire capacity of HDFS

D. When your workload consists of processor-intensive tasks

You have converted your Hadoop cluster from a MapReduce 1 (MRv1) architecture to a MapReduce 2 (MRv2) on YARN architecture. Your developers are accustomed to specifying map and reduce tasks (resource allocation) tasks when they run jobs. A developer wants to know how specify to reduce tasks when a specific job runs. Which method should you tell that developer to implement?

A. Developers specify reduce tasks in the exact same way for both MapReduce version 1 (MRv1) and MapReduce version 2 (MRv2) on YARN. Thus, executing p mapreduce.job.reduce-2 will specify 2 reduce tasks.

B. In YARN, the ApplicationMaster is responsible for requesting the resources required for a specific job. Thus, executing p yarn.applicationmaster.reduce.tasks-2 will specify that the ApplicationMaster launch two task containers on the worker nodes.

C. In YARN, resource allocation is a function of megabytes of memory in multiple of 1024mb. Thus, they should specify the amount of memory resource they need by executing D mapreduce.reduce.memory-mp-2040

D. In YARN, resource allocation is a function of virtual cores specified by the ApplicationMaster making requests to the NodeManager where a reduce task is handled by a single container (and this a single virtual core). Thus, the developer needs to specify the number of virtual cores to the NodeManager by executing p yarn.nodemanager.cpu-vcores=2

E. MapReduce version 2 (MRv2) on YARN abstracts resource allocation away from the idea of "tasks" into memory and virtual cores, thus eliminating the need for a developer to specify the number of reduce tasks, and indeed preventing the developer from specifying the number of reduce tasks.

You are upgrading a Hadoop cluster from HDFS and MapReduce version 1 (MRv1) to one running HDFS and MapReduce version 2 (MRv2) on YARN. You want to set and enforce a block of 128MB for all new files written to the cluster after the upgrade. What should you do?

A. Set dfs.block.size to 128M on all the worker nodes, on all client machines, and on the NameNode, and set the parameter to final.

B. Set dfs.block.size to 134217728 on all the worker nodes, on all client machines, and on the NameNode, and set the parameter to final.

C. Set dfs.block.size to 134217728 on all the worker nodes and client machines, and set the parameter to final. You do need to set this value on the NameNode.

D. Set dfs.block.size to 128M on all the worker nodes and client machines, and set the parameter to final. You do need to set this value on the NameNode.

E. You cannot enforce this, since client code can always override this value.

A user comes to you, complaining that when she attempts to submit a Hadoop job, it fails. There is a directory in HDFS named /data/input. The Jar is named j.jar, and the driver class is named DriverClass. She runs command:

hadoop jar j.jar DriverClass /data/input/data/output

The error message returned includes the line:

PrivilegedActionException as:training (auth:SIMPLE) cause.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exits: file :/data/input

What is the cause of the error?

A. The Hadoop configuration files on the client do not point to the cluster

B. The directory name is misspelled in HDFS

C. The name of the driver has been spelled incorrectly on the command line

D. The output directory already exists

E. The user is not authorized to run the job on the cluster