how to update config parameter of hadoop mapred-site.xml without restarting the cluster

Question

I would like to add/update the following parameter

 <property>
    <name>mapred.map.tasks.speculative.execution</name>
    <value>false</value>
  </property>

in mapred-site.xml in hadoop. How can I do this without restarting the cluster?

Answer 1

You can set this per-job in your job's configuration. If you want to change this behaviour globally you have to restart your cluster. There is no way to change it without restarting.

Answer 2

According to the Hadoop - The Definitive Guide

Be aware that some properties have no effect when set in the client configuration. For example, if you set mapred.tasktracker.map.tasks.maximum in your job submission with the expectation that it would change the number of task slots for the tasktrackers running your job, you would be disappointed, because this property is honored only if set in the tasktracker’s mapred-site.xml file. In general, you can tell the component where a property should be set by its name, so the fact that mapred.tasktracker.map.tasks.maximum starts with mapred.tasktracker gives you a clue that it can be set only for the tasktracker daemon. This is not a hard and fast rule, however, so in some cases you may need to resort to trial and error, or even to reading the source.

The properties which are honored in the client side and are job specific don't a need a cluster restart, but on the other hand cluster specific configurations like mapred.tasktracker.map.tasks.maximum would require the cluster to be restarted.