Login with  Log in with facebook
Hiring Manager? SIGN UP HERE
Apr/13

11

Tuning different hadoop parameters

dfs.replication

sets the file replication factor. Default values 3. To add or modify this property go to hdfs-site.xml which by default is located in $HADOOP_HOME/conf/hdfs-site.xml 

<property>     
   <name>dfs.replication<name>     
   <value>4<value>     
</property>

dfs.block.size

HDFS is designed to manage and hold large amounts of data therefore the block size in HDFS is a thousands times larger than a traditional file system. dfs.block.size setting is used to divide files into blocks by hdfs. The default value is 64 MB but can it be set to much larger value. It's not uncommon to see dfs.block.size value north of 1 GB.

Following examples sets dfs.block.size to 128MB.

 

<property>     
<name>dfs.block.size<name>     
<value>134217728<value>     
<property>

Schedule a Demo

Schedule a Demo with us

Name *
Email *
Phone *
Company *
Details