Yes, you can do it using output commiters. Output Committers Hadoop makes sure a job either succeds or fails gracefully. This is [...]
Remove from mapred.exclude Remove from hdfs.exclude $ hadoop mradmin -refreshNodes $ hadoop dfsadmin -refreshNodes $ [...]
If your cluster does not have excludes file, add it in hdfs-site.xml dfs.hosts.exclude [...]
Intermediate data is not written in hdfs but in local disk.
If all replicas of one or more blocks of a file become unavailable, a file is considered corrupt and any attempt to access this file will [...]