HADOOP:HDFS: Recover files deleted in HDFS from .Trash

When files/directories are deleted, Hadoop moves files to .Trash directory  if “TrashPolicyDefault: Namenode trash configuration: Deletion interval ” enabled and interval is set.

hadoop fs -rm -r -f /user/root/employee

15/07/26 05:12:14 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 360 minutes, Emptier interval = 0 minutes.

Moved: ‘hdfs://sandbox.hortonworks.com:8020/user/root/employee’ to trash at: hdfs://sandbox.hortonworks.com:8020/user/root/.Trash/Current

# hadoop fs -ls /user/root/employee

ls: `/user/root/employee’: No such file or directory

Notes on .Trash:

The Hadoop trash feature helps prevent accidental deletion of files and directories. If trash is enabled and a file or directory is deleted using the Hadoop shell, the file is moved to the .Trash directory in the user’s home directory instead of being deleted. Deleted files are initially moved to the Current sub-directory of the .Trash directory, and their original path is preserved. Files in .Trash are permanently removed after a user-configurable time interval. The interval setting also enables trash checkpointing, where the Current directory is periodically renamed using a timestamp. Files and directories in the trash can be restored simply by moving them to a location outside the .Trash directory.

Where is the configuration located ?

# grep -ri -a1 trash /etc/hadoop/conf/

/etc/hadoop/conf/core-site.xml: <name>fs.trash.interval</name>

/etc/hadoop/conf/core-site.xml- <value>360</value>

in CDS you can enable/disable and configure interval:

http://www.cloudera.com/content/cloudera/en/documentation/cloudera-manager/v4-latest/Cloudera-Manager-Managing-Clusters/cmmc_hdfs_trash.html

From AMBARI you can change the settings using below path:

HDFS==>Configs==>Advanced Core Site ==>fs.trash.interval

Advertisements

Author: rajukv

Hadoop(BigData) Architect and Hadoop Security Architect can design and build hadoop system to meet various data science projects.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s