Hadoop: FAQ on hdfs snapshot

$ hdfs dfs -mkdir /user/test
$ hdfs dfs -put output23.txt /user/test
$ hdfs dfs -du /user/test
67335  /user/test/output23.txt
$ hdfs dfsadmin -allowSnapshot /user/test
Allowing snaphot on /user/test succeeded
$hdfs dfs -createSnapshot /user/test
Created snapshot /user/test/.snapshot/s20150924-050641.662

How to recover the file from .snapshot directory ?

$hdfs dfs -rm /user/test/output23.txt
15/09/24 05:08:31 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 360 minutes, Emptier interval = 0 minutes.
Moved: ‘hdfs://sandbox.hortonworks.com:8020/user/test/output23.txt’ to trash at: hdfs://sandbox.hortonworks.com:8020/user/hdfs/.Trash/Current

Confirm File is deleted:
 hdfs dfs -ls /user/test/output23.txt
ls: `/user/test/output23.txt’: No such file or directory

hdfs dfs -cp /user/test/.snapshot/s20150924-050641.662/output23.txt /user/test

15/09/24 05:12:02 WARN hdfs.DFSClient: DFSInputStream has been closed already
$ hdfs dfs -ls /user/test/output23.txt
-rw-r–r–   1 hdfs hdfs      67335 2015-09-24 05:12 /user/test/output23.txt

Can we delete files from .snapshot copy ?

Try to delete file from .snapshot
As .snapshot is a readonly copy it will not allow to delete the files.

hdfs dfs -rm /user/test/.snapshot/s20150924-050641.662/output23.txt
rm: Failed to move to trash: hdfs://sandbox.hortonworks.com:8020/user/test/.snapshot/s20150924-050641.662/output23.txt: “.snapshot” is a reserved name.

How to list snapshottable directories ?

hdfs lsSnapshottableDir
drwxr-xr-x 0 hdfs hdfs 0 2015-09-24 05:12 1 65536 /user/test

How to rename a snapshot ?

hdfs dfs -renameSnapshot /user/test s20150924-050641.662 b4patch.24sep15

$ hdfs dfs -ls /user/test/.snapshot
drwxr-xr-x   – hdfs hdfs          0 2015-09-24 05:06 /user/test/.snapshot/b4patch.24sep15

How to show difference between two snapshots ?

$ touch testfile
$ hdfs dfs -put testfile /user/test
$ hdfs dfs -ls /user/test
Found 2 items
-rw-r–r–   1 hdfs hdfs      67335 2015-09-24 05:12 /user/test/output23.txt
-rw-r–r–   1 hdfs hdfs          0 2015-09-24 05:32 /user/test/testfile

$ hdfs dfs -createSnapshot /user/test
Created snapshot /user/test/.snapshot/s20150924-053313.181
$ hdfs dfs -renameSnapshot /user/test s20150924-053313.181 afterPatch.24sep15

$ hdfs snapshotDiff /user/test b4patch.24sep15 afterPatch.24sep15
Difference between snapshot b4patch.24sep15 and snapshot afterPatch.24sep15 under directory /user/test:
M    .
+    ./output23.txt
+    ./testfile
–    ./output23.txt

How to delete the snapshot ?

hdfs dfs -deleteSnapshot /user/test afterPatch.24sep15
hdfs dfs -deleteSnapshot /user/test b4patch.24sep15




Author: rajukv

Hadoop(BigData) Architect and Hadoop Security Architect can design and build hadoop system to meet various data science projects.

One thought on “Hadoop: FAQ on hdfs snapshot”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s