Common HDFS shell commands

Just the list of very commonly used HDFS shell commands...

ls

hadoop fs –ls /

hadoop fs –ls /user/

lsr

Usage: hadoop fs -lsr
Recursive version of ls. Similar to Unix ls -R.

cat

Usage: hadoop fs -cat URI [URI …]

Copies source paths to stdout.

hadoop fs -cat hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2

hadoop fs -cat file:///file3 /user/hadoop/file4

put

Usage: hadoop fs -put URI

Similar to put command, except that the source is restricted to a local file reference.

get

Usage: hadoop fs -get URI

Similar to get command, except that the destination is restricted to a local file reference.

cp

Usage: hadoop fs -cp URI [URI …]

Copy files from source to destination. This command allows multiple sources as well in which case the destination must be a directory.
Example:

• hadoop fs -cp /user/hadoop/file1 /user/hadoop/file2
• hadoop fs -cp /user/hadoop/file1 /user/hadoop/file2 /user/hadoop/dir

mkdir

Usage: hadoop fs -mkdir

Takes path uri's as argument and creates directories. The behavior is much like unix mkdir -p creating parent directories along the path.

Example:

• hadoop fs -mkdir /user/hadoop/dir1 /user/hadoop/dir2

mv

Usage: hadoop fs -mv URI [URI …]

Moves files from source to destination. This command allows multiple sources as well in which case the destination needs to be a directory. Moving files across filesystems is not permitted.
Example:

• hadoop fs -mv /user/hadoop/file1 /user/hadoop/file2

rm

Usage: hadoop fs -rm URI [URI …]

Delete files specified as args. Only deletes non empty directory and files. Refer to rmr for recursive deletes.
Example:

• hadoop fs -rm hdfs://nn.example.com/file /user/hadoop/emptydir

rmr

Usage: hadoop fs -rmr URI [URI …]

Recursive version of delete.
Example:

• hadoop fs -rmr /user/hadoop/dir

tail

Usage: hadoop fs -tail [-f] URI

Displays last kilobyte of the file to stdout. -f option can be used as in Unix.

• hadoop fs -tail pathname