第6章. Hadoop HDFS命令介紹

書上6.3與6.5節命令有誤，以下已經更正

6.1 啟動Hadoop Multi-Node Cluster

start-all.sh

6.2 建立與查看HDFS目錄
Step1 建立HDFS目錄

hadoop fs -mkdir /user
hadoop fs -mkdir /user/hduser
hadoop fs -mkdir /user/hduser/test

Step2 查看使用者HDFS目錄

hadoop fs -ls

Step3 查看HDFS完整目錄

hadoop fs -ls /
hadoop fs -ls /user
hadoop fs -ls /user/hduser

Step4 查看所有HDFS子目錄

hadoop fs -ls -R /

Step5 一次建立所有HDFS子目錄

hadoop fs -mkdir -p /dir1/dir2/dir3
hadoop fs -ls -R /

6.3 從本機複製檔案到 HDFS
Step1 複製本機(local)檔案到 HDFS

hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test

書上的命令-copyFromLocal後面少了空白，以下才是正確

hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test/test1.txt

hadoop fs -ls /user/hduser/test

Step2 列出HDFS目錄下的檔案

hadoop fs -cat /user/hduser/test/README.txt
hadoop fs -cat /user/hduser/test/README.txt|more

Step3 複製重複檔案至HDFS目錄

hadoop fs -copyFromLocal /usr/local/hadoop/README.txt /user/hduser/test

Step4 強迫複製重複檔案至HDFS目錄

hadoop fs -copyFromLocal -f /usr/local/hadoop/README.txt /user/hduser/test

Step5 複製多個本機檔案至HDFS目錄

hadoop fs -copyFromLocal /usr/local/hadoop/NOTICE.txt /usr/local/hadoop/LICENSE.txt  /user/hduser/test

Step6 copyFromLocal複製目錄至HDFS目錄

hadoop fs -copyFromLocal /usr/local/hadoop/etc /user/hduser/test
hadoop fs -ls /user/hduser/test

Step7 查看目錄下所有的檔案

hadoop fs -ls -R /user/hduser/test/etc

Step8 使用put複製檔案至HDFS目錄

hadoop fs -put /usr/local/hadoop/README.txt /user/hduser/test/test2.txt

Step9 使用put命令接受stdin(標準輸入)

echo abc | hadoop fs -put -  /user/hduser/test/echoin.txt
hadoop fs -cat /user/hduser/test/echoin.txt

Step10 使用put命令將本機目錄的列表，儲存到HDFS檔案

ls /usr/local/hadoop | hadoop fs -put - /user/hduser/test/hadooplist.txt
hadoop fs -cat /user/hduser/test/hadooplist.txt

6.4 將HDFS上的檔案複製到本機
Step1 將HDFS上的檔案複製到本機(local)

mkdir test
cd test
hadoop fs -copyToLocal /user/hduser/test/hadooplist.txt
ll

Step3 hadoop fs -get複製到本機(local)

hadoop fs -get  /user/hduser/test/README.txt localREADME.txt

6.5 複製與刪除HDFS檔案
Step1 複製HDFS檔案
書上的命令有誤，以下才是正確

hadoop fs -mkdir /user/hduser/test/temp
hadoop fs -cp /user/hduser/test/README.txt /user/hduser/test/temp
hadoop fs -ls /user/hduser/test/temp

Step2 刪除HDFS檔案

hadoop fs -ls /user/hduser/test
hadoop fs -rm /user/hduser/test/test2.txt

Step3 刪除HDFS目錄

hadoop fs -ls  /user/hduser/test
hadoop fs -rm -R /user/hduser/test/etc

6.6 Hadoop HDFS Web UI 介面瀏覽HDFS

http://master:50070

 

Share on Google Plus

About kevin

This is a short description in the author block about the author. You edit it by entering text in the "Biographical Info" field in the user admin panel.