試驗1HDFSJavaAPI編程

1、實驗1 HDFS Java API編程、實驗目的1、理解HDFS在Hadoop體系結(jié)構(gòu)中的角色;2、熟悉HDFS操作常用的Java API。、實驗平臺操作系統(tǒng)：Lin uxHadoop版本：或以上版本JDK版本：或以上版本Java IDE : Eclipse三、實驗內(nèi)容禾U用HDFS的Java API編程實現(xiàn)以下指定功能：1、文件讀寫：通過FSDataOutputStream 將文件內(nèi)容寫入 HDFS通過FSDatalnputStream從HDFS中讀取文件內(nèi)容。Java代碼：package .*;public class MyTest1 public static void main(Str

2、ing args) throws IOException Con figurati on conf = new Con figurati on();(”,"");(”,"");FileSystem fs = (conf);Path filePath = new Path("/user/hadoop/");public static boolea n test(C on figurati on conf. String path)throws IOExcepti on FileSystem fs = (conf);return (new

3、 Path(path);public static void copyFromLocalFile(C on figurati on conf.String localFilePath, String remoteFilePath) throws IOExceptio n FileSystem fs = (conf);Path localPath = new Path(localFilePath);Path remotePath = new Path(remoteFilePath);/*第一個參數(shù)表示是否刪除源文件，第二個參數(shù)表示是否覆蓋*/(false, true, localPath, re

4、motePath);();public static void appendToFile(Configuration conf, String localFilePath,String remoteFilePath) throws IOExcepti on FileSystem fs = (conf);Path remotePath = new Path(remoteFilePath);/*創(chuàng)建一個文件讀入流 */FilelnputStream in = new FilelnputStream(localFilePath);/*創(chuàng)建一個文件輸出流，輸出的內(nèi)容將追加到文件末尾*/FSDataOu

5、tputStream out = (remotePath);/*讀寫文件內(nèi)容*/byte data = new byte1024;int read = -1;while (read = (data) > 0) (data, 0, read);();();();public static void main( Stri ng args) Con figurati on conf = new Con figurati on();(”,"");(”,"");(”，"NEVER");(”，"true");String

6、 localFilePath = "/home/hadoop/" else + " 不存在.");/*進行處理*/ if (!fileExists) public class MyTest5 /*下載文件到本地判斷本地路徑是否已存在，若已存在，則自動進行重命名*/public static void copyToLocal(Configuration conf. String remoteFilePath,String localFilePath) throws IOException FileSystem fs = (conf);Path remote

7、Path = new Path(remoteFilePath);File f = new File(localFilePath);/*如果文件名存在，自動重命名(在文件名后面加上 _0, _1 .) */if () + " 已存在.");In teger i = 0;while (true) f = new File(localFilePath + "_" + ();if (!() localFilePath = localFilePath + "_" + ();break; i+; "將重新命名為:"+ loca

8、lFilePath);/*讀取文件內(nèi)容*/public static void cat(C on figurati on conf. String remoteFilePath)throws IOExcepti on FileSystem fs = (conf);Path remotePath = new Path(remoteFilePath);FSDataI nputStream in = (remotePath);BufferedReader d = new BufferedReader( new In putStreamReader(i n);Stri ng line = nu II;

9、while (li ne = () != n ull) ();();();public static void main(String args) Con figurati on conf = new Con figurati on();(”,"");Stri ng remoteFilePath = "/user/hadoop/"/*刪除文件*/public static boolea n rm(C on figurati on conf, Stri ng remoteFilePath)throws IOExcepti on FileSystem fs = (conf);Path remotePath = new Path(remoteFilePath);boolea n result = (remotePath, false);();return result;public static void main(String args) Con figurati on conf = new Con figurati on();("", "");Stri ng remoteFilePath = "/user/hadoop/" / HD