使用Hadoop的Archive处理小文件
某个hive外部表的文件:
归档:
hadoop archive -archiveName libarchive.har -p /user/asmp/hive/asmp/tt_repair_deed_tmp2 -r 3 * /user/asmp/hive/asmp/test
报错:
The resolved paths set is empty. Please check whether the srcPaths exist, where srcPaths = [/user/asmp/hive/asmp/test/analysis, ]
新建文件夹analysis把文件移动到此文件夹中
再次执行归档即可
浏览:
hadoop fs -ls har:/user/asmp/hive/asmp/libarchive.har/analysis
解档:
hadoop distcp har:/user/asmp/hive/asmp/libarchive.har/analysis /user/asmp/hive/asmp/test