コピー数を変更するコマンド:hadoop fs -setrep



Command Modify Number Copies



1、hadoop fs -setrep

setrep Usage: hadoop fs -setrep [-R] [-w] Changes the replication factor of a file. If path is a directory then the command recursively changes the replication factor of all files under the directory tree rooted at path. Options: The -w flag requests that the command wait for the replication to complete. This can potentially take a very long time. The -R flag is accepted for backwards compatibility. It has no effect. Example: hadoop fs -setrep -w 3 /user/hadoop/dir1 Exit Code: Returns 0 on success and -1 on error.

2.サンプルデモ

(1)ファイルのコピー数を表示します。2番目の列の3は、3つのコピーを意味します

[root@xxxxx ~]# hadoop fs -ls /test 18/01/23 01:24:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Found 4 items -rw-r--r-- 3 root hbase 100797385 2018-01-22 22:26 /test/a.tar -rw-r--r-- 3 root hbase 100854145 2018-01-22 22:26 /test/a.zip -rw-r--r-- 3 root hbase 38009648 2018-01-22 22:27 /test/kafka-2.10-0.10.2.0.x86_64.rpm -rw-r--r-- 3 root hbase 2374608 2018-01-22 22:27 /test/zookeeper-3.4.6-1.x86_64.rpm [root@xxxxx ~]#

(2)/ testディレクトリのコピー数を変更します

[root@xxxxx ~]# hadoop fs -setrep -w 2 /test 18/01/23 01:24:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Replication 2 set: /test/a.tar Replication 2 set: /test/a.zip Replication 2 set: /test/kafka-2.10-0.10.2.0.x86_64.rpm Replication 2 set: /test/zookeeper-3.4.6-1.x86_64.rpm Waiting for /test/a.tar ... WARNING: the waiting time may be long for DECREASING the number of replications. . done Waiting for /test/a.zip ... done Waiting for /test/kafka-2.10-0.10.2.0.x86_64.rpm ... done Waiting for /test/zookeeper-3.4.6-1.x86_64.rpm ... done [root@xxxxx ~]#

(3)再度部数を確認すると、2列目が2になります

[root@xxxxx ~]# hadoop fs -ls /test 18/01/23 01:25:12 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Found 4 items -rw-r--r-- 2 root hbase 100797385 2018-01-22 22:26 /test/a.tar -rw-r--r-- 2 root hbase 100854145 2018-01-22 22:26 /test/a.zip -rw-r--r-- 2 root hbase 38009648 2018-01-22 22:27 /test/kafka-2.10-0.10.2.0.x86_64.rpm -rw-r--r-- 2 root hbase 2374608 2018-01-22 22:27 /test/zookeeper-3.4.6-1.x86_64.rpm [root@xxxxx ~]#

コピーが不十分な場合は、コピーを変更するだけです