HBase Migration Guide
Last updated: 2020-02-28 19:04:47PDF
An HBase table is built based on Hadoop HDFS. Therefore, HBase data can be migrated with DistCp of Hadoop HDFS or relevant tools provided by the HBase layer based on the HBase table structure.
As shown above, there are multiple HBase migration schemes, among which snapshot-based migration is recommended.
Snapshot-based HBase Migration
- Create a table in the new cluster with the same structure as in the original cluster.
hbase shellto create a snapshot in the original cluster.
$ ./bin/hbase shell hbase>snapshot 'myTable', 'myTableSnapshot'
'myTable'is the HBase table name, and
'myTableSnapshot'is the HBase snapshot name. You can run
list_snapshotsto check whether the snapshot is successfully created or run
delete_snapshotto delete it.
- Export the snapshot from the source cluster to the target one.
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot myTableSnapshot -copy-to hdfs://10.0.0.38:4007/hbase/snapshot/myTableSnapshot
10.0.0.38:4007is the value of
$activeip:$rpcportof the target cluster. When the snapshot is exported, the system will start a MapReduce job. You can add
-mappers 16 -bandwidth 200to specify the mapper and bandwidth. Here,
200indicates 200 MB/sec.
- Restore the snapshot to the target HDFS in the target cluster by running the following command in the target cluster.
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot myTableSnapshot -copy-from /hbase/snapshot/myTableSnapshot -copy-to /hbase/
- Restore the corresponding HBase table and data from HDFS in the target cluster.
disable "myTable" restore_snapshot 'myTableSnapshot' enable 'myTable'
- You can test the migration result by using a simple HBase table.