Backup and Disaster Recovery Overview
Cloudera Manager provides an integrated, easy-to-use management solution for enabling data protection in the Hadoop platform. Cloudera Manager provides rich functionality aimed towards replicating data stored in HDFS and accessed through Hive across data centers for disaster recovery scenarios. When critical data is stored on HDFS, Cloudera Manager provides the necessary capabilities to ensure that the data is available at all times, even in the face of the complete shutdown of a data center.
Cloudera Manager also provides the ability to schedule, save and (if needed) restore snapshots of HDFS directories and HBase tables.
- Select - Choose the key datasets that are critical for your business operations.
- Schedule - Create an appropriate schedule for data replication and/or snapshots – trigger replication and snapshots as frequently as is appropriate for your business needs.
- Monitor - Track progress of your snapshots and replication jobs through a central console and easily identify issues or files that failed to be transferred.
- Alert - Issue alerts when a snapshot or replication job fails or is aborted so that the problem can be diagnosed expeditiously.
Replication capabilities work seamlessly across Hive and HDFS – replication can be setup on files or directories in the case of HDFS and on tables in the case of Hive—without any manual translation of Hive datasets into HDFS datasets or vice-versa. Hive metastore information is also replicated which means that the applications that depend upon the table definitions stored in Hive will work correctly on the replica side as well as the source side as table definitions are updated.
Built on top of a hardened version of distcp—the replication uses the scalability and availability of MapReduce and YARN to parallelize the copying of files using a specialized MapReduce job or YARN application that diffs and transfers only changed files from each Mapper to the replica side efficiently and quickly.
Also available is the ability to do a “Dry Run” to verify configuration and understand the cost of the overall operation before actually copying the entire dataset.
- Cloudera Manager Admin Console port: Default is 7180.
- HDFS NameNode port: Default is 8020.
- HDFS DataNode port: Default is 50010.
- WebHDFS port: Default is 50070.
See Configuring Ports for Cloudera Manager for more information, including how to verify the current values for these ports.