— No in-place upgrade from CDH3 to CDH4
Upgrading from CDH3 to CDH4 is fully supported, but in-place upgrading of CDH3 packages to CDH4 packages is not supported. You must uninstall the CDH3 packages before installing CDH4 packages. See the CDH4 upgrade documentation for more information.
— The cluster will not restart if the cluster is using configuration files in /etc/hadoop/conf.empty
Before CDH4.2, the the default configuration files were installed in /etc/hadoop/conf.empty. As of CDH4.2, they are in /etc/hadoop/conf.dist. The cluster deployment instructions tell you to copy these default configuration files to a custom directory of your own choosing, set the alternatives mechanism to point to that custom directory, and edit them there to create the working configuration for your cluster.
If you did not follow these instructions when you installed an earlier version of CDH, your configuration files may still be in /etc/hadoop/conf.empty, and if that is the case, the cluster will not restart after you upgrade it to CDH4.2.
Resolution: Use workaround.
- To avoid the problem: Before upgrading:
- Copy your configuration files from /etc/hadoop/conf.empty to a custom directory of your own choosing.
- Set the alternatives mechanism to point to that custom directory as shown in step 2 of these directions.
- If the problem has already occurred: If you have already upgraded to CDH4.2 and the cluster has failed to start, do the two steps above, and then rename the files whose names have been changed (the files you edited when creating the configuration) so that they have their original names: for example, rename core-site.xml.rpmsave to core-site.xml.
— Upgrading HDFS from CDH3 to CDH4 with security enabled may fail
12/11/05 19:42:54 ERROR namenode.NameNode: Exception in namenode join java.io.IOException: During upgrade failed to load the editlog version -19 from release 0.20.203. Please go back to the old release and restart the namenode. This empties the editlog and saves the namespace. Resume the upgrade after this step. at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.check203UpgradeFailure(FSEditLogLoader.java:645) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:144) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:93) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:685) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:641) at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:321) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:234) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:498) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:390) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:354) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:389) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:423) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:590) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:571) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1134) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1193)
Resolution: None; use workaround
- To avoid the problem: Follow these directions.
- If the problem has already occurred and you are seeing the error described above, proceed as follows: 1) Uninstall the CDH4 software and reinstall CDH3. 2) Start up the NameNode under CDH3. This will result in the fsimage being rewritten anew, with no edit log entries. 3) Shut down the NameNode as you normally would while it is still in startup safe mode to ensure that no edits are written to the edit log. 4) Follow the CDH3 to CDH4 upgrade instructions. Skip the instructions about putting the NameNode into safe mode, since you have already done that.
— In an MRv2 (YARN) deployment, CDH4.2.0 and CDH4.1.x servers are not compatible with CDH4.0.x clients
Resolution: None; use workaround.
Workaround: Upgrade clients to CDH4.1.0+
— Version numbers of bigtop-jsvc and bigtop-tomcat packages for Beta customers
In CDH4.0 and CDH4.1, the bigtop-jsvc and bigtop-tomcat packages have version numbers based on the CDH4 release version numbers rather than individual Apache Software Projects version numbers. This is a change from CDH4 beta releases, so customers upgrading from beta versions of CDH4 may need to manually remove the bigtop-jsvc and bigtop-tomcat packages before beginning the upgrade procedure.
— Users must build native libraries when installing from tarballs
When installing Hadoop from Cloudera tarballs, users must build their own native libraries. The tarballs do not include libraries that are built for the different distributions and architectures.
— Change in logging format as of CDH4 affects upgrades from CDH3 to CDH4
As of CDH4, the logging format — for processes other than the MRv1 JobTracker and TaskTracker— has changed from DRFA (Daily Rolling File Appender) to RFA (Rolling File Appender). This means that you cannot reuse log4j.properties from CDH3 releases for CDH4 installations.
Workaround: Cloudera recommends using the default log4j.properties file that ships with CDH4.
— Default configuration directory is conf.dist as of CDH4.2
As of CDH4.2, the default configuration directory is /etc/hadoop/conf.dist, not /etc/hadoop/conf.empty as it was in earlier releases. This means that if you created your configuration in /etc/hadoop/conf.empty, you must copy it to a custom directory before you upgrade, and update the alternatives framework to point to this directory, as instructed in the first step of Deploying HDFS on a Cluster. If you fail to do this, and leave your custom configuration in /etc/hadoop/conf.empty, you cluster will not restart after you upgrade to CDH4.2.
|<< Previous: Performance||Next: Apache Hadoop Common >>|