The CDH4 ZooKeeper package uses /var/lib/zookeeper as the default data directory instead of /var/zookeeper, which is the default data directory for the CDH3 ZooKeeper package. When upgrading from CDH3, you must update the configuration so that that the data directory points to the directory where your data is.
After upgrading from CDH3 to CDH4 you MUST either:
a) Edit the /etc/zookeeper/zoo.cfg file and change the dataDir and dataLogDir properties to point to the directory with your CDH3 data — for example, change it from the CDH4 default /var/lib/zookeeper to the CDH3 default /var/zookeeper) directory if that is where your CDH3 data resides. — or — b) Move the directory and its contents from /var/zookeeper to its new location in /var/lib/zookeeper. This can be done with sudo -u zookeeper cp -r /var/zookeeper/* /var/lib/zookeeper.
In any case, after upgrading the ZooKeeper packages to CDH4, verify that the dataDir (and potentially dataLogDir) specified in /etc/zookeeper/conf/zoo.cfg point to a valid ZooKeeper directory.
If you previously modified your CDH3 zoo.cfg configuration file (/etc/zookeeper.dist/zoo.cfg), the upgrade from CDH3 to CDH4 will rename and preserve a copy of your modified zoo.cfg as /etc/zookeeper.dist/zoo.cfg.rpmsave. You can compare this to the new /etc/zookeeper/zoo.cfg and resolve any differences that should be carried forward (typically where you have changed property value defaults).
If your CDH3 zoo.cfg file has not been modified since installation, it will be auto-deleted and re-created.