I have my SC/CC and CLC on two seperate RHEL 7 hosts. I'm using overlay storage mode. If I hard reboot the SC/CC and even sometimes if I initiate an orderly reboot now I keep breaking storage and end up with the following in /var/log/eucalyptus/cloud-error.log on my SC.
ERROR [BindingCache:main] [com.eucalyptus.binding.BindingCache$BindingFileSearch$DeleteRecursively.apply(BindingCache.java):195] Unable to delete file: /var/run/eucalyptus/classcache
The /var/log/eucalyptus/cloud-output.log shows the SC waiting for contact from the CLC.
INFO | Waiting for system view with database...
The euctl command shows a valid block storage manager value zone-1.storage.blockstoragemanager = overlay
I've tried restarting EUCA services on the CLC after my SC is back up with no luck. I seem to be able to fix it by de-registering and re-registering the SC and re-issuing the euctl command for overlay storage. Am I doing something wrong by unceremoniously shutting down my SC and is there a more elegant way to fix it?
If you have multiple network interfaces you should set the bind address in eucalyptus.conf on hosts running a eucalyptus-cloud service (e.g. CLC, SC)
CLOUD_OPTS="--bind-addr=192.168.123.123"
You should also verify that multicast works in your environment as this is used for communication between hosts.