How to reset your OpenStack services to "Up" in your IBM Cloud Manager dashboard for a vSphere cloud

While trying test and manage components in your vSphere cloud, you may see "Services Down" in some parts of the dashboard. For example, under "Host Aggregates":

I basically found that restarting all of the "nova*" services on my controller to be the answer to this problem.

The one-line answer is to log into your controller node as root and run the following:

for i in `systemctl -a | grep \.service | grep " active"| grep nova | awk '{ print $1 }'`; do systemctl restart $i;echo $i; done

So it's iterating through the results of the 'systemctl -a' command that contain active services containing the word "nova", and restarting each of those services. After you run the above, you should see that it shows "Services Up" for all availability zones on all hosts (since in a vSphere cloud, these services are all running on the controller node, rather than on the VMware nodes themselves).

There's a new cloud in town, Part 4 How to reset ICMWO 4.3 to reinstall a cloud

After making sure ICMWO (IBM Cloud Manager With OpenStack) had the correct fixpack installed, the installation and configuration of ICO 2.5 succeeded and is working fine. I'm able to deploy individual VMs and Heat stacks to an OpenStack cloud. I've only created a small vSphere cloud, and I believe that's the reason that I haven't had much success getting ICMWOS to work with that cloud (I can launch instances, but those instances can't see their operating system). But I've learned several useful pieces of information through the process. I'll list the most important one here, and I'll write a new post for each of the others.

How to "reset" ICMWO to reinstall a cloud

While you're kicking the tires (or even installing into a production environment), you will certainly encounter the need to try to re-deploy a cloud. This "reset" functionality isn't made available from the GUI, and really involves doing some things outside of ICMWO. Happily, ICMWO doesn't install anything on your vCenter or ESXi servers (the controller uses the appropriate vSphere APIs through the vCenter server to do all the dirty work). So, to reset things so you can re-deploy a vSphere cloud, you need to:

1. Delete and re-create the "controller" node that you previously specified. This is the server that ICMWO deployed OpenStack to. You created this server specifically for this purpose based on the topology requirements of ICMWO. My entire environment is running under VMWare Workstation, so I simply took a snapshot of this VM once I had the OS installed and configured, so I could revert to that snapshot before each successive attempt.

2. Next, you need to delete the TWO Chef resources associated with the controller. There is a NODE and a CLIENT that have been created for the controller. To delete those, you need to run the following two commands (where "vmc.mynet.foo" is the FQDN of the controller for your VMWare cloud):

knife client delete vmc.mynet.foo

knife node delete vmc.mynet.foo

3. Finally, to delete the cloud from ICMWO Deployer GUI (https://icmwos.mynet.foo:8443), you need to log into the ICMWO server (via the console, ssh, VNC, etc.) as the same user you use to log into the Deployer GUI and delete a directory. The name of the directory contains the name of the cloud that you specified in ICMWO when you deployed the cloud and the datetimestamp when it was created. The directory is under:

$HOME/icm/clouds

And the name will be "cloudName_datetimestamp". So to delete the cloud named "vc55" from my GUI, I needed to run this command:

rm -rf ~/icm/clouds/vc55_2015-09-29_173201

And now ICMWO is ready to allow you to try to deploy a cloud to that node. I don't know if this name is stored anywhere else, so the safest route in my opinion is to use a different name for the new cloud.