There's an updated cloud in town Part 3: Reinstalling everything

The prerequisite checker doesn't check the ICMWO fixpack

ICO 2.5 has a great prerequisite check script, but I found out the hard way that it doesn't check to make sure you have the correct ICMWO fixpack installed (fp 2 or later is required). Specifically, what I got was a crash of the script, rather than anything useful. Even after I made it so the script didn't crash, the error still didn't directly point to the fact that I had forgotten to install the fixpack. Here's the failure message I received at the end of the prereq_checker.sh output:

Traceback (most recent call last):

  File "/mnt/hgfs/ico25/ico_install/installer/read-params.py", line 347, in <module>

    if not validator.validate(param_dict, prereq, log_file) and not os.path.exists("/tmp/skip-checking"):

  File "/mnt/hgfs/ico25/ico_install/installer/validator/__init__.py", line 74, in validate

    openstackServices.validate(params, response1, messageValidator)

  File "/mnt/hgfs/ico25/ico_install/installer/validator/openstackServices.py", line 70, in validate

    if not user_response:

UnboundLocalError: local variable 'user_response' referenced before assignment

I edited line 70 of openstackServices.py to allow it to at least complete correctly, and this was the error message at that point:

 -  Checking that the following services are available: ['cinder', 'glance', 'nova', 'neutron|nova network']

   -  Status: Failed

   -  The following services must be available and running : ['cinder', 'glance', 'nova', 'neutron|nova network']

ERROR: The cinder service API is not enabled

   -  User response: Ensure that the openstack services are available

All of those services were running, but it jarred me into remembering that I hadn't installed FP3. Once I installed the fixpack, the prereq checker script and install were able to complete successfully.

Verifying you ran ICM_config_ico.sh correctly

To verify that you ran the ICM_config_ico.sh script correctly on your servers, open the OpenStack Dashboard (https://openstack_controller) and navigate to IDENTITY->Domains, then click the down arrow far to the right of a domain and select Edit Domain to get this dialog:

Specifically, notice that there are 5 tabs: Domain Information, Domain Members, Domain Groups, Domain Administrators, Quota and Availability Zones. This is how it should look. If it doesn't, you did something wrong in running the ICM_config_ico.sh script. I think my problem was that I ran it twice on my controller - once with the "master_controller" option and once with the "controller" option. I should have only run it ONCE, with the "master_controller" option.

You will need to go to the Availability Zones tab and add the "nova" availability zone to the Default domain.

Also navigate to IDENTITY->Projects and click the down arrow to the far right of the "admin" project, then select Edit project and you should see this dialog:

Notice that it has an Availability Zones tab. Again, that is not there if the scripts didn't run correctly.

And now for the project (similar to the domain above), you will need to go to the Availability Zones tab and add the "nova" availability zone to the projects you'll be working with through ICO.

There's an updated cloud in town Part 2: Still Installing ICO 2.5

A few more hurdles overcome as I get closer to getting ICO 2.5 installed.

Some RHEL 7 notes

The firewall in RHEL7 (and 7.1) is not iptables. Instead, it's the firewalld service that's controlled by systemd. I'm not sure which install option causes it to be configured because it wasn't running on all of my RHEL 7.1 systems. Anyway, to turn it off, you can run:

systemctl stop firewalld
systemctl disable firewalld

In my case, it was blocking port 53 (dns), which I needed open to configure the vCenter server (next section). I first just used Applications->Sundry->Firewall to open port 53, then realized that I could just turn it completely off in my test environment so I don't hit any more problems with it.

Installing vSphere 6.0 Without a Windows Machine

I decided to also install vSphere 6.0 to use that as a testbed, and that has a few challenges. Specifically, the vCenter Server Appliance (VCSA) no longer ships directly as a .OVA file. It is now an ISO file that you're supposed to mount and run on a Windows machine to remotely install the vCenter Server Appliance on a remote ESXi server.

I didn't want to get a Windows machine involved if at all possible, and it turns out to be fairly straightforward to do this. You will find 99% of the instructions in this great article:

http://www.unixarena.com/2015/05/how-to-deploy-vcsa-6-0-on-vmware-workstation.html

Specifically, the .OVA file can be found in the .ISO file that you download from VMWare. It just doesn't have a .ova extension. So you need to extract the file, change the name to include the .ova extension, and then you're mainly off to the races. HOWEVER, you have to do ONE MORE THING to actually get it working. Specifically, you need to add this additional line to the end of the .vmx file after you import the .ova file:

guestinfo.cis.appliance.net.dns.servers="172.16.30.8"

Set the value appropriately for your network. If you don't add this, the VM will start up, but will have the error:

Failed to configure network

And I couldn't find a way to fix that in the VM as it stood. I updated the DNS settings, rebooted the server, did lots of other things, etc., and it still just showed that error. So I knew I would have to recreate the VM from the OVA file, but needed to figure out how to set the DNS server of the VM from the VMX file.

So I mounted the VCSA ISO file on Linux and ran the following command at the root of it:

grep -r guestinfo.cis *

Somewhat amazingly, that came back within seconds and I found all of the settings from the linked article, and then I searched for "dns" and found the above REQUIRED setting.

I didn't have a "good" DNS server on my network, so I quickly created a DNS server on one of my RHEL7.1 systems. It's REALLY easy to do this if you have all of your hosts in the /etc/hosts file. You just need to run the command:

service dnsmasq start

systemctl start dnsmasq

(Edit 9/26: I changed the above command to use the systemd mechanism for starting the service)

And that's it. You now have a DNS server.

You still NEED a Windows machine for vSphere

I thought that the vSphere Web Client would allow me to just use a browser, but that's not quite right. the web interface requires Flash, and really only supports Windows or MacOS clients. So I've had to bring a Windows machine into the mix anyway.

There's an updated cloud in town Part 1: Installing ICO 2.5

IBM Cloud Orchestrator 2.5 was recently released, so I'm installing it in my lab. This new release is based on IBM Cloud Manager with OpenStack 4.3, so quite a bit has changed. This series of posts will discuss some of the issues with installation and overall thoughts as I go through.

Wrong Version of IBM Cloud Manager 

IBM Cloud Manager is included in the eAssembly for ICO, but it's not the required level! The Cloud Manager files included with ICO are JUST version 4.3. However, version 4.3 with at least fixpack 2 is what's required. So after installing Cloud Manager, you will need to go to IBM Fix Central to download Cloud Manager 4.3 fixpack 3, which was released after ICO 2.5.

Cloud Manager: Additional YUM Repository Needed for KVM

If you're deploying a KVM cloud to Red Hat 7 or 7.1, you will need to enable the Red Hat "Optional" repository to have access to several python packages, including python-zope-interface and python-pyasn1-modules.

If your target machines don't have access to the Red Hat Subscription Network, you can get around this by downloading Fedora 19 and add it as a Yum repository. But the Fedora ISO doesn't include the python-pyasn1-modules rpm. So you'll need to download that and add it to a local repository on the ICM Deployer machine. I used the 'createrepo' command to create the repository under /opt/ibm/cmwo/yum-repo/operatingsystem/redhat7.1/x86_64/optional .

I downloaded the Fedora 19 ISO from here: http://www.itsprite.com/free-linux-download-fedora-19-iso-cddvd-images/

And I downloaded the python-pyasn1-modules RPM from here: ftp://ftp.icm.edu.pl/vol/rzm5/linux-fedora-secondary/releases/19/Everything/armhfp/os/Packages/p/python-pyasn1-modules-0.1.6-1.fc19.noarch.rpm I think a better link is: https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/19/Everything/x86_64/os/Packages/p/python-pyasn1-modules-0.1.6-1.fc19.noarch.rpm

I didn't need any other packages - the Fedora 19 ISO and the python-pyasn1-modules RPM gave my install everything it needed.