Monday, September 28, 2015

There's an updated cloud in town Part 3: Reinstalling everything

The prerequisite checker doesn't check the ICMWO fixpack

ICO 2.5 has a great prerequisite check script, but I found out the hard way that it doesn't check to make sure you have the correct ICMWO fixpack installed (fp 2 or later is required). Specifically, what I got was a crash of the script, rather than anything useful. Even after I made it so the script didn't crash, the error still didn't directly point to the fact that I had forgotten to install the fixpack. Here's the failure message I received at the end of the output:

Traceback (most recent call last):
  File "/mnt/hgfs/ico25/ico_install/installer/", line 347, in <module>
    if not validator.validate(param_dict, prereq, log_file) and not os.path.exists("/tmp/skip-checking"):
  File "/mnt/hgfs/ico25/ico_install/installer/validator/", line 74, in validate
    openstackServices.validate(params, response1, messageValidator)
  File "/mnt/hgfs/ico25/ico_install/installer/validator/", line 70, in validate
    if not user_response:
UnboundLocalError: local variable 'user_response' referenced before assignment

I edited line 70 of to allow it to at least complete correctly, and this was the error message at that point:

 -  Checking that the following services are available: ['cinder', 'glance', 'nova', 'neutron|nova network']
   -  Status: Failed
   -  The following services must be available and running : ['cinder', 'glance', 'nova', 'neutron|nova network']
ERROR: The cinder service API is not enabled
   -  User response: Ensure that the openstack services are available

All of those services were running, but it jarred me into remembering that I hadn't installed FP3. Once I installed the fixpack, the prereq checker script and install were able to complete successfully.

Verifying you ran correctly

To verify that you ran the script correctly on your servers, open the OpenStack Dashboard (https://openstack_controller) and navigate to IDENTITY->Domains, then click the down arrow far to the right of a domain and select Edit Domain to get this dialog:

Specifically, notice that there are 5 tabs: Domain Information, Domain Members, Domain Groups, Domain Administrators, Quota and Availability Zones. This is how it should look. If it doesn't, you did something wrong in running the script. I think my problem was that I ran it twice on my controller - once with the "master_controller" option and once with the "controller" option. I should have only run it ONCE, with the "master_controller" option.

You will need to go to the Availability Zones tab and add the "nova" availability zone to the Default domain.

Also navigate to IDENTITY->Projects and click the down arrow to the far right of the "admin" project, then select Edit project and you should see this dialog:

Notice that it has an Availability Zones tab. Again, that is not there if the scripts didn't run correctly.

And now for the project (similar to the domain above), you will need to go to the Availability Zones tab and add the "nova" availability zone to the projects you'll be working with through ICO.

Thursday, September 24, 2015

There's an updated cloud in town Part 2: Still Installing ICO 2.5

A few more hurdles overcome as I get closer to getting ICO 2.5 installed.

Some RHEL 7 notes

The firewall in RHEL7 (and 7.1) is not iptables. Instead, it's the firewalld service that's controlled by systemd. I'm not sure which install option causes it to be configured because it wasn't running on all of my RHEL 7.1 systems. Anyway, to turn it off, you can run:

systemctl stop firewalld
systemctl disable firewalld

In my case, it was blocking port 53 (dns), which I needed open to configure the vCenter server (next section). I first just used Applications->Sundry->Firewall to open port 53, then realized that I could just turn it completely off in my test environment so I don't hit any more problems with it.

Installing vSphere 6.0 Without a Windows Machine

I decided to also install vSphere 6.0 to use that as a testbed, and that has a few challenges. Specifically, the vCenter Server Appliance (VCSA) no longer ships directly as a .OVA file. It is now an ISO file that you're supposed to mount and run on a Windows machine to remotely install the vCenter Server Appliance on a remote ESXi server.

I didn't want to get a Windows machine involved if at all possible, and it turns out to be fairly straightforward to do this. You will find 99% of the instructions in this great article:

Specifically, the .OVA file can be found in the .ISO file that you download from VMWare. It just doesn't have a .ova extension. So you need to extract the file, change the name to include the .ova extension, and then you're mainly off to the races. HOWEVER, you have to do ONE MORE THING to actually get it working. Specifically, you need to add this additional line to the end of the .vmx file after you import the .ova file:""

Set the value appropriately for your network. If you don't add this, the VM will start up, but will have the error:

Failed to configure network

And I couldn't find a way to fix that in the VM as it stood. I updated the DNS settings, rebooted the server, did lots of other things, etc., and it still just showed that error. So I knew I would have to recreate the VM from the OVA file, but needed to figure out how to set the DNS server of the VM from the VMX file.

So I mounted the VCSA ISO file on Linux and ran the following command at the root of it:

grep -r guestinfo.cis *

Somewhat amazingly, that came back within seconds and I found all of the settings from the linked article, and then I searched for "dns" and found the above REQUIRED setting.

I didn't have a "good" DNS server on my network, so I quickly created a DNS server on one of my RHEL7.1 systems. It's REALLY easy to do this if you have all of your hosts in the /etc/hosts file. You just need to run the command:

service dnsmasq start

systemctl start dnsmasq

(Edit 9/26: I changed the above command to use the systemd mechanism for starting the service)

And that's it. You now have a DNS server.

You still NEED a Windows machine for vSphere

I thought that the vSphere Web Client would allow me to just use a browser, but that's not quite right. the web interface requires Flash, and really only supports Windows or MacOS clients. So I've had to bring a Windows machine into the mix anyway.

Wednesday, September 23, 2015

There's an updated cloud in town Part 1: Installing ICO 2.5

IBM Cloud Orchestrator 2.5 was recently released, so I'm installing it in my lab. This new release is based on IBM Cloud Manager with OpenStack 4.3, so quite a bit has changed. This series of posts will discuss some of the issues with installation and overall thoughts as I go through.

Wrong Version of IBM Cloud Manager 

IBM Cloud Manager is included in the eAssembly for ICO, but it's not the required level! The Cloud Manager files included with ICO are JUST version 4.3. However, version 4.3 with at least fixpack 2 is what's required. So after installing Cloud Manager, you will need to go to IBM Fix Central to download Cloud Manager 4.3 fixpack 3, which was released after ICO 2.5.

Cloud Manager: Additional YUM Repository Needed for KVM

If you're deploying a KVM cloud to Red Hat 7 or 7.1, you will need to enable the Red Hat "Optional" repository to have access to several python packages, including python-zope-interface and python-pyasn1-modules.

If your target machines don't have access to the Red Hat Subscription Network, you can get around this by downloading Fedora 19 and add it as a Yum repository. But the Fedora ISO doesn't include the python-pyasn1-modules rpm. So you'll need to download that and add it to a local repository on the ICM Deployer machine. I used the 'createrepo' command to create the repository under /opt/ibm/cmwo/yum-repo/operatingsystem/redhat7.1/x86_64/optional .

I downloaded the Fedora 19 ISO from here:

And I downloaded the python-pyasn1-modules RPM from here: I think a better link is:

I didn't need any other packages - the Fedora 19 ISO and the python-pyasn1-modules RPM gave my install everything it needed.

Monday, July 6, 2015

Loading the SCCD 7.5.1 Demo Content Package into SCCD (ICD) 7.5.3

This tip is specifically for SCCD 7.5.3.


When trying to install the SCCD 7.5.1 Content Pack into SCCD 7.5.3, you'll get an error about the column named GMAPSDISTANCEMATRIX. Specifically, the error will state that there is no default value and no values for that column in the data. The first round of errors come from the Data/SCCD_SI_MAPMANAGER.xml file in the content pack. If you manually download the content pack, you can view the file to see that the data it's complaining about really isn't there. And once you fix the errors found in that file, there are other errors.


Since this is just demo data on a demo system, I figured it was OK to modify the offending Maximo objects, and the modifications worked to eliminate the error and allow me to go through all of the scenarios. You should be on a demo system, so you should be OK with that stipulation.

To workaround the problem, you need to make two different types of database modifications within Maximo.

First, turn on Admin mode following these instructions:

Next, you need to mark that the following attributes are NOT required:


Go To Applications->System Configuration->Platform Configuration->Database Configuration and search for the MAPMANAGER object:

Select the object found and look for the each one of the listed attributes Attribute. You'll need to go to the second page of attributes to find SPATIALDISTANCEMATRIX.

You can then de-select "Required?" flag which will make it a non-required attribute (which corresponds to a column in the database). Once you've done this for the four attributes, click the Save button.

Now open the ASSET object and you need to add an attribute named AMCREW. Go into the ASSET object and add a new row named AMCREW. All of the default values work fine. This attribute won't hold any data - it just needs to be there.

Then go back to "List View" and select "Apply Configuration Changes on the left under "More Actions":

Once the process completes, you should see dialogs similar to the following:

At this point you can successfully install the content pack.

Tuesday, June 30, 2015

IBM Performance Management 8.1.1 is GA!

IBM® Performance Management is a comprehensive solution that helps manage the performance and availability for complex applications that might be running in a data center, public cloud, or hybrid combination. This solution provides you with visibility of your applications, ensuring optimal performance and efficient use of resources.
The Performance Management solution collects data from both Performance Management agents and Tivoli® Monitoring agents. Data is displayed in the Application Performance Dashboard for both Performance Management agents and their hybridized Tivoli Monitoring counterparts.

The IBM Performance Management solution has four offerings:

  • IBM Application Performance Management Advanced combines IBM MonitoringIBM Application Performance Management, and IBM Application Diagnostics into one offering.
  • IBM Application Performance Management for end user experience, transaction tracking, and resource monitoring of all your application components. Application Performance Management is useful for managing your critical applications in production.
  • IBM Monitoring for resource monitoring of infrastructure, application components, and cloud workloads. Resource monitoring helps you identify and address slow transactions, capacity issues, and outages.
  • IBM Application Diagnostics for code level visibility into your applications and the health of your application servers. Use the diagnostics dashboards to find performance bottlenecks in application code.

A new piece of documentation included with this release is the Scenarios guide. There is currently only one scenario documented, but it is very detailed and represents the most common scenario by far.

Friday, June 19, 2015

Live Video Presentation: ITM Nugget: Overview of how to retrieve enterprise wide information in one simple output

IBM's Mark Leftwich has made a great video on how you can pull enterprise wide information in one simple step. These techniques and examples are great for audits, sanity checking monitoring and problem diagnosis work:

Thursday, June 18, 2015

A great document from IBM on integrating SCCD processes with traditional Maximo asset management functions

IBM Control Desk (previously named SCCD - SmartCloud Control Desk) is Maximo, but because it provides several additional customizations to Maximo, many traditional Maximo developers are intimidated by it. But that shouldn't be the case. Like other Maximo additions (Maximo for Oil and Gas, etc.), ICD is mainly a collection of additional pages (officially named "applications") that access the same Maximo database that you know and love. It does deal with some different objects (tables) in the database, but it's all the exact same architecture. And while it does add new workflows and a few other things, these are all really just "normal" Maximo customizations. It just happens to be that ICD contains quite a few of these customizations, and that's why IBM packages it as its own product.

Have a look at this paper for more information on the use of automated vs manual processes when dealing with ICD and asset management: