The latest version of OpenStack (Rocky) can leverage bare metal servers directly

The source link from ZDNet:

https://www.zdnet.com/article/new-openstack-cloud-release-embraces-bare-metal/

Now you can provision bare metal servers through OpenStack. The linked article describes some of the use cases, and provides additional links to more information.

Using a certificate with an Azure deployment in ICO 2.5

The ICO 2.5 documentation clearly states that you cannot create a service that uses certificates from ICO, and that is still true. HOWEVER, you can deploy into an EXISTING service that has a certificate associated. The trick here is that you have to first manually create the service in Azure, then manually attach the certificate in Azure. Then when you select the "Deploy cloud service into Azure region" Self-Service option, select the "Use an existing Cloud Service" option.

A feature that I would love to see in OpenStack

A graphical user interface for the commands.

Yes, Horizon is pretty nice, but there are lots of things that it doesn't allow you to do. Additionally, the CLI commands are pretty intimidating. Here's the documentation for the current release: http://docs.openstack.org/cli-reference/content/openstackclient_commands.html . Notice how each command has at least 20 options, each with its own flags, etc. And Horizon doesn't actually call these commands. It uses the REST API (and/or other API), which is completely separate.

So what I'd like to see is something graphical that lets you first pick a command ('nova', 'glance', etc.), then you have a drop-down choice list for the next parameter, and the next, etc. Many of these commands require output from other commands as their input. So the idea would be that the interface is running each of the required commands to build the appropriate list of options. And after building the command, this tool could output that for you to use in scripts, etc.

If it was an easy task, it would already exist, and it doesn't. There's also quite a lot of movement in the code, so keeping this tool up-to-date would be quite a challenge. But I personally think it would be worth it.

ICO 2.5: Where does the heat template for the "Deploy LAMP stack" come from?

ICO 2.5 comes with a few samples in the Self-Service Catalog, including one named "Deploy LAMP stack" (under "Deploy customized cloud services"). Once you have your linux_img image configured correctly and your linux_key key pair created, you can request this offering to deploy a LAMP stack among three machines: Apache, MySQL and PHP. I found the heat template command logs on each machine in /var/log/cloud-init-output.log and /tmp/install.log, and wanted to find where exactly that template was coming from. It took me far too long to find it because it's an inline string in the JavaScript code that can be seen with Business Process Designer. Specifically, you can find it in the Embeddable Deploy Lamp Template service in the SCOrchestrator OpenStack Services toolkit, in the Build Parameters element on the diagram, here:

You can see it in the green text as part of the "Implementation" in the bottom panel.

How to reset your OpenStack services to "Up" in your IBM Cloud Manager dashboard for a vSphere cloud

While trying test and manage components in your vSphere cloud, you may see "Services Down" in some parts of the dashboard. For example, under "Host Aggregates":

I basically found that restarting all of the "nova*" services on my controller to be the answer to this problem.

The one-line answer is to log into your controller node as root and run the following:

for i in `systemctl -a | grep \.service | grep " active"| grep nova | awk '{ print $1 }'`; do systemctl restart $i;echo $i; done

So it's iterating through the results of the 'systemctl -a' command that contain active services containing the word "nova", and restarting each of those services. After you run the above, you should see that it shows "Services Up" for all availability zones on all hosts (since in a vSphere cloud, these services are all running on the controller node, rather than on the VMware nodes themselves).

There's a new cloud in town, Part 4 How to reset ICMWO 4.3 to reinstall a cloud

After making sure ICMWO (IBM Cloud Manager With OpenStack) had the correct fixpack installed, the installation and configuration of ICO 2.5 succeeded and is working fine. I'm able to deploy individual VMs and Heat stacks to an OpenStack cloud. I've only created a small vSphere cloud, and I believe that's the reason that I haven't had much success getting ICMWOS to work with that cloud (I can launch instances, but those instances can't see their operating system). But I've learned several useful pieces of information through the process. I'll list the most important one here, and I'll write a new post for each of the others.

How to "reset" ICMWO to reinstall a cloud

While you're kicking the tires (or even installing into a production environment), you will certainly encounter the need to try to re-deploy a cloud. This "reset" functionality isn't made available from the GUI, and really involves doing some things outside of ICMWO. Happily, ICMWO doesn't install anything on your vCenter or ESXi servers (the controller uses the appropriate vSphere APIs through the vCenter server to do all the dirty work). So, to reset things so you can re-deploy a vSphere cloud, you need to:

1. Delete and re-create the "controller" node that you previously specified. This is the server that ICMWO deployed OpenStack to. You created this server specifically for this purpose based on the topology requirements of ICMWO. My entire environment is running under VMWare Workstation, so I simply took a snapshot of this VM once I had the OS installed and configured, so I could revert to that snapshot before each successive attempt.

2. Next, you need to delete the TWO Chef resources associated with the controller. There is a NODE and a CLIENT that have been created for the controller. To delete those, you need to run the following two commands (where "vmc.mynet.foo" is the FQDN of the controller for your VMWare cloud):

knife client delete vmc.mynet.foo

knife node delete vmc.mynet.foo

3. Finally, to delete the cloud from ICMWO Deployer GUI (https://icmwos.mynet.foo:8443), you need to log into the ICMWO server (via the console, ssh, VNC, etc.) as the same user you use to log into the Deployer GUI and delete a directory. The name of the directory contains the name of the cloud that you specified in ICMWO when you deployed the cloud and the datetimestamp when it was created. The directory is under:

$HOME/icm/clouds

And the name will be "cloudName_datetimestamp". So to delete the cloud named "vc55" from my GUI, I needed to run this command:

rm -rf ~/icm/clouds/vc55_2015-09-29_173201

And now ICMWO is ready to allow you to try to deploy a cloud to that node. I don't know if this name is stored anywhere else, so the safest route in my opinion is to use a different name for the new cloud.

There's an updated cloud in town Part 3: Reinstalling everything

The prerequisite checker doesn't check the ICMWO fixpack

ICO 2.5 has a great prerequisite check script, but I found out the hard way that it doesn't check to make sure you have the correct ICMWO fixpack installed (fp 2 or later is required). Specifically, what I got was a crash of the script, rather than anything useful. Even after I made it so the script didn't crash, the error still didn't directly point to the fact that I had forgotten to install the fixpack. Here's the failure message I received at the end of the prereq_checker.sh output:

Traceback (most recent call last):

  File "/mnt/hgfs/ico25/ico_install/installer/read-params.py", line 347, in <module>

    if not validator.validate(param_dict, prereq, log_file) and not os.path.exists("/tmp/skip-checking"):

  File "/mnt/hgfs/ico25/ico_install/installer/validator/__init__.py", line 74, in validate

    openstackServices.validate(params, response1, messageValidator)

  File "/mnt/hgfs/ico25/ico_install/installer/validator/openstackServices.py", line 70, in validate

    if not user_response:

UnboundLocalError: local variable 'user_response' referenced before assignment

I edited line 70 of openstackServices.py to allow it to at least complete correctly, and this was the error message at that point:

 -  Checking that the following services are available: ['cinder', 'glance', 'nova', 'neutron|nova network']

   -  Status: Failed

   -  The following services must be available and running : ['cinder', 'glance', 'nova', 'neutron|nova network']

ERROR: The cinder service API is not enabled

   -  User response: Ensure that the openstack services are available

All of those services were running, but it jarred me into remembering that I hadn't installed FP3. Once I installed the fixpack, the prereq checker script and install were able to complete successfully.

Verifying you ran ICM_config_ico.sh correctly

To verify that you ran the ICM_config_ico.sh script correctly on your servers, open the OpenStack Dashboard (https://openstack_controller) and navigate to IDENTITY->Domains, then click the down arrow far to the right of a domain and select Edit Domain to get this dialog:

Specifically, notice that there are 5 tabs: Domain Information, Domain Members, Domain Groups, Domain Administrators, Quota and Availability Zones. This is how it should look. If it doesn't, you did something wrong in running the ICM_config_ico.sh script. I think my problem was that I ran it twice on my controller - once with the "master_controller" option and once with the "controller" option. I should have only run it ONCE, with the "master_controller" option.

You will need to go to the Availability Zones tab and add the "nova" availability zone to the Default domain.

Also navigate to IDENTITY->Projects and click the down arrow to the far right of the "admin" project, then select Edit project and you should see this dialog:

Notice that it has an Availability Zones tab. Again, that is not there if the scripts didn't run correctly.

And now for the project (similar to the domain above), you will need to go to the Availability Zones tab and add the "nova" availability zone to the projects you'll be working with through ICO.

ICO all-in-one deployment doesn't install Workload Deployer

The ICO 2.4 and 2.4.0.1 Deployment Server installers are definitely great pieces of software. They check prerequisites correctly and install without a hitch. They are really, really nice.

This post is about one of the options in the 'ds wizard' script options. Specifically, when you take the path:

[0] New IBM Cloud Orchestrator deployment.
[0] Demo
[1] All In One
[2] All In One IBM Cloud Orchestrator Deployment
   - Deploys IBM Cloud Orchestrator all-in-one in one machine.


This path actually deploys the template sco-devstack (/opt/ico_install/topology_templates/sco_devstack.json), which does NOT install the IBM Workload Deployer component. The effect that this has is that you do NOT have a PATTERNS option in the Administrator GUI (https://machine.fqdn:8443/), and so you can't create, view or use any patterns.

So if you want an all-in-one deployment, you should take a different option. Specifically, the path you should take through the wizard is:

[0] New IBM Cloud Orchestrator deployment.
[0] Demo
[1] All In One
[2] All In One IBM Cloud Orchestrator Deployment 
[0] All In One with addtional KVM compute node
   - Deploys IBM Cloud Orchestrator all-in-one KVM topology with an additional KVM compute node where components use a shared 
   - internal database.

And when it asks you for the FQDNs of the three servers, specify the FQDN of the ONE server that you'll be using in each of those three places.

A BIG note about running ICO 2.4.0.1

I successfully installed ICO 2.4.0.1 using 2 machines:

1: Deployment Server (where I could then run the 'ds wizard' command)

2: My all-in-one deployment to a KVM hypervisor.

Both machines are running RHEL 6.5 and have lots of CPU, RAM and disk.

My HUGE problem after install was that the Deployment Server automatically started an unnecessary dnsmasq process. And because of this, I could not create any new instances. Every time I tried, I got errors similar to the following in /var/log/nova/compute.log (and similar in network.log also):

2015-01-27 13:29:14.987 19521 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Remote error: ProcessExecutionError Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf env CONFIG_FILE=["/etc/nova/nova.conf"] NETWORK_ID=3 dnsmasq --strict-order --bind-interfaces --conf-file=/etc/dnsmasq.conf --pid-file=/var/lib/nova/networks/nova-br1002.pid --listen-address=10.10.1.3 --except-interface=lo --dhcp-range=set:franknet,10.10.1.3,static,255.255.255.0,120s --dhcp-lease-max=256 --dhcp-hostsfile=/var/lib/nova/networks/nova-br1002.conf --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro --domain=novalocal --no-hosts --addn-hosts=/var/lib/nova/networks/nova-br1002.hosts
Exit code: 2
Stdout: u''
Stderr: u"2015-01-27 13:29:13.983 10307 INFO nova.openstack.common.periodic_task [req-0d06a702-1f2a-4b95-a8f3-0f8665d4c83b None None] Skipping periodic task _periodic_update_dns because its interval is negative\n2015-01-27 13:29:13.985 10307 INFO nova.network.driver [req-0d06a702-1f2a-4b95-a8f3-0f8665d4c83b None None] Loading network driver 'nova.network.linux_net'\n2015-01-27 13:29:13.986 10307 DEBUG nova.servicegroup.api [req-0d06a702-1f2a-4b95-a8f3-0f8665d4c83b None None] ServiceGroup driver defined as an instance of db __new__ /usr/lib/python2.6/site-packages/nova/servicegroup/api.py:65\n2015-01-27 13:29:14.107 10307 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.6/site-packages/stevedore/extension.py:156\n2015-01-27 13:29:14.121 10307 DEBUG stevedore.extension [-] found extension EntryPoint.parse('file = nova.image.download.file') _load_plugins /usr/lib/python2.6/site-packages/stevedore/extension.py:156\n\ndnsmasq: failed to create listening socket for 10.10.1.3: Address already in use\n"
[u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n    incoming.message))\n', u'  File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', u'  File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', u'  File "/usr/lib/python2.6/site-packages/nova/network/floating_ips.py", line 193, in deallocate_for_instance\n    super(FloatingIP, self).deallocate_for_instance(context, **kwargs)\n', u'  File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 563, in deallocate_for_instance\n    instance=instance)\n', u'  File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 246, in deallocate_fixed_ip\n    address, instance=instance)\n', u'  File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 980, in deallocate_fixed_ip\n    self._teardown_network_on_host(context, network)\n', u'  File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 249, in inner\n    return f(*args, **kwargs)\n', u'  File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 1906, in _teardown_network_on_host\n    self.driver.update_dhcp(elevated, dev, network)\n', u'  File "/usr/lib/python2.6/site-packages/nova/network/linux_net.py", line 1004, in update_dhcp\n    restart_dhcp(context, dev, network_ref)\n', u'  File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 249, in inner\n    return f(*args, **kwargs)\n', u'  File "/usr/lib/python2.6/site-packages/nova/network/linux_net.py", line 1118, in restart_dhcp\n    _execute(*cmd, run_as_root=True)\n', u'  File "/usr/lib/python2.6/site-packages/nova/network/linux_net.py", line 1211, in _execute\n    return utils.execute(*cmd, **kwargs)\n', u'  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 165, in execute\n    return processutils.execute(*cmd, **kwargs)\n', u'  File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 196, in execute\n    cmd=sanitized_cmd)\n', u'ProcessExecutionError: Unexpected error while running command.\nCommand: sudo nova-rootwrap /etc/nova/rootwrap.conf env CONFIG_FILE=["/etc/nova/nova.conf"] NETWORK_ID=3 dnsmasq --strict-order --bind-interfaces --conf-file=/etc/dnsmasq.conf --pid-file=/var/lib/nova/networks/nova-br1002.pid --listen-address=10.10.1.3 --except-interface=lo --dhcp-range=set:franknet,10.10.1.3,static,255.255.255.0,120s --dhcp-lease-max=256 --dhcp-hostsfile=/var/lib/nova/networks/nova-br1002.conf --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro --domain=novalocal --no-hosts --addn-hosts=/var/lib/nova/networks/nova-br1002.hosts\nExit code: 2\nStdout: u\'\'\nStderr: u"2015-01-27 13:29:13.983 10307 INFO nova.openstack.common.periodic_task [req-0d06a702-1f2a-4b95-a8f3-0f8665d4c83b None None] Skipping periodic task _periodic_update_dns because its interval is negative\\n2015-01-27 13:29:13.985 10307 INFO nova.network.driver [req-0d06a702-1f2a-4b95-a8f3-0f8665d4c83b None None] Loading network driver \'nova.network.linux_net\'\\n2

Once I killed the offending dnsmasq process, I could successfully create instances.

SOURCE: http://fedoraproject.org/wiki/QA:Testcase_launch_an_instance_on_OpenStack