Why don't databases have foreign keys any more?

If you've looked at a database from a vendor lately (Tivoli included; some example products that include databases are CCMDB, TADDM, ITCAM for WAS, ITCAM for RTT to name a few), you'll notice that there are very few (or NO) relationships between the tables. What this means is that you can't get a nice Entity Relationship Diagram from them, and that makes it quite a bit harder to write reports using the data in them. 

So why is it this way?

More and more, software developers are using Object-relational mapping (ORM) components to abstract the software from the database. Here is a pretty comprehensive list of object-relational mapping software. What this software does is abstract the data relationships from BOTH the software AND from the database, putting that relationship information into various files that are used by the selected ORM implementation. The impact this has is that it is difficult as a consumer/user of the software to write reports directly against the database - because the first step is having to reverse engineer the usage of the tables in the database.

What can be done about it?

I can think of a couple of approaches:

1. The best solution I can think of is that you need to ask the vendor for ERDs for all databases that will be used to store collected metrics. 

2. Since there may be some valid intellectual property-related arguments against no. 1 above, the next best approach would be to ask the vendor for the SQL needed to produce specific reports.

3. If neither of the above works, then reverse engineering is the only approach left. I've had success in this area by turning up the debugging on the software and looking for SQL "SELECT" statements in the log files. 

Data visualization using Google Spreadsheets, Yahoo Pipes and Google Maps

OK, so this isn't a Tivoli-related post, but since a big part of what we all want to do is visualize data, I thought that this post that I found through reddit.com was an exellent description of how to use several free web-based utilities to gather and display data. Here's the link:

http://ouseful.wordpress.com/2008/10/14/data-scraping-wikipedia-with-google-spreadsheets/

Here is the author's summary:

So to recap, we have scraped some data from a wikipedia page into a Google spreadsheet using the =importHTML formula, published a handful of rows from the table as CSV, consumed the CSV in a Yahoo pipe and created a geocoded KML feed from it, and then displayed it in a Yahoo map.

I haven't come up with an implementation that gathers Tivoli data, but I can think of one:

The main thing you would need is to get HTML that can be consumed by Google Spreadsheets. One way to do this would be with a JSP (on WAS) or PHP (on IBM HTTP Server) or other server-side language that makes SOAP or direct database calls to retrieve data from TDW and display it in HTML. Once you have that, you could then follow the steps in the article to viualize the data.

Creating TPM Signatures for Multi_sz registry entries

When creating some software signatures for Windows the use of registry entries to identify an application/operating system on a system. If you check out many of the existing Windows signatures, you will see that they use registry entries extensively.

One issue I came across was around the use of multi_sz type registry entries. This came up when scanning a Windows server, the operating system was not defined. In order to define the operating system the following keys were needed:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProductName=Microsoft Windows Server 2003
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\CSDVersion=Service Pack 1
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ProductOptions\ProductType=ServerNT

And the multi_sz
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ProductOptions\ProductSuite=Blade Terminal Server

For this last key, the Blade and Terminal Server were on separate lines. After looking at other signatures that also used multi_sz, I could not see what was required to separate the lines, so I tried a space, \n and a couple others that never worked.

I then tried the dcmexport command and looked at other multi_sz definitions. They were useless as the character was not a normal ASCII character.

Then I found Windows signatures located in $TIO_HOME/eclipse/plugins/windows-operating-system/xml. In there I found the code & # 1 4 5 ; (remove the spaces) used in existing definitions. I then created the software signature, associated it with the Software Definition Windows Server 2003 Standard Edition SP1 (for lack of a better one for now), ran a scan and the operating system was recognized.

Hope this helps.

TCR Report Packages

Tivoli Common Reporting provides two ways to package your BIRT report design files viz. Report Design Format and Report Definition Format. Both have the same acronym, so we can't abbreviate them.  This article explains the two formats in detail.

Report Design Format

This is the simplest of the two formats. How do you create them?   Simply create a zip file of your BIRT project directory (note! you need to create the zip file of the PROJECT not the ReportDesign file).  That's it. In Windows, you can right click on the project directory and click Send To->Compressed (Zipped) Folder, there you're done!  Import the resulting zip file into TCR and it will import the files with .rptdesign extension as TCR reports. 

The Path to the report design files will be reflected in the TCR navigation tree.  For example, if your report design files are located in /GBSReports/Netcool directory, then the report designs in TCR will appear under GBSReports->Netcool in the TCR Navigation tree. 

Report Definition Format

TCR Documentation does not give good idea about Report Definition Format. Creating Report Definition Format involves creating an XML file that describes the report set structure. The advantage of this format is that you can create a well documented report set and it also provides the ability to share the same report design across multiple levels in the navigation tree.   Even though, this format helps to create a well-documented report set, ironically, TCR does not have a detailed documentation such as details about the XML Schema.   The OPAL ITM62 Report set uses the Report Definition Format and it should give you some idea, but that's all I have come across so far.

Overall, the Report Design Format should be good enough for most of our needs and it is much easier to change the Report set structure as well.  However, Report Definition Format gives more control over Report set organization and one hopes that more information about the format will be available in future releases.

And here's how to use extended attributes in TADDM

UPDATE: Since TADDM 7.1.2 is MUCH friendlier about extended attributes, I changed the title of this post. Details on reporting on extended attributes to come soon.

... you'll want to be able to populate them during discovery. And this requires the use of the SCRIPT: directive in your custom server template. The documentation on this is pretty sketchy, so that's why I'm including a full script here that will set the value of the "FrankCustom" custom attribute for a LinuxComputerSystem.

Prerequisites
1. You must create the extended attribute named FrankCustom for the LinuxUnitaryComputerSystem class (or any class in its inheritance chain).

2. You must enable the LinuxComputerSystemTemplate custom Computer System sensor. You can do this from the TADDM GUI, Discovery drawer, click on Computer Systems, and edit the LinuxComputerSystemTemplate to set it to "enabled".

3. You have to create a file named $COLLATION_HOME/etc/templates/commands/LinuxComputerSystemTemplate with the following line:

SCRIPT:/myscripts/myscript.py

If it's a Jython script you're calling (which it is in this case), the script filename MUST end with a ".py" extension. If you try to use ".jy", it will absolutely fail.

Now, you need to create the file /myscripts/myscript.py with the following contents:



import sys
import java
from java.lang import System

coll_home = System.getProperty("com.collation.home")
System.setProperty("jython.home",coll_home + "/external/jython-2.1")
System.setProperty("python.home",coll_home + "/external/jython-2.1")

jython_home = System.getProperty("jython.home")
sys.path.append(jython_home + "/Lib")
#sys.path.append(coll_home + "/lib/sensor-extensions")
sys.prefix = jython_home + "/Lib"

import traceback
import string
import re
import StringIO
import jarray

from java.lang import Class
from com.collation.platform.util import ModelFactory
from java.util import Properties
from java.util import HashMap
from java.io import FileInputStream
from java.io import ByteArrayOutputStream
from java.io import ObjectOutputStream


#Get the osobject and the result object
os_handle = targets.get("osobject")
result = targets.get("result")
system = targets.get("system")
# two more pieces of information available
#env = targets.get("environment")
#seed = targets.get("seed")

# Execute a command (it's best to give the full path to the command) and
# store its output in the variable named output
output = os_handle.executeCommand("/bin/ls -l /tmp")
s_output = StringIO.StringIO(output)
output_list = s_output.readlines()

# This loop just stores the last line of output from the above command in
# the variable named "output_line"
for line in output_list:
output_line = line
s_output.close()

#Get the AppServer ModelObject for the MsSql server
#app_server = result.getServer()

#Set up the HashMap for setting the extended attributes
jmap=HashMap()
jmap.put("FrankCustom",output_line)
bos=ByteArrayOutputStream()
oos=ObjectOutputStream(bos)
oos.writeObject(jmap)
oos.flush()
data=bos.toByteArray()
oos.close()
bos.close()

#Call setExtendedAttributes on the AppServer
system.setExtendedAttributes(data)


And there you have it. The examples you find in the documentation and on the TADDM wiki just won't lead you to any kind of success because they're missing some very key information.

Feel free to use extended attributes in TADDM

Update: TADDM 7.1.2 has massive improvements in data accessibility, so this post is no longer valid.


Don't let the documentation lead you astray - extended attributes in TADDM should be your LAST CHOICE for customization!

I realize that's a bold statement (especially since there's so much information about creating extended attributes in the documentation), but I'll explain why it's a valid statement:

1. You can't run any reports based on values stored in extended attributes! None. You can say, for example "give me all ComputerSystems where myCustomAttribute contains the string 'foo'". Just can't do it. The only call to get extended attribute values requires the GUID (or GUIDs) of the object(s) you're interested in. This is enough reason to stay away from extended attributes.

2. Extended attributes don't get imported into the CCMDB unless you do a LOT of customization to the TADDM integration adapter.

If you have custom data that you need to store in TADDM, what you need to do is become familiar with the CDM (Common Data Model) and the APIs available in TADDM. They are documented in three different archives that you have to extract (these all exist on your TADDM server):

CDMWebsite.zip - contains information about the CDM, including entity relationship diagrams.
model-javadoc.tar.gz - contains javadoc information for interacting with model objects.
oalapi-javadoc.tar.gz - contains javadoc information for accessing TADDM, from which you may just output XML, or you might obtain references to model objects, whose methods are defined in the above file.

If you're reading these files on your own workstation, make sure to grab a fresh copy from the TADDM server after each fixpack, as Tivoli does update this documentation.

By reading and understanding the documentation above, you can determine the appropriate object types to use to effectively store all of your data.

Getting the TADDM Discovery Client (aka TADDM GUI) to run when you have Java 1.5 and 1.6 installed

Getting the TADDM Discovery Client (aka TADDM GUI) to run when you have Java 1.5 and 1.6 installed

The TADDM GUI uses the Java Web Start technology (i.e. a JNLP file) to run. This allows the application to be launched as a full-screen application (rather than just an applet) over the web. If you ONLY have Java 5 installed , there shouldn’t be a problem. But if you have 1.5 AND 1.6 installed, you’ll probably get an error stating something similar to:

[SunJDK14ConditionalEventPump] Exception occurred during event dispatching:

java.lang.ClassCastException: java.lang.StringBuffer cannot be cast to java.lang.String

at com.collation.gui.client.GuiMain.setApplicationFont(GuiMain.java:511)

at com.collation.gui.client.GuiMain.doLogin(GuiMain.java:442)

at com.collation.gui.client.GuiMain.access$100(GuiMain.java:83)

The reason the problem occurs is that the .JNLP file states that it wants Java 1.5+, which means 1.5 or higher, so javaws will pick the highest one available, which is 1.6. But 1.6 isn't supported by the TADDM GUI. So here's how to fix it.

This file describes what has to be done to fix the problem. I apologize for the PDF link, but trying to get all of the screenshots uploaded was driving me crazy.

Java Info

Java versions are even more confusing than you think they are, so I wanted to clear up at least a little of it here.

Java 1.4.2 is also called Java 2 Platform Version 1.4.

Java 1.5 is also called Java 5

Java 1.6 is also called Java 6

Java 1.5 (aka Java 5) introduced a LARGE number of features that aren't available in version 1.4.2. And from what I've found, it's pretty difficult to have both Java 1.5 (or 1.6) and 1.4.2 installed on the same system. If you NEED to run some apps that require Java 1.4.2 (like the TEP desktop client, for example) and some apps that require Java 1.5, I would truly create a separate Windows virtual machine (using the virtual machine software of your choosing, though I would recommend VMWare) to run version 1.4.2.

After 1.4.2, however, it really appears (so far) that you can install the different versions on the same machine AND have them play well together. My next post will have some information about this.

If some piece of software states that it requires Java 1.5, that *should* mean that the JDK or JRE with a version number of 1.5.0_xx, where xx is the Update Number, will work; it's not guaranteed, but in my experience, it does NORMALLY work. Similar is true for Java 1.6 - version 1.6.0_xx *should* work.

What's the difference between the JDK and JRE

The JRE (Java Runtime Engine) is what most people need. It contains the java executable and all of the other executables that are needed to run Java clients, but it does NOT contain any of the executables needed to WRITE and package Java applications (i.e. javac and jar, among others).

The JDK (Java Developer Kit) contains all of the JRE, plus all of the developer tools needed to write and package Java applications.

What about SE and EE variants?

SE (Standard Edition) and EE (Enterprise Edition) are used to distinguish between types of developer environments.

SE is for creating client applications that will run inside a standalone JVM (Java Virtual Machine) on a user's local machine. Applets and Java Web Start applications, though they are accessed over the web, are examples of applications created using the SE developer kit.

EE contains all of the SE, PLUS it allows a developer to create applications that will run inside an application server, such as WebSphere, WebLogic, JBoss, Oracle App Server, and many others. These applications actually run on a JVM on a server. A user connects to that server to access the application, but the Java application itself is using CPU cycles on the server.

What's Java Web Start

Java Web Start (JavaWS) is a technology that allows application developers to create a web-launchable Java application directly from the browser. A Java Web Start application is defined in a .JNLP file (which is just a text file, so you can open one up to look at it), and is launched using the javaws executable. This is different than an applet in a few ways:

1. An applet generally runs inside the browser itself, or it can open a new window in which it runs *seemingly* outside of the browser. I say "seemingly" because if you close the browser, the applet will still die. Any windows opened by the applet are child windows of the browser and will automatically be closed when the browser is closed. A JavaWS application's control file (the JNLP file) is downloaded by the browser, but is then launched by the javaws executable. So if you close the browser, the application won't close.

2. A JavaWS application is a completely standalone application, which has full control over menus, windowing elements, look-and-feel, etc. of itself. An applet is constrained by the browser in several ways. (This difference is mainly interesting to developers, but I think it's useful for users to be aware of).

3. A JavaWS application can be launched outside of a browser; you can just double-click the javaws executable (or a .JNLP file, for that matter) to launch a JavaWS application. An applet MUST be launched in a browser.

Where to get Java

Sun Java:

ANY VERSION older than the current version: http://java.sun.com/products/archive/index.html

Current Update of 1.5: http://java.sun.com/javase/downloads/index_jdk5.jsp

Current Update of 1.6: http://java.sun.com/javase/downloads/index.jsp

IBM Java:
Links to different platforms and versions: http://www.ibm.com/developerworks/java/jdk/

Converting TDW Timestamps to DateTime in BIRT

We have previously published articles on how to convert TDW timestamps to "regular" timestamps in DB2. In BIRT, you may need to tackle the same problem again. The TDW timestamps can not be used for DateTime arithmetic and Charting functions in BIRT and has to be converted to regular DateTime. Though you can use the SQL to convert these timestamps to regular, in some cases, you may want to use the JavaScript integration of BIRT to convert them as well. Here is how to do it.

1. In your BIRT data set, ensure that you're selecting the TDW Timestamp that needs to be converted to the regular timestamp.
2. In the "Edit DataSet Window", click on Computed Columns and Click New button.
3. Give the column name for the new Computed Column, select the column data type as "Date Time".
4. In the expression field, click on the fX button and enter the following JavaScript code. Replace row["Timestamp"] with the appropriate Column name for the TDW Timestamp.

if (row["Timestamp"] == null)
{
null
}
else {
new Date(
(candleTime=row["Timestamp"],
(parseInt(1900) +
parseInt(candleTime.substr(0,3))) + "/" +
candleTime.substr(3,2)
+ "/" + candleTime.substr(5,2) + " " +
candleTime.substr(7,2) +
":" + candleTime.substr(9,2) + ":" +
candleTime.substr(11,2)
))
}

5. Goto the Preview Results and ensure that the newly Computed Column appears in the output.

This is the kind of method that ITM 6.2 built-in reports use and hope you find it useful.

CCMDB 7.1.1 Install Error: CTGIN2381E

When running the CCMDB 7.1.1 installer from a Windows system to install the code on a UNIX/Linux box (since the installer only runs on Windows, this is what you have to do if your server is UNIX/Linux), you may encounter the error:

CTGIN2381E: Maximo Database upgrade command failed

It turns out that this error message can be caused by lots of different issues, so the text of the error itself isn't necessarily very helpful. In my case, I found that the error was occurring because of an exception being thrown in the middle of the nodeSync() function of the DeploymentManager.py script (that is installed on your Windows deployment machine when you run the CCMDB installer). And what I found is that it just needed a sleep statement to allow the node manager to come up successfully. So the line I added (immediately after the "try:" statement at line 115) was:

lang.Thread.currentThread().sleep(30000)

This causes it to sleep for 30 seconds. You can probably set this timeout to a smaller value (esp. since the nodeSync() function is actually called quite a few times during the install), but 30 seconds worked perfectly for me.

Tivoli Common Reporting (TCR) Space on IBM DeveloperWorks

It's available here:

http://www.ibm.com/developerworks/spaces/tcr

It's got lots of good information and links on TCR. Also, you probably want to poke around a little to see if there's anything else on devworks that might interest you.

You can configure a laptop with 8GB of RAM

I recently got an 8GB kit for my laptop from:
http://www.compsource.com/ttechnote.asp?part_no=KVR667D2S5K28G&vid=229&src=F

and it works GREAT. I've got a Thinkpad t61p (Santa Rosa) running Vista Ultimate x64, and it sees all 8GB. Additionally (and most importantly), VMWare Workstation also sees all of the memory and allows me to allocate it to a virtual machine.

JDBC configuration in TCR v1.1.1

One of the important thing in configuring Tivoli Common Reporting server is the JDBC configuration. This step doesn't seem to be documented in the installation guide properly. The installation guide talks about modifying a python script but it is not needed and there is a simple way to get configure the JDBC driver.

The simplest way to get it working is to copy the JDBC drivers to a particular directory deep down in the TCR installation tree. The location is %TCR_HOME%\lib\birt-runtime-2_2_1\ReportEngine\plugins\org.eclipse.birt.report.data.oda.jdbc_2.2.1.r22x_v20070919\drivers

Once you copied the JDBC drivers to that location and recycled TCR, the driver classes will be available to TCR and you should be able to configure JDBC connections using TCR web interface (Right click on a report -> Edit Data Sources).

Hope this helps.

How to get GBSCMD?

Of late, there has been some confusion about how to get a copy of GBSCMD. GBSCMD is free and getting it is very easy. Just write an email from your company email address to Tony (Tony dot Delgross at gulfsoft.com) and ask for a copy. He will send a copy it to you. That's it. If you want some additional features to be included in it, please feel free to write to me. (venkat at gulfsoft.com).

Tivoli Common Reporting Info

DeveloperWorks has a great site devoted to Tivoli Common Reporting as part of their Spaces initiative. Check it out here.

Parsing the Download Manager file

When using the Download Director from IBM, a file is created called dlmgr.pro. This file is created in the directory that you download your files to. I have often found it annoying with the naming convention of files that I download to determine what a file is after I have downloaded it and left it for a while. I found that the dlmgr.pro file does contain pretty much all the info needed to identify the file information. So I thought I would create a quick perl script to parse the file. It is not the prettiest script, but it does what I need.

The script requires the path and file name to the dlmgr.pro file as an argument, and then creates a file called dlmgr.csv in the current directory.

Click here to download

TPAP/ISMP/CCMDB/Maximo/TAMIT/etc. info

Tivoli is now starting to roll out solutions based on Maximo, so I figured an overview, explanation of the names of things and list of resources would be helpful.

Overview

IBM Tivoli CCMDB 7.1 was the first product introduced by Tivoli based on the newly-revised Maximo platform. is now up to version 7.1.1, and there are now additional Maximo 7.1-based applications available, such as IBM Tivoli Asset Management for IT 7.1, IBM Maximo Asset Management 7.1, Tivoli Business Continuity Process Manager 7.1.0, IBM Tivoli Release Process Manager 7.1.1 and Tivoli Service Request Manager 7.1. (Notice how the naming isn't quite standardized; I imagine they will correct this in the future). All of these products are fundamentally based on the same core software and database, which was previously known as the "Maximo base software". This base software is now known by a couple of names, depending on which documentation you're viewing: Tivoli Process Automation Platform (TPAP) or IBM Service Management Platform (ISMP or SMP)

(I'll use TPAP as my acronym of choice from here on, because it seems to be used in the latest redbooks). TPAP generally consists of the supporting middleware (WAS 6.2, Tivoli Directory Server 6.1, ITUP Composer, DB2 9.2, Rational Agent Controller 7.0, IBM HTTP Server and others), the database schema, and some WebSphere-based applications. I say generally because you do have options to use non-IBM middleware, and other software may end up being used in your implementation.

TPAP is NOT available as a standalone product, similar to the way that Tivoli Framework was/is not available as a standalone product. By itself, TPAP does "provide you with the capability" to do several things, but its usefulness really comes through when there are additional components bundled with it. This is the case with all of the products listed above - they are shipped as TPAP plus something. Exactly what that "something" is depends on the product. If you purchase more than one of the above products, then you will only install TPAP once, then you will use product installer to add additional components on top of what you already have installed.

Example: CCMDB

As one example, when you install CCMDB, you get TPAP plus:

Change Process Manager - This consists of additional applications, roles, workflows, job plans, tools, reports and management controls to support the ITIL-defined Change process.

Configuration Process Manager - This consists of additional applications, roles, workflows, job plans, tools, reports and management controls to support the ITIL-defined Configuration Management process.

TADDM - A standalone application that is used to discover configuration items and their relationships, and this data is uploaded to the TPAP database through the IBM Tivoli Integration Composer.

Example: TAMIT

As another, slightly different example, when you install Tivoli Asset Manager for IT, you get TPAP plus:

customized TPAP-based applications, roles, workflows, job plans, tools, reports and management controls to support all of the functions surrounding the management of IT Assets (such as Contract Management, Procurement and Financial Management).


Tivoli License Compliance Manager - A standalone application that is used to gather inventory/asset information, which is then fed into the TPAP database using the IBM Tivoli Integration Composer.

Tivoli License Compliance Manager for z/OS - similar to above, for z/OS.

Installing Multiple Products

As you install additional products, you will see additional options in the Go To menu of the TPAP interface. This interface is accessible by pointing your browser to http://your_server/maximo. A large issue currently is finding documentation on the integration of the functions provided when you have multiple products installed. I imagine IBM will publish some documentation on this as the product suite matures, and I plan to add blog entries as I develop more generic solutions/opinions for the integration issues from the implementations we're currently performing. A good place to check for information from IBM is at http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=maximo.

Etc.

There are some other process managers listed in Tivoli's documentation, but note that ONLY those process managers at version 7.1 or above are actually part of the current TPAP. The others are generally version 1.1, which are note completely outdated.

You will see the ISMP Toolbox referenced in several places, and that can be found on OPAL here. It's really a great collection of documentation related to IBM's service management strategy.

Thanks to everyone who came by our booth at Pulse

The conference was much larger than it has been in years past, and we got a chance to visit with old friends and to make some new ones. And for everyone who didn't get to go, here's a picture of us with our booth.

From left to right: Tony, Venkat, IV, Martin, Frank, Jason

Sample Emailing Script

Here is a simple custom emailing script using Perl. You need to install Mail::Sendmail module from CPAN to get it to work. You also need to change the values of $mailhost and $from field values.

Here is a typical usage of this program.

/itm_email.pl somebody@gulfsoft.com "Sample Subject" "Sample message body"


#!/usr/bin/perl

# Send emails via this platform independent module
use Mail::Sendmail;

main();

sub main {
my ($to, $subject, $message) = @ARGV;
send_email($to, $subject, $message);
}

# Sends email using Mail::Sendmail module
sub send_email {
my ($to, $subject, $message) = @_;
my ($mailhost, $from, %mail, @to);

$mailhost = "smtp.mydomain.com";
$from = "itm_alerts\@mydomain.com";

%mail = (
Smtp => $mailhost,
From => $from,
To => $to,
Subject => $subject,
Message => $message,
);
sendmail(%mail);
}


Hope you find it useful.

TCM 4.2.3 FP06

Not sure how I missed this but FP06 is now out (since March 28).

Some interesting features:

1. Windows 2008 is supported as an Endpoint
2. AIX 6.1 is supported as an Endpoint
3. WSUS 3.0 support

There are also some other changes that you can read further about at ftp://ftp.software.ibm.com/software/tivoli_support/patches/patches_4.2.3/4.2.3-TIV-TCM-FP0006/4.2.3-TIV-TCM-FP0006.README

Tivoli System Automation for Multiplatforms Overview

TSAM is not a new product, but it's definitely one that is becoming more popular these days, especially with the introduction of the Balanced Data Warehouse, which relies on its capabilities. So I thought a general overview might be useful. This high-level overview doesn't go into the gory details, as those are very well documented in the product documentation and in several redbooks and redpieces (links at the end of this post). Rather, this overview is meant to be a little more explanatory than the existing marketing material.

TSAM competes directly with Veritas Cluster Services (VCS). The purpose of both products is the same: to provide high availability to resources. A large portion of the functionality of TSAM is provided by Reliable Scalable Cluster Technology (RSCT) component, which is included as part of AIX, and is available for Windows, Red Hat and SuSE Linux. RSCT, along with its prerequisites (such as General Purpose File System), provides the mechanisms that allow you to define resources and have them monitored. There is no graphical interface for the configuration of RSCT or TSAM, so all of the configuration must be done from the command line.

RSCT provides the ability to "harvest" resources in your environment, so it will find and identify all of your network adapters, disks, filesystems, etc. You don't have to define *those* components. RSCT also provides the *ability* to define arbitrary logical resources and the *ability* to define automations to react to changes in the environment. RSCT also allows you to group resources into Resource Groups, which allow you to manage multiple resources as a single logical entity. RSCT also allows you to define Relationships between resources, allowing you to specify, for example, that these two filesystems must ALWAYS be available on the same node. (Update: these features are actually provided by TSAM) RSCT also allows you to define actions that should be taken when different resource change states. However, if you wanted to leverage RSCT directly, you would have to do a LOT of detailed customization for each component that you want actually automate to make highly available.

This is where TSAM fits in to make life easier for you. You've still got a good bit of customization to do, but it's a LOT less that you would otherwise have to do with the base RSCT. TSAM sits on top of RSCT and provides the mechanism that actually allows you to define arbitrary resources, resource groups and relationships, and based on your configuration (or "policy"), TSAM configures all of the automation necessary to make your resources highly available among the nodes in a cluster.

Specifically, TSAM allows you define resources of type "IBM.Application" (these are your applications) for which you configure Start, Stop, and Monitor commands. TSAM also provides the feature that automates the reflex actions required when your resources change state (for example, when an application goes Offline). With the introduction of these features (on top of RSCT as the base technology), TSAM fully automates the high availability of your applications based upon your configuration.

Much (certainly not all) of TSAM is scripts that are wrappers for the underlying RSCT commands. So when debugging TSAM, you'll primarily use RSCT commands. This is important to note, since it means that you must be intimately familiar with RSCT to be able to successfully deploy and manage TSAM. And TSAM makes a point of not replicating the RSCT documentation, so many of the commands you'll use to configure TSAM are documented in the RSCT manuals.

TSAM also provides an Operations Console, which is implemented as a WebSphere application that plugs into the WebSphere-based Integrated Solutions Console. This is really JUST an Operations Console, in that it ONLY allows operator-level management of your environment (e.g. bring an application online or offline, view information). This console provides you with NO configuration capabilities. You need to create all of your Resources, Resource Groups, Relationships, etc. from the command line. But it's not all painful. TSAM also provides the capability to export your "Automation Policy" (essentially, your entire configuration) to an XML file, and also to import from the same file. So this allows IBM and others to provide sample automation XML files that you can download, modify for your environment, and add to your existing policy.

All of the above is provided by the TSAM Base Automation Component. TSAM also provides an End to End (E2E) Automation Adapter, which allows you to manage and automate actions to be performed on resources across multiple clusters. This E2E component supports not only TSAM clusters, but also HACMP, Linux clusters, and even VCS clusters on Solaris.

TSAM also provides a TEC EIF publisher so you can send events to TEC (or the Omnibus EIF Probe) to allow you to monitor the status of your TSAM environment.

Links:

Lots of Documentation and Downloads:
http://www.ibm.com/software/tivoli/products/sys-auto-linux/downloads.html

Version 2.1 Redbook:
http://www.redbooks.ibm.com/redbooks/pdfs/sg247117.pdf

Main Product Page:
http://www-306.ibm.com/software/tivoli/products/sys-auto-multi/index.html

Installing BIRT on Windows Vista?

Well, installing BIRT is a no-brainer.  Just download the BIRT All-in-one from eclipse.org and unzip it. Very simple, right?  Prepare for a little bit of surprise when you do it in Windows Vista.   While trying to unzip on my Windows Vista computer, Vista displayed the following error. 

 

The source file name(s) are larger than is supported by the file system. Try moving to a location which has a shorter path name or try renaming to shorter name(s) before attempting this operation".

 

This error occurs due to 255 character limit on file names and C:\users\administrator\downloads\birt-report-designer-all-in-one-2_2_2.zip takes about 75 characters! This does not leave much room for the files within the zip causing the above error.

 

The solution is very simple. Rename the zip file to something like b.zip and move it up to the root directory (C:\).   Now try extracting it, it works like a charm. 

 

And, don’t ask me how the heck this worked in Windows XP and NOT in Vista? J  

 

 

Some other useful VMWare tips

Since Venkat posted some nice tips for VMWare, I thought I would add a couple of my own.

Pre-allocate Disk
There are significant performance increases in pre-allocating the disk. Obviously this take more space, but it can really save having a fragmented image. This can also save on having to defragment all the time.

Split Disk
When building the image, make sure to split the disk into 2 GB files.

Run VM image from separate physical drive than the host
The host drive is going to be busy enough, if you have a second drive, use it! Also if you are using a laptop, there is a very good chance that the internal drive is a 4200 or 5400 RPM drive. These are really slow and to slow to run anything other than one VM that you are going to use Notepad on.

External Drive Solutions
I have tried a couple different drives running through USB, but this really is not much better than running on the host drive. USB is too slow for this type of use. You could use firewire, but not many x86 machines have firewire. One of the newer things to come out is eSATA (http://en.wikipedia.org/wiki/Serial_ATA#eSATA_in_comparison_to_other_external_buses). I purchased a 7200 RPM 2.5" drive with an eSATA enclosure and eSATA PCMCIA card for about $250.00. This will work without an external power supply, which is great

Here are the parts
- SEAGATE 160GB Momentus 7200.2 SATA Notebook HDD w/ G-Force Protection, NCQ, 8MB Cache
- Vantec eSATA 2-Port SATA PCMCIA Adapter (there is also an express card version)
- Vantec Nexstar 3 eSATA Aluminum 3.5" Enclosure, eSATA & USB2.0

This drive gets about 55Mb/s transfer versus my internal 5400 RPM drive at about 25Mb/s.

Image Snapshots
Snapshots are supposed to only cause a very minor amount of performance impact on the VM image, but if you don't need the snapshot, get rid of it.

Disk Space
Make sure there is more than 30% disk space available on the physical disk. There are many write ups on this topic that state when you drop below 30% free, there is significant slow down in performance.

Defrag
Make sure that there is a regular defrag of the physical drive along with defragmenting the VM drive.

A couple links for your viewing pleasure
http://www.virtualization.info/2005/11/how-to-improve-disk-io-performances.html

http://www.scribd.com/doc/281249/Vmware-server-tips-tricks-vmworld-2006

Hope this helps.

Martin

Some useful VMware parameters

If you are running VMWare server/workstation, you will find the following Vmware parameters useful.

mainMem.useNamedFile = "False"

This parameter is helpful if you have a slower disk like flash drive or drivers like ntfs-3g.

tools.syncTime = "TRUE"

Syncs the VM's clock with the host. Helpful to avoid time slips while running VMs.

MemTrimRate = "0"

Vmware, by default, trims the unused VMs memory and gives it back to host OS. This parameter disable the behavior. While this results in increased memory usage per VM, it also improves the VM performance.

Sched.mem.pshare.enable = "False"

Disables page sharing across VMs. This is again useful on systems with large amount of memory.

prefvmx.minVmMemPct = "100"

If you have a huge amount of memory, you might want to "pre-allocate" all allocated memory. It gives good performance boost but obviously limits the number of VMs you can run on the host.

Hope you find these parameters useful.

Strange errors with the Warehouse Proxy Agent if you have a bad initial configuration

I was just installing all of the pieces of ITM 6.2, ITCAM for Web Resources 6.2 and ITCAM for Response Time 6.2 on Windows with DB2 9.1 and I ran into a painful problem. The problem is that the WPA trace log had these errors:

(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxdbb.cpp,2360,"checkUTF8ClientEncoding") Database client encoding is not UTF8.
+47F29C88.0003 You need to set the OS environment variable DB2CODEPAGE=1208 for DB2 or
+47F29C88.0003 NLS_LANG=_.AL32UTF8 for ORACLE
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxdbb.cpp,1728,"createStatusLogTable") Table WAREHOUSELOG exists already
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxdbb.cpp,1735,"createStatusLogTable") Index WHLOG_IDX1 on WAREHOUSELOG exists already
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxdbb.cpp,1746,"createStatusLogTable") Index WHLOG_IDX2 on WAREHOUSELOG exists already
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxdbb.cpp,1757,"createStatusLogTable") Index WHLOG_IDX3 on WAREHOUSELOG exists already
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxbase.cpp,250,"setError") Error 219/3/-443(FFFFFE45)/0 executing SQLColumns
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxbase.cpp,266,"setError") Error "[IBM][CLI Driver][DB2/NT] SQL0443N Routine "SYSIBM.SQLCOLUMNS" (specific name "COLUMNS") has returned an error SQLSTATE with diagnostic text "SYSIBM:CLI:-727". SQLSTATE=38553
+47F29C88.0009 "
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxdbb.cpp,792,"initializeDatabase") Initialization with Datasource "ITM Warehouse" failed
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxsrvc.cpp,722,"testDatabaseConnection") testDatabaseConnection failed
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxsrvc.cpp,653,"setupExportServer") A retry will be attempted in 10 minute(s).
(Tuesday, April 1, 2008, 3:35:20 PM-{9AC}khdxsrvc.cpp,654,"setupExportServer") Please check the troubleshooting guide for any error not described above.

And that meant that no data was getting in the warehouse.

The root of the problem is that I had NOT set the environment variable "DB2CODEPAGE=1208" before I started installing, and at this point some tables were created incorrectly. To fix the problem, I had to:

1. Stop the WPA

2. drop the UTF8TEST and WAREHOUSELOG tables from the WAREHOUS database (don't worry - the WPA recreates them when it starts back up)

3. Restart the WPA

And at this point, all was well.

Customizing ITM 6.1 tep.jnlp to work with Java 1.5

The default tep.jnlp file works great in certain environments, and not at all in others. Basically, it works if you only have Java 1.4.2 installed, or if you have Java 1.4.2 and Java 1.5 installed. But things don't work so well if you have Java 1.6 installed. This change fixes that problem.

What I did was change the line that reads:

j2se version="1.4+" size="64m" args="-showversion -noverify"

to the following:

j2se version="1.5*" size="64m" args="-showversion -noverify"

If you save this file with a new name, you must also update the line in the file that references itself (i.e. change 'href="tep.jnlp"' to 'href="newfilename.jnlp"').

And I found that ONE of these two files (the original or the changed) will work for almost all users. They just need to try one, and if that doesn't work, try the other. The problem is that the Java 1.4.2 browser plugin doesn't support the changed syntax. So you can just tell your users to try the first, then the second.

Edit: If a user only has Java 1.6 installed, the changed file will work and will automatically download and install Java 1.5.

GBSCMD V3.5 new features

In the past, we have posted few articles about gbscmd. We have updated the tool with few additional cool features such as clearing offline entries, executing commands on the agents and refreshing TEC EIF Integration component. This article explains how to use these features.

Clearing Offline Entries

This is one of the long requested requirement. Whenever the servers are decommissioned from monitoring environment, one has to remove the servers by selecting "Clear Offline" from TEP. However, there are no CLI equivalent for this function in tacmd, which is a handicap if you want to automate the decommisioning process. You can use gbscmd clearoffline feature to perform this function from command line.

Executing commands on agents

I wrote an earlier blog article about this feature. This feature is very handy to run non-interactive commands on the remote monitoring system. However, this feature does not capture the output of the command.

Refreshing TEC EIF Adapter component

If your site relies on tecserver.txt for setting situation severities or updates the MAP files frequently, any changes to these files require hub TEMS restart. Instead, you can use the refresheif feature of gbscmd to cycle the EIF component alone.

Examples

The following example restarts a Windows OS agent.
gbscmd command --auth ~/itm61.auth --command "net stop kntcma_primary && net start kntcma_primary" --managedsystem Primary:ITM61:NT

The following example restarts a Unix OS agent.
gbscmd command --auth ~/itm61.auth --command "/opt/IBM/ITM/bin/itmcmd agent stop ux;/opt/IBM/ITM/bin/itmcmd agent start ux" --managedsystem uxitm61:KUX

The following example clears an offline entry, which can be handy when servers are decommissioned from the monitoring environment.

gbscmd clearoffline --auth ~/itm61.auth --managedsystem uxitm61:KUX

More Info

You can download the manual from here. Please contact Tony (Tony.Delgross at gulfsoft.com) if you have any questions or need a copy of gbscmd.

TPM SPB Import Automation Package

Posted by: martinc on Mar 06, 2008 - 04:07 PM

With TPM it is possible to import SPBs with the TCM integration, Software Package Editor or manually from the Web UI. I wanted to add another.

What happens if you have a bunch of SPBs that you want to import and you do not want to use the integration? Use the SPE or Web UI? That would take forever!

So I thought I would whip up a quick automation package to do just that. Click here to download the automation package.

The package is made up of 3 workflows

GBS_FindAllSpb.wkf
Function: Retrieves a list of all SPBs based on the FileRepository rootPath and the RelSearchPath

Arguments:

FileRepositoryID - This is the ID of the file repository to use for the search. Yes this could be changed to the name if you want. RelSearchPath - This would be the path to search relative to the FileRepository. For example: if you enter "/" (no quotes) this will search the rootPath of the file repository and all sub-directories for spb files if you enter "/ActivePerl" (no quotes) this will search the rootPath of the file repository and all sub-directories under ActivePerl for spb files Calls: GBS_GetSpbVariables.wkf GBS_GetSpbVariables.wkf Function: Retrieves various package attributes used to define the Software Module to TPM. Arguments: SpbRelPath - This is the relative path to the SPB found in GBS_FindAllSpb. SpbFileName - The file name to retrieve the package attributes from. FileRepositoryID - This is the ID of the file repository to use for root path Calls: GBS_ImportSPB.wkf GBS_ImportSPB.wkf Funtion: Using the variables passed in from GBS_GetSpbVariables.wkf, creates a XML file in the same directory as the SPB and then uses the built-in workflow called DcmXmlImport to import the XML file. Arguments: SpName - value retrieved from the field Name in the SPB SpVersion - value retrieved from the field Version in the SPB SpVariables - value(s) retrieved from the default_variables field in the SPB SpbRelPath - This is the relative path to the SPB SpbFileName - File name of the SPB FileRepositoryID - This is the ID of the file repository to use for root path Extra Notes It is also possible to just run GBS_GetSpbVariables.wkf if there is only one SPB to define. Just pass the required fields into the workflow execution and only that SPB will be defined With this template, it is very easy to make some changes that would suit your environment. Any comments or feedback is appriciated. Martin Carnegie

TPM 5.1.0.2 FP0003 and 5.1.1 FP0001 are available!

Posted by: martinc on Mar 05, 2008 - 04:54 PM

Fix Pack 3 for TPM 5.1 and Fix Pack 1 for 5.1.1 are now available! So what does it do?

The most important thing is that this allows for current 5.1.0 users to upgrade to 5.1.1. In order to do this, you must first be at 5.1.0 FP002. When the install is done, you will be at 5.1.1.1. Also brings the 5.1.1 to 5.1.1.1.

5.1.0 - FP0002 readme
5.1.0.2-TIV-TPM-WIN-FP0003.README.HTM

5.1.1 - FP0001 readme
5.1.1-TIV-TPM-WIN-FP0001.README.HTM

IBM Automation Package for IBM Tivoli Monitoring
There is supposed to be an updated automation package on OPAL, but I do not see it yet. Very curious to see what this one offers as the old one was, um, lacking.

VMware Virtual Infrastructure 3 support
I know that there have been a few people asking about this one

Automation package for management of large numbers of zones, regions, and depot servers
This is great! Nothing more tedious than manually creating these.


Here are the list of defects for each version
5.1.0
5.1.0-TIV-TPM-FP0002.DEFECTS.HTM

5.1.1
5.1.1-TIV-TPM-FP0001.DEFECTS.HTM

Unfortunately the one that is missing still is the Vista agent.

I will get these downloaded and see how the install goes. I am keeping my fingers crossed :)

Edit:
To download go to
Tivoli Provisioning Manager Fix Pack 5.1.1-TIV-TPM-FP0001


Tivoli Provisioning Manager Fix Pack 5.1.0.2-TIV-TPM-FP0003

TPM 5.1.1 Classroom Comments

Posted by: martinc on Mar 05, 2008 - 07:42 PM

I recently taught a custom class we developed for TPM 5.1.1 and thought I would provide some feedback on TPM 5.1.1

Performance
Well first of all, I just want to say a word on performance. I taught almost the same class on 5.1.0 FP01 and 5.1.1 using all the same hardware and I was impressed with the performance. I had some students in this class from the class I taught a year ago on 5.1.0 and they stated that everything was running faster. In fact I had more students this time and there was almost no lag when running various tasks.

Comments on the use/functionality
Web Replay - although this was not really part of the class, we had some extra time to look at it. We felt that the tool was interesting and probably good as a training tool and possibly for some tasks. Generally speaking, I (or someone that has been using the interface for a while) can manage to get things done faster than Web Replay. This is still a good addition though.

Computer information
The layout of the Computer information is very clean. The General tab displays a very good summary of important information about the computer.

Compliance and Remediation (also applies to patching)
The 3 step process layout makes the creation of a compliance check very easy.

Here is a snapshot of the Compliance tab for a group (Click on the image to see a larger one).



Notice the easy Step 1 2 and 3. Also under the Step one there is a quick checkbox to add the OS Patches and Updates Check. Simple!

Discovery
The integration of SMB and SSH discovery in one discovery configuration is also a biggy. Seems minor, but it is a pain to set up two seperate discovery scans that hit the same ip addresses.

Another nice feature in the discovery is that it will populate the operating system name. To do this in previous versions, you either had to run an inventory scan or manually add it. This saves some time.

Depot (CDS) Installation
In 5.1.0 in order to install a depot on a system that was already a TCA, you would have to first uninstall the TCA and then run the depot install. In 5.1.1, this has been fixed. The install will recogize that the TCA is already installed and only install the depot subagent.

SOA Rename
One fun thing while developing the course was that I noticed that everything was changed from SOA to SDI. SDI is Scalable Distribution Infrastructure. This name sure makes a lot more sense for the function than SOA.

Other
Just some Acronyms for you to make your day.

CAS – Common Agent Services
CDS - Content Delivery Service
CIT – Common Inventory Technology
DCM – Data Centre Model
DMS – Device Management Service
JES – Job Execute Service (also referred to as DMS)
SAP – Service Access Point
SDI – Scalable Distribution Infrastructure
SIE – Software Installation Engine
SOA – Service Oriented Architecture
SPE – Software Package Editor
TPM – Tivoli Provisioning Manager
TPMfOSD – TPM for Operating System Deployment
TPMfOSDEE – TPMfOSD Embedded Edition
TPMfSW – TPM for Software
WSDL – Web Services Definition Language

ITM 6.2 Workspace Parameter - more portal tuning

Posted by: jlsham on Mar 01, 2008 - 05:00 AM

itm61
If you have more than 200 of any type of agent, you start to see messages in your views about the number of systems being queried is too high.
Well, in your cq.ini - look for

#KFW_REPORT_NODE_LIMIT=xxx

Uncomment this line and set it to a more meaningful value for the number of agents you need to query.

Restart the portal and you're done.

Modified applet.html file to resolve one TEP IE browser connection error.

Posted by: napomokoetle on Feb 18, 2008 - 05:11 PM

This short blog is related to the blog I wrote a while ago title "Upgrade from ITM6.1 FP6 to ITM6.2 may break logon through IE TEP".
I had not posted the applet.html file I'm posting herein with the original blog I wrote on the problem because IBM/Tivoli had insisted the problem was unique to the environment I was working on at the time.




Well I guess the error has proven not to be unique to the environment I was working on since other folks keep seeing the same problem and requesting I pass them the modified applet.html file. Posting the file here will save me from searching all over for the file every time I get asked for it by those who would like to try it out.

TO RECAP:


Environment:
Upgraded from ITM6.1 FP6 to ITM6.2 RTM:

TEPS & TEMS installed on same W2K3 (Dual 3.06 GHz Intel Xeon (Hyper-threaded0) host with 3808MB ram.

PROBLEM: From an XP remote host, TEP in IE 6.0.2800 browser reports “cannot connect” error.
When I clicked on the Java WebStart created icon I got the same error as in IE browser.

Plugin150.trace reports:

(475405aa.0c474f80-(null)AWT-EventQueue-2:Bundle,0,"Bundle.Bundle(String,String,Locale,String)") Resource bundle: id = kjri, baseName = candle.kjr.resources.KjrImageBundle, actual locale used:


(475405b4.10ed7f00-(null)Thread-10:DataBus,0,"DataBus.make()") EXCEPTION: Unable to create DataBus object: org.omg.CORBA.INITIALIZE: Could not initialize (com/borland/sanctuary/c4/EventHandler) unexpected EOF at offset=0 vmcid: 0x0 minor code: 0 completed: No


(475405b4.10ed7f00-(null)Thread-10:QueryModelMgr,0,"QueryModelMgr.QueryModelMgr()") EXCEPTION: InstantiationException --> Unable to instantiate DataBus object


(475405b4.10ed7f00-(null)Thread-10:QueryModelMgr,0,"QueryModelMgr.make()") EXCEPTION: InstantiationException --> Unable to instantiate QueryModelMgr object


(475405b4.10ed7f00-(null)Thread-10:LogonDialog,0,"LogonDialog.processOK()") EXCEPTION: Unable to connect to CandleNet Portal server: java.net.ConnectException


(475405b6.33611380-(null)AWT-EventQueue-2:CNPClientMgr,0,"CNPClientMgr.terminate(CNPAppContext,boolean)") Performing normal exit of TEP Workplace


(475405b6.33611380-(null)AWT-EventQueue-2:UserManager,0,"UserManager.loadPermissionXRef()") Error loading User Permission Cross-Reference tables: KFWITM219E Request failed during creation.

---

Solution:
After some multiple cycles troubleshooting the error with IBM, they eventually made a couple changes to the applet.html for me to try out. Please make a backup copy of your current applet.html, and then replace the version found in your ..\cnb directory with the attachment on the link below. If this makes no difference in your testing, then you will need to open an ITM 6.2 PMR with IBM/Tivoli so that support can work this issue to a successful conclusion with you.

Click here to download the modified applet.html

Adios,
J. Napo Mokoetle

Fine Tuning the Portal Server

Posted by: jlsham on Feb 12, 2008 - 11:34 PM

The following are a few of the parameters used to fine tune the Tivoli Enterprise Portal, the most popular one is the expansion to show more than 25 systems in the navigator.
Tuning parameters can be set in the cnp.bat or cnp*.sh, these are located in the

Windows:
\CNP

or

Linux/Unix

../../cj/bin/cnp*.sh


For example: /opt/IBM/ITM/*interp*/cj/bin/cnp.sh

cnp.databus.pageSize = #of rows to get for any workspace table. Default is 100 rows.

cnp.navigator.branch.pagesize = Navigator Expansion - this is the popular one, default value is 25.

cnp.navigator.branch.threshold = Warning threshold for Navigator branch requests. Default value is 100.

Simple Postemsg-like solution for ITM 6.x

Posted by: venkat on Jan 23, 2008 - 09:44 PM

One of the most convenient features of classic Tivoli event adapter is the postemsg command. Using this you can send custom events to TEC from your scripts/CLI. In ITM 6.1, there is no such equivalent command available though one can design such a solution in just three steps. This article discuss a way to setup postemsg like solution in ITM 6.x

Overview

The idea is to use gbscmd sendmessage command to write a custom message to ITM Universal Message Console and develop a situation to forward any messages written to Universal Message Console.

Step 1: Create a situation

Now develop a simple situation to forward Universal Message Log entries with the following formula. Hint. Create a situation under "Tivoli Enterprise Monitoring Server".

(Originnode == AND Category == GBSMSG)

It is important to include "Originnode" in your condition. Otherwise, the situation will not fire. Distribute the situation to *HUB. Associate and start the situation.

Step 2: Write to Universal Message Console using Gbscmd

You can send a message to Universal Message Console using the following command.

gbscmd sendmessage --auth --message "some test message" --category "GBSMSG" --severity 2

Step 3: Verify the results

Ensure that the message appears in Situation Event Console and if you have Netcool/TEC Integration enabled, the message should also appear in the respective consoles.

Posted by Venkat Saranathan

Testing database connectivity using JDBC

Posted by: venkat on Dec 28, 2007 - 09:20 PM

Many a time, you might want to test if you could connect to a database or not. The simplest way is of course having a database client installed and use it. However, this method may not be possible in all cases. For example, on an ITM warehouse proxy, you'll have only Db2 JDBC type 4 drivers, (which are nothing but two files, db2jcc.jar and db2jcc_license_cu.jar). How do you test connectivity on such systems? Here is a Jython script.

Why Jython?

In addition to the self-serving opportunity of me learning Jython, Jython can reuse the same JDBC drivers that TDW uses. So there is no need to setup Perl DBI or database client. We could write the code in Java, but that's an overkill because of compilation requirements. Setting up Jython is very easy. Please refer to this page for detailed installation steps.

Code

Here is the sample code. Just be sure to change the connection settings.

Note: Do NOT cut-paste the code from below listing. Jython, like python, is sensitive to tabs. So, download the code by clicking here.

from com.ziclix.python.sql import zxJDBC
import java.sql

conn = "jdbc:db2://server:50000/warehous"
user = "user"
passwd = "passwd"
driver = "com.ibm.db2.jcc.DB2Driver"

try:
db = zxJDBC.connect(conn, user, passwd, driver)
print "Connection successful"
except zxJDBC.DatabaseError, e:
print "Connection failed:", e

TPM 5.1.1 Documentation

Posted by: martinc on Dec 21, 2007 - 04:55 PM

IBM has provided a document on the components to download for install and updated the Info-Center
Download information http://www-1.ibm.com/support/docview.wss?rs=1015&uid=swg24017302

There is information for all the OS platforms, you just have to scroll down. For some reason it is not quite set up like previous pages. Oh well.

Info Center: http://publib.boulder.ibm.com/infocenter/tivihelp/v20r1/index.jsp

TPM 5.1.1 is now available

Posted by: martinc on Dec 21, 2007 - 02:46 PM

The Christmas present everyone was waiting for :)
So what is new and improved?

The biggest focus on the new version was the installer. Anyone who has tried to install TPM 5.1 knows that it just did not work. I know that installs for me were < 50% successful on the first try. Even with the exact same VM images from attempt to attempt. In the new version, I did not have a failure on the 4 times that I installed. Sweet!

There is also more support for different topologies. In the previous version, getting TPM to use a remote database was a post install step. Now it is part of the installer.

Some of the new features are:
Agent less inventory scanning - there is no longer a requirement to have the agent installed on a system to perform an inventory scan. The scan will copy the required files to the target, initiate the scan and return the results to the DCM

Web Replay - This was available in a "beta" in 5.1 but is now part of the install. Web replay allows for some automation of tasks. Some tasks within TPM take many steps to complete and for the most part do not require any input. A recording of the steps can be done to "automate" some steps and stop at prompts for others.

Unknown device management - When discovering devices and an error is encountered, the device is dropped and not recorded in the DCM. Now the device will be recorded but will require some manual steps to complete the discovery. This is useful when a device is discovered but the user name and password are not correct. A new discovery can be done with the correct information on the unknown device.

Enhanced Discovery - When discovering SMB or SSH devices, the discovery can be done in one discovery configuration rather than two separate discovery configurations. This is mainly a time saver so that the same subnet does not need to be scanned twice.

There have also been many changes in the user interface to improve performance and readability.

My Comments
So far I have been pretty impressed by the installer and some of the changes made to the interface. I know that the agent less inventory scanning has been something that people were asking for and it does work! As I said before, the installer did work very nice. One of the big changes in the install is that by default it only uses local OS authentication (much like the fast start install). There is documentation and scripts to allow for easy configuration of TDS or MSAD (read only).

I will get some screen shots and other comments in the new year.

So Merry Christmas and Happy New Year!

Martin

Tivoli Common Reporting (TCR) uses the ZeroG InstallAnywhere installer

While many Tivoli products use the InstallShield Multi-Platform (ISMP) installer, TCR uses Zero G installer.

This little difference is important if you want to reinstall the product. Specifically, ISMP uses the vpd.properties file as its installation registry. Zero G, on the other hand, uses a file named .com.zerog.registry.xml (notice that it begins with a dot, so it's a hidden file). So if you need to delete the product and start over, you need to edit or delete this file, also. If you only remove the install directory, when you run the installer, it will just tell you that Tivoli Common Reporting is already installed, and won't reinstall it.

Using BIRT for TDW Reporting

There has been alot of interest in BIRT lately with the release of Tivoli Common Reporting 1.1.1 to GA, so I'm posting a link to part of my TUG presentation on using BIRT..

TUG BIRT Presentation

Actually searching all files when searching for text

For quite a while I have always been annoyed by the search in Windows. I thought it was great that you could use the search to look for text inside a file, just like grep. Of course in usual Microsoft fashion, search all files does not mean to actually search all files.

So finally I became so annoyed with this, I went to see if there was a solution.

I did find the solution and I thought I would share the link
Using the "A word or phrase in the file" search criterion may not work

Hope this helps someone.

Where Wizards Fear To T"h"read

Since PERL is a tool of choice for many of us Tivolians when it comes to automations and integrating systems with Tivoli products, I thought it could be of help to others to throw out a few gotchas I've encountered programming Perl threads. Please note that I'm not referring to "forking".

Though Perl threads can help speed up things in a Tivoli automation or integration script, they can also add headaches and frustration to ones life. The following three pointers should help reduce some of the strive...

1. Using nested data structures with PERL THREADS.

When using threads it is usual to share data. Sharing complex data structures works very well for a single threaded (single process) PERL script. But once the main process starts producing child threads, then hell breaks loose with the complex data structure as PERL can as of the writing of this document, deal with ONLY one level deep references to a SHARED data structure like an array or hash.
If you encounter and error like:

Invalid value for shared scalar at ...

then you've nested too much. Keep your reference to shared arrays and hashes to one-level down and you'll be ok.


2. Creating an ALARM signal handler within a child thread.
Often in Tivoli a person will call "w" command/s within a PERL program to perform some operation on a Tivoli resource. But it often happens that a "w" command hangs or takes too long to complete. What many often do is create an ALARM to timeout the command so the process continues execution when some specified time has elapsed. This usually works OK in a single thread program, but things start going out of hand when there's more than one thread executing in a PERL program. What happens with threads is that instead of timing-out the particular intended call within a CHILD thread, the whole PERL program quits executing! That is the main thread and its children threads die!!! The one work-around I've always found effective is to include code that looks as follows in the MAIN program flow. Make sure this code appears in the main thread and NOT in a child-thread.


$SIG{ALRM} = sub {}; #Just here to workaround the threads "Alarm clock" bug


That amazingly does the trick!


3. Handling the CTRL C interrupt within a threaded Perl program.
To avoid resource contention, often a person would prevent more than one instance of a PERL program from running by creating a lock-file at the begin of the program and then removing the file just before end of the program. But a problem comes when for some known or unknown reason, the PERL program receives an INT signal and terminates after creating the lock-file, but before executing to a point where the lock-file gets removed. Thus preventing any subsequent run of the program. It's easy to circumvent this situation and handle the signal in a single thread PERL program, but it can be a pain doing like-wise in a multi-threaded PERL program where a simple $SIG{INT} may do anything but catch the interrupt. For instance the norm would be to do something like:

$SIG{INT} = sub { "code that removes the lock-file"; die};

Trying to handle the interrupt this way in a multi-threaded PERL program may actually result with the program dumping core. Including the following chunk of code/subroutine most often does magic and beautifully handles the interrupt signal without fail:



use sigtrap qw[handler sig_handler INT TERM QUIT PIPE];
use Swith;

sub sig_handler {
my($sig) = shift;
print "Caught signal $sig";

switch($sig) {
case ["INT","TERM","QUIT"] {
unlink "/tmp/dashlock";
print "Exiting";
exit(0);
}
case "PIPE" {
print "Continuing";
}
}
}



I'm sure the above is not all there is on PERL threading gotchas, I however hope the pointers save you time and heart-ache should you have to deal with Perl threading in the near future and encounter similar issues.

That's all folks!

Adios,
J. Napo Mokoetle
"Even the wisest man is yet to learn something."

Eclipse plugin to access the TADDM API

I realize that this won't be interesting to many people, but I wrote an Eclipse plugin to access the TADDM API.

You can download the plugin from http://www.gulfsoft.com/downloads/TADDM_API_plugin.zip . The plugin requires Eclipse 3.3 and the Web Tools Platform.

To install the plugin:

1. Close Eclipse if it is running
2. Extract the TADDM_API_plugin.zip file into your eclipse directory. It will place the file com.gulfsoft.wst.xml.ui_1.0.0.200711131411.jar in your plugins directory.
3. All of the other help for the plugin is available from>Help->Help Contents in Eclipse. The help is found under the topic TADDM API Plugin.

ITM AIX Premium agents - an Overview

I recently got a chance to implement AIX Premium agents for one of our customers in a production environment. This article briefly discusses about our experience with these agents and also discusses about the pros & cons of these agents.

Installation

Installation of these agents is similar to Unix OS agents. There is a minor gotcha though. IBM currently offers "System P" SE edition as a free download with 1 year support, but don't confuse them with the "AIX Premium" agents which are available for current ITM customers. The "System P SE Edition" consists of agents for AIX Base, VIO and CEC whereas the "Premium" one consists of "AIX Premium", CEC and HMC agents. (Check with your IBM Sales rep about your entitlement).

Use the "AIX Premium agents" C101HIE.tar from Passport advantage and installation is very similar to usual agent installation. Make sure that you install the agent support files on your TEMS/TEPS servers.

What is the difference?

So what is new with "AIX Premium" agents? Of course the workspaces are different and the attributes provide AIX specific information such as LPAR, entitlement, Volume Group information and paging space that are NOT available with the generic Unix OS agents. You should be able to define situations to monitor these resources just like you would do for Unix OS Agents. This information could be very useful for AIX administrators.

You can find more information about this agent at the IBM Information Center for AIX Premium agent.

Rollout Considerations

The advantage of getting AIX specific information is really nice and most of admins would like it better than the Unix OS agents. However, there are couple of factors that you might want to look into before deciding whether to go forward with System P agents. One thing is the level of support and fixes available right now. Currently the UNIX OS agent is part of the "core" ITM release strategy and gets updated with every fixpack whereas AIX Premium agent is pretty much like the special type of agents such as DB2 agent, SAP agent etc. Since this is a fairly new type of agent, we don't know whether it will be integrated into IBM Fixpack release strategy.

One other issue is the added complexity of managing another type of agent. If you are happy with the current UNIX OS agent, then you could probably experiment it in your test environment and see if you need the features of AIX Premium agents.

An Introduction to Netcool Omnibus Components

We have already discussed lot of things about Omnibus in our previous articles but here we are going to explain the basic components of Netcool Omnibus and their functions.

What is an ObjectServer?

ObjectServer is the central piece of Omnibus and can be considered analogous to the TEC Server. The ObjectServer receives events from various probes
and processes them according to automation rules. The ObjectServer, however, is a light-weight process when you compare with TEC. Moreover, Omnibus provides
easy to use high availbility features out of the box, so it is normal to have two or more ObjectServers running in an environment.

What is a Probe?

Probes can be thought as an equivalent of Tivoli Event Adapters. They receive events from various sources, such as SNMP agents, Unix log file, etc. Omnibus
probes are much more intelligent than TEC event adapters in the sense, it can detect/filter duplicate information at the source and it is capable of advanced programming capabilities such as looping, regular expressions, etc.

Gateways

Gateways in Omnibus are different from the Gateways in Framework. In Omnibus, they act as a bridge between an Object Server and other ObjectServers, third-party applications or databases. Their function is to replicate ObjectServer data from one ObjectServer to other components such as another Object Server (for redundancy), or to third-party applications such as Remedy (for trouble-ticket management), or to databases such as Oracle (for event storage).

License Server

License Server is a central piece of Omnibus environment that dispenses necessary server and client licenses as and when requested. Without the license server, the Netcool components will not start. If the license server goes down AFTER the component is started, it should be back up within 24 hours or the Netcool components will shutdown. There are High Availability features supported for License Server and most of the time it is pretty stable. However, IBM understands the pain of managing one more component and ever since the acquisition, IBM is focussing on moving away from license server and expect this to be out in future releases.

Process Agent

The Process Agent plays a very important role in the overall architecture. It is responsible for starting and stopping Netcool components and it also restarts these components in case they died abnoramlly. Process agents are also responsible for executing automation requests coming in from remote systems. However, ironically the Process Control Agent does not manage processes on Windows, it is used only for executing requests received from remote systems. The process control agent works very well and it is very nice to see all the components started and managed by a single program. May be ITM could take a leaf out of it and use a similar solution!

Proxy Server

The Proxy Server is an optional component that can be used for better scalability and for firewalls. The Proxy Server acts as a proxy for ObjectServer, receives events from various probes and sends them over a single connection to the real ObjectServer. This will reduce the number of connections that the real ObjectServer has to handle. An equivalent in Tivoli world, Gateway comes to my mind!


These are the basic components you should know about. Stay tuned for more articles on Netcool Omnibus in the coming days.

Situation Status Field in ITM6.1

The situation_status field in TEC uses single letter status to denote the current status of the situation. Understanding the different values of this field is important if you need to write TEC rules for incoming events so that you don't end up taking multiple actions for the same event. This article lists the different values for Situation_Status field. The article credit should go to IV Blankenship.

According to IV in one of our earlier blog articles, the following are the valid values for situation_status field.

N = Reset
Y = Raised
P = Stopped
S = Started
X = Error
D = Deleted
A = Ack
E = Resurfaced
F = Expired

Here is the link to the earlier blog aricle/comments.

ITM FP05 Fixes & Omnibus Fixpacks

There are few interesting patches released at Tivoli Patches site including an interim fix for ITM 6.1 FP05 and a fixpack for Netcool Omnibus/Webtop. Here are the links to readme files.

ITM Interim Fix to Fixpack 05

Netcool Omnibus Fixpack 03

Thanks to Martin Carnegie for bringing this to our attention.

Using the APDE with JDBC Type 4 drivers

The APDE (Application Package Development Environment) can either be used on the TPM server or can be installed on a remote computer for the development of workflows. In Fix Pack 2, documentation was provided on how to configure the APDE to use JDBC Type 4 drivers instead of having to install the DB2 client on the remote computers. The problem was that this documentation jumps all over the place. So I thought I would document the steps I used to install the APDE and now I will share them with you.

Background
The APDE is the development environment in TPM for building custom workflows (and other things). Even though you could develop these workflows in the Workflow Editor or even notepad, the APDE provides an excellent IDE that allows for easier creation of custom workflows.

Steps required for DB2 connectivity on Windows
These instructions are for installing the APDE on a Windows system. The same principles should apply for other OSs.

1. Install Java 1.4.2+ confirm that the environment is setup by opening a command prompt and type java -version (Download from www.java.com
2. Download Eclipse from www.eclipse.org. This has to be 3.1.2. Version 3.2 does not work. (Download from www.ecplise.org
3. Extract the eclipse zip file to C:\Program Files (actually directory could be anywhere)
4. Copy the apde.zip and apde.upgrade.zip to remote computer
5. Extract apde.zip and apde.upgrade.zip
6. Copy/move the contents of apde.zip to the eclipse directory and overwrite existing files
7. Copy/move the contents of apde.upgrade.zip to the eclipse directory and overwrite existing files
8. Create a directory under the C:\Program Files\eclipse directory called TPMCFG
9. Copy the files db2jcc.jar, db2jcc_license_cisuz.jar, db2jcc_license_cu.jar from DB2_HOME\java on the TPM server to the TPMCFG directory
10. Copy the crypto.xml and dcm.xml files from TIO_HOME\config to the TPMCFG directory
11. Edit the new dcm.xml file and modify the file to the following values:
...



...
jdbc:db2://:/tc

The server name will be the name of the server where your TPM TC database resides (should be the TPM server). The db2 port is the port db2 is on, most likely this is 50000 or 60000 (this seems to happen more on Linux)
12. Run the eclipse.exe from C:\Program Files\eclipse
13. Go to Window -> Preferences -> Automation Package -> Database
14. Press the Import button and select the dcm.xml file from the TPMCFG directory. Confirm all the settings are correct. I had to modify the password as this did not seem to import correctly.
15. In the Import driver, I had to select the db2jcc.jar file and press the Add button or I received db connection errors
16. Press OK and restart the APDE

If you want to set up the APDE to allow for dynamic installs of the automation packages. The Deployment Engine information needs to be configured. Go to the Deployment Engine section and change the Host Name to the TPM server name. The remote and local directories will also need to be set.

Now it should be all good to go!

Martin Carnegie

Tivoli Provisioning Manager - Fix Pack 2

I am currently going through the Fix Pack 2 install for TPM (should be the same for TPM for Software) and thought I would make some notes.

The fix became available on June 3rd, but I was too fast on the draw and found out that the files were not uploaded totally. Oops. Then on the 5th, I saw that the files were now quite a bit bigger. Each of the files for the various OSs are about 1.8GB, OUCH!

For the patch information, go to http://www-1.ibm.com/support/docview.wss?uid=swg24016022&loc=en_US&cs=UTF-8〈=all

The install was quite easy and I did not see any errors. One thing to note, make sure you are logged on as the tioadmin account. This is a lesson I learned when installing FP01. I know the docs say this, but I thought that as Administrator (yes only Windows so far) I should be able to do it. Boy was I wrong. At some point the install of TPM changes the security to various files that does not even allow Administrator access.

So what is new?

Endpoint Tasks
This seemed to be something missing from GA and FP01 that was available in TCM. This allows for the creation of a Task much like we saw in TCM. In fact I think that it is much easier. The basic steps are:
1. Create a script and put it in the LocalFileRepository (and a sub-directory if you want)
2. In the TPM web interface, go to Task Management and there is a new section called Endpoint Tasks.
3. Select Edit -> Create Endpoint Task
4. Enter the task name and description
5. Select the files to send from the repository. You can define different files for different operating systems in the same task.
6. Define and parameters for the task
7. View the summary and then press Finish and the task is ready to go!

One thing to note, this only works if SOA is enabled. In my quick search of the docs, I did not see this as a requirement, but the message I saw when I tried to send without SOA enabled sort of pointed to the problem. The message was "COPDEX123E The workflow threw a NotSupportedInDE exception. The message is This workflow is a template workflow for TMF and TCA EndPointTask only."

"Automatic" agent upgrade
In previous versions, upgrading the TCA was a bit of a pain. Usually it was better just to remove the existing agent and reinstall with the new version. This would obviously be a nightmare in any real environment. So now there is a vorkflow called "TCA_Agent_Upgradeable.wkf" that can be wrapped as a Favorite Task that allows for the upgrading of agents. For the couple workstations I tried this on, the upgrade worked. For more information on the process check out Upgrading the common agent" on Info Center

Patching for HP-UX and AIX
I have not been able to try either of these, but I thought I would list them anyway. I know that AIX patching was there before, but according to people I talked to at the TTUG, it did not work. I was told that FP02 is where you have to be to make this work. HP-UX patching is new though and I am sure that people are looking for this.

Java Web Start for the Software Package Editor (SPE)
This is a nice improvement. Now you do not need to have Eclipse installed on your computer to use the SPE.

You do require Java to be install with the minimum level of 1.4.2. Once Java is installed, open your web browser and go to "https://:9045/SPEWebStart/spe.jnlp". This will start the Java Web Start, download the required files and then start the SPE. Once the SPE is started, you will have to go to the Settings -> Preferences to configure the Web Server Name, port and path. You will also have to configure if you are using SSL and default paths to use. Check out Launching and configuring the Software Package Editor on Info Center for more information.

New SOAP commands for Packaging
Some new SOAP commands were created to allow for the creation, distribution, install and uninstall of a software package. There does not seem to be much information in the docs about this yet. The only one I have found is around creating the SPB. I will keep looking and post more later.

Unified Patch Management
You can now manage patches in TCM from TPM and be rid of that "Automation Server" that TCM used. This is a possible scenario that I can see using TPM as a first entry point. I personally think that the patch management facility works fairly well in TPM. The one that came with TCM was a real pain to install and use. So by implementing TPM and importing your TCM environment into TPM you can manage all the patches from TPM and still do all the other stuff from TCM.

Note: This is only for Windows right now.

Defects Fixed
There are around 720 defects that have been fixed! For a complete listing, go to http://www3.software.ibm.com/ibmdl/pub/software/tivoli_support/patches/patches_5.1.0/5.1.0-TIV-TPM-FP0002/5.1.0-TIV-TPM-FP0002.DEFECTS.HTM

Extra Notes
Now that I have FP02 installed, it is time to start working through some of the features in more detail. Hopefully this blog has given you a little help in seeing what is new.

I see that IBM is putting a lot of work in this product and I think they are making progress in many areas.

The one area that still needs work is the installer. Unfortunately, this cannot be addressed in a fix pack. I have heard rumors that there will be a new installer in the next version (5.2??), but these are only rumors.

Some things that I see help improve the success of an install are pre-installing the DB2 and WebSphere products. These seem to be the two trouble areas, especially if you are using machines that are not up to the minimum requirements (like a test lab).

I also think that a fully manual process should be available to install TPM. I have done this and pretty much got everything to work, but it was a fight as I was working in areas that are not even close to documented. All I can say is thank you VMWare for snapshots :)

So now you are wondering if TPM is ready for your environment. Well here is the straight answer, yes and well no.

If you are new to the Tivoli environment and you want TPM or TCM, go to TPM now. There is no point in using TCM. There are some issues with TPM, but learning TCM now and then learning TPM later is going to really hurt. Learn TPM now, get the users on it and don't look back.

If you are a current TCM user, then I would definitely have TPM in the test lab right now at least. The biggest problem for current TCM users that I have seen is that there has been so many processes built around TCM (web pages, reports, data sharing/use) that it is going to take a long time to move everything over. Also the whole push vs pull thing is a big difference to TCM people. We have spent years managing the client expectations that when we submit something, it will start almost right away. Now with TPM, this is no longer true. There is some configurations that can be done to decrease the waiting (polling) but this can severely impact the performance of the product. So there is going to be quite a bit of work to change these expectations, unless something is done to allow for a "push now" option.

Ok this is enough for now, my fingers are bleeding from all the typing. I will keep you posted on anything I find new and interesting. If you have any questions, feel free to comment or send me an email (martin dot carnegie at gulfsoft dot com)