Never watch 8 episodes of the Sopranos

This is not technical, just a word of advice. I watched 8 episodes of the Sopranos on DVD back to back. I started swearing at truly inapprpriate moments. Probably good to throw a Brady Bunch episode in the mix.

How to enable file transfer feature in ITM 6.2.2 FP 2

One of the coolest and long awaited feature introduced in ITM 6.2.2 FP2 is the file transfer feature. Using this, you should be able to transfer monitoring scripts, dependency files and config files down to the monitored agent. ITM provides necessary CLI to transfer files to/from the agent using "tacmd getfile" and "tacmd putfile".

However this feature is not enabled in the TEMS by default. TO enable this feature, you need to do the following steps.

1. Edit $CANDLEHOME/config/*_ms_*.config

2. Add the following enviornment variable at the end of the config file.

KT1_TEMS_SECURE='Y'

3. Restart the TEMS.

4. Now run "tacmd login" followed by "tacmd getfile" or "tacmd putfile".

Hope this helps.

Who needs an agent?

What agent should I build and offer for free next? Let me know and I
will see what I can do.

Silent Install for Agent Builder Files

To install an agent from the Agent Builder, you use the option to Create a Solution Installer Image

There is also a silent installation option. A silent.txt, is included with the final package - configure this silent file with all the connection details required. If you require additional template options - I will cover that later.

Run the appropriate installer for the operating system with the -silent option. 

For example:

setupwin32.exe -silent -options response_file

A better alternative to free virtualization.

A recently had to build a new server, really big server... 48 cores, 128gb RAM and 16tb of local storage. I wanted to use VMware ESXi server - the free bare metal hypervisor, but it is limited to 32 cpu's. I looked at Microsoft's HyperV and I have never been less impressed by a product. So I looked at Citrix Xenserver 5.6 - free version. It supports up to 64 cpu's and is a true bare metal hypervisor. There are desktop clients to manage from a workstation, it manages disk stores very niceley and has tools for all the guest OSes. It's not quite as friendly as VMware, but IMHO much more usable and easier to setup clusters.

Hushing those Navigator Updates in TEP

Add these to the cq.ini


KFW_CMW_DETECT_AGENT_ADDR_CHANGE=N
The Navigator function detects when the IP@ for an Agent is discovered. If the Agent environment is constantly changing or has improper configurations that generate excessive Navigator tree rebuilding, consider adding this environment variable to have any discovery of changes or additions of IP address ignored.

KFW_CMW_DETECT_AGENT_HOSTNAME_CHANGE=N
This variable is like the one for detecting Agent address change except that it prevents the Navigator rebuilding if an agent hostname is changed.

KFW_CMW_DETECT_AGENT_PROPERTY_CHANGE=N
Similar to the above except that it prevents the Navigator rebuilding if an agent affinity or affinity version changes.

Configuring the TBSM 4.2.1 Discovery Library Toolkit to work with TADDM 7.1.1 and later

EDIT: Corrected the information after re-testing on a clean machine.

On a Windows TBSM server (I haven't tested this on other platforms), you need to copy the taddm-api-client.jar AND platform-api.jar files from the TADDM SDK into TWO different directories on the TBSM data server to get the Discovery Library Toolkit to work. Specifically, you need to copy taddm-api-client.jar to:

D:\ibm\tivoli\tbsm\XMLToolkit\sdk\clientlib

and copy platform-model.jar to:

D:\ibm\tivoli\tbsm\XMLToolkit\sdk\lib

I found that if you only copy the first file, the toolkit won't work at all, and won't generate any errors or messages of any use.

Scheduled Wakeup in Ubuntu

Well, this is hardly related to Tivoli, but just thought of sharing this cool stuff. I have a 5-year old desktop running Ubuntu Lucid used mainly for running scheduled jobs off of cron. So, basically it plays some music everyday for couple of hours and sits idle for the rest of the day. Ideally, I wanted to put the system on standby all the time and waking it up only when the scheduled job needs run. It is relatively easy to do it in Ubuntu (especially if your BIOS supports it).

Here are the exact steps needed on my Lucid Lynx. Please note that you need to have Kernel 2.6.22 or later for this to work.

1) Install the Power management interface tools.
sudo apt-get install powermanagement-interface
2) Copy the following code somewhere in your filesystem and save it as "suspend_x_hours.sh".

#!/bin/bash
# This script puts the system under standby mode for x hours

usage() {

echo "usage: $0 <n-hours>"

echo "where <n-hours> is the number of hours to be on standby"

exit 0

}
if [ $# -ne 1 ]

then

usage

fi

PATH=$PATH:/usr/sbin
hours=$1
echo 0 > /sys/class/rtc/rtc0/wakealarm
echo `date '+%s' -d "+ $hours hours"` > /sys/class/rtc/rtc0/wakealarm
pmi action suspend

3) Schedule the script in root's crontab. e.g the following crontab entry runs at 8PM and puts the system in sleep for 10 hours, waking it up at 6:00 AM.

00 20 * * * /home/venkat/bin/suspend_x_hours.sh 10 2>/dev/null

That's it. It takes only about 10 seconds to resume from sleep and it even restores your SSH sessions when it comes back from sleep!

Hope you find it useful.

ITM Tip: Disabling default situations during install time

Many ITM sites want to disable default ITM situations so that unnecessary alerts are not sent to the operations.  In old days, we usually disable the situations by running a simple script. One such example of the script is posted below.

http://blog.gulfsoft.com/2008/03/disable-all-default-situations.html

This task is much easier with ITM 6.2.2.  While seeding application support, ITM asks you to whether you want to add the default managed system groups to the situation distribution. If you answer no, the default situations will not be distributed to any managed system unless you explicitly assign them! However, this feature is applicable only for fresh installation of ITM 6.2.2 and may not apply to those who are upgrading from old version  of ITM.

GbsNcoSql V2.0

I hope you read the earlier article about the GbsNcoSql tool at the link below.

 

http://blog.gulfsoft.com/2010/01/now-you-have-alternative-to-freetds.html

 

In the first version, the tool could perform only SELECT statements, but now we have updated the tool to execute non-SELECT statements such as INSERT, DELETE and UPDATEs as well. For those who subscribed to this tool, I will send out an updated version this week.

 

Again, this tool is 100% free. Please send an email request from your work email to tony delgross (tony dot delgross at gulfsoft.com). 

 

One disclaimer:  This tool uses FreeTDS libraries to access Omnibus Object server database. While this works, please understand that IBM does not support this method.

IBM - Recording RPT 8 HTTP scripts

Tivoli is doing a great job of updating their support information, including this very detailed article on using RPT with ITCAMfT:

IBM - Recording RPT 8 HTTP scripts

Some possible responses to TADDM Error CTJTD3602E

IBM's documentation contains the following description of this error:

CTJTD3602E: The Change Manager is still processing. Wait and retry the discovery at a later time.

Explanation: The Change Manager is still running to process recent changes discovered. The discovery cannot be started until the Change Manager completes.

Operator Response: None.

Administrator Response: Allow time for the Change Manager to complete its processing before starting a discovery.

Programmer Response: None.

What we have found is that this condition can possibly occur for at least a couple of different reasons. The two situations we've found have been:

1. There was a deadlock on the database that we had to clear. A "little while" after we cleared the lock, we were able to successfully run a discovery again without receiving the error message.

2. The change manager partially ran, but didn't update the CHANGE_SERVER_STATUS table. Specifically, it left the value of the STATUS column set to 16 for the last discovery. To fix this, we had to run the following SQL:

update DB2TADDM.CHANGE_SERVER_STATUS set status = 17

This updates the STATUS column for all rows in the table. You could limit it with a WHERE clause, but in our case, it was valid as is.

We then needed to stop and restart TADDM, and the error message went away.

Identifying newly added systems in Tivoli Monitoring using GBSCMD

Hope you have been using the GBSCMD utility developed by Gulf Breeze for Tivoli Monitoring product.

 

In the latest version of GBSCMD (V3.7.1), there is a cool new feature added to the program. With "gbscmd listnewsystems", it can list the recently deployed systems in the environment along with the deployed time and the type of the agent.  With this feature, you can setup "after-install" policy scripts in ITM  just like Framework with little scripting.

 

Note: IV Blankenship wrote a blog article (http://www.gulfsoft.com/blog_new/print.php?sid=322) about this idea sometime back. 

 

Please note that the list of new systems is lost when TEMS is recycled, so you need to make sure this information is saved in a persistent store like a database or flatfile.

 

You can download the latest documentation for GBSCMD from the following link.

 

http://www.gulfsoft.com/downloads/gbscmdReferenceManual.pdf

 

Not yet subscribed to GBSCMD. Please contact us from your company email and we will send you a copy. Don't worry we will use the email only to send you future updates to GBSCMD. For those, who already subscribed to GBSCMD, we will send an update this week.

 

Are you going to Pulse 2010?
Come see Gulf Breeze at Pulse so we can show why we are the best.

Interesting Visualization

Anyone in IT cares about data at some point, so I try to bookmark visualizations that I think are noteworthy. The one I saw today is:

http://www.michaelvandaniker.com/labs/browserVisualization/

I'm not interested in the data that's behind it; I just think it's a neat way to present it.

http://manyeyes.alphaworks.ibm.com/manyeyes/

This is IBM's community Many Eyes project, which can be used by anyone.

http://code.google.com/apis/visualization/documentation/gallery.html

This is Google's Visualization API, also open to anyone.

http://graphviz.org/

Graph/node/relationship visualization.

GbsNcoSql User Guide

You can download the complete documentation for GbsNcoSql from the following link.

 

http://www.gulfsoft.com/downloads/GbsNcoSqlUserGuide.pdf

 

Hope you find it useful.

Now you have an alternative to FreeTDS - GbsNcoSql

Thanks to Venkat Saranathan, who developed this tool and wrote the description. He also wrote 'gbscmd' for use with ITM 6.x.

Many times, you need to analyze the Omnibus data using scripts. For this purpose, Netcool Omnibus provides nco_sql utility that lets you run the Omnibus queries from the command line. However, one of the main limitations of the nco_sql utility is that it is nearly impossible to parse the output. The difficulties include single record spanning across multiple lines, no delimiter option to name a few.

We at Gulf Breeze worked on this requirement sometime back and developed an utility to address some of these shortcomings. The utility is written in Java and can be run with the Java Runtime that comes with Omnibus. Some of the benefits/features of this utility includes the following.

• Currently, it can run any Omnibus "SELECT" queries only. Though the utility could potentially be modified to run other queries, the idea is to use the nco_sql utility for all other type of queries.
• Consistent one-record per line output
• Ability to specify your own delimiter with "-d" switch.
• Ability to suppress header output with "-n" switch.
• Ability to specify queries in a file or read from command line.
• No need to compile any FreeTDS drivers. It comes with the necessary FreeTDS drivers.
• Authorization information kept in a separate file and can be specified with -a switch. You don't need to specify the password in your scripts.
• Platform independent and works with the IBM and Sun JREs.

Here is a sample usage of this utility.

usage: java -jar GbsNcoSql.jar -a [-n] [-d ] [-f | -q ]

Where,

-a File name containing authorization information
-d Delimiter
-f File name containing SQL query text
-h Displays the help information
-n No header output
-q Query text

Here is a sample data.

$ java -jar GbsNcoSql.jar -a my.auth -q "SELECT Node,Tally from alerts.status"
Node,Tally
sys1,1
sys21,1
sys3,1
192.168.1.50,190

You can create an authorization manually in text editior. Here is how my authorization file looks. Please note that in the value for server is the hostname NOT the object server name.

$ cat my.auth

server=somehost

port=4100
user=root
password=mypass

Click this link to download.

Perl one-liner to convert ITM timestamps to normal timestamps

One of the annoying aspects of ITM troubleshooting is that the timestamps in the logs are written in epoch time in hexadecimal instead of 'human-readable' format. I don't know the exact reason behind it, but the whole purpose of logs is to aid in troubleshooting and timestamps are critical piece of information in troubleshooting.

Anyway, to convert the log timestamps to normal timestamp, I have been using the following one-liner.

perl -lane 'if (/^(.)([\dA-F]+)(\..*)/) { printf "%s%s%s\n", $1, scalar(localtime(oct("0x$2"))),$3; }' <log-file>

The one-liner can be used to read from pipe as well as below.

tail -100 <log-file> perl -lane 'if (/^(.)([\dA-F]+)(\..*)/) { printf "%s%s%s\n", $1, scalar(localtime(oct("0x$2"))),$3; }'

Hope this makes troubleshooting little easier.

GBSCMD V3.6.4 - Overview

GBSCMD is a free Gulf Breeze offering for performing ITM operations from command line. It is complimentary to tacmd tool and performs many operations that are not provided by tacmd. GBSCMD uses ITM Webservices feature extensively and some of the benefits of GBSCMD are listed below.

• Feature support for backlevel ITM versions. Most of the features works from ITM 6.1 on wards.
• Provides a way to execute remote commands on agents.
• Provides a way to get agent data in CSV format.
• Clear offline managed systems from Command line.
• Provides a postemsg like feature to send events to situation event console.

If you would like to learn about GBSCMD tool, here are some links to the past articles about GBSCMD. You can google search "gbscmd site:blog.gulfsoft.com".

http://blog.gulfsoft.com/2008/03/gbscmd-v35-new-features.html

http://blog.gulfsoft.com/2008/03/simple-postemsg-like-solution-for-itm_15.html

We have been tweaking the tool from time to time to introduce new features and address bug fixes and this blog discuss some of the changes introduced during the recent versions. The latest and greatest version of the tool as of this writing is version 3.6.4. If you would like to get the latest version, please feel free to contact me at venkat at gulfsoft.com or Tony Delgross at gulfsoft.com. If you have some suggestion on features that you would like to see in GBSCMD, please feel free to write to me as well.

Here are the changes introduced to GBSCMD since version 3.6.1

Version 3.6.1

This version introduced support for row filtering of SOAP Call results with --afilter option. The following example uses --afilter option to get the disk data from an agent and filters the results to include only C: drive information.

./gbscmd ct_get --auth itm62.auth --Object NT_Logical_Disk --target Primary:ITM62:NT --afilter Disk_Name;EQ;C:

You can also get the last 24 hour history data from the agent using --history switch to CT_Get. Some of the other changes include ability to see the results in XML format using --xml option (Thanks IV for making that change!).

Version 3.6.2

This version introduced new SOAP calls for starting and stopping situations at RTEMS level using --starttemssit and --stoptemssit options. If you are running backlevel of ITM and would like the ability to start and stop situations, you can use this feature. It also addressed some timeout issues while getting large SOAP data. (e.g. listing of all managed systems in the entire enterprise for a large ITM Setup).

Version 3.6.3

This version uses new SOAP call for listing situations running on individual agents.

Version 3.6.4

This version introduces column filtering of SOAP results with --attribute option. For example, if you want to get the disk data and only interested in Disk Name and Free Megabytes attributes, the following command line will get you the information.

./gbscmd ct_get --auth itm62.auth --Object NT_Logical_Disk --target Primary:ITM62:NT --attribute Disk_Name --attribute Free_Megabytes

This version also adds --version switch to identify the current GBSCMD version.

The complete documentation is available for download from the following link.

http://www.gulfsoft.com/downloads/gbscmdReferenceManual.pdf

If you have any questions or suggestions, please feel free to mention them in the comments section.

How to get ITM agent data in CSV format using GBSCMD

Here is a commonly asked question from our customers. How do I get the real time agent data exported to excel format (CSV) from command line for further analysis? Even though this can be done from Portal using a combination of logical workspaces and table view, it is real simple  with gbscmd tool.

 

With GBSCMD, you just invoke ct_get subcommand providing agent name and attribute group you're interested in.  For example, if you want to get the list of current disk usage information from a Windows OS agent, you can use the following command.

 

./gbscmd ct_get --auth itm62.auth --Object NT_Logical_Disk --target Primary:ITM62:NT  >> disk_usage.csv

 

If you would like to get this information on set of windows agents, you can easily loop thru them one after another like the example below.

 

for agent in `cat myagents.list`
do
 ./gbscmd ct_get --auth itm62.auth --Object NT_Logical_Disk --target $agent  >> disk_usage.csv
done
   
Hope you find this tip useful.

Multiple Logfile Monitoring Agent - KG2

As promised, here is a link to download a logfile monitoring agent. It will work on Unix, Linux or Windows and can monitor up to 10 logfiles per instance of the agent. This is a very generic agent with no filtering. The agent will take up to the first 1024 characters and put them into one field. As always, I've run this in the lab it works great - your setup may vary. The watchdog is setup and the memory threshold is set at 100mb for all OS platforms.

Step 1 - Download and unzip. The entrie package is 26mb in size, it contains all of the supported platforms.

ZIP version

http://www.gulfsoft.com/downloads/blog_downloads/KG2.zip

or the tar/gzip version

http://www.gulfsoft.com/downloads/blog_downloads/KG2.tgz


Step 2 - Run the installIraAgentTEMS.bat or .sh on your HUB TEMS and Remote TEMS.
Step 3 - Run the installIraAgentTEPS.bat or .sh on your TEP server (recycle the TEPS afterwards)
Step 4 - Populate your depot using tacmd addbundles -i /path... or just copy the zip file to the destination and run the installIraAgent.bat or .sh.
Step 5 - Configure it. Create an instance and add at least file name to the agent.
Step 6 - Once the agents appear in your portal server, you can create a situation that uses the "Scan for String within String" method to search for specific keywords.

Step 7 - Have fun.

Updated Tivoli Patch Listing Script

A little while back I posted a perl script (http://blog.gulfsoft.com/2009/05/script-to-retrieve-tivoli-patches.html) to list updated patches on ftp.software.ibm.com. It appears that only some patches are getting updated there and all patches are now being handled on a new service called Fix Central.

I have now updated the script to look at the Fix Central site to provide this same function.The file can be downloaded from http://www.gulfsoft.com/downloads/blog_downloads/patches_list_fc.zip