Perl one-liner to convert ITM timestamps to normal timestamps

One of the annoying aspects of ITM troubleshooting is that the timestamps in the logs are written in epoch time in hexadecimal instead of 'human-readable' format. I don't know the exact reason behind it, but the whole purpose of logs is to aid in troubleshooting and timestamps are critical piece of information in troubleshooting.

Anyway, to convert the log timestamps to normal timestamp, I have been using the following one-liner.

perl -lane 'if (/^(.)([\dA-F]+)(\..*)/) { printf "%s%s%s\n", $1, scalar(localtime(oct("0x$2"))),$3; }' <log-file>

The one-liner can be used to read from pipe as well as below.

tail -100 <log-file> perl -lane 'if (/^(.)([\dA-F]+)(\..*)/) { printf "%s%s%s\n", $1, scalar(localtime(oct("0x$2"))),$3; }'

Hope this makes troubleshooting little easier.

GBSCMD V3.6.4 - Overview

GBSCMD is a free Gulf Breeze offering for performing ITM operations from command line. It is complimentary to tacmd tool and performs many operations that are not provided by tacmd. GBSCMD uses ITM Webservices feature extensively and some of the benefits of GBSCMD are listed below.

• Feature support for backlevel ITM versions. Most of the features works from ITM 6.1 on wards.
• Provides a way to execute remote commands on agents.
• Provides a way to get agent data in CSV format.
• Clear offline managed systems from Command line.
• Provides a postemsg like feature to send events to situation event console.

If you would like to learn about GBSCMD tool, here are some links to the past articles about GBSCMD. You can google search "gbscmd site:blog.gulfsoft.com".

http://blog.gulfsoft.com/2008/03/gbscmd-v35-new-features.html

http://blog.gulfsoft.com/2008/03/simple-postemsg-like-solution-for-itm_15.html

We have been tweaking the tool from time to time to introduce new features and address bug fixes and this blog discuss some of the changes introduced during the recent versions. The latest and greatest version of the tool as of this writing is version 3.6.4. If you would like to get the latest version, please feel free to contact me at venkat at gulfsoft.com or Tony Delgross at gulfsoft.com. If you have some suggestion on features that you would like to see in GBSCMD, please feel free to write to me as well.

Here are the changes introduced to GBSCMD since version 3.6.1

Version 3.6.1

This version introduced support for row filtering of SOAP Call results with --afilter option. The following example uses --afilter option to get the disk data from an agent and filters the results to include only C: drive information.

./gbscmd ct_get --auth itm62.auth --Object NT_Logical_Disk --target Primary:ITM62:NT --afilter Disk_Name;EQ;C:

You can also get the last 24 hour history data from the agent using --history switch to CT_Get. Some of the other changes include ability to see the results in XML format using --xml option (Thanks IV for making that change!).

Version 3.6.2

This version introduced new SOAP calls for starting and stopping situations at RTEMS level using --starttemssit and --stoptemssit options. If you are running backlevel of ITM and would like the ability to start and stop situations, you can use this feature. It also addressed some timeout issues while getting large SOAP data. (e.g. listing of all managed systems in the entire enterprise for a large ITM Setup).

Version 3.6.3

This version uses new SOAP call for listing situations running on individual agents.

Version 3.6.4

This version introduces column filtering of SOAP results with --attribute option. For example, if you want to get the disk data and only interested in Disk Name and Free Megabytes attributes, the following command line will get you the information.

./gbscmd ct_get --auth itm62.auth --Object NT_Logical_Disk --target Primary:ITM62:NT --attribute Disk_Name --attribute Free_Megabytes

This version also adds --version switch to identify the current GBSCMD version.

The complete documentation is available for download from the following link.

http://www.gulfsoft.com/downloads/gbscmdReferenceManual.pdf

If you have any questions or suggestions, please feel free to mention them in the comments section.

How to get ITM agent data in CSV format using GBSCMD

Here is a commonly asked question from our customers. How do I get the real time agent data exported to excel format (CSV) from command line for further analysis? Even though this can be done from Portal using a combination of logical workspaces and table view, it is real simple  with gbscmd tool.

 

With GBSCMD, you just invoke ct_get subcommand providing agent name and attribute group you're interested in.  For example, if you want to get the list of current disk usage information from a Windows OS agent, you can use the following command.

 

./gbscmd ct_get --auth itm62.auth --Object NT_Logical_Disk --target Primary:ITM62:NT  >> disk_usage.csv

 

If you would like to get this information on set of windows agents, you can easily loop thru them one after another like the example below.

 

for agent in `cat myagents.list`
do
 ./gbscmd ct_get --auth itm62.auth --Object NT_Logical_Disk --target $agent  >> disk_usage.csv
done
   
Hope you find this tip useful.

Multiple Logfile Monitoring Agent - KG2

As promised, here is a link to download a logfile monitoring agent. It will work on Unix, Linux or Windows and can monitor up to 10 logfiles per instance of the agent. This is a very generic agent with no filtering. The agent will take up to the first 1024 characters and put them into one field. As always, I've run this in the lab it works great - your setup may vary. The watchdog is setup and the memory threshold is set at 100mb for all OS platforms.

Step 1 - Download and unzip. The entrie package is 26mb in size, it contains all of the supported platforms.

ZIP version

http://www.gulfsoft.com/downloads/blog_downloads/KG2.zip

or the tar/gzip version

http://www.gulfsoft.com/downloads/blog_downloads/KG2.tgz


Step 2 - Run the installIraAgentTEMS.bat or .sh on your HUB TEMS and Remote TEMS.
Step 3 - Run the installIraAgentTEPS.bat or .sh on your TEP server (recycle the TEPS afterwards)
Step 4 - Populate your depot using tacmd addbundles -i /path... or just copy the zip file to the destination and run the installIraAgent.bat or .sh.
Step 5 - Configure it. Create an instance and add at least file name to the agent.
Step 6 - Once the agents appear in your portal server, you can create a situation that uses the "Scan for String within String" method to search for specific keywords.

Step 7 - Have fun.

Updated Tivoli Patch Listing Script

A little while back I posted a perl script (http://blog.gulfsoft.com/2009/05/script-to-retrieve-tivoli-patches.html) to list updated patches on ftp.software.ibm.com. It appears that only some patches are getting updated there and all patches are now being handled on a new service called Fix Central.

I have now updated the script to look at the Fix Central site to provide this same function.The file can be downloaded from http://www.gulfsoft.com/downloads/blog_downloads/patches_list_fc.zip

Tivoli Policy Driven Software Distribution (TPDSD) Quick Overview

General Comments
We have been hearing about this product since Pulse 2008. The idea was to target a product on the desktop segment rather than one the covers both the server and desktop markets. This was done as these are two different worlds when it comes to management. IBM was hearing from customers that they wanted a more hands-off approach to dealing with desktops, in other words “set and forget” (my term I just made up, so don’t blame IBM for this possible bad phrasing). This was also what other vendors have been doing in the desktop market.

I have been involved with the beta of this product since it was available and will cover a few components of the product.

During the beta, this product was known as DTM. At release, it was renamed to Tivoli Policy Driven Software Distribution (TPDSD), which at least let you know that it was policy driven, but seemed to state that it only did software distribution, which it does, but also does more. It looks like this could be renamed in the next release to Tivoli Endpoint Manager (TEM). Which may seem to be a better name to some, it is actually really confusing to others that have been involved with Framework as this was the name of the component that responsible for managing information regarding the endpoints. For the purposes of this blog, I will use the current name of TPDSD. Who knows, maybe it will change again before TEM comes out ;)

Operating System Support
With the first release of TPDSD, there is very limited support for just Windows on both the server and target. The server can run on Windows 2003 and 2008 on 32 or 64 bit. The agents are supported on Windows XP, Vista, 2003 (32/64), 2008 (32/64).

The next release is supposed to have more support for both the servers and agents. There is even talk of MacOS support.

Agent Communications
In TPMfSW you were able to handle tasks to the targets using either RXA or the TCA. TPDSD only uses the agents for these tasks and the only time RXA is used is for the remote agent installation.

The great news is that the agent is now natively compiled, so no more JRE! The new agent is approx 30MB and that is the core agent and all other required support files.

Like TPMfSW, the agent also has various polling intervals. There are intervals for:
1. Checking local cached policies
2. Checking for new policies
3. Sending reports

All of these variables can be set at installation time or by an agent configuration policy.

What is a Policy?
The policies within TPDSD are what are used to do anything to an agent. If you want to install a piece of software, that is a policy. Scan the system, another policy. Configure the agent, you guessed it, a policy. When a policy is created, it is assigned to a target or a group of targets and will stay with that target until it is withdrawn or the machine is rebuilt.

Policies can be setup to either require or prohibit software from being installed. In order to require software to be installed, you need to first define the software to TPDSD and then define how the software is to be recognized on the target. For the recognition of software, there quite a few different ways that can be used to define how to detect software on a target.
1. If it is an MSI, you can define the GUID as the key. When importing an MSI, this is done automatically
2. If using a SPB, the state code is used when importing. This would take the name and version of the SPB and verify that it is in a successful install state on the target (IC--- for those familiar with TCM)
3. File existence – check for a specific file in a directory
4. Registry existence – check for a specific registry key

There are 35 different checks that can be done and multiple checks can be combined to determine if a product is installed or not. This can make the detection very flexible.

To prohibit software, you still need to create the software definition which would also define how to remove the software. The definition needs to exist so that the policy can be created and the uninstall instructions need to exist to actually do the removal.

Self Service Catalog
The self service catalog is a web interface that can be used be the client system to request the installation of software. The catalog contains a searchable section and a “Popular Software” section. The “Popular Software” could be something like a core set of products or free software that is allowed to be installed. This interface also allows the end user to remove software that is currently installed.

When the user requests software from the catalog, it is submitted and a new policy is created for the software to the specific target. This means that the agent will keep checking if the software is installed even after the initial request is completed.

The “set and forget” is the policy part of the product. You set a policy for a target, or group of targets, to have or not have software and the product takes care of maintaining the state of the target.

Integration with TPMfOSD
TPMfOSD is not included with TPDSD, but can be easily integrated using a couple global variables. It is very easy to include the TPDSD agent as a software module so that it will automatically be installed and connected with the TPDSD server. Once installed, the agent can be set to automatically install application or perform other configurations based on policies.

Conclusion
This product is looking really good and I cannot wait for the next release. Right now, due to the limited OS support and a few other things, it is not really ready for the primetime, and I do not believe that IBM is even stating that it is. It is really close and the concepts will continue from this version to the next. The next version will include much more OS support and I have heard of many new (and cool, as in good) features that people have been saying are missing. I do believe that this is a way better solution that has been designed from the ground up for the distributed environment, unlike TPMfSW, which will good, really was designed more for a datacenter.

Shameless Plug
Look for Gulf Breeze Software Partners at Pulse 2010. I will also be doing a presentation called “End to End computer management with Tivoli Policy Driven Software Distribution”. This session will be demoing the use of TPDSD along with TPMfOSD to perform a bare-metal install all the way to installing applications via polcy.

Hope to see you there!

How to find the GUID of an object in the TADDM GUI

The TADDM GUI only shows you a subset of the information that's stored in the database. The rest of the information can be found by accessing the database directly or using the command line ($COLLATION_HOME/dist/sdk/bin/api.sh). If you use the command line, the easiest way to get information about an object is if you have its GUID, with:

api.sh find [--depth num] --guid THE_GUID

but you need to get the GUID first. It's actually easy - just drag and drop an icon from the GUI into Notepad, and what you'll see will be similar to:

<DragInformation><Node><Name>SVC-2145-GYUR0XSVC01-IBM</Name><ClassName>com.collation.platform.model.topology.storage.StorageSubSystem</ClassName><SubType>com.collation.platform.model.topology.storage.StorageSubSystem</SubType><CollationType>topology.storage.StorageSubSystem</CollationType><ID>459DBBF7B68832D98A1C414EA1E5E2EF</ID><IconName>storagesub</IconName></Node></DragInformation>

The part in bold is the GUID. Easy as that.