Migrating endpoints to a new TMR

This question keeps coming up in the TME10 list so I thought I would go through a couple methods that I have used. You can determine which one you want to use for your environment.

Note: This is mainly looking from a Windows endpoint perspective, but easy enough to apply to any OS.

Using wadminep
By making use of the wadminep command, you can issue a sequence of commands to delete the lcf.dat file and restart the endpoint with new configuration settings. The two arguments to use are remove_file and reexec_lcfd. These can be wrapped in a script (sh, perl) and executed against one or many targets. This method is easiest to implement, but is probably slower as it processes on endpoint at a time.

The basic steps are
wadminep remove_file lcf.dat
wadminep reexec_lcfd -g "<gateway_name>+<gateway_port>"

You can either hard code the gateway name and port, or make these variables.

Using a software package
It is possible to create a software package that can be distributed to the targets to perform the same basic functions of the wadminep commands. The good thing with the software distribution method is that you can hit more targets at the same time and also use the MDIST2 features to help with the distribution (result info, timeouts, bandwidth, etc).

The only issue with using a software package is that you cannot just run a script that stops and starts the endpoint. If you run a script that stops the endpoint, it will also stop the script from executing and it will result in either an interrupted or failed state. To work around this issue, the package will need to execute a script that will spawn a new script and release. Since the spawned script will execute right away, a delay is needed, which is where the sleep command comes in handy.

This package will consist of 2 batch files and the sleep command. The sleep command may need to be included as this is not available on all Windows systems.

Batch File 1 - start_mig_ep.bat
This script will remove the old dat file from the endpoint and then spawn the restart_ep.bat script

=====================================================================
@echo off

call "%SYSTEMROOT%\Tivoli\lcf\1\lcf_env.cmd"

REM Create backup directory in case the endpoitn needs to be moved back to the old TMR
if not exist "%LCF_DATDIR%\mig_bak" mkdir "%LCF_DATDIR%\mig_bak"

REM move the LCF files
if exist "%LCF_DATDIR%\lcf.dat" move /y "%LCF_DATDIR%\lcf.dat" "%LCF_DATDIR%\mig_bak\lcf.dat"
if exist "%LCF_DATDIR%\lcfd.log" move /y "%LCF_DATDIR%\lcfd.log" "%LCF_DATDIR%\mig_bak\lcfd.log"

REM Delete the bak/bk files. In some newer versions of endpoint, these files are created
del /f/q "%LCF_DATDIR%\*.bk"
del /f/q "%LCF_DATDIR%\*.bak"

start "Restart EP" "$(target_dir)\restart_ep.bat"
=====================================================================

Batch File 2 - restart_ep.bat
This file will use sleep to give some time for the endpoint to report that it is completed and then continue with the endpoint restart. When the endpoint is restarted, the gateway parameter will be passed with a gateway from the new TMR.

=====================================================================
@echo off

echo setting sleep for 30 seconds > "$(target_dir)\restart_ep.log"

call "$(target_dir)\sleep" 30 >> "$(target_dir)\restart_ep.log"

REM Stop the lcfd
call net stop lcfd >> "$(target_dir)\restart_ep.log"

REM Start the lcfd
call net start lcfd "/g$(gw_name)+9494" "/d3" >> "$(target_dir)\restart_ep.log"
=====================================================================

Once the scripts are created, the software package can be created to send the files to the target and then execute the start_mig_ep.bat script. Some notes on the software package that will be required:
1. Variables will be required for the scripts
- target_dir: used as the destination for where the files will be sent and the script executed from. I have set this to be a subdirectory of the Tivoli endpoint install location. This is done using the environment variable LCF_DATDIR (case sensitive if on UNIX). The value I used is $(LCF_DATDIR)\..\..\..\temp (if installed in C:\Tivoli\lcf\dat\1, this would be C:\Tivoli\temp)

- gw_name: used to define the gateway to be used in the distribution. This will allow for different distributions to be sent to different gateways. This could be the ip address, short name or FQDN.

2. Use of the substitute variables. Each of the scripts have the Tivoli variables in them. In order for the scripts to be updated, the Substitute Variables checkbox needs to be checked. This is in the advanced file properties for the files.


Migrate_ep SPD file contents
=====================================================================
"TIVOLI Software Package v4.2.3 - SPDF"

package
name = migrate_ep
title = "No title"
version = 1.0
web_view_mode = hidden
undoable = o
committable = o
history_reset = n
save_default_variables = n
creation_time = "2009-01-14 10:00:00"
last_modification_time = "2009-01-14 10:00:00"

default_variables
target_dir = $(LCF_DATDIR)\..\..\..\temp
gw_name = gateway_name
end


log_object_list
location = $(target_dir)
unix_user_id = 0
unix_group_id = 0
unix_attributes = rwx,rx,
end

move_removing_host = y
no_check_source_host = y
lenient_distribution = y
default_operation = install
server_mode = all,force
operation_mode = not_transactional
post_notice = n
before_as_uid = 0
skip_non_zero = n
after_as_uid = 0
no_chk_on_rm = y
versioning_type = swd
package_type = refresh
sharing_control = none
stop_on_failure = y

add_directory
stop_on_failure = y
add = y
replace_if_existing = y
replace_if_newer = n
remove_if_modified = n
location = C:\CID\SRC
name = migrate_ep
translate = n
destination = $(target_dir)
descend_dirs = n
remove_empty_dirs = y
is_shared = n
remove_extraneous = n
substitute_variables = y
unix_owner = root
unix_user_id = 0
unix_group_id = 0
preserve_unix = n
create_dirs = y
remote = n
compute_crc = n
verify_crc = n
delta_compressible = d
temporary = n
is_signature = n
compression_method = deflated
rename_if_locked = y

add_file
replace_if_existing = y
replace_if_newer = n
remove_if_modified = n
name = sleep.exe
translate = n
destination = sleep.exe
remove_empty_dirs = y
is_shared = n
remove_extraneous = n
substitute_variables = y
unix_owner = root
unix_user_id = 0
unix_group_id = 0
preserve_unix = n
create_dirs = y
remote = n
compute_crc = n
verify_crc = n
delta_compressible = d
temporary = n
is_signature = n
compression_method = deflated
rename_if_locked = y
end


add_file
replace_if_existing = y
replace_if_newer = n
remove_if_modified = n
name = restart_ep.bat
translate = n
destination = restart_ep.bat
remove_empty_dirs = y
is_shared = n
remove_extraneous = n
substitute_variables = y
unix_owner = root
unix_user_id = 0
unix_group_id = 0
preserve_unix = n
create_dirs = y
remote = n
compute_crc = n
verify_crc = n
delta_compressible = d
temporary = n
is_signature = n
compression_method = deflated
rename_if_locked = y
end


add_file
replace_if_existing = y
replace_if_newer = n
remove_if_modified = n
name = start_mig_ep.bat
translate = n
destination = start_mig_ep.bat
remove_empty_dirs = y
is_shared = n
remove_extraneous = n
substitute_variables = y
unix_owner = root
unix_user_id = 0
unix_group_id = 0
preserve_unix = n
create_dirs = y
remote = n
compute_crc = n
verify_crc = n
delta_compressible = d
temporary = n
is_signature = n
compression_method = deflated
rename_if_locked = y
end

end


execute_user_program
caption = "Execute migrate batch file"
transactional = n

during_install
path = $(target_dir)\start_mig_ep.bat
inhibit_parsing = n
timeout = 300
unix_user_id = 0
unix_group_id = 0
user_input_required = n
output_file = $(target_dir)\start_mig_ep_out.log
error_file = $(target_dir)\start_mig_ep_err.log
output_file_append = n
error_file_append = n
reporting_stdout_on_server = n
reporting_stderr_on_server = n
max_stdout_size = 10000
max_stderr_size = 10000
bootable = n
retry = 1

exit_codes
success = 0,0
failure = 1,65535
end

end

end

end

=====================================================================

System Downtime Monitoring Using Universal Agent

ITM provides system uptime monitoring out-of-the box.  You just have to select the uptime attribute and you can use the attribute in situation formula. It is that simple. Okay, what if you want to monitor the system downtime? It may sound like little difficult but with Universal Agent it is possible to guage this value with a simple MDL and script combination.  This article explains how to do it. It also provides an example how to use some of the Time functions of UA.  

How does it work?

To calculate downtime, we need to write a simple script that outputs the current date and time and the previous value of the date and time. We use the UA to get the capture these values.  So, how do we get the previous value? UA provides this feature out-of-the box and it is documented!  Just use the environment variable $PREV_VALUE in your script. Unfortunately this value is not persistent across UA restarts, so your script should store the last time it ran in a file somewhere.  You can also use the UA functions to convert the script output to ITM timestamp. So after this,  you will get Current Time, the previous time the script ran as attributes in the portal. You can write a simple situation that uses Time Delta function to calculate the difference between the two times and alert.  

MDL

A Simple MDL listing is given below. It is given as an example only. Perform your own testing to ensure its working. 

//APPL V02_SYSTEM_DOWNTIME

//NAME DOWNTIME K 300 AddTimeStamp Interval=60

//SOURCE SCRIPT /opt/gbs/bin/downtime.sh 

//ATTRIBUTES 

Hostname (GetEnvValue = HOSTNAME)

CurrentDate D 10

CurrentTime D 10

PrevDate D 10

PrevTime D 10

CurrentDateTime (CurrentDate + CurrentTime)

PrevDateTime (PrevDate + PrevTime)

CurrentTimeStamp (TivoliTimeStamp = CurrentDateTime)

PrevTimeStamp (TivoliTimestamp = PrevDateTime)

Script

Here is a sample shell script that retrieves current and prev time stamp values. 

#!/bin/sh

# Latest Timestamp

current_value=`date "+%m/%d/%Y %H:%M:%S"`

# If the PREV_VALUE exists, displays current and prev values,

# else retrieve PREV_VALUE from persistent file

if [ "x$PREV_VALUE" != 'x' ]

then

   echo $current_value $PREV_VALUE

else

   prev_value=`cat /tmp/downtime.txt`

   echo $current_value $prev_value

fi

# finally, store the current timestamp in persistent file

echo $current_value > /tmp/downtime.txt 

Drawbacks

Does this solution provide accurate downtime estimate? No, it doesn't. For example you may get potential false alerts if the UA goes down for some reason. Also, it provides the time difference between the script last ran and script's latest run not exactly the time between system reboots. But these are minor drawbacks to live with! 

Questions, comments? Please feel free to post them.  

TPM 5.1.1.2 Custom Inventory Scan

Well it is finally available. The custom inventory scan is one of the features that existing Tivoli Configuration Manager (TCM) customers required in TPM before they could move to using TPM completely (yes there are still more, but this was pretty important)

A while back, I created a blog entry on creating custom inventory scans with TCM (see http://blog.gulfsoft.com/2008/03/custom-inv-scan.html) that I thought I would look at moving to TPM. This is a basic VBS script that will read in a registry key and record to the values to a MIF file. This MIF file is then imported into the DCM and can be used in a query from the Reports section of the TPM UI.

The steps involved are:
1. Define the requirements for the data to determine table columns
2. Create the table
3. Create the pre and post scripts to create the output file (can be MIF or XML)
4. Create the inventory extension properties file and import
5. Deploy the scripts to the target
6. Run the custom discovery created by the import in step 3
7. Run the custom report created by the import in step 3

Define Requirements
For this sample, the requirements for the data to be inserted are the Registry Hive, Key(s) and Value.

Create the Table
The table will be made up of the requirements defined above and columns to uniquely identify the computer and discovery id. Also the fix pack documentation recommends adding foreign keys for cascade deletes. For this example, I set the various fields to VARCHAR and defined a possible length for them. I also assigned it to the tablespace IBM8KSPACE, well, just because I felt like it ;)

Table Creation Syntax for DB2
create table db2admin.registrydata1(REG_PATH VARCHAR(256), REG_VALUE VARCHAR(32), REG_DATA VARCHAR(128)) in IBM8KSPACE;
alter table db2admin.registrydata add column SERVER_ID BIGINT;
alter table db2admin.registrydata add column DISCOVERY_ID BIGINT;
alter table db2admin.registrydata add foreign key (SERVER_ID) references db2admin.SERVER(SERVER_ID) on DELETE CASCADE;
alter table db2admin.registrydata add foreign key (DISCOVERY_ID) references db2admin.DISCOVERY(DISCOVERY_ID) on DELETE CASCADE;

Create the pre and post scripts
One of the possible issues is around passing arguments to the scripts, but then again, this was not that easy in TCM. The examples in the FP docs do not talk about passing arguments in the extension properties file (discussed in the next step). From the limited testing I did, this does not seem possible.

The pre script will be used to generate the MIF file to be retrieved. One issue I have with the pre script is that it does not accept arguments. So this means that to execute the VBS script that I want to use, I have to create a wrapper BAT file to pass the arguments.

The post script does not really need to do anything. It could be as simple as just an exit 0.

prescript.windows.bat
==============================================
cscript //nologo c:\test\registrydata.vbs "Software\Martin" c:\test\regdata.windows.mif
==============================================

postscript.windows.bat
==============================================
echo Running post script
==============================================

regdata.windows.mif (sample output)
==============================================
START COMPONENT
NAME = "REGISTRY VALUE DATA"
DESCRIPTION = "List registry value and data entries"
START GROUP
NAME = "REGISTRYDATA"
ID = 1
CLASS = "DMTF|REGISTRYDATA|1.0"
START ATTRIBUTE
NAME = "REG_PATH"
ID = 1
ACCESS = READ-ONLY
TYPE = STRING(256)
VALUE = ""
END ATTRIBUTE
START ATTRIBUTE
NAME = "REG_VALUE"
ID = 2
ACCESS = READ-ONLY
TYPE = STRING(32)
VALUE = ""
END ATTRIBUTE
START ATTRIBUTE
NAME = "REG_DATA"
ID = 3
ACCESS = READ-ONLY
TYPE = STRING(128)
VALUE = ""
END ATTRIBUTE
KEY = 1,2,3
END GROUP
START TABLE
NAME = "REGISTRYDATA"
ID = 1
CLASS = "DMTF|REGISTRYDATA|1.0"
{"Software\\Martin","test","test"}
{"Software\\Martin","test1","test1"}
END TABLE
END COMPONENT
==============================================


Create the inventory extension properties file
The extension file is used to define the table that is used to define various properties required to populate TPM with the definition, tables, scripts and output file to be used in the discovery process. The properties file for this example is as follows:

==============================================
#name the extension
extName=REGISTRYDATA

#Description of the extension
extDescription=GBS Collect Registry data

#Custom Table Names
TABLE_1.NAME=REGISTRYDATA

#file for Windows platform
WINDOWS=yes
pre_windows=C:\\test\\prescript.windows.bat
out_windows=c:\\test\\regdata.windows.mif
post_windows=c:\\test\\postscript.windows.bat
==============================================

extName is used to define the DCM object name for the new extension.
extDescription is used in the description field for the discovery configuration and report (if created)

Multiple table names can be used by using sequential numbers after the TABLE_. For the purposes of this demo, only one table is used

Operating system flags – This can be defined for WINDOWS, AIX, HPUX, SOLARIS and LINUX

Pre/Post scripts and Output files – are used for each operating system. Used the prefixes pre_, out_ and post_ with the OS definitions of windows, aix, hpux, solaris and/or linux

Save the file as whatever naming convention is desired. For the purposes of this example the file was created in C:\IBM\tivoli\custom\inventory\registrydata\registrydata.properties

Import the properties file
To import the properties, use the command %TIO_HOME%\tools\inventoryExtension.cmd. This command can be used to create, delete and list inventory extensions. The syntax used for this example was:

inventoryExtention.cmd create –p C:\IBM\tivoli\custom\inventory\registrydata\registrydata.properties –r yes

The “p” parameter defines the file to be used and the “-r yes” is used to tell the import to also create the custom report for the inventory extension.

Command Output:
=========================================
C:\IBM\tivoli\tpm\tools>inventoryExtension.cmd create -p C:\IBM\tivoli\custom\inventory\registrydata\registrydata.properties -r yes
2008-12-30 09:59:03,890 INFO log4j configureAndWatch is started with configura
ion file: C:\ibm\tivoli\tpm/config/log4j-util.prop
2008-12-30 09:59:04,062 INFO COPINV006I Parsing the command line arguments ...
2008-12-30 09:59:04,796 INFO COPINV007I ... command line arguments parsed.
2008-12-30 09:59:04,796 INFO COPINV008I Start processing ...
2008-12-30 09:59:04,812 INFO Start parsing property file
2008-12-30 09:59:08,780 INFO Finished parsing property file
2008-12-30 09:59:08,874 INFO COPINV021I The specified extension: REGISTRYDATA has been successfully registered.
2008-12-30 09:59:08,874 INFO Creating discovery configuration...
2008-12-30 09:59:14,984 INFO Discovery REGISTRYDATA_Discovery successfully created
2008-12-30 09:59:15,390 INFO Report REGISTRYDATA_Report succesfully created
2008-12-30 09:59:15,484 INFO COPINV009I ... end processing.
2008-12-30 09:59:15,484 INFO COPINV005I The command has been executed with return code: 0 .
=========================================

Note the names of the Discovery Configuration and the Report created. TPM does not need to be restarted for this to take effect.

Deploy the scripts to the target
The next step is to deploy the script(s) to the target(s). In TCM, this was done by creating a dependency on the inventory object. Currently this is not something that is documented, but could be done with a software package, a workflow or possibly modifying the CIT package (not sure if this would be advisable). I am going to leave this out for now as I think this will need to be investigated further. So for now, this was just a manual copy of the files to the remote target. Copy the files to C:\TEST as defined in the properties file.


Run the custom discovery
Once the import is completed, a new discovery configuration is created. This discovery configuration will be labeled according to the extName field in the properties file and suffixed with “_Discovery”.




One of the main differences between a normal inventory discovery and the custom is on the Parameters tab of the discovery. There is a new field called Inventory Extension.




Run the custom report (if created)
Once the scan is completed, execute the custom report. This will display the results for the data collected. The report is created under the Discovery section.




In the results the data for the scanned registry section for each computer will be displayed.



Other Random Notes

There are also 3 tables (that I found so far). The names are INVENTORY_EXTENSION, INVENTORY_EXTENSION_FILE and INVENTORY_EXTENSION_TABLE. The table INVENTORY_EXTENSION_FILE contains the pre, post and output files. I was able to modify the entries to use a different file on the target without having to re-import the properties file.

ITM 6.2.1 Agent Availability Monitoring - What's new?

If you have been using MS Offline Situations in ITM 6.x so far, you know the drawbacks of such monitoring.  For example, You will get hundreds of such situations when a RTEMS goes offline even if the agent itself running. You  can use process monitoring situations (for the process kntcma.exe), but it has its own drawbacks. ITM 6.2.1 introduces a new approach to this problem and this article explains the new solution.

In ITM 6.2.1, we can use the Tivoli Proxy Agent Services to monitor the availability of the agents. For example, if a Windows OS agent goes down, the Tivoli Proxy Agent Services can restart it.   In case the OS agent went down too many times, you can easily monitor the condition using a situation. Just use the Alert Message attribute in the Alerts Table attribute group and check if the agent exceeded restarted count.   Here are few other conditions that you can monitor with the Alerts Message attribute.

• Agent Over utilizing CPU
• Agent Over Utilizing Memory
• Agent Start Failed
• Agent Restart Failed
• Agent Crashed (abnormal stop)
• Agent Status Check Script Failed
• Managed/Unmanaged agent removed from the system

With the combination Tivoli Proxy Agent Services auto restart feature and set of situations to monitor the above exceptions,  you can devise an effective agent availability monitoring solution with fewer and only relevant alerts reaching the console.  Do you have questions about the above solution? Please feel free to write back.

Merry Christmas!

Creating a Web Services/SOAP client in eclipse

The Eclipse development environment is very powerful, so I figured I would create a simple SOAP client in it from a given WSDL file. As it turns out, it's a little painful, so I wanted to write up the steps I had to go through.

I used Eclipse 3.3.2 (Europa) JEE: Eclipse 3.3.2 (Europa) JEE on Windows XP in a VM

I started with an empty Java project named "FirstJava".

First import your WSDL file into your project.
- Select the "src" folder of your proejct, right-click and select "Import"
- Choose General/File System
- Select the folder containing your file and click OK
- now place a check mark next to your WSDL file and click OK

Now download Tomcat. There is a built-in "J2EE Preview" that works for some things, but not for this.

Now create a new Target Runtime and server associated with Tomcat.
- Select the project and select Project->Properties, then select "Targeted Runtimes" on the left of the dialog.



- Click the "New" button
- Select Apache->Tomcat 5.5 AND ALSO CHECK "Also create new local server"



- Input the appropriate values as needed.


- Select the newly-create runtime


- Click OK to save the properties.

Create a new Web Services Client
- Right-click your WSDL file name and select New->Other..., then in the dialog displayed, select Web Services->Web Service Client).



- Your WSDL file name should be filled in at the top of the dialog


- In this dialog, use the slider underneath "Client Type" to specify "Test client" (move it all the way to the top).
- Click Finish.
- This will create a bunch of new code in your current project, plus it will create a new project (named "FirstJavaSample" in my case) with the JSPs you'll be able to (hopefully) run to test your client.


- This will give you an error about the JSP not supporting org.apache.axis.message.MessageElement[]. Just click OK several times until the error box goes away. We'll fix that later.

If all went well, you should see something like the following:


Now we have to fix the errors.

Create a JAR file named FirstJava.jar containing the contents of the bin directory of your FirstJava project.

Copy that file to the Tomcat "Common/lib" folder (C:/Apache/Tomcat5.5/Common/lib on my system).

You will additionally need to find these files under the eclipse/plugins directory and copy them to the Tomcat Common/Lib folder:

axis.jar
saaj.jar
jaxrpc.jar
javax.wsdl15_1.5.1.v200705290614.jar
wsdl4j-1.5.1.jar

(If you can't find them on your system, use Google to find and download them. One of them - I don't recall which - was tricky to find for me because it was actually in another JAR file.)

Now stop the Tomcat server by selecting it in the "Servers" view and clicking the stop (red square) button.


Now re-run your application by opening the FirstJavaSample project and finding the file named "TestClient.jsp". Right-click that file and select Run As->Run On Server, select your Tomcat server and click Finish.

You should now see that things are working correctly.


You may need to edit the generated JSP files to add input fields and such, but that's specific to your particular file.

Good luck, and happy coding.

Enabling JMX in TBSM

Since TBSM runs on top of Tomcat, you can enable JMX access to TBSM's Tomcat server to give you some insight to how the JVM is doing. To do this, you'll need to edit the $NCHOME/bin/rad_server file to add a line (in the appropriate place, which should be easy to spot once you're in the file):

JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8999 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"

This specifies that no authentication is needed, which is fine in a test environment, but for a production environment, you would want to enable SSL and/or authentication (Google will find lots of links for you).

You do need to restart TBSM to have the changes take effect. Once it starts back up, you can view the JMX data using the jconsole command that is included with the Java 1.5 (and above) JDK. When you startup jconsole, specify the hostname of your TBSM server and port 8999 (specified above), with no user or password. That will give you a nifty GUI that looks like this:



One of the available tabs is "Memory", which will give you some (possibly) useful information about memory utilization in the JVM.


As you can see, there are other tabs, which you should investigate to see what additional information is available.

TADDM TUG Presentation

I'm giving this presentation on TADDM for the NYC TUG today and the Philadelphia TUG tomorrow, and wanted to make it available online.

Full size: https://drive.google.com/open?id=0B2lRAtNC_A9BYkZ1THlEb3lMeEU