Monday, March 10, 2008

Windows 2003 SP1 Tivoli Installation Woes

Honestly, I never thought I would face so many tricky issues in Windows 2003 Tivoli installation. After all, Windows 2003 is one of the primary operating systems for Tivoli product and unlike Linux/Unix, there are not many places things could go wrong. But in the end, this is one of the most troublesome installations I ever went through and thought of sharing the lessons learnt with everyone.

Framework installation Access Violation Error
The first problem occurred much sooner than I had expected. Framework installation failed with the following error.

stderr: 0 0 [main] D:\Tivoli\trip\sleep.exe 1000

Found that the Windows 2003 Data Execution Protection (DEP) feature is blocking the execution. The solution was to enable DEP only for system program and not for all programs. Right click My Computer -> Properties -> Advanced . Under Performance, Click Settings -> Data Execution Protection -> Turn on DEP for essential Windows Programs or Services only. However this might raise few eye brows in security area and the best method is to add the Tivoli setup.exe program to the DEP exclusion list.

Activity Planner Initialization failed

Things went relatively smooth for the rest of the framework installation and most of ITCM installation. However, when I started activity planner with wstartapm the apmlog0 showed the following error. The same error occurred when I tried to do wlstpln to list the activity plans.

AMN0121E Activity planner initialization failed. Check whether the Activity planner user has been created correctly and/or the user and password maintained by Activity Planner are synchronized with the corresponding value of the operating system.

One of the very few clear error messages from Tivoli, right? Wrong. I know the password was correct but I did a wsetapmpw to reset the password again. Still the same error. Little more research revealed that the Tivoli administrator account for the apm userid (tivapm) was never created during the installation. So, I had to manually create a Tivoli administrator called swd_admin_ and add the APM userid login (tivapm) to it. I also had to add all the APM roles and senior role to the account. Now APM is working fine.

Multi-step Activity Plan hangs on the first step

So, I got everything working. Now I needed to some test distributions to make sure everything is okay. I sent down an inventory scan to one endpoint and it worked fine, the result was reflected both in the MDist2 as well as APM databases.

Next, I created an activity plan to deliver two software packages one after another conditioned by the targets. (CT). The first package was delivered correctly, the result was updated in the mdist2 database but it never got updated in the APM database, which showed the package in Started state.

Might be a random occurrence? I tried again after clearing out everything and bumping the log levels for RIM, planner and distribution manager. Same thing. The wtrace output (wtrace -jk $DBDIR > /tmp/somefile) showed the following error for notification manager (run_nm) method. The notification manager is a Software Distribution component that receives results from mdist2 and handles dispatching of the results to other components such as activity planner, writing to log and sending a TEC event. etc.

loc-ec 1927 M-hdoq 1-1902 0 e=128
Time run: [Wed 15-Mar 17:21:02]

Object ID: 1230685615.1.710#SWDManager::Swdmgr#
Method: run_nm
Principal: VMTMRWIN2003\GBSASPNET@vmtmrwin2 (9940560/0)
Helper pid 1464 Path: C:\apps\tivoli\bin\\w32-ix86\TME\SWDIS\SWDMGR\swdmgr.exe

This proved to be hard to debug. After digging more in Tivoli knowledge base, I got one of the weirdest solutions. The solution was to enable the Windows Firewall and Internet Connection Sharing Service. After enabling the service, I re-sent the distribution and now everything started working.

Finally, I got all the desired features working. Hopefully there is no more Windows 2003 quirkiness. Wish me luck.


Unknown said...

Well, thanks to you, I finally figured out what was wrong with my Tivoli Framework Installation. I've worked for Tivoli for 12 years, and I have never installed on the OS--correction, I have installed once before, and when it failed, I switched machines/OS's! LOL. Today, I HAD to install it on a Win2003 server, and I assumed everything would go ok... NOT! You were exactly right, and if it weren't for you, I would have wasted many, many more hours and not found the right solution. Thank you for this post. This was a ridiculous problem to have with no WARNING in the release notes that I found.

technofetyszysta said...

You saved my life.I've just lost a few hours fighting with different combinations of framework an OS patches trying to add a managed node on w2k3 via TRIP. Than I realized it must be something with windos.

Venkat said...

Glad you found it useful!