Tuesday, July 23, 2013

APM UI - Customizing with Agent Builder data - "the Basics"

The IBM SmartCloud Application Performance Management UI (APM UI) provides a fast and simple interface into data provided by IBM Tivoli Monitoring (ITM).  It can be rendered on anything from a web browser to a smart phone.  As expected, it also provides customization capabilities.  In this post, we'll walk through end-to-end the process of introducing data/metrics from a custom ITM Agent Builder agent into the APM UI.

You're encouraged to take a look at this presentation by Cheng Quan Li of IBM Development which provides a nice overview of Dashboard Customization in APM UI.

Additionally, the documentation for APM UI customization (v.7.6.0.1 was the current version at the time of writing) can be found here.

For the purposes of this customization "basics", we're going to keep things very simple.  No fancy charts or graphs - we will simply display tabular data.  Our intent is to get comfortable with the process of customizing APM UI.  In later articles we may dig deeper into making your presentation more flashy.

We have a custom-built ITM Agent Builder agent (GBS Supermarket) for this exercise.  The data coming from this agent represents the check-out lanes at a typical supermarket.  Each lane has a name as well as performance data such as how many customers are waiting in line and the average customer wait time.

Here's what our agent looks like within the Tivoli Enterprise Portal Server (TEPS):


Click the title to read more.



Adding an ITM Agent Builder into the APM UI involves several steps and prerequisite information.

  1. Gather ITM data definitions in order to populate the "datasets" portion of your JSON files
  2. Build/create custom JSON files
  3. Run customBuilder[.sh|.bat] to load custom JSON files into APM UI

Data Definitions (datasets)

There are 2 methods we've found to best gather the ITM data definitions (attribute groups and attributes) necessary to populate the "datasets" section of your customized JSON file(s).

Using the TEPS

The ITM product code for our GBS Supermarket agent is 71 (as in the convenience store 7-11.  Hey, how creative can you be with two characters?)  Using the TEPS server to gather our data definitions involves the following steps:
  1. Navigate to $CANDLEHOME/<OS interpreter>/cq/data/
  2. Open the file dock[ProductCode]
  3. Retrieve the *TABLE and *COLUMN attributes

Our TEPS runs on Linux.  Our $CANDLEHOME is /opt/IBM/ITM.  So the following commands give us the information we need:

# cd /opt/IBM/ITM/lx8263/cq/data/
# cat dock71 | egrep "TABLE|COLUMN"

*TABLE: K71SUPERMA
*COLUMN: ORIGINNODE
*COLUMN: TIMESTAMP
*COLUMN: LANENAME
*COLUMN: AVGPEOPLEW
*COLUMN: AVGWAITTIM
[*snip]

 * Note: There are additional entries for attribute groups such as the Performance Object Status, but we're going to ignore those.

Using the TEMS

Using the TEMS server to gather our data definitions involves the following steps:
  1. Navigate to $CANDLEHOME/tables/[TEMS_NAME]/ATTRIBLIB
  2. Open the file k[ProductCode].atr
  3. Retrieve the TBL and COLU attributes

Our TEMS runs on Linux and is named HTEMS140.  So the following commands give us the information we need:

# cd /opt/IBM/ITM/tables/HTEMS140/ATTRLIB/
# cat k71.atr | egrep "tabl|colu"

tabl K71SUPERMA
colu ORIGINNODE
colu TIMESTAMP
colu LANENAME
colu AVGPEOPLEW
colu AVGWAITTIM
[*snip]

 * Note: There are additional entries for attribute groups such as the Performance Object Status, but we're going to ignore those.


Creating the JSON file(s)

Creating custom APM UI pages starts by writing one or more JSON files.

Do yourself a favor and find your new favorite JSON validation and formatting website, such as:


As you're building your JSON, periodically paste the code into a site like above to see where you missed or added a comma or bracket!

Your first file always must be named "summary.json".  The contents of that file defines what is seen on the first page.

In its simplest form, if you click somewhere on the summary page, you are taken to a new page, which is often referred to as the detail page.  One or more JSON files can be combined to create the details page.  However for this example, we will use just one file named "details.json".

JSON file layout


A basic layout of the JSON file is as follows:

{
    "title": "PAGE TITLE",
    "width": 320,
    "height": 280,
    "datasets": [
        {
            "id": "_table_",
            "columns": [
                "_column1_",
                "_column2_",
                "_column3_"
            ]
        }
    ],
    
    "widgets": [
        {
            "class": "ibm.apm.widgets.YOUR-WIDGET-TYPE",
            "wid": "w1",
            "params": {
                "x": 0,
                "y": 0,
                "width": 320,
                "height": 280,
                "labelWidth": 80,
                "meta": [
                    {
                        "id": "_table_._column1_",
                        "alias": "Description of Column1"
                    },
                         {
                        "id": "_table_._column3_",
                        "alias": "Description of Column2"
                         },
                         {
                        "id": "_table_._column3_",
                        "alias": "Description of Column3"
                         }
                        
                  ]
               }
          }
     ]
}

There can be multiple definitions within datasets, multiple classes of widgets as well as multiple meta definitions for the attributes.

As noted earlier, our Summary and Detail pages will be very simple to avoid adding confusion with the multitude of possibilities with different widgets, etc.

We'll use the GridWidget to present our GBS Supermarket attributes in a 320 pixel by 280 pixel Summary page.  Upon clicking the Summary page, you'll be taken to the Detail page, which is the same information using the same GridWidget, but in a larger 450 pixel by 350 pixel view.

summary.json

{
    "title": "GBS Supermarket",
    "width": 320,
    "height": 280,
    "datasets": [
        {
            "id": "K71SUPERMA",
            "columns": [
                "LANENAME",
                "AVGPEOPLEW",
                "AVGWAITTIM"
            ]
        }
    ],
    "widgets": [
        {
            "class": "ibm.apm.widgets.GridWidget",
            "wid": "w1",
            "params": {
                "x": 0,
                "y": 0,
                "width": 320,
                "height": 280,
                "labelWidth": 80,
                "meta": [
                    {
                        "id": "K71SUPERMA.LANENAME",
                        "alias": "Lane Name",
                        "align": "left"
                    },
                    {
                        "id": "K71SUPERMA.AVGPEOPLEW",
                        "alias": "Customers Waiting (Avg)",
                        "align": "left"

                    },
                    {
                        "id": "K71SUPERMA.AVGWAITTIM",
                        "alias": "Average Wait Time (sec)",
                        "align": "left"

                    }
                ]
            }
        }
    ]
}




detail.json

{
    "title": "GBS Supermarket",
    "width": 450,
    "height": 350,
    "datasets": [
        {
            "id": "K71SUPERMA",
            "columns": [
                "LANENAME",
                "AVGPEOPLEW",
                "AVGWAITTIM"
            ]
        }
    ],
    "widgets": [
        {
            "class": "ibm.apm.widgets.GridWidget",
            "wid": "w1",
            "params": {
                "x": 0,
                "y": 0,
                "width": 450,
                "height": 350,
                "labelWidth": 80,
                "meta": [
                    {
                        "id": "K71SUPERMA.LANENAME",
                        "alias": "Lane Name",
                        "align": "left"
                    },
                    {
                        "id": "K71SUPERMA.AVGPEOPLEW",
                        "alias": "Customers Waiting (Avg)",
                        "align": "left"
                    },
                    {
                        "id": "K71SUPERMA.AVGWAITTIM",
                        "alias": "Average Wait Time (sec)",
                        "align": "left"
                    }
                ]
            }
        }
    ]
}


Loading custom JSON files (customBuilder[.sh|.bat])

On the APM UI server, create a directory to hold your summary.json and detail.json.  We used /opt/IBM/APM/k71/

You will need to know a userid and password for the APM UI as well as the port if it is not the standard 16311.

Locate customBuilder.sh (customBuilder.bat on Windows).
It is within <installation path>/installedApps/TIPCell/isc.ear/apmTIP.war/customCfg/

On our Linux APM UI server the path to customBuilder.sh is /opt/IBM/tivoli/tipv2/profiles/TIPProfile/installedApps/TIPCell/isc.ear/apmTIP.war/customCfg/


# ./customBuilder.sh -i k71 -u tipadmin -p ****** -d /opt/IBM/APM/k71
Notice: id must start with "_" and it is prefixed automatically.
Please input a keyword to query datasources in itm.HTEMS140.172.16.17.143
To list all datasources, press the ENTER key: [enter]

1. Tivoli OMNIbus Server Agent
2. Queue-Sharing Group
3. Universal Data Provider
4. Web Response Time
5. DB2
6. Universal Agent
7. Warehouse Proxy
8. Tivoli OMNIbus Data Warehouse Agent
9. Active Directory
10. MS Host Integration Server
11. Robotic Response Time
12. i5/OS
13. Internet Service Monitors
14. Internet Server Availability Assessment Agent
15. Windows Services and Processes Config File
16. QI Agent
17. QI Broker
18. Transaction Collector
19. GBS Supermarket
20. Cognos Resource Monitor
21. UISolution.ITMSDNT
22. Microsoft Hyper-V Server
23. Transaction Reporter
24. SNMP HP-UX Systems
25. Performance Analyzer Warehouse Agent
26. Microsoft Exchange Server
27. Agentless Linux OS
28. UISolution.ITMSDBASE
29. IBM Tivoli Monitoring 5.x Endpoint Agent
30. All Managed Systems
31. IBM Tivoli Monitoring 5.x Endpoint
32. Microsoft HIS
33. MS .NET Framework
34. CIM Solaris Systems
35. System Management Agent Systems
36. MS SharePoint Portal Server
37. Linux OS
38. SNMP AIX Systems
39. Sun Management Center Systems
40. UNIX OS
41. Microsoft Lync Server
42. Linux KVM Agent
43. VMware VI
44. Monitored Network Devices
45. Network Devices Monitoring Agent
46. Microsoft IIS
47. Agentless AIX OS
48. CFile
49. Agentless Solaris OS
50. CognosContentStore
51. Lotus Workplace Server
52. MS BizTalk Server
53. WebSphere App Server
54. Windows OS
55. WebSphere Agent
56. WebSphere XS Zone
57. MQSERIES
58. Summarization and Pruning Agent
59. Microsoft SQL Server
60. VMware VI Agent
61. UDB Agent
62. WebSphere Portal Server
63. WebSphere ESB
64. Generic Configuration
65. Clients
66. Applications
67. MS Cluster Server
68. Application Management Console
69. Client Response Time
70. WebSphere XD Cell
71. NetApp Agent
72. UNIX Logs
73. SNMP Windows Systems
74. UISolution.ITMSDLZ
75. WebSphere Process Server
76. Tivoli Enterprise Portal Server
77. WebSphere Message Broker
78. Internet Services
79. Agentless HP-UX OS
80. UISolution.manager
81. WPSICPING
82. WMI Windows Systems
83. SNMP Linux Systems
84. Agentless Windows OS
98. Input a new keyword
99. Quit
19
 Category Summary Total Added 1,Updated 0,Failed 0
 Datasource Summary Total Added 1,Updated 0,Failed 0
 GroupWidget Summary Total Added 2,Updated 0,Failed 0
 Page Summary Total Added 2,Updated 0,Failed 0
 Application Template Summary Total Added 1,Updated 0,Failed 0


Results in APM UI

Now log in to your APM UI to see the results of your work!

First we'll create a new Application grouping for the Application Overview Page.

Click the [Edit] button on the upper-right.


Click the + button on the upper-right


Provide the Application Name and a Description, then click the + to add an Application Resource.

Find and select "GBS Supermarket" from the list of available Components:

Select the instance of the GBS Supermarket (K71) agent from those found to be installed and click [Add]

Click Save to see "GBS Supermarket" in the Application Overview:

Click on [Resources] next to GBS Supermarket and your Summary page will be rendered.


Click anywhere within the tabular view and your Detail page will be rendered.


Making Changes and Updates

If you wanted to change the size of the widget or to alter the alignment of data within the table, simply make the necessary edits to either the summary.json or detail.json and re-run the customBuilder[.sh|.bat] command to import the latest definitions.  Refresh your browser window and see the results.

Wrap Up

This turned into a lengthy article, but we've just begun to scratch the surface of the customization capabilities within APM UI.  Some other topics we're considering include:

  • More advanced customization with Agent Builder data (different Widgets, etc.)
  • Customizing out-of-the-box dashboards (Linux, Windows, etc.)


Feel free to leave a comment and let us know what you'd like to see next!


5 comments:

Blue Medora Staff said...

Wow - what a fantastic write-up Jamie. You and Gulf Breeze are to be commended for contributing this to the Tivoli APM community!

J Weeda said...

Thanks for this excellent post. I'm looking forward to future the articles you're considering to post.

Jamie Carl said...

Thanks guys! I just published Part 2, hopefully even more to come soon!

J Weeda said...

btw, do you know how to amend the portal page, meaning the page from where you select 'Resources' in your example above? Example changes: changing the labels on the buttons. Thanks in advance for your feedback.

Jamie Carl said...

I assume you're talking about Resources, Events, Clients and Transactions on the Application Overview page. Unfortunately, no, I don't know how to customize them. To my knowledge there is no documented/supported method of changing those labels. I do find a several hits on "Resources" in .json files where APMUI is installed, but hacking at those would probably void some warranties!