Moving to the cloud. Pick any two: Cheap, Fast, or Easy

The Cloud offers literally all of your current IT services, plus tons more, some of which most of your IT department has never heard of before. You can quickly and easily move all your existing workloads there, but you'll pay dearly, as many companies are finding out. You can take the time to train your IT staff and meticulously plan for the most efficient way to the cloud, but that's not quick. 

Moving to the cloud correctly truly requires rethinking how you do everything in IT. In all cases, the best route is to only move a subset of workloads or capabilities to the cloud, and different clouds may be better for different workloads. Some things are easier and cheaper to run on-prem. For example, in many cases it can be cost effective to arm your IT and development staff with laptops with 64GB of RAM. Doing so allows each one to run their own private multi-cloud in which they can test away. A brand new laptop with warranty with 64GB of RAM and 6 cores (12 threads) can be found for under $1700 on eBay and has a useful life of 4 years. Such a VM in the cloud (AWS EC2 r5.2xkarge) costs $.20 per hour, which is $5,300 for a three-year term, and doesn’t allow the flexibility of a local system running VMWare Workstation. That's a very specific example, but it illustrates why each and every workload needs to be analyzed or audited before simply moving it to the cloud.

 Some workloads are more suited to specific public clouds. WebSphere applications are a big example. If you want to “lift and shift” these workloads to the cloud, the IBM Cloud should be your first choice. If you have apps that run under Sharepoint, you should absolutely run those applications on Microsoft’s Azure cloud. There are many other workloads that may run equally well on any cloud, and for those, the analysis needs to take other factors into account.

 The point I want to get across is that moving to the cloud requires analysis by a qualified team of experts. I think the best approach is to hire one or more experts and simultaneously train your own team to help them get up to speed. The combination of those two is extremely important, because you don’t want newly-trained people responsible for your entire cloud migration. You want an expert who can guide the team, allowing them to take over responsibilities over time.

If you're scripting on Windows, use PowerShell

My last post on PowerShell was in 2008, so I thought I would write an update. If you're writing scripts on Windows, you should probably be using PowerShell. It seems to have 99 to 100% of the tools (especially parsers) that I ever need. I just recently needed to scrape a web page for some data, so I thought I would spend some time messing with PowerShell to get it going. Well, it only took about 30 minutes to develop the entire script that I needed, with absolutely no external dependencies.

Here's the whole script:

# get the web page. Yes, PowerShell has a 'curl' command/alias
$resp = curl -UseDefaultCredentials http://myhostname/mypagename

# Get all of the rows of the table with an ID of "serverTable"
$rows = $resp.ParsedHtml.getElementById("serverTable").getElementsByTagName("TR")

# Loop through the rows, skipping the header row: 
for ($i=1; $i -lt $rows.Length(); $i++) {

# get the hostname of this row
  $thehost = $rows[$i].getElementsByTagName("TD")[2]

# get the date this host was last rebooted
  $rebootDateString=$rows[$i].getElementsByTagName("TD")[6]

# if the host was rebooted over 20 days ago, print that date

  if ([datetime]::Now.AddDays(-20) -gt [datetime]::parseexact($rebootDateString.innerText,"G", $null))   {

    $rebootDateString.innerText } }

That's it, with no external references and nothing extra to install. It's got date parsing, date arithmetic, HTML parsing and HTTP request capabilities all built in. I realize that this then isn't portable... or is it? There's actually a PowerShell port for Linux available, with instructions here from Microsoft:

https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-linux?view=powershell-6

I know that Python is a hot language these days, but I don't like it as much as PowerShell. I tried to do the above with Python, and it took quite a bit longer, even though I've used Python more than PowerShell. You have to import some classes and then use XPath to find elements in the HTML. PowerShell was just straightforward and easy, at least for me, with my background and expectations. YMMV, but I like PowerShell.

How to view an LtpaToken2 token

Leave a Comment

If you find this article useful, please let us know in the comments.

The Article

If you use any WebSphere-based applications (DASH, Impact, BPM, ODM, etc.), you're using LTPA Tokens. An LTPA Token is a browser cookie named LtpaToken2. You can see it if you turn on developer tools in your browser (F12) after you log into one of these applications. You'll see it in the "COOKIES" request header. The value of that header will look something like this:

s_vi=[CS]v1|2E7C0CDA8507BB19-4000010FA0013DFE[CE]; s_cc=true; s_sq=%5B%5BB%5D%5D; JSESSIONID=00004FX-3-uu2ZoYHx1t9p8fJIb:52e767f1-e67e-4220-8435-fa54d8776107; CSRFCookie=C9495874E4D5BA23D8E1330E4F76EA5C9495874E4D5BA23D8E1330E4F76EA5; LtpaToken2=i/InlYuq2tm3rPdd/3BEzA8m9BCc8WGNR3q6eu7OfeQ7s1ICiMvPv0QCNQar5cCQlyVH5GE0N0VNbJj1Z6sUGe2S3nb1kwwbzdzPWzCbNPPtN3uiPWnfLyXzi5T4p2Pz/URwCfP6zWW2NOob/yQoG5vYg/JAgJag9CWP5tqd9+6FgInahSj3VaYYvu69O4hY+h6e6D+v7mpLTYBRM33TlVugTxOkx64JTMAdwFAfH553Ob2T+sW4aqyiGc7arLodIMlWjiVbkBBEgYZ0PXMyCPKb7JPa+5lFxfMRBK0P1kMsC34OXnQ1jUaedx44U4I5

Notice that each cookie=value pair is separated by a semicolon (;). The LtpaToken2 cookie and value are in bold above. That cookie contains the "principal" (user) name, the "realm" (a named security policy domain), and the expiration date of the token. This token is used by a WebSphere server (whether it's WebSphere ND, WebSphere Liberty, or any other flavor of WebSphere) to make authorization decisions to determine which resources the user has access to. It's also used for authentication across WebSphere servers if the two servers share the same LTPA keys. This sharing of keys is how one server "trusts" an LTPA token created by another server.

For troubleshooting purposes, sometimes you want/need to see what's inside an LTPA token. I found the best article on the web that provides the code to allow you to do exactly that. The page is found here: http://tech.srij.it/2012/04/how-to-decrypt-ltpa-cookie-and-view.html , but it leaves out some basic steps that I'm including here. I'm also copying the Java code at the bottom of this blog post just in case the linked article disappears.

Once you've copied the code into a file named DecryptLTPA.java (the file MUST have this name to match the name of the class defined in the file), you then need to specify several values in the file. Specifically, you need to provide values for the following variables:

ltpaKey : the com.ibm.websphere.ltpa.3DESKey value from ltpa.keys file. This file already exists if you're using WebSphere Liberty. If you're using "full" WebSphere, you need to create this file (with whatever name you want) by exporting the LTPA keys from the WebSphere Administration Console. In the ltpa.keys file, this value will end with the characters "\=" (backslash equals). When you paste the value into the DecryptLTPA.java, remove that backslash.

ltpaPassword : WebAS  - This is the default password on WebSphere Liberty. If you're using "full" WebSphere, the password will be whatever you specify when you export the LTPA keys using the WebSphere Administration Console.

tokenCipher : the entire value of the ltpatoken2 header from the COOKIES header in the Request.

Once you've done the above, the beginning of the definition of the main() function in the DecryptLTPA.java file should look something like:

 public static void main(String[] args) {
  String ltpaKey = "ADbpPpqPf3bnkj0b34sNaNC2FYHYygub3/cGjIn+mR4=";
  String ltpaPassword = "WebAS";
  String tokenCipher = "i/InlYuq2tm3rPdd/3BEzA8m9BCc8WGNR3q6eu7OfeQ7s1ICiMvPv0QCNQar5cCQlyVH5GE0N0VNbJj1Z6sUGe2S3nb1kwwbzdzPWzCbNPPtN3uiPWnfLyXzi5T4p2Pz/URwCfP6zWW2NOob/yQoG5vYg/JAgJag9CWP5tqd9+6FgInahSj3VaYYvu69O4hY+h6e6D+v7mpLTYBRM33TlVugTxOkx64JTMAdwFAfH553Ob2T+sW4aqyiGc7arLodIMlWjiVbkBBEgYZ0PXMyCPKb7JPa+5lFxfMRBK0P1kMsC34OXnQ1jUaedx44U4I5";

You then need to compile the java file with a command similar to the following:

 javac -cp /opt/IBM/tivoli/impact/wlp/usr/servers/ImpactUI/apps/blaze.war/WEB-INF/lib/commons-codec-1.10.jar DecryptLTPA.java

That command points to the commons-codec.1.10.jar file, which contains the definition of the Base64 class referenced by the code. That command was run on a Netcool Impact version 7.1.0.16 server which uses WebSphere Liberty. If you were to compile the code on a DASH version 3.1.3.0 server, the command would look like this:

javac -cp /opt/IBM/tivoli/JazzSM/profile/installedApps/JazzSMNode01Cell/IBM Cognos.ear/p2pd.war/WEB-INF/lib/commons-codec-1.3.jar DecryptLTPA.java

In both cases, my path includes the Java 1.7 SDK binaries and my JAVA_HOME is set to the Java 1.7 SDK directory.

Once you have the file compiled, you can actually run it, which requires a similar command:

 java -cp /opt/IBM/tivoli/impact/wlp/usr/servers/ImpactUI/apps/blaze.war/WEB-INF/lib/commons-codec-1.10.jar:. DecryptLTPA

Notice that you must include "." (present working directory) in the classpath (-cp flag) in addition to the commons-codec jar file.

Once that runs, you should see output similar to:

Algorithm:[AES]

Full token string:[expire:1568250479184$u:user\:customRealm/impactadmin%1568250479184%zoQ7cb1BSWekvxdd3slUpxmCRlBCBmMO1nu8iztv73PKQN3MIybuCx/C9EdKGwoeoguJHKrj0BOOAeXxVLDgIQL5Jz2Tg6LQcIyTpAtRAVMsqWTPFzDBrs85Zxs9kP0zFEiOvEsDUmRXXm92dN6zxWooEyGz453x1VPmqGoZ0ww=]

Token is for:[expire:1568250479184$u:user\:customRealm/impactadmin]

Token expires at:[2019-09-11-21:07:59 EDT]

Token signature:[zoQ7cb1BSWekvxdd3slUpxmCRlBCBmMO1nu8iztv73PKQN3MIybuCx/C9EdKGwoeoguJHKrj0BOOAeXxVLDgIQL5Jz2Tg6LQcIyTpAtRAVMsqWTPFzDBrs85Zxs9kP0zFEiOvEsDUmRXXm92dN6zxWooEyGz453x1VPmqGoZ0ww=]

As you can see, the user (principal) ID and expiration timestamps are easily visible, which was my goal from the beginning.

Full Java code:

import java.security.MessageDigest;
import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.Date;
import java.util.StringTokenizer;

import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.DESedeKeySpec;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

import org.apache.commons.codec.binary.Base64;

public class DecryptLTPA {
 private static final String AES = "AES/CBC/PKCS5Padding";
 private static final String DES = "DESede/ECB/PKCS5Padding";

 public static void main(String[] args) {
  String ltpaKey = "<DES key from ltpa token>";
  String ltpaPassword = "<password used to export ltpa token>";
  String tokenCipher = "<the header text to dercypt>";

  try {
   Base64 b = new Base64();
   byte[] secretKey = null;
   MessageDigest md = MessageDigest.getInstance("SHA");
   md.update(ltpaPassword.getBytes());
   byte[] hash3DES = new byte[24];
   System.arraycopy(md.digest(), 0, hash3DES, 0, 20);
   Arrays.fill(hash3DES, 20, 24, (byte) 0);
   secretKey = decrypt(b.decode(ltpaKey), hash3DES, DES);
   byte[] ltpaByteArray = b.decode(tokenCipher);

   String algorithm, userInfo, expires, signature, ltpaPlaintext;
   try {
    algorithm="DES";
    ltpaPlaintext = new String(decrypt(ltpaByteArray, secretKey, DES));
   } catch (Exception e) {
    algorithm="AES";
    ltpaPlaintext = new String(decrypt(ltpaByteArray, secretKey, AES));
   }

   System.err.println("Algorithm:["+algorithm+"]");
   StringTokenizer st = new StringTokenizer(ltpaPlaintext, "%");
   userInfo = st.nextToken();
   expires = st.nextToken();
   signature = st.nextToken();
   System.err.println("Full token string:[" + ltpaPlaintext + "]");
   Date d = new Date(Long.parseLong(expires));
   SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd-HH:mm:ss z");
   System.err.println("Token is for:[" + userInfo + "]");
   System.err.println("Token expires at:[" + sdf.format(d) + "]");
   System.err.println("Token signature:[" + signature + "]");

  } catch (Exception e) {
   e.printStackTrace();
  }
 }

 public static byte[] decrypt(byte[] ciphertext, byte[] key, String algorithm) throws Exception {
  SecretKey sKey = null;

  if (algorithm.indexOf("AES") != -1) {
   sKey = new SecretKeySpec(key, 0, 16, "AES");
  } else {
   DESedeKeySpec kSpec = new DESedeKeySpec(key);
   SecretKeyFactory kFact = SecretKeyFactory.getInstance("DESede");
   sKey = kFact.generateSecret(kSpec);
  }
  Cipher cipher = Cipher.getInstance(algorithm);

  if (algorithm.indexOf("ECB") == -1) {
   if (algorithm.indexOf("AES") != -1) {
    IvParameterSpec ivs16 = generateIvParameterSpec(key, 16);
    cipher.init(Cipher.DECRYPT_MODE, sKey, ivs16);
   } else {
    IvParameterSpec ivs8 = generateIvParameterSpec(key, 8);
    cipher.init(Cipher.DECRYPT_MODE, sKey, ivs8);
   }
  } else {
   cipher.init(Cipher.DECRYPT_MODE, sKey);
  }
  return cipher.doFinal(ciphertext);
 }

 private static IvParameterSpec generateIvParameterSpec(byte key[], int size) {
  byte[] row = new byte[size];

  for (int i = 0; i < size; i++) {
   row[i] = key[i];
  }

  return new IvParameterSpec(row);
 }
}

An updated version of MxLoader is available

Just released today:

https://www.ibm.com/developerworks/community/groups/community/MxLoader

The best easy and free video editing software available is Shotcut

I've never editing a video before, but I needed to yesterday. I found Shotcut, and I couldn't be happier about how easy it is to use. It's free, easy AND it's a portable app, so you don't have to install anything. They've got a very active forum and tons of how-to videos available. If you need to do some video editing (including multiple tracks, audio overlay, and all the high-ed options), give it a try.

JavaScript regular expression trick

Working on a Netcool Impact implementation recently I ran across a feature of JavaScript regular expressions that really impressed me. I'll compare it to a somewhat similar feature/syntax in Perl.

If you need to test a string for a regular expression in Perl, you can do the following:

if ($mystring =~ /my_regular_expression/) ...

That will return true if $string contains the specified regular expression.

In JavaScript, you can invoke the test() method directly on the regular expression (including the leading and trailing "/") with one parameter, which is the string to test. Here's what the equivalent of the above looks like in JavaScript:

if (/my_regular_expression/.test(mystring)) ...

And to test if it doesn't match, the syntax is:

if (!/my_regular_expression/.test(mystring)) ...

That's it. I just thought it was pretty neat.

APM historical data without a TEMS

IBM APM only directly supports the storage of up to 32 days of data. You've always had the option to store older data in the ITM 6 data warehouse, but setting up an entire ITM 6 infrastructure has always seemed like a whole lot of work. As it turns out, you don't have to set up the entire infrastructure! You oonly have to set up a warehouse proxy agent, summarization and pruning agent and the warehouse database. No TEMS or TEPS, as the WPA and SPA can work in autonomous mode.

Madhavan Vyk recently posted a great article on DeveloperWorks detailing exactly how to configure this:

https://www.ibm.com/developerworks/community/blogs/0587adbc-8477-431f-8c68-9226adea11ed/entry/Integrating_IBM_Cloud_APM_8_x_with_Tivoli_Data_Warehouse?lang=en