Decrypting the idisrv.sth file from IBM Directory Integrator

Background

The idisrv.sth file that's included in IBM Directory Integrator isn't a normal stash file. A normal stash file can be decrypted with a simple Perl script (see my previous post). However, this one is different. Happily, IBM does include a method to read this file, though it's hidden in a Jar file and requires some Java know-how.

Java Code

Here's the Java code required to read the file (in a file named ftStashFile.java - the name of the file must match the name of the class):

import com.ibm.di.server.StashFile;

import java.util.Vector;

public class ftStashFile extends StashFile {

  public static void main(String[] args) throws Exception {

    try {

      Vector thePassword = readPasswords();

      System.out.println(thePassword.firstElement());

    } catch (Exception e) {

      System.out.println("Exception encountered: " + e.toString());

    }

  }

}

The command to compile it is:

javac -cp .:/opt/IBM/TDI/V7.1.1/jars/common/miserver.jar ftStashFile.java

Then the command to run it is:

java -cp .:/opt/IBM/TDI/V7.1.1/jars/common/miserver.jar:/opt/IBM/TDI/V7.1.1/jars/common/tdiresource.jar:/opt/IBM/TDI/V7.1.1/jars/3rdparty/IBM/icu4j-4_4_2.jar ftStashFile

NOTE: You need to have the idisrv.sth file in the same directory as ftStashFile.class when you run it.

NOTE2: You will have problems if the JDK you're using is older than the JDK that was used to compile the IDI jar files. You'll get an error similar to this if that's the case:

  bad class file: ../jars/common/miserver.jar(com/ibm/di/server/StashFile.class)

    class file has wrong version 55.0, should be 52.0

    Please remove or make sure it appears in the correct subdirectory of the classpath.

And that's it. I've got a video going through how I figured this out here for more details.

JD-GUI Download: 

https://java-decompiler.github.io/

Perl script for decrypting a GSKit-created stash file:

https://blog.gulfsoft.com/2025/01/decrypting-properties-in-ibm-directory.html

Keystore Explorer (open-source tool similar to iKeyMan):

https://keystore-explorer.org/downloads.html

Decrypting properties in IBM Directory Integrator

If you look at the usage from the IDI encryption utility (cryptoutils.sh), you'll see this:

[root@jazzsm9 serverapi]# ./cryptoutils.sh 
CTGDKD446I Usage:
-input <input file>
-output <output file>
-mode <encrypt|decrypt|encrypt_config|decrypt_config>
-keystore <keystore file>
-storepass <keystore password>
-alias <encryption key alias>
[ -keypass <key password> ]
[ -transformation <encryption transformation> ]
[ -storetype <keystore type> ]
[ -cryptoproviderclass <security provider used for encryption> ]

Unfortunately, none of those "-mode" options will let you decrypt values in any of the *.properties files (e.g. global.properties, solution.propterties, etc.) So how do you do it?

To get the answer, you need to find the online documentation here to find that there are two additional options that aren't listed above. They are:

encrypt_props
decrypt_props

Once you know that, you're over the largest obstacle. But now you have several additional flags with values to provide, and the documentation doesn't give you an example of doing exactly this. So here's the example:

/opt/IBM/TDI/cev10/serverapi/cryptoutils.sh \

-input /opt/IBM/TDI/ftsoldir/solution.properties \

-output /tmp/foo \

-mode decrypt_props \

-keystore /opt/IBM/TDI/ftsoldir/testserver.jks \

-storepass server \

-alias server

In the above case, I wanted to decrypt the encrypted values in my solution.properties file. My solution directory is /opt/IBM/TDI/ftsoldir. Notice also that you MUST provide the certificate alias that points to the server certificate in the solution directory. By DEFAULT (meaning: all of this can be changed), the alias of that certificate is "server", it is stored in the $SOLDIR/testserver.jks keystore, and the password of the keystore is "server". The name of the keystore and the alias are specified in these two properties in solution.properties:

com.ibm.di.server.encryption.keystore = testserver.jks
com.ibm.di.server.encryption.key.alias = server

If, however, you forget the password, that's not a good thing. Normally you can decrypt an IBM stash file with a perl script like this:

#!/usr/bin/perl
use strict;
die "Usage: $0 <stash file>n" if $#ARGV != 0;
my $file=$ARGV[0];
open(F,$file) || die "Can't open $file: $!";
my $stash;
read F,$stash,1024;
my @unstash=map { $_^0xf5 } unpack("C*",$stash);
foreach my $c (@unstash) {
last if $c eq 0;
printf "%c",$c;
}
printf " ";

However, that doesn't work on the IDI stash file (idisrv.sth) because this isn't a GSKit stash file. From the docs:

The stash file contains the Server keystore password values encrypted with AES128 with a fixed key.

Check back later to find out later how to read this stash file - I think I've figured out how to decrypt it.

I did figure it out, and here's the video.

And NOTE: This only matters if you want to CHANGE the password of the keystore and keep the keystore. If all you need to do is add a server certificate to the keystore for a new web connection, you can simply use the "Get Certificate" button in the HTTP Server Connector to get the server's certificate and add it to the keystore.

IBM Identity Manger: Adding a new attribute

Background

The documentation on this is lacking on this topic, and the example LDIF file provided is in a format that will cause idsldapmodify to fail with a painfully obtuse error message. So I'm documenting this here to help the next person avoid some headache.

Details

Here's the documentation on how to add an attribute. In there, it provides this LDIF file as an example:

dn: cn=schema changetype: modify add: attributetypes attributetypes: ( myAttribute-oid NAME ( 'myAttribute' ) DESC 'An attribute I defined for my LDAP application' EQUALITY 2.5.13.2 SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 {200} USAGE userApplications ) - add: ibmattributetypes ibmattributetypes: ( myAttribute-oidDBNAME ( 'myAttrTable' 'myAttrColumn' ) ACCESS-CLASS normal LENGTH 200 )

However, if you try to import that LDIF file with the idsldapmodify command given, you'll get the following error:

ldapmodify: invalid format (line 5 of entry: cn=schema)

After a bit of pain, I figured out that the whitespace in the lines are the problem. You need to either concatenate the continued lines together OR have some white space at the front of the continued lines. Additionally, there should be a space in betwee "oid" and "DBNAME". So if you change the file to the following, you will no longer get the error:

dn: cn=schema
changetype: modify
add: attributetypes
attributetypes: ( myAttribute-oid NAME ( 'myAttribute' ) DESC 'An attribute I defined for my LDAP application'    EQUALITY 2.5.13.2 SYNTAX 1.3.6.1.4.1.1466.115.121.1.15    {200} USAGE userApplications )
-
add: ibmattributetypes
ibmattributetypes: ( myAttribute-oid DBNAME ( 'myAttrTable' 'myAttrColumn' ) ACCESS-CLASS normal LENGTH 200 )
What about updating the objectclass?
That's what I was wondering! Simply adding an attribute doesn't really help you much until you add it as a MUST or a MAY attribute to an objectclass. IBM has an almost complete guide to doing that here, But it is incomplete. For one thing, the format of the data is wrong again. The long attribute values should all be on a single line, like this:
dn: cn=schema
changetype: modify
replace: objectclasses
objectclasses: ( 2.16.840.1.113730.3.2.2 NAME 'inetOrgPerson' DESC 'Defines entries representing people in an organizations enterprise network.' SUP 'organizationalPerson' Structural MAY ( audio $ businessCategory $ carLicense $ departmentNumber $ employeeNumber $ employeeType $ givenName $ homePhone $ homePostalAddress $ initials $ jpegPhoto $ labeledURI $ mail $ manager $ mobile $ pager $ photo $ preferredLanguage $ roomNumber $ secretary $ uid $ userCertificate $ userSMIMECertificate $ x500UniqueIdentifier $ mytest ) )
Additionally, where did that value for "objectclasses" come from?? Well, that came from this command:
ldapsearch -b cn=schema -s base objectclass=* objectclasses | grep \'inetOrgPerson\'
Running that, you will get a result similar to:
objectclasses=( 2.16.840.1.113730.3.2.2 NAME 'inetOrgPerson' DESC 'Defines entries representing people in an organizations enterprise network.' SUP organizationalPerson STRUCTURAL MAY ( audio $ businessCategory $ carLicense $ departmentNumber $ displayName $ employeeNumber $ employeeType $ givenName $ homePhone $ homePostalAddress $ initials $ jpegPhoto $ labeledURI $ mail $ manager $ mobile $ o $ pager $ photo $ preferredLanguage $ roomNumber $ secretary $ uid $ userCertificate $ userPKCS12 $ userSMIMECertificate $ x500UniqueIdentifier ) )
Modify that to include your new attribute, then put that into the LDIF file. Then run idsldapmodify to import that LDIF file into your server. After that, you'll have your new attribute added to the inetOrgPerson objectclass.

IDI v10 uses PKCS12 keystores

IDI Version 10 ships with PKCS12 keystores rather than JKS keystores, but the file names still end with .jks, which can be quite confusing, especially if you're still running on an OS that has an older version of java as the system java. If you have your shell configured with the defaults and you just run something like:

keytool -list -keystore testserver.jks -v

You'll get the following error:

keytool error: java.io.IOException: Invalid keysstore format
java.io.IOException: Invalid keystore format
...

The simple fix for this is to provide the full path the the IDI-provided keytool command, such as:

/opt/IBM/TDI/cev10/jvm/jre/bin/keytool -list -keystore testserver.jks -v

Which will show you details about all of the certificates in the testserver.jks keystore.

FYI: the default password for testserver.jks is "server" and for serverapi/testadmin.jks, it is "administrator"

IDI Development: Promoting Code From TEST to PROD

Background

The documentation on this topic is slim on the gory details, so I figured I would write a little bit about it. 

Situation

If you have different TEST/DEV and PROD environments (as you should), you will be creating solutions in TEST/DEV, then you need to somehow get those solutions to PROD. There are two difference scenarios to consider at this point.

1. Solutions with no external properties files

This is the easy scenario. And the way to make it the easiest is if you add your PROD IDI Server to the list of Servers in your TEST/DEV CE (in the bottom left hand pane of your CE):

Adding a remote server is easy if you simply have all of the defaults in place and all servers are at the same version of IDI. If you've updated certificates and/or have different versions of IDI, it's harder. So for this post, I'm going to assume you're on the easy path. So to add a remote server, you just need to first add a server to create a new server document. To keep things simple, name it the same as the remote hostname. Then right-click that server and select "Open Configuration". In the configuration, point to the remote IDI server like so:

My remote server is named jazzsm9, and the IDI server is running on the default port of 1099. Just the first (Server API Address) field needs to be filled in. It will actually fill in the last field (ActiveMQ Management port) automatically for you based on the port you specify in the first field. once you add it, the server should show up with a green circle with an inscribed white triangle next to it, like this:

Now you can simply Export your solution (Project) directly from the CE to the remote server. 

To do this:

1. Right-click your project and select "Export..."

2. Select "IBM Security Verify Directory Integrator-> Runtime Configuration (e.g. rs.xml)"

3. On the dialog that follows, make sure to select "Server", select the appropriate server, and specify a name for the solution. The name you pick will be the name of the XML file created in that remote server's $SOLUTION_DIR/configs directory. 

Here's a link to a short video showing the process.

2. Solutions WITH external properties files

This process is more involved because you not only have to copy the solution over to the remote server, you ALSO MUST COPY OR CREATE THE EXTERNAL PROPERTIES FILES ON THE REMOTE SERVER. This is one of the important parts that the existing documentation leaves out. The project/solution contains REFERENCES to external properties files, but does not actually contain the properties files themselves nor the properties in those files. This means that you will have to either copy the properties files over to the other server in the correct location or create them in the correct location. This isn't a huge step, but it's one you need to keep in mind when promoting code from TEST to PROD, especially the first time you do it for a particular project/solution.

IDI Operations: Always Start the IDI Server and CE with the "-s" Flag

Keeping it simple, you should always start the ibmdisrv or ibmditk processes with the "-s" flag to specify the solution directory to use. If you don't do this, then you're just using whatever happens to be set in your environment, which could lead to quite a bit of confusion. As an example, you could have your CE (the eclipse IDI GUI) running with a DIFFERENT solution directory than your Default ibmdisrv process. This could lead to a case where properties that resolve perfectly in the GUI don't resolve when you go to run an AL. You can avoid this by always using full paths to properties files, but then that makes your solutions much less portable.

IDI Operations: Finding the Solution Directory of a Running ibmdisrv Process

Background

I am working with a client that has had ITIM and TDI/IDI installed and running for over a decade, with different groups responsible for the administration of each. They now have around ten ibmdisrv processes running, with some of them configured to run the same Listener ALs (so the one that starts first wins). None of the servers are started with the "-s" flag, so it's a little difficult to figure out their solution directories. 

Each ibmdisrv and ibmditk process needs to have a Solution Directory (aka SOLDIR, SolDir, or current working directory) defined. All default logging is done in the solution directory, and all relative paths used are relative to the solution directory. Basically, the solution directory is very important, and this post will show you how you can find it for a running process.

The solution shown here is for Linux, because that is the most common OS I've seen for running IBM Directory Integrator. The second most popular platform is Windows. If you're running Windows, you can get similar data using the Process Explorer GUI tool.

Solution

The first thing to look at is the process listing for the IDI server java process. The easiest way to find all of these processes running on your server is with this command:

# ps -ef | grep IDILoader

root       5693   5683  0 Jan01 pts/0    00:07:57 /opt/IBM/TDI/ceV10/jvm/jre/bin/java -cp /opt/IBM/TDI/ceV10/IDILoader.jar -Dlog4j2.configurationFile=file:etc/log4j2.xml --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED com.ibm.di.loader.ServerLauncher -d

We see from that output that the PID of this java process is 5693 and the Parent PID (PPID) is 5683. The most straightforward way I've found to find the SolDir for this server is using this command:

# lsof -Pp 5693 | grep cwd

The output should be similar to:

java    5693 root  cwd    DIR              253,0      4096  68731158 /opt/IBM/TDI/prod_V7.1.1

Easy as that, we see that the SolDir for this server is /opt/IBM/TDI/prod_V7.1.1.

Additional Details

We can get more details about the PPID with this command:

# ps -fwwp 5683

UID         PID   PPID  C STIME TTY          TIME CMD

root       5683   5462  0 Jan01 pts/0    00:00:00 /bin/sh /opt/IBM/TDI/ceV10/ibmdisrv -d

What we see here is that the /opt/IBM/TDI/ceV10/ibmdisrv script was called with only the "-d" flag to start this process. When that script runs, it calls the bin/setupCmdLine.sh script, which eventually calls the script bin/defaultSolDir.sh to set the solution directory of the process. (A great post about this and more can be found at this link.) While this is truly great information to know, one drawback is that the solution directory can be overridden with the TDI_SOLDIR environment variable. That's why I like the lsof approach better on Linux. The behavior of lsof on AIX is different and doesn't point to the solution directory like it does on Linux, so this approach may be better there. In either case, it's good to know both.