Beyond the Zero Day: Reverse Engineering Malicious Class Files

http://blogs.rsa.com/beyond-the-zero-day-reverse-engineering-malicious-class-files/

By Erik Heuser, RSA Advanced Cyber Defense Services Advisory Practice Consultant
In part 1 of this blog, “Beyond the Zero Day” we focused on detecting malicious JVM [Java Virtual Machine] activity and identifying the ‘blob’ that was downloaded.  No subsequent network activity was detected after the download, but that doesn’t discount successful malware delivery and deployment.  We can certainly seize and forensically examine the host, but that might require massive time investment for an organization and we don’t even know what we’re looking for yet.  The first place to start is by examining the Class file that kicked off the HTTP GET for our ‘blob’.
Reverse Engineering a Malicious Class File
When analyzing malicious JAR’s or Class files we’ll need a few tools.  To analyze JAR’s, I use JD-GUI, however this is a class file and JD-GUI doesn’t support raw Class files.  We’ll need to use JAD [Java Decompiler] to examine the plain text for the Class.  When using JAD or JD-GUI it’s important to note that the malicious actor is aware of these tools and actively plans for their use.  You’re going to encounter bytecode errors in decompilation and work around them, if possible, to get the applet to run.  You will also run into files that produce so many bytecode errors it’s impossible to debug them.  The Black Hole exploit kit JAR’s are compiled in such a way it makes it extremely hard to perform a dynamic analysis.  The code is also usually run through an obfuscation tool, making the flow difficult to follow and hiding certain strings like URL’s, other Class files, and decoding/decryption keys.
To reverse the Class I use Eclipse, a popular Java IDE.  After starting a new project and adding my Class file, I have to go through and manually fix the bytecode errors.  Luckily in this instance there are few errors and only several lines without functionality need to be commented out.

This is being launched as an applet. Generally, malware writers embed a variable in the applet to be passed to the JAR/Class almost as a key.  We easily found it with NetWitness and need to add it to the Class itself.  You’re typically looking for GetParameter().

Before we set out breakpoints and debug the Class, we need to examine the source and find interesting functions/strings to watch for.  We find quite a few large encoded strings within the Class.

We can also see a section that downloads from a URL and appears to vivify an array with 31 entries it populates with values in two’s complement.  In the while loop we can see the variable abyte0[] being applied with the ‘^=’ operator, which is the Java bitwise exclusive or [XOR] and assignment operator.   This operation is a classic stream decoder and matches up to the 31 byte repeating structures within our ‘blob’.

After examining these artifacts, I find it helpful to toggle breakpoints at the beginning of each of the functions and especially around the functionality of the stream decoder.  After we’re satisfied with our breakpoints and have no more errors, we can go ahead and debug as an applet.  While stepping through the Class, we can watch the variables as it decodes the strings.

The “com.sun.org.glassfish.gmbal.” section is cutoff, it adds “ManagedObjectManagerFactory”.  S1 has an interesting string, 0xCAFEBABE is the file magic for Class files.  This is a hex encoded Class file.   The “un.invoke.anon.AnonymousClassLoad” loads the new Class file, it was originally intended for patching and dynamic Class generation.  The Class needs to be decompiled with JAD and is fairly interesting.

System.setSecurityManager(null) allows the applet full access to the machine.  This allows the unsigned executable to be copied outside of the JVM sandbox and execute.
Before the decoder section, the code has to generate a URL to fetch the malware, one of the many loops with encoded strings assembles the URL.  Entering the decoder section we can see the URL and temp directory where the malware will be copied and executed along with the 31 byte XOR key.  Calculating the two’s complement value and transforming it to hex gives us the key; 8215E1AABBFEDBF24030AF734FB793D0A112CBBD2994EE25BF2F835DC2E5BC.

The URL matches up with what we saw in NetWitness and we now have the key to decode the ‘blob’.  Cursory sandbox analysis revealed no network activity, just like we observed in NetWitness.  In part 3, I’ll be covering how to move forward with the incident by examining the host and it’s memory.
Erik Heuser is an advisory Practice Consultant for the RSA NetWitness Incident Response /Discovery (IR/D) Practice at RSA. In this capacity, Erik is responsible for delivering holistic incident response services using state-of-the-art host and network-based technologies. In addition, Erik performs threat research and develops content / techniques that can be used by clients to identify compromise and mitigate risk.