In this post, we will explore why RASP, or Runtime Application Self-Protection, is not always effective in protecting your Java applications and can be bypassed. Introduction Open source security has long been problematic, yet many organizations continue to overlook its importance. Existing sandboxing solutions such as seccomp and LSMs (SELinux,
In this post, we will explore why RASP, or Runtime Application Self-Protection, is not always effective in protecting your Java applications and can be bypassed.
Open source security has long been problematic, yet many organizations continue to overlook its importance. Existing sandboxing solutions such as seccomp and LSMs (SELinux, AppArmor) function at the application/process level and are not supply chain aware.
With all the magic we use from modern frameworks, we must be prepared. We have all seen this with incidents like Log4Shell, Spring4Shell, and Text4Shell.
Everyone who followed Spring4Shell, was probably surprised to find out that Spring could lead to remote code execution in the first place, as it should not have that capability (to be more specific, Spring was used as a 'proxy' to deploy a web shell using classloader manipulation, for more details, I have published a detailed article about it two years ago : https://medium.com/geekculture/spring-core-rce-cve-2022-22965-a-deep-understanding-f0bd02113769).
Anyway, what we are implementing is Full Trust, and there is a huge gap between what we expect from a package and its actual potential.
Have you ever inspected a log4j library for malicious code before using it? I think most of us simply cross our fingers and trust these libraries. Imagine the consequences if libraries like log4j or Spring were compromised...
Likely, a sophisticated attacker would avoid starting an operating process as it may be detected by an EDR, so they might just disable all the security mechanisms implemented in the application. We need the in-app context to detect elaborated attacks.
These concerns are not purely theoretical; we have witnessed this with the xz backdoor. Imagine the potential damage to the healthcare or financial industries. We are talking about the possibility of damaging a nation's financial system. What about insiders? How can you be sure that no one is tampering with your application?.
Static and traditional dynamic tools cannot detect such activities, and this is where Application Detection and Response solutions (ADRs) or RASP comes in.
RASP, or Runtime Application Self-Protection, is a security technology introduced by Gartner a decade ago. With RASP, the application is empowered to protect itself without relying solely on perimeter-oriented technology, like WAFs.
It is very difficult to protect applications without context. RASP sees data in its effective format, by instrumenting the application and tracking the data as it flows from the point of request reception through to the Java runtime libraries, it can detect and block a plethora of attacks.
I will redirect our focus today to Java applications.
Java is complex and offers a comprehensive set of over-powerful APIs to both users and developers, which attackers can exploit. To safeguard these sensitive APIs from misuse, RASP integrates with the application, deploys several sensors/actuators to achieve full visibility, and blocks attacks in real time, including embedded backdoors (yet it depends on the solution you are using). The generated security events can then be forwarded to a security data lake, a SIEM, a ticketing system, or b integrated with an XDR platform.
Vulnerabilities like Spring4Shell, Log4Shell, and many other zero-days could have been or can be prevented using RASP.
There are different ways to implement RASP, in this article, I will focus on instrumentation-based solutions.
Here’s how it works: initially, an agent conducting the instrumentation is developed using bytecode manipulation libraries such as ASM, ByteBuddy, or Javassist.
When you start a Java program, your code is not executed immediately. Instead, Java creates a virtual machine, then loads and starts the agent, which registers a class transformer. This transformer modifies Java classes to add sensors, in other words hooking.
Thus, when the application runs and a class is loaded, the agent or transformer is notified, and hooks are added to that specific class.
Now, each time code interacts with this class or API, RASP becomes aware and determines whether the call should proceed or not.
Here is what it looks like.
The ProcessBuilder class creates operating system processes. Typical and naive RASP solutions would modify its start method and add a security check just before the actual execution to decide whether it should proceed. For example, they might examine how the call was reached, check if HTTP parameters or headers are part of the process binary or program, and then make a decision. As I will explain in more detail later, placing your hooks here is not recommended and can be bypassed.
When a program operates on a computer, it must access memory to store and retrieve data. However, the operating system manages physical memory (the actual memory chips within the computer), and direct access by programs is restricted. Each process operates within its own address space, allowing different processes to utilize the same virtual addresses while corresponding to distinct physical addresses, hence storing different data in memory.
Process Injection involves executing code within the context of another, typically benign, process, potentially granting access to that process's memory. Executing code through process injection can evade detection by security products like Endpoint Detection and Response (EDR) systems because the malicious activity appears to operate under a legitimate process, such as a Java process
One very important thing about the JVM, is that it suppose that the environment in which it runs is secure. In this section we will see how one can hijack the JVM from another process using process injection as disused earlier and wipe out the RASP probes.
Image if an attacker can get to remove guards and execute code in the name of the java process, he can even evade EDRs, as security engineers may think that java process is already patched with RASP and there is nothing that could happen and mess with it.
Ptrace is a system call that allows a process to trace another process that is being executed. It provides a way for a process to inspect and manipulate the state of another child process.
Ptrace is often used by debuggers and performance profiling tools to monitor and control the execution of processes. It can be used for both legitimate and malicious purposes.
To illustrate this in a real-world context, consider an Android malware that was reported to be spying on QQ and WeChat messengers. Basically, the malware used ptrace to inject itself into the WeChat process, and then exploited DexClassLoader, which is a class loader that dynamically loads classes from .jar files. Once loaded, the malware within the jar would hook into activity changes made by the user, with the intention of stealing personal information and transmitting it to a remote server.
RASP works by instrumenting sensitive classes including Java API class library and even OSS frameworks. For instance, to guard against RCEs, one might consider instrumenting classes like java.lang.ProcessBuilder (we will discuss later why this is not enough).
Our strategy involves utilizing JNI functions to redefine classes that have already been loaded and instrumented (e.g., java.lang.ProcessBuilder#start). By replacing the instrumented version with the original class that lacks instrumentation, we effectively disable the RASP sensor, essentially undoing the initial patch (repatching the patch).
The objective of this section is to develop a library that, once loaded, will employ the JNI interface to interact with the JVM and modify already loaded classes.
Let's proceed with our target class, for example, java.lang.ProcessBuilder, and convert the class into a hexadecimal representation. We will then convert it back to its byte array representation when we are ready to redefine the class at a later stage .
Next, we will define a function that activates upon the library's injection. This function should run in a new thread (for more details why, please refer to the JVM Section at the end of this post).
Afterward, we call JNI_GetCreatedJavaVMs to obtain a reference to the VM instances, which is typically 1.
Next, we attach to the JVM using AttachCurrentThread().
We then call GetEnv() to obtain a reference to the JVMTI interface.
Next, we locate our target class using the JNIEnv interface and redefine it with our version, by converting the previously mentioned hexadecimal representation back into bytes.
Finally, we redefine our class using the JVMTI interface and attach it to the current thread.
To demonstrate the attack in action, we use CVE-2022-22963, a SpEL (Spring Expression Language) Injection vulnerability in Spring Cloud Function. Spring Cloud represents a suite of tools designed to facilitate the development of distributed systems. Specifically, Spring Cloud Function is a serverless framework developed within the Spring Boot ecosystem.
In the spring-cloud-function-web module, routing can be specified through the spring.cloud.function.definition or spring.cloud.function.routing-expression message headers, which enable the use of Spring Expression Language (SpEL). Given that SpEL permits method calls, it can be exploited to execute commands, for example, by invoking new Runtime.exec(cmd) when StandardEvaluationContext is employed. StandardEvaluationContext is a versatile and configurable EvaluationContext implementation, utilizing standard strategies and reflection to resolve properties, methods, and fields.
As one can see below, adding a spring.cloud.function.routing-expression parameter to the POST request header, Spring Cloud Function will directly pass the parameter value into a SpEL interpreter which causes SpEL injection.
Let's proceed by running the application with the RASP agent.
As shown below, the agent successfully detected and blocked the attack, throwing a Security Exception: Expression Language Injection detected .
The attack was successfully detected and blocked due to the instrumentation of org.springframework.expression.spel.standard.SpelExpressionParser. To counteract this, we will inject a library into the Java process that, once loaded, will replace this class with its original, non-instrumented version.
However, RASP agents are usually designed by default to detect such modifications. To circumvent this notification, we can modify the sun.instrument.TransformerManager class to disable it. This can be achieved, for example, by implementing an early return in the sun.instrument.TransformerManager.transform method.
Now we see that we were able to bypass the first guard, but we have another issue, the application is also patched against RCEs.
Let's also reapply our patch to the same class (java.lang.ProcessBuilder).
As shown below, we successfully bypassed the RASP protection, enabling the creation of a disguised native subprocess. This clearly shows that if the environment hosting the application is not secure, then the application itself remains vulnerable, regardless of how securely it has been coded.
Of course, this scenario assumes that the attacker has the capability to inject the library into the process. An attacker might exploit a vulnerability to upload a file, such as a temporary file, and then load the library using System.load() (we will cover it later).
RASP solutions would typically uses input tracing to see if the input provided by a potentially malicious user reached a sensitive sink. What I did to bypass one solution it is to issue the command parameter in a reverse order and once my code gets the chance to be executed it will reverse it back. However, it looked like RASP solutions also has some blacklisting, to get around it is to create a symbolic link, and bypass it. The code snippet is shown below and you can reproduce here:
if(request.getParameter("%{parameter}i") != null){
String tempFile = "/tmp/" + new Random().nextInt();
String command = request.getParameter("%{parameter}i");
StringBuilder reversed = new StringBuilder();
for (int i = command.length() - 1; i >= 0; i--) {
reversed.append(command.charAt(i));
}
String reversedCommand = reversed.toString();
java.nio.file.Files.createSymbolicLink(java.nio.file.Paths.get(tempFile), java.nio.file.Paths.get(reversedCommand.split(" ")[1]));
Process p = Runtime.getRuntime().exec(reversedCommand.split(" ")[0] + " " + tempFile);
System.out.println("RASP Bypassed command: " + reversedCommand + "executed successfully");
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
DataInputStream dis = new DataInputStream(in);
String disr = dis.readLine();
while ( disr != null ) {
out.println(disr);
disr = dis.readLine();
}
}If you remember the process injection bypass, it required to have enough privilege to attach to the java process, however, when the application is not properly configured, e.g. when it accepts large request headers size, and you get SPEL injection that you can abuse, you can do it step by step, first load your library in a temp directory, and then load in the JVM using System.load(..), again if the RASP solutions does not insert guards around the libraries being loaded, you get the bypass.
https://github.com/mouadk/Insomnihack2024/tree/main/bypass-using-jni-injection
To instrument Java classes within the JVM, RASP agents are typically initialized using the -javaagent flag, which specifies the agent's JAR file. Alternatively, the Attach API can be used to load the agent after the JVM has already been started.
Using the Attach API one can achieve the same thing we did earlier with native code through the JVMTI interface, but from plain Java. We can repatch sun.instrument.TransformerManager , java.class.ProcessBuilder and org.springframework.expression.spel.standard.SpelExpressionParser classes and disable the RASP probes as follows:
If we attempt to run the application once more with the RASP agent while attaching our own agent, we can effectively disable the notification to the class transformer, thus bypassing RASP.
First we got blocked.
Then we deployed our agent.
And then bypassed it (cf stack trace below , the agent did not get notified and failing to convert java.lang.ProcessImpl means we were able to create and start the process).
RASP works by adding guards around security-sensitive APIs such as process creation and file operations. However, when implementing RASP or any other sandboxing solution, you should consider all possible attack scenarios, because attackers will probably try to get around it.
We should learn from the past. This is very important. If we take the process creation use case. The java security manager, which has been around for a while (now set to be removed), is already bypassable using Reflection. In fact, the java security manager check that the calling code is allowed to create a subprocess in java.lang.ProcessBuilder.start() which delegates to ProcessImpl.start.
But an attacker can abuse reflection and invoke ProcessImpl.start () (not public so normally not accessible through plain java). So guards should be closer to the native call and not ProcessBuilder.start (e.g. ProcessImpl.init or ProcessImpl.forkAndExec - even with that, you don't have any guarantee) . Here is how one can bypass it:
String processImplClassName = "java.lang.ProcessImpl";
Class<?> processImplClass = Class.forName(processImplClassName);
Method startMethod = processImplClass.getDeclaredMethod("start", String[].class, Map.class, String.class, ProcessBuilder.Redirect[].class, boolean.class);
startMethod.setAccessible(true);To illustrate the method, let's use Spring4Shell and show that using reflection, we can bypass a RASP Agent.
Another way to bypass RASP (if we assume that it has properly implemented guards around java.lang.ProcessImpl), is to use Unsafe API to invoke the native method java.lang.ProcessImpl#forkAndExec (OpenRASP for example add the guards at the constructor level).
java.lang.ProcessImpl#forkAndExecAs shown below, we have successfully bypassed the RASP Agent. The code required to replicate this bypass is available at https://github.com/mouadk/Insomnihack2024/tree/main/bypass-using-unsafe-abuse.
Note that with newer versions of Java, additional parameters are required for the pass to work. These include --add-opens java.base/java.lang=ALL-UNNAMED and --add-exports=java.base/jdk.internal.misc=ALL-UNNAMED(the JVM has been hardened since).
The goal of this section is to show how one can use the unsafe API to operate on the memory directly to bypass RASP (Here RASP is used as an attack vector).
I won't go too much into details about unsafe API but here is what you need to remember for this section:
Unsafe.getLong(long address) can be used to read a value from a given memory address.java.lang.instrument uses a native JVM TI agent called the JPLISAgent. The JPLISAgent manages the initialization of all the Java programming language Agents. JPLISAgent structure is shown below.
As one can see above, it has a handle to the Instrumentation instance which means, if we can get a hand on it, we can get the object as discussed earlier and abuse its API to redefine classes.
Java will create a JPLISAgent when it is loaded:
Access to JVMTI functions is done through JVM TI interface pointer, the JvmtiEnv, the environment pointer (JvmtiEnv is a subclass of JvmtiEnvBase).
As shown below, The VM stores a pointer value associated with each environment. This pointer value is called environment-local storage. It is a void pointer so it can point to anything.
After a small inspection, it points to JPLISEnvironment from which we can obtain the JPLISAgent and thus the Instrumentation instance. More specifically, here what happens: When java is started with an agent to be loaded, it will create and initialize a new JPLISAgent. The initialization consists of setting the EnvironmentLocalStorage with the JPLISEnvironment.
The JPLISEnvironment wraps the corresponding agent.
Specifically, if we know the base address of the module libjvm ( One can deduce the base address by reading /proc/self/maps) we can locate the offset of JvmtiEnvBase::_head_environment which is static (e.g. 0x00000000019b53b8 in jdk17), and add to that the base address we identified and then we get a pointer to the JvmtiEnv. and Then navigate the memory space to reach the InstrumentationImpl object address and reconstruct it to redefine classes (yes we will use the RASP agent itself to bypass it, this was already explored by other people in the past e.g.: https://github.com/BeichenDream/Kcon2021Code/tree/master).
(lldb) image dump symtab libjvm.dylib
[150621] 150621 Data 0x00000000019b53b8 0x000000010473d3b8 0x0000000000000008 0x001e0000 JvmtiEnvBase::_head_environment
Note that every time JvmtiEnvBase or JvmtiEnv is initialized, it will Add the environment to the end of the environment list and set the global variable JvmtiEnvBase::_head_environment. If you have more than one agent loaded, you can for example navigate the list and choose the agent in interest, for example by inspecting its codeSource, jarFile ect.
Now, If you have access to JvmtiEnv, you get access to the agent address and object address of type sun.instrument.InstrumentationImpl, which can then be used to repatch instrumented RASP Classes and bypass it. I used this to bypass a RASP solution supposed to protect against Spring4Shell. An exploit can be found at:
There is a guy who made an interesting observation : when allocating memory outside the heap, slightly above, there are many pointers pointing to libjvm. However, with ASLR usually enabled, he attempted to statistically deduce the base address. For example, if RASP has inserted probes preventing you from directly reading the base address, you can employ similar methods.
The final bypass method we'll discuss today involves escaping the JVM to execute code in a different environment, such as another JVM.
I used for this the JShell API, introduced in JDK 9, a command-line tool used to interactively evaluate Java declarations and statements.
Specifically, JShell creates a child target JVM to execute user code. By leveraging the internal implementation of JShell, we can execute Java code in a JVM where the RASP agent is not loaded, thus remaining completely undetected.
https://github.com/mouadk/Insomnihack2024/tree/main/bypass-using-jvm-escape
Java introduced the Java Platform Module System (JPMS) in JDK 9, allowing packages to be encapsulated within modules. This change means that a method from one package calling another in a different package now involves inter-module communication, with a method in a package within one module calling a method in another package within a different module.
This shift ensures that internal APIs, even those previously public, are encapsulated and made inaccessible to application code, even with the use of reflection. This encapsulation applies to both critical public methods and fields as well as private ones.
Most bypass techniques rely on the Reflection API to access private fields or methods. With strong encapsulation, such access is no longer possible unless explicitly permitted.
However, projects not using modularity, like Spring Boot, can still be manipulated.
Lastly, there are VM options that can relax this strict encapsulation, which should be used cautiously. For example, the option --add-opens java.base/java.lang=ALL-UNNAMED grants deep, unrestricted reflective access to the critical java.lang package, effectively disabling the security benefits of module encapsulation.
The Unsafe API has the capability to do almost anything, making it a 'superpower' API.
Now, the Java team is likely aware of how dangerous this API is, but it's unclear how long it will remain accessible.
The JDK has become more secure, especially with the introduction of the module system, and continues to introduce changes to mitigate the risks associated with the Unsafe API.
For example, defining classes or anonymous classes via this API is no longer supported. While the unofficial package sun.misc.Unsafe has been migrated to jdk.internal.misc.Unsafe, a version of sun.misc.Unsafe still exists. This version is encapsulated but allows for unrestricted reflective access.
It is not scheduled for removal but is expected to be gradually stripped of its more hazardous capabilities, such as direct memory access.
Hardening the Java Development Kit (JDK) is an ongoing, continuous process. The Java team is working to enhance the security of the platform with each release.
For instance, JDK 12 extended reflection filters to further restrict access to some critical and internal fields.
As mentioned earlier, JDK 9 introduced the module system, and JDK 17 enhanced this with encapsulation of JDK internals.
JDK 20 added more fine-grained control over LDAP and RMI protocols, addressing potential vulnerabilities associated with these technologies.
Dynamic loading of agents is disallowed by default starting with JDK 21.
In JDK 22, the Foreign Function & Memory (FFM) API was finalized. This API is used to efficiently invoke code outside the JVM and to safely access memory not managed by the JVM, offering performance comparable to, if not better than, the JNI and sun.misc.Unsafe API.
However, it is not known when the memory access methods in the publicly accessible Unsafe API will be removed.
To fully understand how RASP works and how it can be bypassed, it's important to delve into the initialization process of Java and how your agent gets loaded, among other aspects. I added this section for those who are motivated deep dive into the subject. I hope it helps.
The Java Native Interface (JNI) enables Java applications to interact with native libraries written in other programming languages, such as C, C++, or assembly. Functions and type definitions can be found in the jni.h header file.
In order to use JNI functions, the JNI API defines two data structures, JavaVM and JNIEnv, both points to an array of interface functions.
As shown below, the JavaVM interface, or more specifically, the JNIInvokeInterface, provides a distinct set of methods that manage VM-related operations, such as destroying the JVM and adding threads.
To obtain the JavaVM pointer in native code, the exported function JNI_GetCreatedJavaVMs can be used.
On the other hand, the JNIEnv (JNINativeInterface) allows a native application to to access VM features. The JNIEnv is only valid in the current thread and provides access to a variety of methods, such as FindClass, DefineClass etc.
To access the JNI core functions, a thread must first call AttachCurrentThread() to attach itself to the VM (previously set) and obtain a pointer to the JNIEnv interface. Once attached to the VM, a native thread operates similarly to a regular Java thread running inside a native method. The native thread remains attached to the VM until it calls DetachCurrentThread() to detach itself.
It is crucial for the attached thread to have sufficient stack space. For instance, when using pthreads, the stack size can be specified in the pthread_attr_t argument of pthread_create.
To summarize, JNI defines two key data structures: JavaVM and JNIEnv. The JNIEnv interface offers most of the JNI functions and is essential for resolving Java classes, calling Java methods, etc. Therefore, if an attacker gains access to JNIEnv, bad things can happen.
The JVM Tool Interface (JVM TI) serves as a native programming interface designed for use by tools, including profiling, debugging and monitoring tools.
The JVMTI interface facilitates bytecode instrumentation, allowing for the redefinition of classes already loaded into the JVM.
Similar to how JNI functions are accessed, access to JVMTI functions is facilitated through an interface pointer.
A JVMTI environment can be obtained through the JNI Invocation API's GetEnv function, as illustrated below:
JavaVM* jvm
jvmtiEnv *jvmti;
...
(*jvm)->GetEnv(jvm, &jvmti, JVMTI_VERSION_1_1);If we summarize, native code can obtain a pointer to the JVM instance running in the current process (typically there is only one) using JNI_GetCreatedJavaVMs. Next, the GetEnv function can be used to obtain the JNIEnv or JVMTI interface.
In this section, we will delve into how the JVM is initialized and how JavaVM and JNIEnv interface pointers are set.
The entry point of the JVM is implemented in the main.c file.
At a certain point during its execution, the main function delegates the continuation of the JVM startup process to another function, named JLI_Launch(...)
CreateExecutionEnvironment() is invoked to determine where the JVM library is located .
Next, LoadJavaVM(...) is invoked. LoadJavaVM(...) is responsible for dynamically loading the JVM library into the memory of the running process. The exact JVM library loaded depends on the operating system and the architecture of the target platform.
Once the JVM library is loaded, the LoadJavaVM function retrieves the addresses of JNI functions (e.g JNI_CreateJavaVM).
These addresses are assigned to a structure of type InvocationFunctions defined as follows:
After LoadJavaVM() completes, InitializeJVM() is called. It initiates the Java Virtual Machine using JNI_CreateJavaVM() (defined in jni.cpp) which loads and initializes a Java VM and returns a pointer to the JNI interface pointer.
The invocation of JNI_CreateJavaVM_inner function completes the initialization of the JVM.
To ensure that only one instance of the JVM is created per process, an atomic lock mechanism is employed. This restriction is necessary because the JVM utilizes global variables.
The function Thread::create_vm() is subsequently called.
The Thread::create_vm() (defined in threads.cpp ) initialize various things; it will create a JavaThread instance and link an OS thread with a Java Thread object.
When JavaThread is initialized, it will set the JNIEnv in the current thread.
As shown above, it first retrieves the JNI functions (jni_functions()), that is, the JNINativeInterface_:
and then set it in the current thread.
Now, the created JNIEnv instance is stored in the thread's _jni_environment variable.
and it can be accessed using thread->jni_environment().
The JavaVM (main_vm) is initialized with the jni_InvokeInterface as a global variable in jni.cpp:
When JNI_CreateJavaVM finishes, the main class is loaded.
and finally, the static main() method is called from the Main class.
That's it. It took me some time to write this article, and I hope you found it beneficial. If you think you have found a mistake, please let me know!.
RASP or not RASP, the in-app context in the only way to protect comprehensively applications from being manipulated by attackers (known vulns, zero-days, backdoors ...).
Jeff Williams
jvm-raspBeichenDreamcodeplutosh2-stack
GlassyAmadeus
