Hiding in the Trees
Evading a (terrible) Random Forest Classifier
Machine learning components of security products come in many forms and are increasingly relevant to offensive security researchers. In Evading the Machine I worked through a basic evasion attack against a logistic regression classifier. In that example we saw that 'the model' is really just a plane inside a three-dimensional space that's described by the 'features' we selected. One of the weaknesses of logistic regressions is that as the number of dimensions increases, the spatial intuition for the model's explainability begins to break down. You cannot for example easily plot and visualize 5 or 6 dimensions, nor can you easily visualize the corresponding hyperplane that represents the decision boundary, and additionally achieving explainability becomes a challenge. Some solutions exist for these and other problems concerning the curse of dimensionality, but we can also mitigate some of those concerns all together by choosing a different machine learning algorithm.
A different and common example of machine learning classifiers are tree-based algorithms. Tree-based algorithms like the Random Forest Classifier we'll discuss in this blog offer additional benefits like being highly performant, deterministic, interpretable, and that potentially makes them a valuable capability in security products. In this blog we'll work through the construction of a simple Random Forest classifier, and examine the differences in the intuition required to achieve an evasion attack on these models. The complete code and graphics for this blog are available in the repository.
Building a Random Forest Classifier
For this example I reused the data from Evading the Machine, and jumped right into building the classifier. The scikit-learn library provides a convenient implementation of Random Forests that we can use for this example.
# Load dataset
try:
df = pd.read_csv("pe_features.csv")
except FileNotFoundError:
print("Error: 'pe_features.csv' not found.")
return
X = df.drop(columns=["label"])
y = df["label"]
# Convert to numpy arrays to avoid feature name warnings during classification
X = X.values
y = y.values
# Train/test split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.20, random_state=42
)
print(f"Training on {len(X_train)} samples, testing on {len(X_test)} samples.")
# Initialize and train Random Forest
# n_estimators is the number of trees in the forest
rf_model = RandomForestClassifier(n_estimators=3, random_state=42, min_impurity_decrease=0.001) # min_impurity_decrease was set so I could reuse injector_1.cxx
rf_model.fit(X_train, y_train)A Random Forest classifier is an ensemble of decision trees, each trained on a random subset of the data. The final prediction is made by aggregating the predictions of all the individual trees. This approach helps to reduce over-fitting and improve the model's generalization ability. In the snippet above, we specify that our random forest should be composed of 3 trees. This is for simplicity, but we could increase the number of trees to improve the model's accuracy.
Unlike some model architectures, most tree-based algorithms are not sensitive to the scale of the input features. This means that we don't need to apply and save a separate scaling transformation to the data before training the model.
Once we've implemented the training script above, we can run it and examine how the model performs on the test set.
This model is able to achieve an accuracy of 89%, which is not particularly good but it works well enough for this example.
Learning About the Classifier
For a Random Forest, the "decision boundary" is complex. It can be described as a series of axis-aligned splits that carve the feature space into hyper-rectangles. But because fundamentally a forest is just a collection of decision trees, we can actually "read" the logic of the classifier by looking at the individual trees.
The intuition here is that the model will invoke each of the trees in the forest to make a prediction, and the final prediction is made by aggregating the predictions of all the individual trees. This means that we can navigate the trees to understand the decision boundary of the model, and how a given binary's features are used to make a prediction.
In this example our forest consists of 3 trees, and we can examine the output of each tree individually to assess how a given binary will be classified. To simplify this process, classify.py is implemented to take in a path to a binary, and output a trace of the classification path through the trees. For convenience, I'll reuse the injector binary from Evading the Machine.
We build the injector binary using clang and then run it through classify.py to see how it is classified by the forest. The output shows that the injector binary is classified as malware. This is because the decision path through the forest leads to a leaf node with a distribution (annotated as [Benign, Malicious]) of [0. 1.] in Tree 0, [0.03125 0.96875] in Tree 1, and [1. 0.] in Tree 2. Note that Tree 2 actually classified the sample as Benign. The mean of the second number in each of those arrays becomes the ensemble's (ie. the forest's) prediction as malware with a probability of 0.6562.
We can follow this decision path to understand how the classifier makes its predictions in the .png available for the forest. By default all left branches are True, and all right branches are False.
Repeating this process for the other trees in the forest results in a highly interpretable intuition for the decision boundary of the model. However, because there's multiple trees, each with many branches, reversing a set of properties to make the binary benign is not straightforward. The model's decision boundary is complex and requires a deep understanding of the feature space to reverse engineer.
Evading the Classifier
What we can do now is extract the paths from the Random Forest that result in a Benign classification. We can do this by identifying the the output points (ie the Leaf Nodes) in each tree that result in Benign classifications, and then reverse engineer the logic of the forest to identify the features that lead to these leaves. We can AND together the nodes that lead to the Benign Leaf Nodes to build our intuition for the requisite properties that a binary needs to be classified as benign. A similar technique using classification paths has been used by CrowdStrike to improve the detection of sample similarities. For our purposes, we care about extracting information from the trees to facilitate the development of an evasive binary. I implemented this logic in evasive_analysis.py, and the core logic is below.
The script will generate a bunch of statements like the one below, broken out by tree. For simplicity, I'll focus on achieving the benign classification for Tree 0, but this same technique can, and sometimes must, be applied to multiple trees.
And we can either think really hard about how to simplify the logic, or pass it into your favorite large-language model to take a guess. I did the latter and got the following.
We can then modify our binary so that it fits the properties of our target path to the benign leaf node, and at that point we can expect that the classifier will output a benign label.
For this example I was able to reuse the injector_1.exe.
We see that the injector_1.exe has a weighted entropy of 4.1983 (which is between 4.0754 and 5.0910), a strings density of 14.1905 (which is greater than 10.0563), a log size of 4.0315 (which is less than 5.0572) and therefore satisfies our simplified path to Leaf 34 in Tree 0. In this case, these properties also achieve a benign classification for Tree 2 and therefore a benign label from the forest.
Considerations
This post explored how tree-based models differ from linear classifiers. While a logistic regression offers a single, smooth decision boundary, a Random Forest creates a "jagged" landscape of Boolean logic. We saw that while this complexity can make manual intuition harder, it also makes the model more transparent to programmatic reverse-engineering.
One major consideration is the depth and complexity of the forest. In this example, we used a very shallow forest with only 3 trees. Modern production classifiers may use hundreds of trees and thousands of features (including imports, API call sequences, and byte-level histograms). This creates a massive search space for evasion, where satisfying the "Benign" paths for a majority of trees becomes a significant optimization challenge.
Furthermore, this analysis assumes "White Box" access, where we have knowledge of the features the model is expecting, we have the model file (.pkl), and we can inspect the model's internal thresholds. In a more constrained "Black Box" scenario, an attacker can't directly backtrack paths and must instead rely on probing the model with variations of a binary to map out the decision surface indirectly.
Conclusion
By moving from a simple linear model to a Random Forest, we’ve seen that machine learning evasion is less about "tricking" an AI and more about understanding the specific logical constraints of a data-driven system. We demonstrated that if you can extract the underlying decision paths, you can deterministically derive the exact properties needed to nudge a malicious binary into a benign leaf in the trees (ie. a "hyper rectangle" of the feature space). Again, this intuition offer insights to offensive security practitioners on how to improve their understanding of machine learning features in security products, and (for defenders) how the mechanisms of a classifier can be improved to mitigate the risks of these attacks.
Last updated