Deriving Simulation Results

Introduction

This module aims at showing how to trace the securiCAD simulation results back to the research publications they are based on. We will look at a very limited number of attack steps and derive their impact on each other back to their publications.

The main purpose of this document is to demonstrate how simulation results can be back-tracked.

Starting Point

Model

Although it is quite possible to follow this document without using securiCAD in parallel, we recommend that you load the derivingSimulationResults.sCAD model to follow and see the different simulation results while reading.

Main Object

Our main object to investigate will be the Service object labeled “Gen COTS Srv” with the id number 20 in our model. In the Modeling From Scratch module we were mainly looking at the Write attack on a Datastore object and found out that this Service object was part of an attack path making that happen. Now we will instead look directly into the attack steps DeployExploit and FindExploit on the Service object.

Main Object Simulation Results

When looking at the “TTC” graph of each attack step in a calculated model, we see a collection of figures regarding time and probability for a particular attack step to succeed.

These figures are all describing the attackers success rate achieving this attack step and for the purpose of tracing the results, we will only look at the “Success Rate” parameter above the diagram.

At first, we will look at the DeployExploit attack step.

The DeployExploit attack step. Total success rate is 52%

 

Shortly, we will also look at the FindExploit attack step since it is a prerequisite to succeed with deploying an exploit.

The FindExploit attack step. Total success rate is 54%
The FindExploit attack step. Total success rate is 54%

 

The main difference between these is not in the “Success Rate” parameter, but instead in the shape of the probability distribution; the FindExploit probability picks up earlier since easier/quicker methods of acquiring an exploit are relatively quick in the beginning but stalls at a lower success rate than the more tedious ones. The FindExploit plot is showing the combined probability for finding an exploit regardless of how it was done.

Attack Step Relationship

This relationship is described in the “securiLang Quick Reference” material and is further detailed in the “SecuriLang Technical Documentation”, both available upon request. Please contact support@foreseeti.com.

In the section describing the Service object in the Technical Documentation we see that DeployExploit is coming from FindExploit. (We also see that it is influenced by the DEP and the ASLR defenses on the Host running the Service, but since they are set to On in our model, they don’t have an impact in our example model.)

 

Single Attack Step Probability

What we mean with “Single” attack step probability is when only one attack step is directly influencing/leading/contributing to the next attack step, like in this case. Some attack steps are a direct/immediate consequence of others (for instance, compromising a Host will immediately lead to compromising all Client objects connected to it) while others, like in this case, will require some extra effort from the attacker. In other words, provided that the attacker has found an exploit, further effort is required in order to also deploy it in a Service.

This is mentioned in a table along with the attack step definition, also from the Technical Documentation. In our case, we have the following table which also comes with a reference to the research paper providing the figures.

 

Checking a Single Attack Step Probability

According to securiCAD

Any probability in the attack step tree showing the attack paths and the simulation results are based on the success rate of previous attack steps. Because of this, the easiest way to check the probability between two attack steps is to disconnect the attacker from its original position and connect it directly to the previous attack step. In our case, we would connect it directly to the “Gen COTS Srv” Service object and select the attack step “FindExploit”.

Doing that and recalculating will give the following TTC graph for the DeployExploit attack step of the “Gen COTS Srv” object.

 

According to Research

Back-tracking the “12.6” TTC median value back to research is first done by looking at the “Sommestad T. H., 2012” reference. It points out a published research paper with the “Estimates of success rates of remote arbitrary code execution attacks” title. In this paper, we find the following table on success rate probabilities for server side attacks.

 

The highlighted line in the above table is representing the situation where there is an AccessControl connected to our service, there is no Deep Packet Inspection (IDS) in the model but the NX (DEP) and ASLR defenses are enabled.

This table is showing the probability levels for different scenarios/settings given that the attacker will spend 5 days on the attack. The figure we will use to calculate our TTC50% value is the Expected (mean) value saying 24%.

We assume that the time it takes to succeed in deploying exploit follows an exponential distribution parameterized by lambda. According to the table, within a week (assuming 5 working days), the success rate is X%, which translates to CDF(5 days) = X%.

Therefore the equation becomes ​\( 1 – e^{-5\times\lambda} = X\% \)​ which will give us lambda as

\[ \lambda = {{ln(1-0.24)}\over{-5}} = 0.05488736914 \]

The median value, shown as the TTC 50% level in securiCAD, is then calculated via

\[ TTC 50\% = {ln(2)\over\lambda } = 12.6 \ days \]

which is also the mean value found in the securiLang Technical Documentation.

FindExploit; A More Complex Attack Step

Now, we shall look at an attack step with more contributing attack steps influencing it; the FindExploit attack step.

Please remove the direct connection between the attacker and FindExploit on the Service object and reconnect it to Compromise on the Windows 7 Host.

Attack Steps Related to FindExploit

From the securiLang Technical Documentation we see that FindExploit can be achieved in several ways.

Attack steps related to FindExploit

 

The attack steps FindExploit, DevelopExploit and DevelopZeroDay are related to actually acquiring an exploit. UserAccess is required to finding out what service this is which in turn will reveal what exploit will do the trick. Exploits are in turn divided into two main categories; those who have a released patch to them and those who have not. Therefore, in order to make use of an exploit targeting a vulnerability which has a patch to it, it is required that the Service is not properly patched which is the reason that the Patched defense also comes into play.

Deriving the FindExploit Simulation Result

This part of this module will bring us to the core of what we want to investigate; deriving the result of a particular attack step based on the results of previous attack steps and defenses influencing it.

The Result of FindExploit

The following attack step result is what we want to investigate. We will, as before, use the “Success Rate” parameter as the indicator of the different contributions. The reason to this is that since the different contributing attack steps might, and often will, have different probability distributions, the combination of them that securiCAD does when presenting the result to us might be a bit tricky to handle manually. Therefore, we will look at the “Success Rate” which is showing the probability when most of the distributions have subsided to their final values.

The FindExploit attack step. Total success rate is 54%
The FindExploit attack step. Total success rate is 54%

 

Attack Step Tree of FindExploit

When having run a simulation, selected our Service’s FindExploit attack step and looking at the attack step tree, we shall see the following content. Please make sure to use the “All Paths” mode when investigating results like this.

Attack step tree of FindExploit

 

Nomenclature Recap

In this attack step tree we see all different phenomena that an attack step tree can have. Let’s brush up our memory a bit.

  • The && symbols on the vertical lines tell us that all attack steps at this level are required in order to achieve the attack step above it. (Often referred to as “AND” steps.)
  • Unlabeled vertical lines hold attack steps that may contribute on their own without requiring other attack steps to contribute to the next level. (Often referred to as “OR” steps.)
  • The “Caused by…” label is used when several attack steps are combined and contribute to the upper level attack steps. You may also see it as the “Caused by…” label is used to distinguish between attack steps of “AND” type and “OR” type even though they actually operate on the same level in the tree.
  • Labels in italics represent imperfect defenses that allows for an attack step to be successful.

The “(20)Gen COTS Srv” object is our Service and the “(24)Gen COTS Srv” is the SoftwareProduct object connected to it. They are not the same object even though their names are identical apart from their id numbers.

UserAccess

Reading from the top of the tree, we see that there is an && label on the vertical line connected to the “(20)Gen COTS Srv.UserAccess” attack step and the next “Caused by…” attack step. This means that the attack step UserAccess is required for FindExploit to be successful in the first place. In our example, UserAccess is immediately 100% successful which means that it will not interfere with the simulation we want to investigate.

Unpatchable and ZeroDay

On the next level, we find one “Caused by…” label leading to attack steps related to Patchable vulnerabilities, which we will soon take a closer look at, and three attack steps not related to patches. They are “(24)Gen COTS Srv.DevelopExploitForPublicUnpatchableVulnerability”, “(24)Gen COTS Srv.DevelopZeroDay” and “(24)Gen COTS Srv.FindExploitForPublicUnpatchableVulnerability”.

In all three cases, patches will not help since either the vulnerability is known but there is no patch (yet) or the vulnerability is not known/published at all.

Patchable

The two attack steps labeled “…ExploitForPublicPatchableVulnerability” are both connected “behind” the “(20)Gen COTS Srv.Patched” defense in italics since they are dependent on the Service not being entirely patched in order to contribute to the next step.

Success Rates in the Attack Step Tree

The “Success Rate” results of the involved attack steps (and the defense) are as follows;

  • “(20)Gen COTS Srv.Patched” is 25% successful within 100 days. Actually, the defense is set to 75% and what we see here is the probability of bypassing it.
  • “(24)Gen COTS Srv.DevelopExploitForPublicPatchableVulnerability” is 83% successful within 100 days.
  • “(24)Gen COTS Srv.FindExploitForPublicPatchableVulnerability” is 75% successful within 100 days.
  • “(24)Gen COTS Srv.DevelopExploitForPublicUnpatchableVulnerability” is 18% successful within 100 days.
  • “(24)Gen COTS Srv.DevelopZeroDay” is 30% successful within 100 days.
  • “(24)Gen COTS Srv.FindExploitForPublicUnpatchableVulnerability” is 15% successful within 100 days.

What the Research Says About Finding Versus Developing Vulnerabilities

In the above example we have attack steps for both finding and developing a vulnerability (both for the patchable and the unpatchable case). Looking at the research paper regarding this, we find a flow chart describing that an attacker would not go for both finding and developing an exploit for the same vulnerability.

The flow chart and this discussion around it comes from the “A bayesian network model for likelihood estimations of acquirement of critical software vulnerabilities and exploits” research paper published by H. Holm, M. Korman and M. Ekstedt published in 2014 at the Royal Institute of Technology, Stockholm, Sweden.

Exploit acquirement flow chart

 

For the Patchable case, the attacker will choose either the D-H-L track or the D-F-L track. For the Unpatchable alternative, the attacker will choose either the E-J-M track or the E-I-M track.

It will do either one but not both and the securiCAD simulation logic will select the most successful alternative to represent it.

DevelopZeroDay will run a race of its own, like always.

Manual Calculations

According to the previous discussion, we may omit the less successful “(24)Gen COTS Srv.FindExploitForPublicUnpatchableVulnerability” and “(24)Gen COTS Srv.FindExploitForPublicPatchableVulnerability” attack steps from our manual calculations. This leaves us with the following figures.

  • “(20)Gen COTS Srv.Patched” is 25% successful within 100 days.
  • “(24)Gen COTS Srv.DevelopExploitForPublicPatchableVulnerability” is 83% successful within 100 days.
  • “(24)Gen COTS Srv.DevelopExploitForPublicUnpatchableVulnerability” is 18% successful within 100 days.
  • “(24)Gen COTS Srv.DevelopZeroDay” is 30% successful within 100 days.

To combine these into the final “(20)Gen COTS Srv.FindExploit” attack step’s success rate, we need to do the following;

  1. Calculate the contribution from “(24)Gen COTS Srv.DevelopExploitForPublicPatchableVulnerability” provided that it is partly blocked by the “(20)Gen COTS Srv.Patched” defense. This is done via 0,83 x 0,25 = 0,2075
  2. Calculate the contribution from “(24)Gen COTS Srv.DevelopExploitForPublicUnpatchableVulnerability” or “(24)Gen COTS Srv.DevelopZeroDay” which are independent of each other. This is done via 1 – (1 – 0,18)(1 – 0,3) = 1 – 0,82 x 0,7 = 1 – 0,574 = 0,426
  3. We now have the contribution from both the “patchable” track and the “unpathcable” track (with ZeroDay) and shall combine them as well. This is done via 1 – (1 – 0,2075)(1 – 0,426) = 1 – 0,7925 x 0,574 = 1 – 0,4549 = 0,545
  4. Our manual calculations gives us a “Success Rate” result for “(20)Gen COTS Srv.FindExploit” of 54,5% and securiCADgives us a result of 54%. Q.E.D.