Re: Scientists create tool to kill cyber attacks in ‘less than a second’

[Undiscussed Horrific Abuse, One Victim of Many]

https://www.hindawi.com/journals/scn/2021/8933681/

> Abstract
>
> Perimeter-based detection is no longer sufficient for mitigating the
> threat posed by malicious software. This is evident as antivirus (AV)
> products are replaced by endpoint detection and response (EDR) products,
> the latter allowing visibility into live machine activity rather than
> relying on the AV to filter out malicious artefacts. This paper argues that
> detecting malware in real-time on an endpoint necessitates an automated
> response due to the rapid and destructive nature of some malware. The
> proposed model uses statistical filtering on top of a machine learning
> dynamic behavioural malware detection model in order to detect individual
> malicious processes on the fly and kill those which are deemed malicious.
> In an experiment to measure the tangible impact of this system, we find
> that fast-acting ransomware is prevented from corrupting 92% of files with
> a false positive rate of 14%. Whilst the false-positive rate currently
> remains too high to adopt this approach as-is, these initial results
> demonstrate the need for a detection model that is able to act within
> seconds of the malware execution beginning; a timescale that has not been
> addressed by previous work.
> 1. Introduction
>
> Our increasingly digitised world broadens both the opportunities and
> motivations for cyberattacks, which can have devastating social and
> financial consequences [1
> <https://www.hindawi.com/journals/scn/2021/8933681/#B1>]. Malicious
> software (malware) is one of the most commonly used vectors to propagate
> malicious activity and exploit code vulnerabilities.
>
> Due to the huge numbers of new malware appearing each day, the detection
> of malware samples needs to be automated [2
> <https://www.hindawi.com/journals/scn/2021/8933681/#B2>].
> Signature-matching methods are not resilient enough to handle obfuscation
> techniques or to catch unseen malware types and as such, automated methods
> of generating detection rules, such as machine learning, have been widely
> proposed [3 <https://www.hindawi.com/journals/scn/2021/8933681/#B3>–6
> <https://www.hindawi.com/journals/scn/2021/8933681/#B6>]. These
> approaches typically analyse samples when the file is first ingested,
> either using static code-based methods or by observing dynamic behaviours
> in a virtual environment.
>
> This paper argues that both of these approaches are vulnerable to evasion
> from the attacker. Static methods may be thwarted by simple
> code-obfuscation techniques whether rules are hand-generated [7
> <https://www.hindawi.com/journals/scn/2021/8933681/#B7>] or created using
> machine learning [8
> <https://www.hindawi.com/journals/scn/2021/8933681/#B8>]. Dynamic
> detection in a sandboxed environment cannot continue forever, either it is
> time-limited [9 <https://www.hindawi.com/journals/scn/2021/8933681/#B9>]
> or ends after some period of inactivity [10
> <https://www.hindawi.com/journals/scn/2021/8933681/#B10>]. This fixed
> period allows attackers to inject benign activity during analysis and wait
> to carry out malicious activity once the sample has been deemed harmless
> and passed on to the victim’s environment. The pre-execution filtering of
> malware is the model used by antivirus but this is insufficient to keep up
> with the ever-evolving malware landscape and has led to the creation of
> endpoint detection and response (EDR) products which allow security
> professionals to monitor and respond to malicious activity on the victim
> machine. Real-time malware detection also monitors malware live on the
> machine thus capturing any malicious activity on the victim machine even if
> it was not evident during initial analysis. This paper proposes that once a
> threat is detected, due to the fast-acting nature of some destructive
> malware, it is vital to have automated actions to support these detections.
> In this paper, we investigate automated detection and killing of malicious
> processes for endpoint protection.
>
> There are several key challenges to address in detecting malware
> on-the-fly on a machine in use by comparison with detecting malicious
> applications that are detonated in isolation in a virtual machine. These
> are summarised below:(1)*Signal Separation:* Detection in real time
> requires that the malicious and benign activities are separated in order
> that automated actions can be taken on only the malicious processes.(2)*Use
> of Partial Traces:* In order to try and mitigate damage, malware needs to
> be detected as early as possible, but, as shown in previous work [11
> <https://www.hindawi.com/journals/scn/2021/8933681/#B11>], there is a
> trade-off between the amount of data collected and classification accuracy
> in the first few seconds of an application launching and the same may be
> true for individual processes.(3)*Quick Classification:* The inference
> itself should be as fast as possible in order to further limit the change
> of malicious damage once the process is deemed malicious.(4)*Impact of
> Automated Killing in Supervised Learning:* Supervised learning averages
> the error rate across the entire training set but when the classification
> results in an action, this smoothing out of errors across the temporal
> dataset is not possible.
>
> This paper seeks to address these key challenges and provides preliminary
> results including a measure of “damage prevented” in a live environment for
> fast-acting destructiveware. As well as the results from these experiments,
> this paper contributes an analysis of the computational resources against
> detection accuracy for many of the most popular machine-learning algorithms
> used for malware detection.
>
> The key contributions of this paper are as follows:(i)The first general
> malware detection model to demonstrate damage mitigation in real-time using
> process detection and killing(ii)Benchmarking of commonly used ML
> algorithm implementations with respect to computational resource consumption
> (iii)Presentation of real-time malware detection against more user
> background applications than have previously been investigated; increasing
> from 5 to 36 (up to 95 simultaneous processes)
>
> The next section outlines related work, followed by a report of the three
> methodologies that were tested to try and address these challenges 3 in
> which the method for evaluating these models is also explained (6.5). The
> experimental setup is described in Section 5.2.1
> <https://www.hindawi.com/journals/scn/2021/8933681/#sec5.2.1>, followed
> by results in Section 6
> <https://www.hindawi.com/journals/scn/2021/8933681/#sec6>.
> 2. Related Work2.1. Malware Detection with Static or Post-collection
> Behavioural Traces2.1.1. Static Sources
>
> Machine learning models trained on static data have shown good detection
> accuracy. Chen et al. [5
> <https://www.hindawi.com/journals/scn/2021/8933681/#B5>] achieved 96%
> detection accuracy using statically extracted sequences of API calls to
> train a Random Forest model. However, static data have been demonstrated to
> be quite vulnerable to concept drift [12
> <https://www.hindawi.com/journals/scn/2021/8933681/#B12>, 13
> <https://www.hindawi.com/journals/scn/2021/8933681/#B13>]. Adversarial
> samples present an additional emerging concern; Grosse et al. [14
> <https://www.hindawi.com/journals/scn/2021/8933681/#B14>] and Kolosnaji
> et al. [8 <https://www.hindawi.com/journals/scn/2021/8933681/#B8>]
> demonstrated that static malware detection models achieving over 90%
> detection accuracy could be thwarted by injecting code or simply altering
> the padded code at the end of a compiled binary, respectively.
> 2.1.2. Post-Collection Dynamic Data
>
> Dynamic behavioural data are generated by the malware carrying out its
> functionality. Again machine learning models have been used to draw out
> patterns between malicious and benign software using dynamic data. Various
> dynamic data can be collected to describe malware behaviour. The most
> commonly used data are API calls made to the operating system, typically
> recorded in short sequences or by frequency of occurrence. Huang and
> Stokes’s research [3
> <https://www.hindawi.com/journals/scn/2021/8933681/#B3>] reports the
> highest accuracy in recent malware detection literature with a very large
> dataset of more than 6 million samples to achieve an accurate detection
> rate of 99.64% using a neural network trained on the input parameters
> passed to API calls, their return values, and the co-occurrence of API
> calls. Other dynamic data sources include dynamic opcode sequences (e.g.,
> Carlin et al. [9 <https://www.hindawi.com/journals/scn/2021/8933681/#B9>]
> achieved 99% using a Random Forest), hardware performance counters (e.g.,
> Sayadi [15 <https://www.hindawi.com/journals/scn/2021/8933681/#B15>]
> achieved 94% on Linux/Ubuntu malware using a decision tree), network
> activity and file system activity (e.g., Usman et al. [16
> <https://www.hindawi.com/journals/scn/2021/8933681/#B16>] achieved 93%
> using a decision tree in combination with threat intelligence feeds and
> these data sources), and machine activity metrics (e.g., Burnap et al. [17
> <https://www.hindawi.com/journals/scn/2021/8933681/#B17>] achieved 94%
> using a self-organising map). Previous work [18
> <https://www.hindawi.com/journals/scn/2021/8933681/#B18>] demonstrated
> the robustness of machine activity metrics over API calls in detecting
> malware collected from different sources.
>
> Dynamic detection is more difficult to obfuscate but typically the time
> taken to collect data is several minutes, making it less attractive for
> endpoint detection systems. Some progress has been made on early detection
> of malware. Previous work [11
> <https://www.hindawi.com/journals/scn/2021/8933681/#B11>]) was able to
> detect malware with 94% accuracy within 5 seconds of execution beginning.
> However, as a sandbox-based method, malware which is inactive for the first
> 5 seconds is unlikely to be detected with this approach. Moreover, the
> majority of dynamic malware detection papers use virtualised environments
> to collect data.
> 2.2. Real-Time Malware Detection with Partial Behavioural Traces
>
> OS = operating system; HPCs = Hardware performance counters; DT = Decision
> Tree; MLP = Multi-layer perceptron; NN = Neural Network; RF = Random Forest.
>
> Previous work has begun to address the four challenges set out in the
> introduction. Table 1
> <https://www.hindawi.com/journals/scn/2021/8933681/tab1/> summarises the
> related literature and the problems considered by the researchers.
> Table 1  <https://www.hindawi.com/journals/scn/2021/8933681/tab1/>
> Real-time malware detection literature problems considered.
>
> To the best of our knowledge, challenge *(1) signal separation* has only
> previously been addressed by Sun et al. [23
> <https://www.hindawi.com/journals/scn/2021/8933681/#B23>] using
> sequential API call data. The authors execute up to 5 benign and malicious
> programs simultaneously achieving 87% detection accuracy after 5 minutes of
> execution and 91% accuracy after 10 minutes of execution.
>
> Challenge *(2) to detect malware using partial traces as early as
> possible* has not been directly addressed. Some work has looked at early
> run-time detection; Das et al. [20
> <https://www.hindawi.com/journals/scn/2021/8933681/#B20>] used an FPGA as
> part of a hybrid hardware-software approach to detect malicious Linux
> applications using system API calls which are then classified using a
> multilayer perceptron. Their model was able to detect 46% of malware within
> the first 30% of its execution with a false-positive rate of 2% in offline
> testing. These findings however were not tested with multiple benign and
> malicious programs running simultaneously and do not explain the impact of
> detecting 46% of malware within 30% of its execution trace in terms of
> benefits to a user or the endpoint being protected. How long does it take
> for 30% of the malware to execute? What has occurred in that time?
>
> Greater attention has been paid to challenge *(3) quick classification*,
> insofar as this problem also encompasses the need for lightweight
> detection. Some previous work has proposed hardware based detection for
> lightweight monitoring. Syadi et al. [15
> <https://www.hindawi.com/journals/scn/2021/8933681/#B15>] use high
> performance counters (HPCs) as features to train ensemble learning
> algorithms and scored 0.94 AUC using a dataset of 100 malicious and 100
> benign Linux software samples. Ozsoy et al. [21
> <https://www.hindawi.com/journals/scn/2021/8933681/#B21>] use low-level
> architectural events to train a multilayer perceptron on the more widely
> used [25 <https://www.hindawi.com/journals/scn/2021/8933681/#B25>] (and
> attacked) Windows operating system. The model was able to detect 94% of
> malware with a false-positive rate of 7% using partial execution traces of
> 10,000 committed instructions. The hardware-based detection models,
> however, are less portable than software-based systems due to the ability
> of the same operating system to run on a variety of hardware configurations.
>
> Both Sun et al. [23
> <https://www.hindawi.com/journals/scn/2021/8933681/#B23>] and Yuan [22
> <https://www.hindawi.com/journals/scn/2021/8933681/#B22>] propose
> two-stage models to address the need for lightweight computation. The first
> stage comprises a lightweight ML model such as a Random Forest to alert
> suspicious processes, the second being a deep learning model which is more
> accurate but more computationally intensive to run. Two-stage models, as
> Sun et al. [23 <https://www.hindawi.com/journals/scn/2021/8933681/#B23>]
> note, can get stuck in an infinite loop of analysis in which the first
> model flags a process as suspicious but the second model deems it benign
> and this labelling cycle continues repeatedly. Furthermore, if the first
> model is prone to false negatives, malware will never be passed to the
> second model for deeper analysis.
>
> Challenge *(4) the impact of automated actions* has been discussed by Sun
> et al. [23 <https://www.hindawi.com/journals/scn/2021/8933681/#B23>]. The
> authors also propose the two-stage approach as a solution to this problem.
> The authors apply restrictions to the process whilst the deeper NN analysis
> takes place followed by the killing of malicious-labelled processes. The
> authors found that the delaying strategy impacted benignware more than
> malware and used this two-stage process to account for the irreversibility
> of the decision to kill a process. The authors did not assess the impact on
> the endpoint with respect to the time at which the correctly classified
> malware was terminated.
> 3. Methodology-Three Approaches
>
> As noted above, supervised learning models average errors across the
> training set but in the case of real-time detection and process killing, a
> *single* false positive on a benign process amongst 300 true-negatives
> would cause disruption to the user. The time at which an malware is
> detected is also important, the earlier the better. Therefore, the
> supervised learning model needs to be adapted to take account of these new
> requirements.
>
> Tackling this issue was attempted in three different ways and all three
> are reported here in the interests of reporting negative results as well as
> the one which performed the best. These were:(1)Statistical methods to
> smooth the alert surface and filter out single false-positives(2)Reinforcement
> learning, which is capable of incorporating the consequences of model
> actions into learning(3)A regression model based on the feedback of a
> reinforcement learning model made possible by having the ground-truth labels
>
> Figure 1 <https://www.hindawi.com/journals/scn/2021/8933681/fig1/> gives
> a high-level depiction of the three approaches tested in this paper.
>
> Figure 1  <https://www.hindawi.com/journals/scn/2021/8933681/fig1/>
> High-level depiction of three approaches taken.
> 3.1. Statistical Approach: Alert Filtering
>
> It is expected that transitioning from a supervised learning model to a
> real-time model will see a rise in false-positives since one single alert
> means benign processes (and all child processes) are terminated, which
> effectively renders all future data points as false positives. Filtering
> the output of the models, just as the human brain filters out transient
> electrical impulses in order to separate background noise from relevant
> data [26 <https://www.hindawi.com/journals/scn/2021/8933681/#B26>], may
> be sufficient to make supervised models into suitable agents. This is
> attractive because supervised learning models are already known to perform
> well for malware detection, as confirmed by the previous paper and other
> related work [11 <https://www.hindawi.com/journals/scn/2021/8933681/#B11>
> , 20 <https://www.hindawi.com/journals/scn/2021/8933681/#B20>, 27
> <https://www.hindawi.com/journals/scn/2021/8933681/#B27>, 28
> <https://www.hindawi.com/journals/scn/2021/8933681/#B28>]. A disadvantage
> of this approach is that it introduces additional memory and computational
> requirements both in order to calculate the filtered results and to track
> current and historic scores; therefore, a model which integrates the
> expected consequences of an action into learning is also tested:
> reinforcement learning.
> 3.2. Reinforcement Learning: Q-Learning with Deep *Q* Networks
>
> The proposed automated killing model may be better suited to a
> reinforcement learning strategy than to supervised learning. Reinforcement
> learning uses rewards and penalties from the model’s environment. The
> problem that this paper is seeking to solve is essentially a supervised
> learning problem, but one for which it is not possible to average
> predictions. There are no opportunities to classify the latter stages of a
> process if the agent kills the process, and this can be reflected by the
> reward mechanism of the reinforcement learning model (see Figure 1
> <https://www.hindawi.com/journals/scn/2021/8933681/fig1/>). Therefore,
> reinforcement learning seems like a good candidate for this problem space.
>
> Two limitations of this approach are that (1) reinforcement learning
> models can struggle to converge on a balanced solution, and the models must
> learn to balance the exploration of new actions with the re-use of known
> high-reward actions; commonly known as the exploration-exploitation
> trade-off [29 <https://www.hindawi.com/journals/scn/2021/8933681/#B29>]
> (2) in these experiments, the reward is based on the malware/benignware
> label at the application level rather than being linked to the actual
> damage being caused; therefore, the signal is a proxy for what the model
> should be learning. This is used because, as discussed above, the damage
> caused by different malware is subjective.
>
> For reinforcement learning, loss functions are replaced by reward
> functions which update the neural network weights to reinforce actions (in
> context) that lead to higher rewards and discourage actions (in context)
> that lead to lower rewards; these contexts and actions are known as
> state-action pairs. Typically, the reward is calculated from the perceived
> value of the new state that the action leads to, e.g., points scored in a
> game. Often this cannot be pre-labelled by a researcher since there are so
> many (maybe infinite) state-action pairs. However, in this case, all
> possible state-action pairs can be enumerated, which is the third approach
> tested (regression model, outlined in the next section).
>
> The reinforcement model was still tested. Here the reward is  for a
> correct prediction,  for an incorrect prediction where  is the total number
> of processes impacted by the prediction. For e.g., if there is only one
> process in a process tree but 5 more will appear over the course of
> execution, a correct prediction gives a reward of , and incorrect
> prediction gives a reward of .
>
> There are a number of reinforcement learning algorithms to choose from.
> This paper explores q-learning [30
> <https://www.hindawi.com/journals/scn/2021/8933681/#B30>–33
> <https://www.hindawi.com/journals/scn/2021/8933681/#B33>] to approximate
> the value or “quality” (*q*) of a given action in a given situation.
> Q-learning approximates q-tables, which are look-up tables of every
> state-action pair and their associated rewards. A state-action pair is a
> particular state in the environment coupled with a particular action, i.e.,
> the machine metrics of the process at a given point in time with the action
> to leave the process running. When the number of state-action pairs becomes
> quite large, it is easier to approximate the value using an algorithm. Deep
> *Q* networks (DQN) are neural networks that implement q-learning and have
> been used in state-of-the-art reinforcement learning arcade game playing,
> see Mnih et al. [34
> <https://www.hindawi.com/journals/scn/2021/8933681/#B34>]. A DQN was the
> reinforcement algorithm trialled here; although it did not perform well by
> comparison with the other methods, a different RL algorithm may perform
> better [35 <https://www.hindawi.com/journals/scn/2021/8933681/#B35>], but
> the results are still included in the interests of future work. The
> following paragraphs will explain some of the key features of the DQN.
>
> The DQN tries out some actions; stores the states, actions, resulting
> states, and rewards in memory; and uses these to learn the expected rewards
> of each available action, with the highest expected reward being the one
> that is chosen. Neural networks are well-suited to this problem since their
> parameters can easily be updated, tree-based algorithms like random forests
> and decision trees can be adapted to this end but not as easily. Future
> rewards can be built into the reward function and are usually discounted
> according to a tuned parameter usually signified by .
>
> In Mnih et al.’s [34
> <https://www.hindawi.com/journals/scn/2021/8933681/#B34>] formulation, in
> order to address the exploration-exploitation trade off, DQNs either
> exploit a known action or explore a new one, with the chance of choosing
> exploration falling over time. When retraining the model based on new
> experiences, there is a risk that previous useful learned behaviours are
> lost; this problem is known as catastrophic forgetting [36
> <https://www.hindawi.com/journals/scn/2021/8933681/#B36>]. Mnih et al.’s [
> 34 <https://www.hindawi.com/journals/scn/2021/8933681/#B34>] DQNs use two
> tools to combat this problem. First, experience replay by which past
> state-action pairs are shuffled before being used for retraining so that
> the model does not catastrophically forget. Second, DQNs utilise a second
> network, which updates at infrequent intervals in order to stabilise the
> learning.
>
> Q-learning may enable a model to learn when it is confident enough to kill
> a process, using the discounted future rewards. For example, choosing not
> to kill some malware at time  may have some benefit as it allows the model
> to see more behaviour at *t* + 1 which gives the model greater confidence
> that the process is in fact malicious.
>
> Q-learning approximates rewards from experience, but in this case, all
> rewards from state-action pairs can actually be pre-calculated. Since one
> of the actions will kill the process and thus end the “experience” of the
> DQN, it could be difficult for this model to gain enough experience. Thus
> pre-calculation of rewards may improve the breadth of experience of the
> model. For this reason, a regression model is proposed to predict the *Q*-value
> of a given action.
> 3.3. Regression Using *Q*-Values
>
> Unlike classification problems, regression problems can predict a
> continuous value rather than discrete (or probabilistic) values relating to
> a set of output classes. Regression algorithms are proposed here to predict
> the q-value of killing a process. If this value is positive, the process is
> killed.
>
> *Q*-values estimate the value of a particular action based on the
> “experience” of the agent. Since the optimal action for the agent is always
> known, it is possible to precompute the “(q-) value” of killing a process
> and train various ML models to learn this value. It would typically be
> quicker to train a regression model which tries to learn the value of
> killing a process than to train a DQN which explores the state-action space
> and calculates rewards between learning, since the interaction and
> calculation of rewards is no longer necessary. The regression approach can
> be used with any machine learning algorithm capable of learning a
> regression problem, regardless of whether it is capable of partial training.
>
> There are two primary differences between this regression approach and the
> reinforcement learning DQN approach detailed in the previous section.
> Firstly, the datasets are likely to be the difference. Since the DQN
> generates training data through interacting with its environment, it may
> never see certain parts of the state-action space, e.g., if a particular
> process  is always killed during training before time , the model is not
> able to learn from the process  data after .
>
> Secondly, *only* the expected value of killing is modelled by the
> regressor, whereas the DQN tries to predict the value of both killing and
> of not killing the process. This means that the equation used to model the
> value of process killing is only an approximation of the reward function
> used by the DQN.
>
> The equation used to calculate the value of killing is positive for
> malware and negative for benignware; in both cases, it is scaled by the
> number of child processes impacted and in the case of malware, early
> detection increases the value of process killing (with an exponential
> decay). Let  be the true label of the process (0 = benign, 1 =
> malicious),  is the number of child processes, and  is the time in seconds
> at which the process is killed; then, the value of killing a process is:
>
> The equation above negatively scores the killing of benignware in
> proportion to the number of subprocesses and scores the killing of malware
> positively in proportion to the number of subprocesses. A bonus reward is
> scored for killing malware early, with an exponential decay over time.
> 4. Evaluation Methodology: Ransomware Detection
>
> Previous, research has not addressed the extent to which damage is
> mitigated by process killing, since Sun et al. [23
> <https://www.hindawi.com/journals/scn/2021/8933681/#B23>] presented the
> only previous work to test process killing and damage with and without
> process killing is not assessed. To this end, this paper uses ransomware as
> a proxy to detect malicious damage, inspired by Scaife et al.’s approach [
> 24 <https://www.hindawi.com/journals/scn/2021/8933681/#B24>]. A brief
> overview of Scaife et al.’s damage measurement is outlined below:
>
> Early detection is particularly useful for types of malware from which
> recovery is difficult and/or costly. Cryptographic ransomware encrypts user
> files and withholds the decryption key until a ransom is paid to the
> attackers. This type of attack is typically costly to remedy, even if the
> victim is able to carry out data recovery [37
> <https://www.hindawi.com/journals/scn/2021/8933681/#B37>]. Scaife et
> al.’s work [24 <https://www.hindawi.com/journals/scn/2021/8933681/#B24>]
> on ransomware detection uses features from file system data, such as
> whether the contents appear to have been encrypted, and number of changes
> made to the file type. The authors were able to detect and block all of the
> 492 ransomware samples tested with less than 33% of user data being lost in
> each instance. Continella et al. [38
> <https://www.hindawi.com/journals/scn/2021/8933681/#B38>] propose a
> self-healing system, which detects malware using file system machine
> activity (such as read/write file counts); the authors were able to detect
> all 305 ransomware samples tested, with a very low false-positive rate.
> These two approaches use features selected specifically for their ability
> to detect ransomware, but this requires knowledge of how the malware
> operates, whereas the approach taken here seeks to use features which can
> be used to detect malware *in general*. The key purpose of this final
> experiment (Section 6.5
> <https://www.hindawi.com/journals/scn/2021/8933681/#sec6.5>) is to show
> that our general model of malware detection is able to detect general types
> of malware as well as time-critical samples such as ransomware.
> 5. Experimental Setup
>
> This section outlines the data capture process and dataset statistics.
> 5.1. Features
>
> The same features as were used in previous work [11
> <https://www.hindawi.com/journals/scn/2021/8933681/#B11>] are used here
> for process detection, with some additional features to measure
> process-specific data. Despite the popularity of API calls noted in Ref. [
> 18 <https://www.hindawi.com/journals/scn/2021/8933681/#B18>], due to
> these findings and Sun et al.’s [23
> <https://www.hindawi.com/journals/scn/2021/8933681/#B23>] difficulties
> hooking this data in real-time, these were not considered as features to
> train the model.
>
> At the process-level, 26 machine metric features are collected; these were
> dictated by the attributes available using the Psutil [39
> <https://www.hindawi.com/journals/scn/2021/8933681/#B39>] python library.
> It is also possible to include the “global” machine learning metrics that
> were used in the previous papers. Although global metrics will not provide
> process-level granularity, they may give muffled indications of the
> activity of a wider process tree. The 9 global metrics are: system-level
> CPU use, user-level CPU use, memory use, swap memory use, number of packets
> received and sent, number of bytes received and sent, and the total number
> of processes running.
>
> The process-level machine activity metrics collected are: CPU use at the
> user level, CPU use at the system level, physical memory use, swap memory
> use, total memory use, number of child process, number of threads, maximum
> process ID from a child process, disk read, write and other I/O count,
> bytes read, written and used in other I/O processes, process priority, I/O
> process priority, number of command line arguments passed to process,
> number of handles being used by process, time since the process began, TCP
> packet count, UDP packet count, number of connections currently open, and 4
> port statuses of those opened by the process (see Table 2
> <https://www.hindawi.com/journals/scn/2021/8933681/tab2/>).
> Table 2  <https://www.hindawi.com/journals/scn/2021/8933681/tab2/>
> 26 process-level features: 22 features + 4 port status values.
> 5.1.1. Preprocessing
>
> Feature normalisation is necessary for NNs to avoid over-weighting
> features with higher absolute values. The test, train, and validation sets (
> ) are all normalised by subtracting the mean () and dividing by the
> standard deviation () of each feature in the training set: . This sets
> the range of input values largely between −1 and 1 for all input features,
> avoiding the potential for some features to be weighted more important than
> others during training purely due to the scalar values of those features.
> This requires additional computational resources but is not necessary for
> all ML algorithms; this is another reason why the supervised RNN used in
> Ref. [11 <https://www.hindawi.com/journals/scn/2021/8933681/#B11>] may
> not be well-suited for real-time detection.
> 5.2. Data Capture
>
> During data capture, this research sought to improve upon previous work
> and emulate real machine use to a greater extent than has previously been
> trialled. The implementation details of the VM, simultaneous process
> execution, and RL simulation are outlined below:
> 5.2.1. Environment: Machine Setup
>
> The following experiments were conducted using a virtual machine (VM)
> running with Cuckoo Sandbox [40
> <https://www.hindawi.com/journals/scn/2021/8933681/#B40>] for ease of
> collecting data and restarting between experiments and because the Cuckoo
> Sandbox emulates human interaction with programs to some extent to promote
> software activity. In order to emulate the capabilities of a typical
> machine, the modal hardware attributes of the top 10 “best seller” laptops
> according to a popular Internet vendor [41
> <https://www.hindawi.com/journals/scn/2021/8933681/#B41>] were used, and
> these attributes were the basis of the VM configuration. This resulted in a
> VM with 4GB RAM, 128GB storage, and dual-core processing running Windows 7
> 64 bit. Windows 7 was the most prevalent computer operating system (OS)
> globally at the time of designing the experiment [25
> <https://www.hindawi.com/journals/scn/2021/8933681/#B25>]. Although
> Windows 10 is now the most popular OS, the findings in this research should
> still be relevant.
> 5.2.2. Simultaneous Applications
>
> In typical machine use, multiple applications run simultaneously. This is
> not reflected by behavioural malware analysis research in which samples are
> injected individually to a virtual machine for observation. The environment
> used for the following experiments launches multiple applications on the
> same machine at slightly staggered intervals as if a user were opening
> them. Each malware is launched with a small number (1–3) and a larger
> number (3–35) of applications. It was not possible to find up-to-date user
> data on the number of simultaneous applications running on a typical
> desktop, so here it was elected to launch up to 36 applications (35 benign
> + 1 malicious) at once, which is the largest number of simultaneous apps
> for real-time data collection to date. From the existing real-time analysis
> literature, only Sun et al. [23
> <https://www.hindawi.com/journals/scn/2021/8933681/#B23>] run multiple
> applications at the same time, with a maximum of 5 running simultaneously.
>
> Each application may in turn launch multiple processes, causing more than
> 35 processes to run at once; 95 is the largest number of simultaneous
> processes recorded; this excludes background OS processes.
> 5.2.3. Reinforcement Learning Simulation
>
> For reinforcement learning, the DQN requires an observation of the
> resulting state following an action. To train the model, a simulated
> environment is created from the pre-collected training data whereby the
> impact of killing or not killing a process is returned as the next state.
> For process-level elements, this reduces all features to zero. A caveat
> here is that in reality, killing the process may not occur immediately and
> therefore memory, processing power, etc., may still be being consumed at
> the next data observation. For global metrics, the process-level values for
> the killed processes (includes child processes of the killed process) are
> subtracted from the global metrics. There is a risk again that this
> calculation may not correlate perfectly with what would be observed in a
> live machine environment.
>
> In order to observe the model performance, a visualisation was developed
> to accompany the simulated environment. Figures 2
> <https://www.hindawi.com/journals/scn/2021/8933681/fig2/> and 3
> <https://www.hindawi.com/journals/scn/2021/8933681/fig3/> show
> screenshots of the environment visualisation for one malicious and one
> benign process.
>
> Figure 2  <https://www.hindawi.com/journals/scn/2021/8933681/fig2/>
> Benignware sample, normalised process-level metrics, 6 observations made
> without process being killed.
>
> Figure 3  <https://www.hindawi.com/journals/scn/2021/8933681/fig3/>
> Malware sample, normalised process-level metrics, no observations made yet.
> 5.3. Dataset
>
> The dataset comprises 3,604 benign executables and 2,792 malicious
> applications (each containing at least one executable), with 2,877 for
> training and validation and 3,519 for testing. These dataset sizes are
> consistent with previous real-time detection dataset sizes (Das et al. [20
> <https://www.hindawi.com/journals/scn/2021/8933681/#B20>] use 168
> malicious, 370 benign; Sayadi et al. [15
> <https://www.hindawi.com/journals/scn/2021/8933681/#B15>] use over 100
> each benign and malicious; Ozsoy et al. [21
> <https://www.hindawi.com/journals/scn/2021/8933681/#B21>] use 1,087
> malicious and 467 benign; Sun et al. [23
> <https://www.hindawi.com/journals/scn/2021/8933681/#B23>] use 9,115
> malicious, 877 benign). With multiple samples running concurrently to
> simulate real endpoint use, there are 24K processes in the training set and
> 34K in the test set. Overall, there are 58K behavioural traces of processes
> in the training and testing datasets. The benign samples comprise files
> from VirusTotal [42
> <https://www.hindawi.com/journals/scn/2021/8933681/#B42>], from free
> software websites (later verified as benign with VirusTotal), and from a
> fresh Microsoft Windows 7 installation. The malicious samples were
> collected from two different VirusShare [43
> <https://www.hindawi.com/journals/scn/2021/8933681/#B43>] repositories.
>
> In Pendelbury et al.’s analysis [13
> <https://www.hindawi.com/journals/scn/2021/8933681/#B13>], the authors
> estimate that in the wild between 6% and 22% of applications are malicious,
> normalising to 10% for their experiments. Using this estimation of Android
> malware, a similar ratio was used in the test set in which 13.5% were
> malicious.
> 5.3.1. Malware Families
>
> PUA = potentially unwanted application, RAT = remote access trojan.
>
> This paper is not concerned with distinguishing particular malware
> families, but rather with identifying malware in general. However, a
> dataset consisting of just one malware family would present an unrealistic
> and easier problem than is found in the real world. The malware families
> included in this dataset are reported in Table 3
> <https://www.hindawi.com/journals/scn/2021/8933681/tab3/>. The malware
> family labels are derived from the output of around 60 antivirus engines
> used by VirusTotal [42
> <https://www.hindawi.com/journals/scn/2021/8933681/#B42>].
> Table 3  <https://www.hindawi.com/journals/scn/2021/8933681/tab3/>
> Malware families with more than 10 samples in the dataset. 315 families
> were represented in the dataset, with 27 having being represented more than
> 10 times. Basic description provided which does not cover the wide range of
> behaviours carried out by some malware families but is intended to indicate
> the range of behaviours in the top 27 families included in the dataset.
>
> Ascribing family labels to malware is nontrivial since antivirus vendors
> do not follow standardised naming conventions and many malware families
> have multiple aliases. Sebastián et al. [44
> <https://www.hindawi.com/journals/scn/2021/8933681/#B44>] have developed
> an open source tool, AVClass, to extract meaningful labels and correlate
> aliases between different antivirus outputs. AVClass was used to label the
> malware in this dataset. Sometimes there is no consensus amongst the
> antivirus’ output or the sample is not recognised as a member of an
> existing family. AVClass also excludes malware that belongs to very broad
> classes of malware (e.g., “agent,” “eldorado,” and “artemis”) as these are
> likely to comprise a wide range of behaviours and so may be applied as a
> default label in cases for which antivirus engines are unsure. In the
> dataset established in this research, 2,121 of the 2,792 samples were
> assigned to a malware family. Table 3
> <https://www.hindawi.com/journals/scn/2021/8933681/tab3/> gives the
> number of samples in each family for which more than 10 instances were
> found in the dataset. 315 families were detected overall, with 27 families
> being represented more than 10 times. These better-represented families
> persist in the train and test sets, but the other families have little
> overlap. 104 of the 154 other families seen in the test set are not
> identified by AVClass as being in the training set.
> 5.3.2. Malicious Vs. Benign Behaviour
>
> Statistical inspection of the training set reveals that benign
> applications have fewer sub-processes than malicious processes, with 1.17
> processes in the average benign process tree and 2.33 processes in the
> average malicious process tree. Malware was also more likely to spawn
> processes outside of the process tree of the root process, often using the
> names of legitimate Windows processes. In some cases, malware launches
> legitimate applications, such as Microsoft Excel in order to carry out a
> macro-based exploit. Although Excel is not a malicious application in
> itself, it is malicious in this context, which is why malicious labels are
> assigned if a malware sample has caused that process to come into being. It
> is therefore possible to argue that some processes launched by malware are
> not malicious, because they do not individually cause harm to the endpoint
> or user, but without the malware they would not be running and so can be
> considered at least undesirable even if only in the interests of conserving
> computational resources.
> 5.3.3. Train-Test Split
>
> The dataset is split in half with the malicious samples in the test set
> coming from the more recent VirusShare repository, and those in the
> training set from the earlier repository. This is to increase the chances
> of simulating a real deployment scenario in which the malware tested
> contains new functionality by comparison with those in the training set.
>
> Ideally, the benignware should also be split by date across the training
> and test set; however, it is not a trivial task to calculate the date at
> which benignware was compiled. It is possible to extract the compile time
> from PE header, but it is possible for the PE author to manually input this
> date which had clearly happened in some instances where the compile date
> was 1970-01-01 or in one instance 1970-01-16. In the latter case
> (1970-01-16), the file is first mentioned online in 2016, perhaps
> indicating a typographic error [45
> <https://www.hindawi.com/journals/scn/2021/8933681/#B45>]. Using Internet
> sources such as VirusTotal [42
> <https://www.hindawi.com/journals/scn/2021/8933681/#B42>] can give an
> indication when software was first seen, but if the file is not very
> suspicious, i.e., from a reputable source, it may not have been uploaded
> until years after it was first seen “in the wild.” Due to the difficulty in
> dating benignware in the dataset collected for this research, samples were
> assigned to the training or test set randomly.
>
> For training, an equal number of benign and malicious processes are
> selected, so that the model does not bias towards one class. 10% of these
> are held out for validation. In most ML model evaluations, the validation
> set would be drawn from the same distribution as the test set. However,
> because it is important not to leak any information about the malware in
> the test set, since it is split by date, the validation set here is drawn
> from the training distribution.
> 5.3.4. Implementation Tools
>
> Data collection used the Psutil [39
> <https://www.hindawi.com/journals/scn/2021/8933681/#B39>] Python library
> to collect machine activity data for running processes and to kill those
> processes deemed malicious. The RNN and Random Forests were implemented
> using the Pytorch [46
> <https://www.hindawi.com/journals/scn/2021/8933681/#B46>] and
> Scikit-Learn [47 <https://www.hindawi.com/journals/scn/2021/8933681/#B47>]
> Python libraries, respectively. The model runs with high priority and
> administrator rights to make sure the polling is maintained when compute
> resources are scarce.
> 6. Experimental Results6.1. Supervised Learning for Process Killing
>
> First, we demonstrate the unsuitability of a full-trace supervised
> learning malware detection model, which achieved more than 96% detection
> accuracy in Ref. [11
> <https://www.hindawi.com/journals/scn/2021/8933681/#B11>]. The model used
> is a gated-recurrent unit recurrent neural network since this algorithm is
> designed to process time-series data. The hyperparameter configuration of
> this model was conducted using a random search of hyperparameters (see
> Table 4 <https://www.hindawi.com/journals/scn/2021/8933681/tab4/> in the
> Appendix for details.)
> Table 4  <https://www.hindawi.com/journals/scn/2021/8933681/tab4/>
> Hyperparameter search space and the hyperparameters of the model giving
> the lowest mean false-positive and false-negative rates.
>
> It is expected that supervised malware detection models will not adapt
> well to process-killing due to the averaging of loss metrics as described
> earlier. Initially, this is verified by using supervised learning models to
> kill processes that are deemed malicious. For supervised classification,
> the model makes a prediction every time a data measurement is taken from a
> process. This approach is compared with one taking average predictions
> across all measurements for a process and for a process tree as well as the
> result of process killing. The models with the highest validation accuracy
> for classification and killing are compared.
>
> Figure 4 <https://www.hindawi.com/journals/scn/2021/8933681/fig4/> illustrates
> the difference in validation set and test set F1-score, true-positive rate,
> and false-positive rate for these 4 levels of classification: each
> measurement, each process, each process tree, and finally showing process
> killing; see Figure 5
> <https://www.hindawi.com/journals/scn/2021/8933681/fig5/> for
> diagrammatic representation of these first 3 levels. Table 5
> <https://www.hindawi.com/journals/scn/2021/8933681/tab5/> reports the F1,
> TPR, and TNR for classification (each measurement of each process) and for
> process killing.
>
> Figure 4  <https://www.hindawi.com/journals/scn/2021/8933681/fig4/>
> F1 scores, true positive rates (TPR), and true negative rates (TNR) for
> partial-trace detection (process measurements), full-trace detection (whole
> process), whole application (process tree), and with process-level
> measurements + process killing (process killing) for validation set (left
> column) and test set (right column).
>
> Figure 5  <https://www.hindawi.com/journals/scn/2021/8933681/fig5/>
> Three levels of data collection: each measurement, each process, each
> process tree.
> Table 5  <https://www.hindawi.com/journals/scn/2021/8933681/tab5/>
> F1-score, true positive rate (TPR), and true negative rates (TNR)
> (all  100) on test and validation sets for classification and process
> killing.
>
> The highest F1-score on the validation set is achieved by an RNN using
> process data only. When process killing is applied, there is a drop of less
> than 5 percentage points in the F1-score, but more than 15 percentage
> points are lost from the TNR.
>
> On the unseen test set, the highest F1-score is achieved by an RNN using
> process data + global metrics, but the improvement over the process data +
> total number of processes is negligible. Overall, there is a reduction in
> F1-score from (97.44, 94.61) to (74.91, 77.66), highlighting the initial
> challenge of learning to classifying individual processes rather than
> entire applications, especially when accounting for concept drift. Despite
> the low accuracy, these initial results indicate that the model is
> discriminating some of the samples correctly and may form a baseline from
> which to improve.
>
> The test set TNR and TPR for classification on the best-performing model
> (process data only) are 79.70 and 82.91, respectively, but when process
> killing is applied, although the F1-score drops by 10 percentage points,
> the TNR and TPR move in opposite directions with the TNR falling to 59.63
> and TPR increasing to 90.24. This is not surprising since a single
> malicious classification results in a process being classed as malicious.
> This is true for the best-performing models using either of the two feature
> sets (see Figure 4
> <https://www.hindawi.com/journals/scn/2021/8933681/fig4/>).
> 6.2. Accuracy Vs. Resource Consumption
>
> Previous work on real-time detection has highlighted the requirement for a
> lightweight model (speed and computational resources). In the previous
> paper, RNNs were the best performing algorithm in classifying
> malware/benignware, but RNNs have many parameters and therefore may consume
> significant RAM and/or CPU. They also require preprocessing of the data to
> scale the values, which other ML algorithms such as tree-based algorithms
> do not. Whilst RAM and CPU should be minimised, taking model accuracy into
> account, inference duration is also an important metric.
>
> Although the models in this paper have not been coded for performance and
> use common python libraries, comparing these metrics helps to decide
> whether certain models are vastly preferable to others with respect to
> computational resource consumption. The PyRAPL library [49
> <https://www.hindawi.com/journals/scn/2021/8933681/#B49>] is used measure
> the CPU, RAM, and duration used by each model. This library uses Intel
> processor “Running Average Power Limit” (RAPL) metrics. Only data
> preprocessing and inference is measured as training may be conducted
> centrally in a resource-rich environment. Batch sizes of 1, 10, 100, and
> 1000 samples are tested with 26 and 37 features, respectively, since there
> are 26 process-level features and 37 when global metrics are included. Each
> model is run 100 times for each of the different batch sizes.
>
> For the RNN, a “large” and a “small” model are included. The large models
> have the highest number of parameters tested in the random search (981
> hidden neurons, 3 hidden layers, sequence length of 17) and the smallest
> (41 neurons, 1 hidden layer, sequence length of 13). These two RNN
> configurations are compared against other machine learning models which
> have been used for malware detection: Multi-Layer Perceptron (feed-forward
> neural network), Support Vector Machine, Naive Bayes Classifier, Decision
> Tree Classifier, Gradient Boosted Decision Tree Classifier (GBDTs), Random
> Forest, and AdaBoost.
>
> 26 features = process-level only, 37 features = machine and process level
> features
>
> Table 6 <https://www.hindawi.com/journals/scn/2021/8933681/tab6/> reports
> the computational resource consumption and accuracy metrics together.
> Decision tree with 38 features is the lowest cost to run, RNN performs best
> at supervised learning classification on the validation set but only just
> outperforms the decision tree with 26 features, which is the best
> performing model at process killing on the validation set at 92.97
> F1-score. The highest F1-score for process killing uses a Random Forest
> with 37 features, scoring 77.85 F1, which is 2 percentage points higher
> than the RF with 26 features (75.97). The models all perform at least 10
> percentage points better on the validation set, indicating the importance
> of taking concept drift into account when validating models.
> Table 6  <https://www.hindawi.com/journals/scn/2021/8933681/tab6/>
> Average resource consumption over 100 iterations for a batch size of 100
> vs. F1-scores on validation and test set for classification and process
> killing across 14 models..
> 6.3. How to Solve a Problem like Process Killing?
>
> From the results above, it is clear that supervised learning models see a
> significant drop in classification accuracy when processes are killed as
> the result of a malicious label. This confirmation of the initial
> hypothesis presented here justifies the need to examine alternative
> methods. In the interests of future work and negative result reporting,
> this paper reports all of the methods attempted and finds that simple
> statistical manipulations on the supervised learning models perform better
> than using alternative training methods. This section briefly describes the
> logic of each method and provides a textual summary of the results with a
> formula where appropriate. This is followed by a table of the numerical
> results for each method. In the following section, let  be a set of
> processes  in a process tree,  be the time at which a prediction is made,
> let  be the prediction for process  at time  where a prediction equal to or
> greater than 1 classifies malware.
> 6.3.1. Mean Predictions
>
> Reasoning: Taking the average prediction across the whole process will
> smooth out those process killing results.
>
> *Not tested.* This was not attempted for two reasons: (1) Taking the mean
> at the end of the process means the damage is done. (2) This method can
> easily be manipulated by an attacker: 50 seconds of injected benign
> activity required 50 seconds of malicious activity to achieve a true
> positive
> 6.3.2. Rolling Mean Predictions
>
> Reasoning: Taking the average over a few measurements will eliminate those
> false positives that are caused by a single false positive over a subset of
> the execution trace. Window sizes of 2 to 5 are tested. Let  be the window
> size:
>
> *Summary of results:* A small but unilateral increase in F1-Score using a
> rolling window over 2 measurements on the validation set. Using a rolling
> window of size 2 on the test set saw a 10 to 20 percentage point increase
> in true negative rate (to a maximum of 80.77) with 3 percentage points lost
> from the true-positive rate. This was one of the most promising approaches.
> 6.3.3. Alert Threshold
>
> Reasoning: Like the rolling mean, single false positives will be
> eliminated but unlike the rolling mean, the alerts are cumulative over the
> entire trace such that a single alert at the start and 30 seconds into the
> process will cause the process to be killed rather than requiring that both
> alerts are within a window of time. Between 2 and 5 minimum alerts are
> tested
>
> *Summary of results:* Again, a small increase across all models, with an
> optimal minimum number of alerts being 2 for maximum F1-score, competitive
> with the rolling mean approach.
> 6.3.4. Process-Tree Averaging
>
> Reasoning: The data are labelled at the application level; therefore, the
> average predictions across the process tree should be considered for
> classification
>
> *Summary of results:* Negligible performance increase on validation and
> test set data (less than 1 percentage point). This is likely because few
> samples have more than one process executing simultaneously.
> 6.3.5. Process-Tree Training
>
> Reasoning: The data are labelled at the application level; therefore, the
> sum of resources of each process tree should be classified at each
> measurement, not the individual processes.
>
> *Summary of results:* Somewhat surprisingly, there was a slight reduction
> in classification accuracy when using process tree data. One explanation
> for this may be that the process tree creates noise around the
> differentiating characteristics that are visible at the process level.
> 6.3.6. DQN
>
> Reasoning: Reinforcement learning is designed for state-action space
> learning. Both pre-training the model with a supervised learning approach
> and not pre-training the model were tested.
>
> *Summary of results:* Poor performance, typically converging to either
> kill or not kill everything, of the few models that did not converge to a
> single dominant action; it does not distinguish malware or benignware well,
> indicating that it may not have learned anything. Reinforcement learning
> may help the problem of real-time malware detection and process killing,
> but this initial implementation of a DQN did not converge to a better or
> even competitive solution to supervised learning. Perhaps, better
> formulation of rewards (e.g., damage prevented) would help the agent learn.
> 6.3.7. Regression on Predicted Kill Value
>
> Reasoning: Though the DQN explores and exploits different state-action
> pairs and their associated rewards, when the reward from each action is
> known in the first place and the training set is limited, as it is here,
> Q-learning can be framed as a regression problem in which the model tries
> to learn the return (rewards + future rewards), the training is faster and
> can be used by any regression-capable algorithm. Let  be the number of
> current and future child processes for  at
>
> *Summary of results:* Improved performance on true negative rate,
> although not perceptible for the highest-scoring F1 models since F1-scores
> reward true positives more than true negatives, this metric can struggle to
> reflect a balance between the true-positive and true-negative rates. The
> highest true-negative rate models are all regression models.
>
> Table 7 <https://www.hindawi.com/journals/scn/2021/8933681/tab7/> lists
> the F1, TPR, and TNR on the validation and test set for each of the methods
> described above. The best-performing model on the test and validation sets
> is reported and the full results can be found in Appendix Table 8
> <https://www.hindawi.com/journals/scn/2021/8933681/tab8/>–10
> <https://www.hindawi.com/journals/scn/2021/8933681/tab10/>. Small
> improvements are made by some models on the validation F1-score, but the
> test set F1-score improves by 4 percentage points in the best instance.
> Table 7  <https://www.hindawi.com/journals/scn/2021/8933681/tab7/>
> Summary of the best process killing models by model training methodology.
> F1, TNR, and TPR for validation and test datasets (full results in Appendix
> Tables 8 <https://www.hindawi.com/journals/scn/2021/8933681/tab8/>–10
> <https://www.hindawi.com/journals/scn/2021/8933681/tab10/>).
> Table 8  <https://www.hindawi.com/journals/scn/2021/8933681/tab8/>
> Summary of process killing models, validation and test set score metrics
> [Table 1 of 3].
> Table 9  <https://www.hindawi.com/journals/scn/2021/8933681/tab9/>
> Summary of process killing models, validation, and test set score metrics
> [Table 2 of 3].
> Table 10  <https://www.hindawi.com/journals/scn/2021/8933681/tab10/>
> Summary of process killing models, validation, and test set score metrics
> [Table 3 of 3].
>
> In most cases, the models with the highest F1-score on the validation and
> test sets are not the same. The highest F1-score is 81.50 from an RF using
> a minimum alert threshold of 2 and both process-level and global process
> metrics.
> 6.4. Further Experiment: Favouring High TNR
>
> Although the proposed model is motivated by the desire to prevent malware
> from executing, the best TNR reported amongst the models above is 81.50%.
> 20% of benign processes being killed would not be acceptable to a user.
> Whilst this research is a novel attempt at very early-stage real-time
> malware detection and process killing, one might consider the usability and
> prefer a model with a very high TNR, even if this is at the expense of the
> TPR.
>
> Considering this, the AdaBoost regression algorithm achieves a 100% TNR
> with a 39.50% TPR on the validation set. The high FNR is retained in the
> test set standing at 97.92%, but the TPR drops even further to just 8.40%.
> The GBDT also uses regression to estimate the value of process killing and
> coupled with a minimum of 4 alerts performs well on the test set but does
> not stand out in the validation set, see Table 11
> <https://www.hindawi.com/journals/scn/2021/8933681/tab11/>.
> Table 11  <https://www.hindawi.com/journals/scn/2021/8933681/tab11/>
> Two models’ F1-score, TNR, TPR for the validation and test set scoring the
> highest TNR on the validation and test sets.
>
> Although less than 10% of the test set malicious processes is killed by
> the AdaBoost regressor, this model may be the most viable despite the low
> TPR. Future work may examine the precise behaviour and harm caused by
> malware that is/is not detected. To summarise results, the most-detected
> families were Ekstak (180), Mikey (80), Prepscram (53 processes), and Zusy
> (49 processes) of the 745 total samples.
> 6.5. Measuring Damage Prevention in Real Time
>
> Although a high percentage of processes are correctly identified as
> malicious by the best performing model (RF with 2 alerts and 37 features),
> it may be that the model detects the malware after it has already caused
> damage to the endpoint. Therefore, instead of looking at the time at which
> the malware is correctly detected, a live test was carried out with
> ransomware to measure the percentage of files corrupted with and without
> the process killing model working. This real-time test also assesses
> whether malware can indeed be detected in the early stages of execution or
> whether the data recording, model inference, and process killing is too
> slow in practice to prevent damage.
>
> Ransomware is the broad term given to malware that prevents access to user
> data (often by encrypting files) and holds the means for restoring the data
> (usually a decryption key) from the user until a ransom is paid. It is
> possible to quantify the damage caused by ransomware using the proportion
> of modified files as Scaife et al. [24
> <https://www.hindawi.com/journals/scn/2021/8933681/#B24>] have done in
> developing a real-time ransomware (only) detection system. The damage of
> some malware types are more difficult to quantify owing to their dependence
> on factors outside the control of the malware. For example, the damage
> caused by spyware will depend on what information it is able to obtain, so
> it is difficult to quantify the benefit of killing spyware 5 seconds after
> execution compared with 5 minutes into execution. Ransomware offers a clear
> metric for the benefits of early detection and process killing.
>
> Although the RF with a minimum of 2 alerts using both process and global
> data gave the highest F1-score on the test set (81.50), earlier experiments
> showed that RFs are not one of the most computationally efficient models by
> comparison with those tested. Therefore, a decision tree is trained on
> process-only data (26 features) in case the time-to-classification is
> important for damage reduction despite the lower F1-score. For this reason,
> the decision tree model is used in this test. The DT also has a very
> slightly higher TPR (see Table 12
> <https://www.hindawi.com/journals/scn/2021/8933681/tab12/>) so a higher
> damage prevention rate may be partially due to the model itself rather than
> just the fewer features being collected and model classification speed.
> Table 12  <https://www.hindawi.com/journals/scn/2021/8933681/tab12/>
> Random Forest and Decision Tree each with a minimum requirement of two
> alerts (“malicious classifications”) to kill a process. F1, TNR, and TPR
> reported on validation and test set.
>
> 22 fast-acting ransomware files were identified from a separate VirusShare
> [43 <https://www.hindawi.com/journals/scn/2021/8933681/#B43>] repository
> which *(i)* do not require Internet connection and *(ii)* begin
> encrypting files within the first few seconds of execution. The former
> condition is set because the malicious server may no longer exist and for
> safety, it is not desirable to connect to it if it does exist. Some malware
> is able to cause significant damage in seconds, in which the timeframes are
> impossible for a human to see, process, react to, and alert in.
>
> The 22 samples were executed for 30 seconds each without the process
> killing model and the number of files modified was recorded. The process
> was repeated with 4 process killing models: DT with min. 2 alerts and 26
> features, RF with min. 2 alerts and 37 features, AdaBoost regressor with 26
> features, and GDBT regressor with min. 4 alerts and 26 features.
>
> It was necessary to run the killing model with administrator privileges
> and to write an exception for the Cuckoo sandbox agent process which
> enables the host machine to read data from the guest machine since the
> models killed this process. The need for this exception highlights that
> there are benign applications with malicious-like behaviours, perhaps
> especially those used for networking and security.
>
> Figure 6 <https://www.hindawi.com/journals/scn/2021/8933681/fig6/> and
> Table 13 <https://www.hindawi.com/journals/scn/2021/8933681/tab13/> give
> the total number of corrupted files across the 22 samples. The damage
> prevention column is a proxy metric denoting how many files were not
> corrupted using a given process killing model by comparison with no model
> being in place. The 22 samples on average each corrupt 910 files within 30
> seconds.
>
> Figure 6  <https://www.hindawi.com/journals/scn/2021/8933681/fig6/>
> Total number of files corrupted by ransomware with no process killing and
> with three process killing models within the first 30 seconds of execution.
> Table 13  <https://www.hindawi.com/journals/scn/2021/8933681/tab13/>
> Total number of files corrupted by ransomware with no process killing and
> with three process killing models within the first 30 seconds of execution.
> Damage reduction is the percentage of files spared when no killing is
> implemented.
>
> The DT model almost entirely eliminates any file corruption with only
> three being corrupted. The RF saves 92.68% of files. The ordinal ranking of
> “damage prevention” is the same as the TPR on the test set, but the
> relationship is not proportional. The same ordinal relationship indicates
> that the simulated impact of process killing on the collected test set was
> perhaps a reasonable approximation of measuring at least fast-acting
> ransomware damage, despite the TPR test set metrics being based on other
> malware families, too.
>
> The DT demonstrates that this architecture is capable of preventing
> damage, but the TNR on the test set of the DT model is so low (66.19) that
> this model cannot be preferred to the RF (81.53 TNR), which still prevents
> over 90% of file damage.
>
> The GBDT prevents some damage, and detects a comparable number of
> ransomware samples (1 in 5). The AdaBoost regressor detected 2 ransomware
> samples of the 22, and in these two cases more than 64% and 45% of files
> were saved, respectively; perhaps, with more execution time, the files
> would be detected but the key benefit of process killing is to stop
> damaging software like these ransomware samples and this algorithm actually
> saw more files encrypted than when no killing model was used; this is
> because there will be a slight variance in the ransomware behaviour and
> execution time each time it runs. The Random Forest is the most plausible
> model, balancing damage prevention and TNR; however, the delay in
> classification may be a result of the requirement to collect more features
> and/or the real-time of the model itself.
> 7. Discussion: Measuring Execution Time in a Live Environment
>
> Although algorithm execution duration was measured above, due to batch
> processing used by the models, the number of processes being classified can
> be increased by an order of magnitude with a negligible impact on execution
> time. The data collection and process killing both have linear, , complexity;
> where  is the number of processes; therefore it is expected that the number
> of processes impacts classification time. The RF with statistical filters
> has complexity  where  is the number of trees in the forest and  is the
> number of alerts considered by the filter; efficient library
> implementations of matrix operations means that the execution time does not
> scale linearly with  for the RF inference. Given this, a further experiment
> was carried out with the RF to measure in a live environment how long the
> data collection, model inference, and process killing takes as the number
> of processes increases. This was tested by executing more than 1000
> processes in the virtual machine whilst the process killing RF runs.
>
> Some processes demand more computational resources than others, and some
> malware in our test set locked pages in memory [50
> <https://www.hindawi.com/journals/scn/2021/8933681/#B50>], which
> prevented the model from having sufficient resources to collect data,
> leading to tens of seconds during which no data were captured and many
> processes were launched. With better software engineering practices, the
> model may be more robust against this kind of malicious activity.
>
> These differences in behaviour can cause the evaluation time to lag as
> demonstrated by the outlier points visible in Figure 7
> <https://www.hindawi.com/journals/scn/2021/8933681/fig7/>. The data show
> a broadly linear positive correlation between the number of processes
> (being monitored or killed) and the time taken for the data collection and
> process killing; this confirms the hypothesis that more processes equates
> to slower processing time. The slowest total processing time was 0.81
> seconds (seen with both 17 and 40 simultaneous processes running), but the
> mean processing time is just under 0.3 seconds with 65 simultaneous
> processes, fitting comfortably within the 1-second goal time. Additional
> code optimisation could greatly improve on these initial results which
> indicate that the processing, even using standard libraries and a
> high-level programming language, can execute reasonably quickly.
>
> Figure 7  <https://www.hindawi.com/journals/scn/2021/8933681/fig7/>
> Mean time to collect data, analyse data with Random Forest, and kill
> varying numbers of processes.
> 8. Implications and Analysis
>
> The experiments in this paper address a largely unexplored area of malware
> detection, by comparison with post-trace classification. Real-time
> processing and response has a number of benefits outlined above and the
> results presented here give tentative indications of the advantages and
> challenges of such an approach.
>
> The initial experiments (Section 6.1
> <https://www.hindawi.com/journals/scn/2021/8933681/#sec6.1>) demonstrate
> that a high-accuracy RNN (as used in [11
> <https://www.hindawi.com/journals/scn/2021/8933681/#B11>]) does not
> maintain high-accuracy when used in real-time with an automated response to
> classify individual processes rather than full application traces, since a
> single false positive classification of sequential data cannot be
> outweighed by later correct predictions.
>
> The next set of experiments (Section 6.1
> <https://www.hindawi.com/journals/scn/2021/8933681/#sec6.1>) showed that
> whilst the RNN achieves one of the highest classification accuracies of a
> set of algorithms tested, it is not one of the best in terms of
> computational resource consumption or latency. However, a clear
> best-algorithm was not evident either since the low-resource consuming
> algorithms (like decision tree) did not always achieve high accuracy.
> Furthermore, all of the supervised learning algorithms were clearly
> unsuited to process killing with the highest F1 score from any algorithm
> being 77.85 on the test set compared with 85.55 for process-level
> classification alone. This 85.55 F1 score is lower than is seen in many
> dynamic malware detection research publications that use full-application
> behavioural traces, indicating the challenges of classification at the
> process level, where malware and benignware may share functionality.
>
> Attempting to improve detection accuracy, three approaches were tested:
> statistical filtering, reinforcement learning, and a regression model
> estimating the utility (q-value) of killing a process. Statistical filters
> using rolling mean or alert thresholds were the only approach to improve on
> the supervised learning model F1 score. Reinforcement learning tended to
> kill processes too early and therefore not explore enough scenarios (and
> thus receive the requisite reinforcement) to allow benign processes to
> continue; this does not mean that future models could not improve upon this
> result. This may be supported by the success of the regression models in
> maintaining a high true-negative rate, given that these models ascribed a
> similar utility to killing processes as the reinforcement learning models.
>
> The accuracy metrics tested thus far simply indicate whether a process was
> ever killed, but do not address whether damage was actually prevented by
> process killing. If damage was not prevented, there is little point to
> process killing and a database of alerts for analysis would be a better
> solution since the risk of killing benignware is eliminated. This is why
> the final set of experiments in Section 6.5
> <https://www.hindawi.com/journals/scn/2021/8933681/#sec6.5> were
> conducted to test the detection models in real time and see if damage could
> be prevented by looking at the number file corrupted by ransomware before
> and after infection. Here, we found that it is possible to prevent 92% of
> files from being encrypted whilst maintaining a true negative rate of 82%.
> This result does not indicate that the system is ready for real-world
> deployment but that perhaps further model analysis probably including
> anomaly detection could raise the true negative rate to a usable point.
> This work also demonstrates the damage that certain malware can carry out
> in a short space of time and reinforces the need for further research in
> this area, since previous work has either focused solely on ransomware [24
> <https://www.hindawi.com/journals/scn/2021/8933681/#B24>] or waited
> minutes to being classification [23
> <https://www.hindawi.com/journals/scn/2021/8933681/#B23>], by which time
> it is too late.
> 9. Future Work
>
> Real-time attack detection has wider applications than endpoint detection,
> as Alazab et al. [51
> <https://www.hindawi.com/journals/scn/2021/8933681/#B51>] argue that
> Internet of Things networks in particular could benefit from real-time
> attack detection using heterogeneous data feed from different sensors
> combined using federated learning approaches.
>
> However, some challenges remain to be solved; behavioural malware analysis
> research using machine learning regularly reports 95% classification
> accuracy. Although useful for analysts, behavioural detection should be
> deployed as part of endpoint defensive systems to leverage the full
> benefits of a detection model. Dynamic analysis is not typically used for
> endpoint protection, perhaps because it takes too long in data collection
> to deliver the quick verdicts required for good user experience. Real-time
> detection on the endpoint allows for observation of the full trace without
> the user having to wait. However, real-time detection also introduces the
> risk that malware will cause damage to the endpoint. This risk requires
> that processes detected as malicious are automatically killed as early as
> possible to avoid harm.
>
> There are some key challenges to implementation, which have been outlined
> in this paper:(i)The need for signal separation drives the use of
> individual processes and only partial traces can be used.(ii)The
> significant drop in accuracy on the unseen test set, even without process
> killing demonstrates that additional features may be necessary to improve
> detection accuracy.(iii)With the introduction of process killing, the
> poor performance of the models on either benignware classification (RF min
> 2 alerts: TNR 81% with an 88% TPR on the test set) or on malware
> classification (GBDT regressor min 4 alerts: 56% TPR with a 94% TNR on the
> test set) means that considerable further work is needed before very early
> stage real-time detection can be considered for real-world use.(iv)Real-time
> detection using full execution traces of processes, however, may be viable.
> This is useful to handle VM-aware malware, which may only reveal its true
> behaviour in the target environment. Although the more complex approach
> using DQNs algorithms did not outperform the supervised models with some
> additional statistical thresholds, the regression models had better
> performance in correctly classifying benignware. Reinforcement learning
> could still be useful for real-time detection and automated cyber defense
> models, but the DQN in these experiments did not perform well.(v)Despite
> the theoretical unsuitability of supervised learning models to state-action
> problems, these experiments demonstrate how powerful supervised learning
> can be for classification problems, even if the problem is not quite the
> one that the model is attempting to solve.(vi)Future work may require a
> more comprehensive manual labelling effort at the process level and perhaps
> labelling sub-sections of processes as malicious or benign.
>
> An additional consideration for real-time detection with automated actions
> is whether this introduces an additional denial-of-service vector using
> process injection for example to trigger process killing. This may also
> however indicate that an attacker is present and therefore aid the user.
> 10. Conclusions
>
> This paper has built on previous work in real-time detection to address
> some of the key challenges: signal separation, detection with partial
> execution traces, and computational resource consumption with a focus on
> preventing harm to the user, since real-time detection introduces this risk.
>
> Behavioural malware detection using virtual machines is a well-established
> research field yielding high detection accuracy in recent literature [3
> <https://www.hindawi.com/journals/scn/2021/8933681/#B3>, 6
> <https://www.hindawi.com/journals/scn/2021/8933681/#B6>, 11
> <https://www.hindawi.com/journals/scn/2021/8933681/#B11>, 20
> <https://www.hindawi.com/journals/scn/2021/8933681/#B20>]. However, as is
> shown here, fixed-time execution in a sandbox may not reveal malicious
> functionality. Real-time malware analysis addresses this issue but risks
> executing malware on the endpoint and requires detection to take place at
> the process level, which is more challenging as the definition of a
> malicious process can be unclear. These two reasons may account for the
> limited literature on real-time detection. Looking forward, real-time
> detection may become more popular if static data manipulation and
> VM-evasion continue to be used and the costs of malicious execution
> continue to rise. Real-time detection does not need to be an alternative to
> these approaches, but could hold complementary value as part of a
> defense-in-depth endpoint security.
>
> To the best of our knowledge, previous real-time detection work has used
> up to 5 simultaneous applications, whereas other users may use far more.
> This paper has demonstrated that up to 35 simultaneous applications (and
> nearly 100 simultaneous processes) can be constantly monitored, where
> previous work [23 <https://www.hindawi.com/journals/scn/2021/8933681/#B23>]
> had tested a maximum of 5. Moreover, these results demonstrated that data
> collection presented a greater limiting factor than machine-learning
> algorithms, which can easily process 1000 samples with negligible impact on
> performance. This result is not too surprising since batch processing
> allows algorithms to achieve O(1) complexity by comparison with O(n) for
> data collection.
>
> Automatic actions are necessary in response to detection if the goal is to
> prevent harm. Otherwise, this is equivalent to letting the malware fully
> execute and simply monitor its behaviour since human response times are
> unlikely to be quick enough for fast-acting malware. From a user
> perspective, the question is not “What percentage of malware was executed?”
> or “Was the malware detected in 5 or 10 minutes?” but “How much damage has
> been done?”.
>
> This paper found that by using simple statistical filters on top of
> supervised learning models, it was possible to prevent 92% of files from
> being corrupted by fast-acting ransomware thus reducing the requirements on
> the user or organisation to remediate the damage, since it was prevented in
> the first instance (the rest of the attack vector would remain a concern).
>
> This approach does not achieve the detection accuracies of state-of-the
> art offline behavioural analysis models but, as stated in the introduction,
> these models typically use the full post-execution trace of malicious
> behaviour. Delaying classification until post-execution negates the
> principal advantages of real-time detection. However, the proposed model
> presents an initial step towards a fully automated endpoint protection
> model, which becomes increasingly necessary as adversaries become more and
> more motivated to evade offline automated detection tools.
> Data Availability
>
> Information on the data underpinning the results presented here, including
> how to access them, can be found in the Cardiff University data catalogue
> at 10.17035/d.2021.0148229014.
> Conflicts of Interest
>
> The authors declare that they have no conflicts of interest.
> Acknowledgments
>
> This research was partly funded by the Engineering and Physical Sciences
> Research Council (EPSRC)-grant references EP/P510452/1 and EP/S035362/1.
> The research was also partly funded by Aureirbus Operations Ltd..
> References
>
>    1. U. Tatar, B. Nussbaum, Y. Gokce, and O. F. Keskin, “Digital force
>    majeure: the mondelez case, insurance, and the (un) certainty of
>    attribution in cyberattacks,” *Business Horizons*, vol. 64, 2021.View
>    at: Publisher Site <https://doi.org/10.1016/j.bushor.2021.07.013> | Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Digital%20force%20majeure:%20the%20mondelez%20case,%20insurance,%20and%20the%20(un)%20certainty%20of%20attribution%20in%20cyberattacks&author=U.%20Tatar&author=B.%20Nussbaum&author=Y.%20Gokce&author=&author=O.%20F.%20Keskin&publication_year=2021>
>    2. S. K. Sahay, A. Sharma, and H. Rathore, “Evolution of malware and
>    its detection techniques,” in *Information and Communication
>    Technology for Sustainable Development*, pp. 139–150, Springer,
>    Singapore, 2020.View at: Publisher Site
>    <https://doi.org/10.1007/978-981-13-7166-0_14> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Evolution%20of%20malware%20and%20its%20detection%20techniques&author=S.%20K.%20Sahay&author=A.%20Sharma&author=&author=H.%20Rathore&publication_year=2020>
>    3. W. Huang and J. W. Stokes, “Mtnet: a multi-task neural network for
>    dynamic malware classification,” in *Proceedings of the 13th
>    International Conference on Detection of Intrusions and Malware, and
>    Vulnerability Assessment-Volume 9721, DIMVA 2016*, pp. 399–418,
>    Springer-Verlag New York, Inc., New York, NY, USA, June 2016.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Mtnet:%20a%20multi-task%20neural%20network%20for%20dynamic%20malware%20classification&author=W.%20Huang%20&author=J.%20W.%20Stokes>
>    4. W. Hu and Y. Tan, “Black-box attacks against rnn based malware
>    detection algorithms,” in *Proceedings of the Workshops at the
>    mThirty-Second AAAI Conference on Artificial Intelligence*, Peking
>    University, Beijing, China, February 2018.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Black-box%20attacks%20against%20rnn%20based%20malware%20detection%20algorithms&author=W.%20Hu%20&author=Y.%20Tan>
>    5. Y. Chen, Z. Shan, F. Liu et al., “A gene-inspired malware detection
>    approach,” in *Journal of Physics: Conference Series*, vol. 1168, IOP
>    Publishing, 2019.View at: Publisher Site
>    <https://doi.org/10.1088/1742-6596/1168/6/062004> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=A%20gene-inspired%20malware%20detection%20approach&author=Y.%20Chen&author=Z.%20Shan&author=F.%20Liu%20et%20al.&publication_year=2019>
>    6. M. Ijaz, M. H. Durad, and M. Ismail, “Static and dynamic malware
>    analysis using machine learning,” in *Proceedings of the 2019 16th
>    International Bhurban Conference on Applied Sciences and Technology
>    (IBCAST)*, pp. 687–691, IEEE, Islamabad, Pakistan, January 2019.View
>    at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Static%20and%20dynamic%20malware%20analysis%20using%20machine%20learning&author=M.%20Ijaz&author=M.%20H.%20Durad&author=&author=M.%20Ismail>
>    7. I. You and K. Yim, “Malware obfuscation techniques: a brief
>    survey,” in *Proceedings of the 2010 International Conference on
>    Broadband, Wireless Computing, Communication and Applications*, pp.
>    297–300, IEEE, Fukuoka, Japan, November 2010.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Malware%20obfuscation%20techniques:%20a%20brief%20survey&author=I.%20You%20&author=K.%20Yim>
>    8. B. Kolosnjaji, A. Demontis, B. Biggio et al., “Adversarial malware
>    binaries: evading deep learning for malware detection in executables,” in *Proceedings
>    of the 2018 26th European Signal Processing Conference (EUSIPCO)*, pp.
>    533–537, IEEE, Rome, Italy, September 2018.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Adversarial%20malware%20binaries:%20evading%20deep%20learning%20for%20malware%20detection%20in%20executables&author=B.%20Kolosnjaji&author=A.%20Demontis&author=B.%20Biggio%20et%20al.>
>    9. D. Carlin, P. O’Kane, and S. Sezer, “A cost analysis of machine
>    learning using dynamic runtime opcodes for malware detection,” *Computers
>    & Security*, vol. 85, pp. 138–155, 2019.View at: Publisher Site
>    <https://doi.org/10.1016/j.cose.2019.04.018> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=A%20cost%20analysis%20of%20machine%20learning%20using%20dynamic%20runtime%20opcodes%20for%20malware%20detection&author=D.%20Carlin&author=P.%20O%E2%80%99Kane&author=&author=S.%20Sezer&publication_year=2019>
>    10. T. Shibahara, T. Yagi, M. Akiyama, D. Chiba, and T. Yada,
>    “Efficient dynamic malware analysis based on network behavior using deep
>    learning,” in *Proceedings of the 2016 IEEE Global Communications
>    Conference (GLOBECOM)*, pp. 1–7, IEEE, Washington, DC, USA, December
>    2016.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Efficient%20dynamic%20malware%20analysis%20based%20on%20network%20behavior%20using%20deep%20learning&author=T.%20Shibahara&author=T.%20Yagi&author=M.%20Akiyama&author=D.%20Chiba&author=&author=T.%20Yada>
>    11. M. Rhode, P. Burnap, and K. Jones, “Early-stage malware prediction
>    using recurrent neural networks,” *Computers & Security*, vol. 77, pp.
>    578–594, 2018.View at: Publisher Site
>    <https://doi.org/10.1016/j.cose.2018.05.010> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Early-stage%20malware%20prediction%20using%20recurrent%20neural%20networks&author=M.%20Rhode&author=P.%20Burnap&author=&author=K.%20Jones&publication_year=2018>
>    12. J. Saxe and K. Berlin, “Deep neural network based malware
>    detection using two dimensional binary program features,” in *Proceedings
>    of the 2015 10th International Conference on Malicious and Unwanted
>    Software (MALWARE)*, pp. 11–20, IEEE, Fajardo, PR, USA, October 2015.View
>    at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Deep%20neural%20network%20based%20malware%20detection%20using%20two%20dimensional%20binary%20program%20features&author=J.%20Saxe%20&author=K.%20Berlin>
>    13. F. Pendlebury, F. Pierazzi, R. Jordaney, J. Kinder, and L.
>    Cavallaro, “{TESSERACT}: eliminating experimental bias in malware
>    classification across space and time,” in *Proceedings of the 28th
>    {USENIX} Security Symposium ({USENIX} Security 19), {USENIX} Association*,
>    pp. 729–746, Santa Clara, CA, USA, September 2019.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title={TESSERACT}:%20eliminating%20experimental%20bias%20in%20malware%20classification%20across%20space%20and%20time&author=F.%20Pendlebury&author=F.%20Pierazzi&author=R.%20Jordaney&author=J.%20Kinder&author=&author=L.%20Cavallaro>
>    14. K. Grosse, N. Papernot, P. Manoharan, M. Backes, and P. McDaniel,
>    “Adversarial perturbations against deep neural networks for malware
>    classification,” 2016, https://arxiv.org/abs/1606.04435 arXiv preprint
>    arXiv:1606.04435.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Adversarial%20perturbations%20against%20deep%20neural%20networks%20for%20malware%20classification&author=K.%20Grosse&author=N.%20Papernot&author=P.%20Manoharan&author=M.%20Backes&author=&author=P.%20McDaniel&publication_year=2016>
>    15. H. Sayadi, N. Patel, S. M. Pd, A. Sasan, S. Rafatirad, and H.
>    Homayoun, “Ensemble learning for effective run-time hardware-based malware
>    detection: a comprehensive analysis and classification,” in *Proceedings
>    of the 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC)*,
>    pp. 1–6, IEEE, San Francisco, CA, USA, June 2018.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Ensemble%20learning%20for%20effective%20run-time%20hardware-based%20malware%20detection:%20a%20comprehensive%20analysis%20and%20classification&author=H.%20Sayadi&author=N.%20Patel&author=S.%20M.%20Pd&author=A.%20Sasan&author=S.%20Rafatirad&author=&author=H.%20Homayoun>
>    16. N. Usman, S. Usman, F. Khan et al., “Intelligent dynamic malware
>    detection using machine learning in ip reputation for forensics data
>    analytics,” *Future Generation Computer Systems*, vol. 118, pp.
>    124–141, 2021.View at: Publisher Site
>    <https://doi.org/10.1016/j.future.2021.01.004> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Intelligent%20dynamic%20malware%20detection%20using%20machine%20learning%20in%20ip%20reputation%20for%20forensics%20data%20analytics&author=N.%20Usman&author=S.%20Usman&author=F.%20Khan%20et%20al.&publication_year=2021>
>    17. P. Burnap, R. French, F. Turner, and K. Jones, “Malware
>    classification using self organising feature maps and machine activity
>    data,” *Computers & Security*, vol. 73, pp. 399–410, 2018.View at: Publisher
>    Site <https://doi.org/10.1016/j.cose.2017.11.016> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Malware%20classification%20using%20self%20organising%20feature%20maps%20and%20machine%20activity%20data&author=P.%20Burnap&author=R.%20French&author=F.%20Turner&author=&author=K.%20Jones&publication_year=2018>
>    18. M. Rhode, L. Tuson, P. Burnap, and K. Jones, “Lab to soc: robust
>    features for dynamic malware detection,” in *Proceedings of the 2019
>    49th Annual IEEE/IFIP International Conference on Dependable Systems and
>    Networks–Industry Track*, pp. 13–16, IEEE, Portland, OR, USA, June
>    2019.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Lab%20to%20soc:%20robust%20features%20for%20dynamic%20malware%20detection&author=M.%20Rhode&author=L.%20Tuson&author=P.%20Burnap&author=&author=K.%20Jones>
>    19. H. Sayadi, A. Houmansadr, S. Rafatirad, H. Homayoun, and P. D. Sai
>    Manoj, “Comprehensive assessment of run-time hardware-supported malware
>    detection using general and ensemble learning,” in *Proceedings of the
>    15th ACM International Conference on Computing Frontiers*, pp.
>    212–215, ACM, Ischia, Italy, May 2018.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Comprehensive%20assessment%20of%20run-time%20hardware-supported%20malware%20detection%20using%20general%20and%20ensemble%20learning&author=H.%20Sayadi&author=A.%20Houmansadr&author=S.%20Rafatirad&author=H.%20Homayoun&author=&author=P.%20D.%20Sai%20Manoj>
>    20. S. Das, Y. Liu, W. Zhang, and M. Chandramohan, “Semantics-based
>    online malware detection: towards efficient real-time protection against
>    malware,” *IEEE Transactions on Information Forensics and Security*,
>    vol. 11, no. 2, pp. 289–302, 2016.View at: Publisher Site
>    <https://doi.org/10.1109/tifs.2015.2491300> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Semantics-based%20online%20malware%20detection:%20towards%20efficient%20real-time%20protection%20against%20malware&author=S.%20Das&author=Y.%20Liu&author=W.%20Zhang&author=&author=M.%20Chandramohan&publication_year=2016>
>    21. M. Ozsoy, C. Donovick, I. Gorelik, N. Abu-Ghazaleh, and D.
>    Ponomarev, “Malware-aware processors: a framework for efficient online
>    malware detection,” in *Proceedings of the 2015 IEEE 21st
>    International Symposium on High Performance Computer Architecture (HPCA)*,
>    pp. 651–661, IEEE, Burlingame, CA, USA, February. 2015.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Malware-aware%20processors:%20a%20framework%20for%20efficient%20online%20malware%20detection&author=M.%20Ozsoy&author=C.%20Donovick&author=I.%20Gorelik&author=N.%20Abu-Ghazaleh&author=&author=D.%20Ponomarev>
>    22. X. Yuan, “Phd forum: deep learning-based real-time malware
>    detection with multi-stage analysis,” in *Proceedings of the 2017 IEEE
>    International Conference on Smart Computing (SMARTCOMP)*, pp. 1-2,
>    IEEE, Hong Kong, China, May 2017.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Phd%20forum:%20deep%20learning-based%20real-time%20malware%20detection%20with%20multi-stage%20analysis&author=X.%20Yuan>
>    23. R. Sun, X. Yuan, P. He et al., “Learning fast and slow:
>    PROPEDEUTICA for real-time malware detection,” *CoRR*, vol. abs/1712,
>    Article ID 01145, 2017.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Learning%20fast%20and%20slow:%20PROPEDEUTICA%20for%20real-time%20malware%20detection&author=R.%20Sun&author=X.%20Yuan&author=P.%20He%20et%20al.&publication_year=2017>
>    24. N. Scaife, H. Carter, P. Traynor, and K. R. Butler, “Cryptolock
>    (and drop it): stopping ransomware attacks on user data,” in *Proceedings
>    of the 2016 IEEE 36th International Conference on Distributed Computing
>    Systems (ICDCS)*, pp. 303–312, IEEE, Nara, Japan, June 2016.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Cryptolock%20(and%20drop%20it):%20stopping%20ransomware%20attacks%20on%20user%20data&author=N.%20Scaife&author=H.%20Carter&author=P.%20Traynor&author=&author=K.%20R.%20Butler>
>    25. GlobalStats, “Market share of windows operating system versions,”
>    2018,
>    http://gs.statcounter.com/os-version-market-share/windows/desktop/worldwide#monthly-201708-201709-bar
>    .View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Market%20share%20of%20windows%20operating%20system%20versions&author=GlobalStats&publication_year=2018>
>    26. J. Benda, A. Longtin, and L. Maler, “Spike-frequency adaptation
>    separates transient communication signals from background oscillations,” *Journal
>    of Neuroscience*, vol. 25, no. 9, pp. 2312–2321, 2005.View at: Publisher
>    Site <https://doi.org/10.1523/jneurosci.4795-04.2005> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Spike-frequency%20adaptation%20separates%20transient%20communication%20signals%20from%20background%20oscillations&author=J.%20Benda&author=A.%20Longtin&author=&author=L.%20Maler&publication_year=2005>
>    27. H.-D. Huang, C.-S. Lee, H.-Y. Kao, Y.-L. Tsai, and J.-G. Chang,
>    “Malware behavioral analysis system: Twman,” in *Proceedings of the
>    2011 IEEE Symposium on Intelligent Agent (IA)*, pp. 1–8, IEEE, Paris,
>    France, April 2011.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Malware%20behavioral%20analysis%20system:%20Twman&author=H.-D.%20Huang&author=C.-S.%20Lee&author=H.-Y.%20Kao&author=Y.-L.%20Tsai&author=&author=J.-G.%20Chang>
>    28. T. Kim, B. Kang, and E. G. Im, “Runtime detection framework for
>    android malware,” *Mobile Information Systems*, vol. 2018, 2018.View
>    at: Publisher Site <https://doi.org/10.1155/2018/8094314> | Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Runtime%20detection%20framework%20for%20android%20malware&author=T.%20Kim&author=B.%20Kang&author=&author=E.%20G.%20Im&publication_year=2018>
>    29. R. S. Sutton and A. G. Barto, *Introduction to Reinforcement
>    Learning*, vol. 135, MIT Press Cambridge, Cambridge, MA, USA, 1998.
>    30. C. J. C. H. Watkins, “Learning from delayed rewards,” King’s
>    University, London, UK, 1989, PhD thesis.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Learning%20from%20delayed%20rewards&author=C.%20J.%20C.%20H.%20Watkins&publication_year=1989>
>    31. C. J. Watkins and P. Dayan, “Q-learning,” *Machine Learning*, vol.
>    8, no. 3-4, pp. 279–292, 1992.View at: Publisher Site
>    <https://doi.org/10.1023/a:1022676722315> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Q-learning&author=C.%20J.%20Watkins%20&author=P.%20Dayan&publication_year=1992>
>    32. R. S. Sutton, “Integrated architectures for learning, planning,
>    and reacting based on approximating dynamic programming,” *Machine
>    Learning Proceedings 1990*, vol. 1990, pp. 216–224, 1990.View at: Publisher
>    Site <https://doi.org/10.1016/b978-1-55860-141-3.50030-4> | Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Integrated%20architectures%20for%20learning,%20planning,%20and%20reacting%20based%20on%20approximating%20dynamic%20programming&author=R.%20S.%20Sutton&publication_year=1990>
>    33. L.-J. Lin, “Self-improving reactive agents based on reinforcement
>    learning, planning and teaching,” *Machine Learning*, vol. 8, no. 3-4,
>    pp. 293–321, 1992.View at: Publisher Site
>    <https://doi.org/10.1007/bf00992699> | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Self-improving%20reactive%20agents%20based%20on%20reinforcement%20learning,%20planning%20and%20teaching&author=L.-J.%20Lin&publication_year=1992>
>    34. V. Mnih, K. Kavukcuoglu, D. Silver et al., “Playing atari with
>    deep reinforcement learning,” 2013, https://arxiv.org/abs/1312.5602 arXiv
>    preprint arXiv:1312.5602.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Playing%20atari%20with%20deep%20reinforcement%20learning&author=V.%20Mnih&author=K.%20Kavukcuoglu&author=D.%20Silver%20et%20al.&publication_year=2013>
>    35. V. Mnih, A. P. Badia, M. Mirza et al., “Asynchronous methods for
>    deep reinforcement learning,” in *Proceedings of the International
>    Conference on Machine Learning*, pp. 1928–1937, PMLR, New York City,
>    NY, USA, June 2016.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Asynchronous%20methods%20for%20deep%20reinforcement%20learning&author=V.%20Mnih&author=A.%20P.%20Badia&author=M.%20Mirza%20et%20al.>
>    36. J. Kirkpatrick, R. Pascanu, N. Rabinowitz et al., “Overcoming
>    catastrophic forgetting in neural networks,” *Proceedings of the
>    National Academy of Sciences*, vol. 114, no. 13, pp. 3521–3526, 2017.View
>    at: Publisher Site <https://doi.org/10.1073/pnas.1611835114> | Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Overcoming%20catastrophic%20forgetting%20in%20neural%20networks&author=J.%20Kirkpatrick&author=R.%20Pascanu&author=N.%20Rabinowitz%20et%20al.&publication_year=2017>
>    37. sophos, 2020,
>    https://www.sophos.com/en-us/medialibrary/Gated-Assets/white-papers/sophos-the-state-of-ransomware-2020-wp.pdf
>    .
>    38. A. Continella, A. Guagnelli, G. Zingaro et al., “Shieldfs: a
>    self-healing, ransomware-aware filesystem,” in *Proceedings of the
>    32nd Annual Conference on Computer Security Applications*, pp.
>    336–347, ACM, Los Angeles, CF, USA, December 2016.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Shieldfs:%20a%20self-healing,%20ransomware-aware%20filesystem&author=A.%20Continella&author=A.%20Guagnelli&author=G.%20Zingaro%20et%20al.>
>    39. P. S. Foundation, “Psutil python library,” 2017.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Psutil%20python%20library&author=P.%20S.%20Foundation&publication_year=2017>
>    40. C. Guarnieri, A. Tanasi, J. Bremer, and M. Schloesser, “The cuckoo
>    sandbox,” 2012, https://cuckoosandbox.org/.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=The%20cuckoo%20sandbox&author=C.%20Guarnieri&author=A.%20Tanasi&author=J.%20Bremer&author=&author=M.%20Schloesser&publication_year=2012>
>    41. Amazon.com, “Amazon laptops,” 2018,
>    https://www.amazon.com/Notebooks-Laptop-Computers/b?ie=UTF8&node=565108
>    .View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Amazon%20laptops&author=Amazon.com&publication_year=2018>
>    42. B. Quintero, E. Martínez, V. Manuel Álvarezv, K. Hiramoto, J.
>    Canto, and A. Bermúdez, “Virustotal,” 2004,
>    https://en.wikipedia.org/wiki/VirusTotal.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Virustotal&author=B.%20Quintero&author=E.%20Mart%C3%ADnez&author=V.%20Manuel%20%C3%81lvarezv&author=K.%20Hiramoto&author=J.%20Canto&author=&author=A.%20Berm%C3%BAdez&publication_year=2004>
>    43. VirusShare.com, “Virusshare.com,” 2017, https://virusshare.com/.View
>    at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Virusshare.com&author=VirusShare.com&publication_year=2017>
>    44. M. Sebastián, R. Rivera, P. Kotzias, and J. Caballero, “Avclass: a
>    tool for massive malware labeling,” in *Proceedings of the
>    International Symposium on Research in Attacks, Intrusions, and Defenses*,
>    pp. 230–253, Springer, New York, NY, USA, September 2016.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Avclass:%20a%20tool%20for%20massive%20malware%20labeling&author=M.%20Sebasti%C3%A1n&author=R.%20Rivera&author=P.%20Kotzias&author=&author=J.%20Caballero>
>    45. SoftAntenna, “Msys2 installer,” 2019,
>    https://www.softantenna.com/softwares/7115-msys2-installer.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Msys2%20installer&author=SoftAntenna&publication_year=2019>
>    46. A. Paszke, S. Gross, S. Chintala et al., in *Automatic
>    Differentiation in Pytorch*, NIPS-W, Long Beach, CA, USA, 2017.
>    47. F. Pedregosa, G. Varoquaux, A. Gramfort et al., “Scikit-learn:
>    machine learning in Python,” *Journal of Machine Learning Research*,
>    vol. 12, pp. 2825–2830, 2011.View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Scikit-learn:%20machine%20learning%20in%20Python&author=F.%20Pedregosa&author=G.%20Varoquaux&author=A.%20Gramfort%20et%20al.&publication_year=2011>
>    48. D. P. Kingma and J. Ba, “Adam: ’a’ method for stochastic
>    optimization,” *CoRR*, vol. abs/1412, p. 6980, 2014.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Adam:%20%E2%80%99a%E2%80%99%20method%20for%20stochastic%20optimization&author=D.%20P.%20Kingma%20&author=J.%20Ba&publication_year=2014>
>    49. C. Belgaid, A. d’Azémar, G. Fieni, and R. Rouvoy, “Pyrapl,” 2019,
>    Software version 0.2.3.1 https://pypi.org/project/pyRAPL/.View at: Google
>    Scholar
>    <https://scholar.google.com/scholar_lookup?title=Pyrapl&author=C.%20Belgaid&author=A.%20d%E2%80%99Az%C3%A9mar&author=G.%20Fieni&author=&author=R.%20Rouvoy&publication_year=2019>
>    50. M. Corporation, “Lock pages in memory,” 2017,
>    https://docs.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/lock-pages-in-memory
>    .View at: Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Lock%20pages%20in%20memory&author=M.%20Corporation&publication_year=2017>
>    51. M. Alazab, S. P. Rm, M. Parimala, P. Reddy, T. R. Gadekallu, and
>    Q.-V. Pham, “Federated learning for cybersecurity: concepts, challenges and
>    future directions,” *IEEE Transactions on Industrial Informatics*,
>    2021.View at: Publisher Site <https://doi.org/10.1109/tii.2021.3119038>
>     | Google Scholar
>    <https://scholar.google.com/scholar_lookup?title=Federated%20learning%20for%20cybersecurity:%20concepts,%20challenges%20and%20future%20directions&author=M.%20Alazab&author=S.%20P.%20Rm&author=M.%20Parimala&author=P.%20Reddy&author=T.%20R.%20Gadekallu&author=&author=Q.-V.%20Pham&publication_year=2021>
>
> Copyright
>
> Copyright © 2021 Matilda Rhode et al. This is an open access article
> distributed under the Creative Commons Attribution License
> <http://creativecommons.org/licenses/by/4.0/>, which permits unrestricted
> use, distribution, and reproduction in any medium, provided the original
> work is properly cited.
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/html
Size: 227020 bytes
Desc: not available
URL: <https://lists.cpunks.org/pipermail/cypherpunks/attachments/20220519/fcd24e39/attachment.txt>