Wrong in the Right Way: Balancing Realism Against Other Constraints in Simulation-Based Training

There is an unspoken but pervasive assumption that the human behavior representations which populate simulation-based training systems must exhibit realistic behavior if the training is to be effective. This assumption motivates the continual calls to improve the realism of computer-generated forces and reflects the concern that human behavior representations (HBRs) that are “brittle” or too predictable will lead to a negative transfer of training. The emphasis on realism as the panacea for simulation-based training has led many HBR researchers to overlook significant issues in the development of human behavior representations for training. Chief among these is that it is ultimately an empirical matter to determine how much realism is needed to meet training objectives in any given situation. While the tradeoff between fidelity and training effectiveness has been explored, the focus has either been on the “look and feel” of the simulation environment or the focus was on simulation research rather than training effectiveness (e.g., [1]).

It is not hard to imagine situations in which deliberately unrealistic behaviors would be needed to support effective training. For example, leveraging the training manager’s allocentric view of the simulation world, a synthetic adversary could be programmed to repeatedly probe a student’s weakness even if an actual adversary could never have such a perspective. As realism has become a primary concern for HBR researchers, many pragmatic issues that surround simulation-based training have been largely ignored. For example, questions about the cost of development and maintenance are rarely addressed, and when they are, the answer is often concerning. For example, Gluck [2] estimates the time and cost of developing a model of a “moderately complex task” at 3.4 years and $400,000. This does not include the follow-up costs of upgrading the model as the systems it interacts with change. Likewise, questions about the representations themselves—whether their behavior can be explained, whether the behavior is repeatable, whether the behavior can be adjusted—are rarely considered, but each is important if training managers are to understand, account for and trust the behaviors they are employing in their simulations.

In this paper, we describe a development methodology that specifically balances the need for sufficient realism in simulation-based training against the competing requirements for affordable, tractable and effective human behavior representations. The essential feature of this method is a focus on representing behavior at a “tactical” level by way of a finite state machine. Although finite state models of human behavior are hardly novel, they provide a perspicuous representation that can be used to communicate the behavior of models to subject matter experts for review and modification. What is novel about our method is the ability to integrate the finite state models with other formalisms, allowing the incremental additions of fidelity where more sophisticated behavior is required. In this way, we seek to address the criticism that finite state models are nothing more than glorified scripts while, at the same time, ensuring that the investments in improving the behavioral realism are targeted and limited. To abuse Einstein’s admonition, a model should be realistic as it needs to be, but not more so. In that same spirit, we note that our goal is not to advance theories of human behavior or to concern ourselves with process-level details when other, more direct methods of representation produce the behavior needed to meet a training objective.

We begin by discussing how concerns about behavioral realism have come to take center stage in the development of simulation-based training while pointing out that this singular focus stands in opposition to other accepted instructional design principles in which training is deliberately modulated to meet both the abilities of the student and the objectives of the instructor. Next, we present the software architecture that underpins our integrating approach to human behavior representation. Finally, to bring the discussion to ground, we describe two ongoing efforts in which we’re using our methodology to develop human behavior representation for simulation-based training environments. One is a “fast jet” simulator to train combat tactics for Air Force pilots, the other is a network “sandbox” to train cyber defenders. In both cases, our models have been designed to support direct and explainable representations of behavior that can be tailored to meet both training objectives and different requirements for realism.

Within the military, the imperative to ‘train as you fight, fight as you train’ is gospel. It follows from the conviction that training must be as realistic as possible if it is to transfer to complex demands of the battlefield. Accordingly, there have been countless calls to improve the realism of the computer-generated forces used for simulation-based training and even entire conferences devoted to the subject. While there might be some question about how behavioral realism is best assessed, no one questions the need for realism in simulation-based training.

But as Hughes and Rolek and point out, it is one thing to improve the realism of systems and entities as physical objects and quite another to address the realism of human behaviors. Whereas the physics that govern the behavior of a system or object are well-known and directly specified (e.g., as kinematic equations), the processes that govern cognitive behavior are not. The field of cognitive modeling—a discipline focused on the computational representation on psychological processes—is an area of active research in which even basic methodological assumptions are up for debate. For this reason, such models are usually developed by research scientists using highly specialized “architectures.” This leads to the high development and maintenance cost for cognitive models we noted previously. It also led Chandrasekaran and Josephson [3] to argue that, given the underspecified nature of cognitive behavior along with the high development costs, it is necessary to consider carefully the requirements a cognitive model must meet so that the appropriate tradeoffs can be weighed. Further, Chandrasakaran and Josephson claim that there is simply no systematic knowledge to drive such a process.

Our experience is consistent with that claim. In place of any sort of systematic method for determining the requirements a simulation places on a cognitive model, scientists and practitioners alike default to various standards of realism, usually assessed in terms of how well a model fits observed human performance on the same task or by subjecting the model to a sort of Turing test. While it seems conservative, this default approach actually obscures an important fact, namely, that many effective modes of training are unrealistic by design. Part-task training, increasing difficulty, deliberate practice, out-driven simulation all forego aspects of realism within the simulated environment to target other pathways for training complex skills [4, 5].

While the trade-offs between simulator realism and training effectiveness have been explored [6], these investigations still presuppose that more realism would be better if only technology or resources allowed it. We are making a different claim, namely, that realism should be subordinated to effectiveness in the development of simulation-based training and, following Chandrasekaran and Josephson [3], that it is ultimately an empirical question to determine how much realism is required. This claim has implications for how we develop and evaluate human behavior representations for simulation-based training. Next, we describe a software infrastructure that supports the incremental addition of realism in the development of behavior representation. We then describe two different projects in which we’ve applied this infrastructure and before we briefly discussing the broader implications of this approach.

Although computer generated forces have employed a modular approach to the simulation of physics-based behavior, human behavior representations tend to be monolithic. This tendency can be traced to traditions in cognitive modeling, in which so-called cognitive architectures [7, 8] have been advanced as computational instantiations of theories that are put to empirical test by comparing the simulation-based predictions of models derived from those architectures against the actual performance of human subjects. As we have argued elsewhere [9], even when cognitive modelers work in an applied context, they are still cognitive scientists whose interests and biases are shaped by their training. At the very least, this leads modelers to use the tools with which they are familiar. In more extreme cases, model development can be mired in what is essentially a proxy war for long standing academic debates as design decisions are second-guessed more for their theoretical implications than their practical benefits. Either way, the result is that architectures are often applied monolithically, even when they are not especially well-suited to the problem at hand.

Rather than view the application of cognitive architecture as an all-or-nothing choice, we see the variety of cognitive architectures as affording different strengths and weaknesses in applied settings [10]. In that spirit, we developed software infrastructure that allows the modeler to choose the ‘right tool for the job’ by promoting the decomposition of agent behaviors so that cognitive architectures can be applied in a piecemeal manner with the requirements of the behavior driving the choice of architecture. We set out to satisfy two design goals with our infrastructure.

The first goal was to allow the development of component models using different modeling formalisms (Fig. 1). Component models can represent any one of the wide variety of human behaviors that underpin intelligence—from perception, through cognition, to action. Examples include component models of vision, auditory processing, recognition, decision-making, and motor planning. Component models can be constructed using different modeling formalisms (e.g., ACT-R, SOAR, and C3TRACE) and programming languages (e.g., MATLAB and R).

Libraries of basic component building blocks created using different modeling formalisms

Full size image

Component models provide the basic blocks for assembling more sophisticated behaviors (Fig. 2). This enables re-use; the same component model can be used for new behaviors. This also enables composability; complex models can be assembled from simpler component models. Behaviors can be built using components from one modeling formalism (Fig. 2, Panel 1), two modeling formalisms (Panel 2), or many modeling formalisms (Panel 3). Most importantly, this approach allows the modeler to make trade-offs between representational fidelity and cost. For example, high-fidelity cognitive architectures such as ACT-R and Soar provide detailed accounts of human behavior at a significant complexity cost, while task network or mathematical models may provide an adequate low-cost alternative.

Composition of increasingly sophisticated agents from basic component building blocks

Full size image

The second design goal was to implement a “workspace” that enables interactions among component models. This is the “glue” that holds the component models together. The workspace supports the definition of persistent local objects (i.e., local to the workspace) that define the input-output relationships among the component models at run time. In this way, the workspace ensures system level modularity; a well-specified interface can allow different human behavior representations to “plug and play” with a given simulation. This functionality is key to supporting the incremental addition of fidelity to human behavior representations. More precisely, having defined the interface among component models, it is possible to “stub out” out an aspect of behavior that could be realized using any number of different formalisms; the choice can then be driven by requirements of the application rather than by constraints imposed by a particular cognitive architecture.

We have taken this incremental approach in the development of two different human behavior representations intended to support simulation-based training. In both cases, we used a task network model to provide the initial stub for the human behavior representation. Task network modeling tools provide a framework for representing human behavior as a decomposition of operator goals or functions into their component tasks, which themselves can be further decomposed. This framework is visualized by way of an intuitive graphical representation of the behavior being modeled in which nodes represent tasks and directed edges represent the flow of control among the tasks. While the flow of control in a task network model can be complicated, there are ultimately two fundamental mechanisms that drive the overt behavior of the model: the “decision logic” used at the branch points in the task network and the time associated with the execution of each task (specified by way of distribution). We now describe each model and how we used these two mechanisms to meet training objectives.

4.1 A Model of Pilot Behavior for the Not So Grand Challenge

The Not So Grand Challenge (NSGC) is a multi-partner research initiative sponsored by the Air Force Research Laboratory [11]. Like previous “model comparison” efforts [12], the goal of the NSGC is to extend the state of the art in agent modeling by providing a uniform interface to a constructive simulation environment and a set of required behaviors so that different approaches to behavior representation can be compared against a common baseline. In addition to understanding the models themselves, the NSGC also provides a context in which to assess development costs, the extent to which models are “data-driven” and the likelihood that a model can generalize to novel scenarios.

Each partner in the NSGC was tasked to develop a model of an adversarial pilot that would fly against another simulated pilot in a constructive fast-jet simulator. The goal was to produce tactical behaviors (e.g., changes in speed, heading altitude, etc.) that were both robust and flexible under different scenarios, with an eye to using the red pilot models in the simulation-based training for humans. We used C3Trace, a government-owned task network modeling tool, to implement a model of the red pilot performing a stern conversion. Figure 3 depicts the top-level view of the tasks that the pilot performs.

The task network model of a pilot performing a stern conversion.

Full size image

The series of tasks that begins with “Red Flies on CATA (1_3)” represents the different states the red pilot occupies when performing a stern conversion—fly straight and level into the engagement until inside of 25 nm; maneuver to gain offset; go into pure pursuit at 120-degree aspect angle. The pilot remains in each state until certain conditions are met (described more fully later). This series of tasks implements a finite-state model of stern conversion; each state summarizes a specific set of actions that the pilot performs until the model transitions to another state.

Although the task network model can serve as a complete representation of the red pilot, we found it useful to integrate the model with a higher-fidelity representation of change detection. The intuition here is that even if the red pilot is implemented as a simple finite state model, the transitions between states need not be driven only by strict rules or threshold conditions. Instead, we used an accumulator model to detect relevant changes in blue’s tactical posture. The accumulator model gets its name from the fact that evidence is accumulated over time (see Fig. 4). Incoming information, in this case kinematic information about the blue aircraft, drives two different accumulator models toward their respective decision bounds. One model detects changes in heading and the other detects changes in “aspect angle” (related but usefully different features of tactical maneuvering). The process terminates when accumulated evidence reaches a decision bound, at which time a categorization decision is made.

An example of how evidence based on blue’s aspect and heading are combined to detect a tactical change

Full size image

Whereas the task network model bears a very clear correspondence to the tactical behavior it represents (i.e., the model recapitulates a real-world process flow), the accumulator model represents a (potentially) data-driven decision-making process. Although domain knowledge is still needed for defining the inputs and decision boundaries, the inner workings of the accumulator model are determined by an off-line training process in which parameters are tuned to achieve the needed performance. In this case, that meant running the accumulator model against baseline scenarios to determine whether the model could consistently and correctly identity tactical changes in blue given different intercept geometries. Having labeled blue behavior in each scenario by hand, and incrementally adjusting the parameters that control how much evidence is needed to reach a decision boundary and how long accumulated evidence persists, the model achieved reasonable performance after a few dozen training runs.

4.2 A Model of a Cyber Attacker

Working on a different project, we developed another model of adversarial behavior. In this case, the goal was to simulate the behavior of a hacker attacking a network in different ways. Once again, we used a task network model to represent the steps attacker uses to launch an attack (e.g., identify a potential target site; probe the site; chose an attack to exploit a discovered vulnerability) (Fig. 5).

The subnetwork of tasks for executing an SQL-injection attack

Full size image

The task network was doubly useful here. First, it provided a direct representation of the behavior which itself is useful in a training context (unlike the fluid tactical dynamics of air-to-air combat, cyberattacks are often automated, making them especially amenable to a finite state representation). Second, we were able to modulate the behavior of the task network. The ability to modulate the agent models in this way highlights the difference between a simulation-based training system and a cyber range used for penetration testing. While there is clearly some benefit in exposing cyber defenders to the full-scale complexity of an attack unfolding in real time across a large cyber range, such realism does not ensure training effectiveness and, in many cases, might hinder it. By contrast, the ability to tailor the attack models, to vary the targets and attack vectors, to adjust the timing of the attacks, and to adjust the “signal-to-noise” ratio ensure that a cyber defender will experience simulation-based training that meet specific training objectives. Each of these adjustments is easily made in the task network model, either by changing the timing associated with each task in the model or by varying the probabilities associated with branch points in the network. Where the utility of penetration testing demands the use of the most sophisticated attacks to uncover as-yet unrecognized vulnerabilities, the utility of a training system requires the ability to modulate the attacks to meet the evolving abilities of the student.

We began development of the two models just described by implementing a task network model. This provided us the “stub” we needed before we considered elaborations to the models. In the case of the NSGC model, the integration of the accumulator model was motivated by the need to demonstrate how aspects of adversarial behavior can be data-driven, insofar as they are learned. While the realism of the resulting behavior is still an important consideration, the chief benefit of such an approach is the potential efficiency gain in model development. Similarly, our model of the cyber attacker allows for modifications that allow for variations in the tempo and salience of the attack. In both cases, realism is a secondary consideration to other requirements that constrain simulation-based training.

While popular imagination might view complete immersion into a hyper-realistic virtual environment as the ultimate in simulation-based training, in our experience, the reality of simulation-based training is one in which both costs and training objectives are unavoidable considerations. This experience militates for both our incremental approach to behavior development and training effectiveness, rather than realism per se, as the standard to meet when evaluating human behavior representations. In both respects, we run counter to the more commonplace approach to human behavior representations for simulation-based training, namely, apply a well-accepted architecture to the problem and judge the realism of the resulting behavior. Instead, following [3], we aim to understand the requirements of the simulation-based training (be they cost, realism, “explainability” etc.) and add features to the human-behavior representation accordingly. At that point, the measure of the human behavior representation should be whether it engenders effective training, which is an altogether different question than whether the behavior is realistic.