A Methodology for Enhancing Crew Resource

Published on June 2016 | Categories: Documents | Downloads: 29 | Comments: 0 | Views: 226
of 12
Download PDF   Embed   Report

Comments

Content


INTRODUCTION
In the 1970s, hundreds of airline passengers
on routine, scheduled flights lost their lives
because each of three aircrews committed an
error. In one incident, the crew failed to take
fuel levels into consideration during problem
solving; in the second incident, the crew did
not monitor the altitude; and in the third inci-
dent, the crew misinterpreted an air traffic
control communication. The crew members who
committed the errors had tens of thousands of
hours of flight experience, yet the errors com-
mitted should have been avoided by even the
most inexperienced pilots. The crews were not
members of a country in which standards of
pilot training and certification were question-
able, and each of these crews worked for a
major air carrier.
Two of the crews were flying domestic opera-
tions within the United States. In both domestic
accidents, the crew members committed the
errors while responding to a potentially unsafe
problem with the plane by taking extra time and
care to troubleshoot or prepare (or both) for
this unplanned circumstance. As a result of their
lapses, their planes ended the flight in one case
by crashing into a stand of trees in Oregon, and
in the other case by crashing into the Florida
Everglades. The third plane, on an international
flight, had results so catastrophic that it sent
shock waves throughout the world. This plane
collided with another aircraft, immediately end-
ing the lives of everyone aboard both planes.
After 20 years, the aviation industry is still
challenged by a haunting question: Why is the
number of take-offs not equal to the number
of safe landings? In the past 20 years, it has
been commonly acknowledged that almost
60% to 80% of aviation incidents and acci-
dents were attributable to human error in the
cockpit (Foushee, 1984). This recognition led
a number of applied psychologists to suggest
an intervention aimed at improving human
A Methodology for Enhancing Crew Resource
Management Training
Eduardo Salas and Carolyn Prince, Naval Air Warfare Center Training Systems Division,
Orlando, Florida, Clint A. Bowers, University of Central Florida, Orlando, Florida, and
Renée J. Stout, Randall L. Oser, and Janis A. Cannon-Bowers, Naval Air Warfare
Center Training Systems Division, Orlando, Florida
Human error is an ever-present threat to the safe conduct of flight. Recently,
applied psychologists have developed an intervention, crew resource management
(CRM) training, designed to help prevent human error in the cockpit. However,
as it is commonly applied within the aviation community, CRM lacks standardiza-
tion in content, design, delivery, and evaluation. This paper presents a discussion
of an applied program of research aimed at developing a methodology for the
design and delivery of CRM training within the Navy. This long-term, theoretical-
ly based program of aviation team research included identification of skills to be
trained, development of performance measures, application of instructional
design principles, and evaluation of the training delivery. Our conclusion indi-
cates that a systematic methodology for developing CRM training can result in
better performance in the cockpit. Actual or potential applications of this
research include any task environment in which teams are interdependent.
Address correspondence to Eduardo Salas, Code 4961, Naval Air Warfare Center Training Systems Division, 12350
Research Parkway, Orlando, FL 32826-3275; [email protected]. HUMAN FACTORS, Vol. 41, No. 1, March 1999,
pp. 161–172.
162 March 1999 – Human Factors
performance and, in particular, teamwork in
the cockpit. This intervention, commonly re-
ferred to as crew resource management (CRM)
training, now has a long history of research
and practice in the air carrier industry (Wiener,
Kanki, & Helmreich, 1993).
On the military side, CRM training devel-
opments have also emerged. (The military
labeled this team training intervention aircrew
coordination training, but we will use the
term CRM in this paper because it is most
common in the airline industry and govern-
ment regulatory agencies.) The U.S. Navy (in
particular, the Marine Corps) enlisted the help
of the Naval Air Warfare Center Training
Systems Division about 10 years ago in im-
proving the safety of its rotary wing fleet. Our
response was to design and conduct a long-
term program of research that began with the-
ory building and moved into development of
measures of performance, design of instruc-
tion, empirical testing, and evaluation. The
purpose of this paper is to describe our efforts
in this regard. To do this, we organized the
presentation around the critical questions that
have guided our research:
1. What is CRM, and, more specifically, what is
CRM training?
2. Which theories provide a basis to develop CRM
training?
3. Which skills underlie effective CRM?
4. Which instructional approaches and strategies
are appropriate to impart CRM skills?
5. What evidence exists to support the effective-
ness of CRM training?
We conclude with a presentation of our
methodology for developing CRM training
and a word about reciprocity between training
science and practice.
What Is CRM and CRM Training?
The three accidents discussed in the previ-
ous section help illustrate a persistent threat to
safe aviation: human error caused by inade-
quate coordination among team members.
Although there were minor mechanical failures
in two of the accidents, each occurred because
of the crew’s error. It is clear that the errors
made in these accidents were not the result of
inadequate technical training. All of those
involved knew how to read the gauges and
how to conduct emergency procedures. It was
also clear that on its own, human redundancy
in the cockpit had not worked to make the sys-
tem more secure; three people together in a
single cockpit still overlooked a basic flight
parameter. This led researchers to expand their
view of what was required for effective avia-
tion beyond technical aspects of the task (i.e.,
flying the aircraft) to include nontraditional
competencies such as teamwork.
According to Lauber (1984), this new way
of thinking (labeled first as cockpit resource
management and later as crew resource man-
agement) is defined as “using all available
resources – information, equipment, and peo-
ple – to achieve safe and efficient flight opera-
tions” (p. 20). Foushee and Helmreich (1988)
added that
CRM includes optimizing not only the person-
machine interface and the acquisition of
timely, appropriate information, but also
interpersonal activities including leadership,
effective team formation and maintenance,
problem-solving, decision-making, and main-
taining situation awareness. (p. 4)
Given this definition, researchers began to
examine what training for CRM might include.
According to Foushee and Helmreich (1988,
p. 4), such training involves “communicating
basic knowledge of human factors concepts
that relate to aviation and providing the tools
necessary to apply these concepts operationally.”
At the time, CRM represented “a new focus on
crew-level (as opposed to individual-level)
aspects of training and operations” (p. 4). The
major goal of the resulting training was to “help
stem the tide of accidents caused by so-called
human error” (Stone & Babcock, 1988, p. 553)
by addressing “situational, sociopsychological
and other factors that influence aircrew perfor-
mance” (Caro, 1988, p. 258).
When it came to actually developing training
to address CRM issues, early attempts held
almost exclusively that CRM training should tar-
get crew members’ attitudes toward teamwork
(Chidester & Foushee, 1988; Helmreich &
Wilhelm, 1991). In keeping with the definition
of CRM, this approach to CRM training empha-
sized the social-psychological factors that influ-
ence crew performance. In fact, this work by
Foushee, Helmreich, and others paved the way
CRM TRAINING 163
for the aviation industry to consider “softer”
human performance and teamwork issues to be
legitimate concerns. This was no small accom-
plishment. The success of these scientists was to
shift the emphasis in aviation training onto
social interactions in the cockpit (i.e., crew
coordination). However, early CRM training
practices did not go far enough; we believe that
there were two fundamental problems.
First, initial CRM training overemphasized
the affective, personality, and attitudinal aspects
of crew coordination (i.e., the “right stuff”) at
the expense of the behavioral aspects of the
problem of crew coordination. Second, little
guidance was available regarding how to train
crew coordination skills. So, although these
early efforts had the effect of enlightening the
aviation community, they did not address the
CRM training problem sufficiently.
In the 10 years or so that have followed, the
lack of a standardized methodology for devel-
oping CRM training (and the associated prob-
lem of disagreement about exactly what needed
to be trained) caused a host of diverse CRM
training programs to be developed. Whereas
early programs emphasized attitudes toward
teamwork, as described earlier, others were
based on personality or skill, and still others on
a combination of these (Byrnes & Black, 1993;
Helmreich & Foushee, 1993; Yamamori &
Mito, 1993). Furthermore, programs have var-
ied in length from 1 h to 2 weeks. Training has
been given in lectures, discussions, videotape
observations, game-playing, classroom role-
play, mishap analyses, and both low- and high-
fidelity simulations. Some courses have used
only one technique (e.g., lecture) and others
have used several. Content has shown diversity
as well and has included topics such as interac-
tion styles, stress reduction, and automation
issues, in addition to the subjects of workload
management, advocacy, and situation aware-
ness that are included in many programs. Some
programs are clearly based on attitude change
or skill development, but others, for which
training has been composed of elements copied
from the programs of other organizations, have
no discernible basis.
In sum, the lack of agreement regarding
what CRM training should include and how it
should be accomplished has led to confusion
and, in many cases, adoption of atheoretical
and suboptimal programs. To address these
deficiencies, our work for the U.S. Navy adopt-
ed a more systematic approach to developing
CRM training that was consistent with our
team training work in similar domains.
As a starting point, we clarified the concept
of CRM by extracting the notion of teamwork
(or crew coordination) from the definitions
offered earlier as the essential ingredient of
CRM. We selected this focus because the evi-
dence – gleaned mostly from accident reports –
suggested that it was the lack of coordination
of team member resources or the underutiliza-
tion of team member resources (or both) that
accounted for most of the errors (Foushee,
1984). Hence, we define CRM as being a set
of teamwork competencies that allow the crew
to cope with situational demands that would
overwhelm any individual crew member (we
elaborate on these competencies in a later sec-
tion). Flowing from this definition, we consid-
er CRM training to be a family of instructional
strategies designed to improve teamwork in
the cockpit by applying well-tested training
tools (e.g., performance measures, exercises,
feedback mechanisms) and appropriate train-
ing methods (e.g., simulators, lectures, videos)
targeted at specific content (i.e., teamwork
knowledge, skills, and attitudes).
It should be noted that our initial emphasis
in CRM training was on skills (in part because
other programs emphasized attitudes). More
recently, our work has been focused more
directly on the knowledge underlying effective
teamwork (see, e.g., Stout, Salas, & Kraiger,
1997), but this work is not covered here. In the
following sections, we describe the program of
research that we conducted to enhance the
training of CRM skills and the resulting method-
ology that we developed. As noted, we do this
by addressing a number of crucial questions
that guided our work.
Which Theories Provide a Basis for CRM
Training Development?
The problem of improving teamwork in a
task as complex as aviation provides a formi-
dable challenge. Clearly, no single approach
is likely to address the entire problem. More-
over, without a strong theoretical foundation –
164 March 1999 – Human Factors
one that rests on an understanding of individ-
ual and team performance, human learning
and skill acquisition, and pedagogy – it is not
likely that effective training will be developed
or fielded. In fact, we submit as others have
that there is nothing more practical than a
good theory (Lewin, as cited in Marrow, 1969).
This statement embraces the philosophy
behind the design of all of the team training
strategies that we have developed and tested
(Salas & Cannon-Bowers, 1997).
Fortunately, there are a number of theoreti-
cal perspectives that can provide a strong
foundation for CRM training development.
For example, Tannenbaum, Beard, and Salas
(1992) described a theoretical model of team
performance that includes consideration of a
variety of factors that might influence the per-
formance of flight crews. This model has
already been used as a foundation for research
in CRM for automated cockpits (see Bowers,
Thornton, Braun, & Salas, 1998).
Recent theoretical advances in cognitive
psychology also offer promise in guiding what
to train and how to train it. For example,
Jentsch (1997) developed a metacognitive
training strategy for junior first officers aimed
at improving their ability to determine the
appropriate time to act. The training focused
on teaching the pilots monitoring skills and
improving their ability to mentally simulate
possible outcomes of either action or inaction
in a particular situation. Pilots who had
received the training performed significantly
better in those situations that required judg-
ment (whether to act or not) than those who
did not receive the training. Similarly, using
shared mental model theory (Cannon-Bowers,
Tannenbaum, Salas, & Volpe, 1995) as a
foundation, Stout, Salas, and Fowlkes (1997)
found that participants who received CRM-
type training improved their knowledge struc-
tures related to teamwork as compared with a
control group that did not receive training. In
addition, the trained group demonstrated 8%
more teamwork skills when it was appropriate
to do so during a simulated mission. Thus,
training positively influenced aviators’ knowl-
edge structures and performance.
Although our work in CRM training has
been influenced in one way or another by all
of the theories just described, we based our
notions about training teamwork in the cock-
pit largely on the body of work conducted in
the 1980s by Salas, Morgan, Glickman, and
colleagues (see, e.g., Morgan, Salas, & Glick-
man, 1994). Briefly, these researchers studied
military command and control teams and
found behaviors that were consistent across
effective teams. Specifically, they found that
using closed-loop communication, predicting
each other’s behavior, performing self-correction,
and providing motivational and task reinforce-
ment led to better team performance (for
details, see McIntyre & Salas, 1995).
Which Skills Underlie Effective CRM?
We often found that aviators who attended
early CRM training commented that the course
was interesting and that they learned a lot
about themselves, but they did not know what
to do to change the way they operated when
they got back in the cockpit. As Ryle (1949)
pointed out, there is a difference between
knowing that something should be done and
knowing how to do it. Therefore, based on our
notions about teamwork, we adopted a behav-
ioral perspective to complement the attitude
and personality-based work that was ongoing
in the commercial sector in the early 1990s.
This meant that from a learning standpoint, we
needed to identify the requisite skills that
enable effective performance in naval aviation.
This training was designed to provide aviators
with behaviors that they could take back to the
cockpit to improve their performance. Thus,
our vision for CRM training emphasizes what
to do rather than how to feel.
In order to fulfill this vision, we synthesized
the scientific literature (Salas, Dickinson,
Converse, & Tannenbaum, 1992), made multi-
ple observations of crews from several commu-
nities performing in full-mission simulations
(Prince & Salas, 1993), and conducted struc-
tured interviews with over 200 aviators from a
variety of operational communities. Through
an integration of information gained from
these three divergent sources, we identified a
set of teamwork constructs with associated
behaviors that we predicted would lead to
effective teamwork in the cockpit (Prince &
Salas, 1993). Along the way, we developed and
CRM TRAINING 165
tested a variety of approaches to identifying fleet-
specific coordination behaviors (see Bowers,
Baker, & Salas, 1994; Bowers, Morgan, Salas,
& Prince, 1993). These tools exist and can be
used to develop skills-based training today
(Salas & Cannon-Bowers, 1997; Salas & Cannon-
Bowers, in press).
In sum, by integrating theoretical models of
teamwork and human learning with training
needs-analysis tools, we addressed the first de-
ficiency of CRM training noted previously by
identifying the specific skills and associated
behaviors that should improve coordination in
the cockpit. The emphasis on specific skills
offers several advantages from the theoretical
and evaluation perspective (described in a
later section). However, training success is also
dependent upon one’s ability to train those
skills, which brings us back to the issue of
how to train CRM skills.
How Can CRM Skills Best Be Trained?
Pertinent to the question of how best to
train teamwork in the cockpit, we contend
that the failure to create opportunities to prac-
tice newly acquired skills was the greatest
shortcoming of early CRM training. In essence,
early approaches failed to take advantage of
literature that relates to human learning and
skill acquisition (Kolb, 1984). Consequent-
ly, our efforts have endeavored to improve
upon the traditional simulator-based practice
experience, which typically lacked specific
measurement and feedback, to more directly
reinforce the targeted teamwork skills. For
example, a program of research on expanded
opportunities for teamwork skill practice
(and feedback) has shown that CRM skills
trained in one situation transfer to another
(Brannick, Prince, Salas, & Stout, 1997; Prince,
Brannick, Prince, & Salas, 1997). In addition,
we have shown that this method of instruction
provides enhanced training transfer as com-
pared with game-playing, a technique current-
ly used in many CRM programs (Brannick,
Prince, et al., 1997; see Jentsch & Bowers,
1998 for a thorough discussion). Once our
instruction was developed, it was necessary
to assess the effectiveness of the training.
The methods we employed to do this are
described next.
What Evidence Exists to Suggest That
CRM Training Is Effective?
As with any other complex human perfor-
mance problem, the issue of establishing a cri-
terion against which to evaluate the success of
CRM training is difficult. At first glance, it is
tempting to use broad performance indices to
evaluate CRM effectiveness. Common exam-
ples include the use of accident rates or the
number of mishaps attributed to CRM errors.
However, there are several concerns regarding
these types of indices: (a) CRM has become
part of aviation vernacular (perhaps accident
investigators are citing it as a cause more fre-
quently); (b) aircraft are becoming increasing-
ly automated, which places considerable
additional coordination demands on crews
(Bowers, Deaton, Oser, Prince, & Kolb, 1995;
Thornton, Braun, Bowers, & Morgan, 1992);
and (c) accidents that happened are a poor
index of accidents that did not happen. There
are numerous anecdotal accounts of accidents
that were prevented with the use of CRM
skills, but do the accidents that did happen
negate these reported benefits?
A better approach is one that is multifac-
eted and considers several levels of evaluation,
including trainee reactions, extent of learning,
extent of performance change, and impact on
organizational effectiveness (Cannon-Bowers
et al., 1989; Ford, Kozlowski, Kraiger, Salas, &
Teachout, 1997). This type of approach
includes a variety of data points for considera-
tion, such as trainee evaluation, specific indices
of learning, evidence of application, and so
forth. A multifaceted approach is not only
more thorough than an approach focusing only
on reactions or attitudes, but it also provides
an improved ability to assess which aspects of
training work and which do not work. Hence,
it is also useful in identifying specific areas in
need of remediation, because it assesses multi-
ple aspects of learning/skill-acquisition.
Using a multifaceted approach, we conducted
formal evaluations of our CRM training interven-
tions in four aircraft communities with a total of
55 crews. In each of the evaluations we used a
control group as a comparison so that we could
actually assess the value of our training. We also
used raters or assessors of performance who were
166 March 1999 – Human Factors
blind to conditions. Finally, we used a measure
of performance that was designed to identify
whether aviators were demonstrating behaviors
(identified through task analysis and assessment
of training manuals and operating procedures)
that had been delineated as more effective by
fleet subject-matter experts (as described in
Fowlkes, Lane, Salas, Franz, & Oser, 1994).
We obtained evidence that our variety of
CRM training is effective, as evidenced by
average performance improvements of 8% to
20% (for empirical evidence see Salas, Fowlkes,
Stout, Milanovich, & Prince, in press; Smith-
Jentsch, Salas, & Baker, 1996; Stout, Salas, &
Fowlkes, 1997). That is, on average we ob-
served 8% to 20% more teamwork behaviors
exhibited in the cockpit by crews that were
trained than by crews that were not trained.
This may not seem like much, but it is actually
impressive considering that many of the par-
ticipants in these studies were experienced
crews who actually changed their cockpit per-
formance based on a relatively short training
intervention. Furthermore, we submit that
performing even one more teamwork behavior
(when required) can make a difference be-
tween mission success and failure (and some-
times between life and death). Similar efforts
in other military communities have also
shown positive results (Cannon-Bowers &
Salas, 1998; Leedom & Simon, 1995; Salas,
Cannon-Bowers, & Johnston, 1997).
In addition, we have conducted studies to
test specific components of the training. For
example, Smith-Jentsch et al. (1996), using a
variety of measurement approaches, demon-
strated the effectiveness (74% performance
improvement for the trained group) of skills-
based assertiveness training, which is a com-
ponent of our CRM program. Jentsch (1997)
also used a multidimensional approach in
assessing the effectiveness of metacognitive
training for junior first officers.
Thus, the data are encouraging. We have
shown that even experienced aviators can
learn new ways to behave in the cockpit and
improve their performance. In the future, we
hope that others will also attempt to evaluate
the CRM training they develop; in fact, this
should be an ongoing process. Given the diffi-
culties noted earlier, it is the responsibility
of all of the experts (researchers and practi-
tioners), the airlines, the fleets, and cognizant
agencies to devise acceptable methods for
evaluating CRM training effectiveness and
then to make the necessary commitment to,
and investment in, conducting these studies.
A METHODOLOGY FOR DEVELOPING
CRM TRAINING
Table 1 shows an overview of the methodol-
ogy that we employed for developing effective
CRM training. It is based on the work we con-
ducted to develop CRM training as well as
guidelines from the literature (Cannon-Bowers
& Salas, 1998; Salas & Cannon-Bowers, in
press; Salas, Cannon-Bowers, & Blickensderfer,
1997; Swezey & Salas, 1992). It is similar to
the event-based approach to training that we
have applied in other domains (Dwyer,
Fowlkes, Oser, Salas, & Lane, 1997; Johnston,
Cannon-Bowers, & Smith-Jentsch, 1995; Oser,
Dwyer, & Fowlkes, 1995). For any particular
community that we worked with, we followed
the steps illustrated in Table 1 to develop theo-
retically rooted, behaviorally based, community-
specific CRM training (for examples, see Prince
& Salas, 1989; Stout, Salas, & Fowlkes, 1997).
The process delineated in Table 1 began
with a thorough analysis of the aircraft mis-
sion and procedures and their impact on crew
coordination demands. The second step was
to assess the coordination demands directly
(for details, see Bowers et al., 1993). This
analysis allowed us to identify specific tasks
that had a team component so that we could
focus attention on these tasks in training.
Next, we employed our theoretical notions
regarding the nature of teamwork competen-
cies to arrive at a set of coordination skills to
be trained. Each skill was then translated into
a training objective and used as the basis to
develop scenarios or exercises that would
allow the skill to be practiced. At the same
time, we developed measures of performance
and associated measurement tools that were
linked to each of the training objectives
(Baker, Prince, Shrestha, Oser, & Salas, 1993;
Dwyer et al., 1997; Prince, Oser, Salas, &
CRM TRAINING 167
Woodruff, 1993). Our goal was to focus on
observable aspects of teamwork behavior so
that raters could readily assess whether the be-
havior was demonstrated sufficiently (Fowlkes
et al., 1994).
We then used the performance measure-
ment data as a basis for developing feedback.
Because feedback mechanisms were crucial to
the training, we spent considerable effort tai-
loring our assessment tools so that they could
form the basis of effective feedback (Fowlkes
et al., 1994). Finally, whenever possible, we
conducted training-effectiveness evaluations
to determine the impact of our training on
cockpit performance.
Description of the Training
Employing the methodology just described,
we developed an instructional strategy for
CRM training with behavioral modeling as its
basis (Salas et al., in press; Stout, Salas, &
Fowlkes, 1997). The training sessions followed
an information-demonstration-practice-feed-
back sequence. The training began with a lec-
ture (delivered live) explaining important
teamwork skills. Next, videotaped models of
effective and ineffective behavior were shown
to the trainees. Following this demonstration
phase, trainees were provided with the oppor-
tunity to practice on carefully crafted scenarios
(as described earlier), and their performance
was tracked. Finally, we provided feedback that
was relevant to the behaviors that we sought to
train.
Using this general strategy, we were able to
train the repertoire behaviors that allowed crew
members to exhibit effective teamwork in the
cockpit, as evidenced by the effectiveness data
described earlier. It should be noted that the
methodology presented here should also be use-
ful in developing other types of CRM training
strategies. In other words, there may be other
ways to train various aspects of teamwork in
the cockpit; our argument is that it is the sys-
tematic process for devising it that helps ensure
success. Further, we are aware that many CRM
training programs have been developed that are
consistent with one or even several of the steps
shown in Table 1. Again, we argue that in order
to develop and evaluate effective CRM training,
all of the activities in Table 1 must be complet-
ed. Otherwise, there is a risk of suboptimizing;
that is, there is a risk of developing training
that addresses only parts of the problem or
lacks a sound theoretical foundation. We also
submit that this methodology may be useful in
guiding the development of team training for
domains other than aviation.
RECIPROCITY BETWEEN TRAINING
RESEARCH AND PRACTICE
Given what we have presented thus far, it is
clear that we have made progress in under-
standing CRM and applying sound training
principles to improve it. However, as with other
areas of training, the results of training research
often do not influence training practice as
much as they could. We suggest that there is
still difficulty infiltrating operational communi-
ties with the results of team performance and
training efforts. This problem is certainly not
limited to CRM training; we have maintained
that the application of research results to prac-
tice is a problem confronted by many areas of
applied psychology (Cannon-Bowers, Tannen-
baum, Salas, & Converse, 1991; Salas, Cannon-
Bowers, & Blickensderfer, 1997).
Too often we find that training developers
fail to make use of the results of training
research. There may be many reasons for this.
In the past, we have contended that it is attrib-
utable to a lack of translation mechanisms. That
is, researchers and practitioners lack a com-
mon forum and language for communicating.
Moreover, given that our work is conducted in
a military context, it is sometimes dismissed as
not generalizing to many commercial tasks,
even though it could offer a substantial reduc-
tion in costs incurred in “re-inventing the
wheel.” Furthermore, by taking advantage of
the research done with military participants –
who often become commercial pilots after
their military careers – it is likely that the state
of the art will improve more quickly than it
has in the past decade.
CONCLUDING REMARKS
In conclusion, the science and practice of
CRM training has evolved considerably over the
past 10 years. There are now viable (and tested)
168
T
A
B
L
E

1
:

A

M
e
t
h
o
d
o
l
o
g
y

t
o

D
e
s
i
g
n

a
n
d

D
e
l
i
v
e
r

C
R
M

T
r
a
i
n
i
n
g
S
t
e
p
D
e
s
c
r
i
p
t
i
o
n
P
r
o
d
u
c
t
s
S
o
u
r
c
e
s
1
.

I
d
e
n
t
i
f
y

o
p
e
r
a
t
i
o
n
a
l
/
m
i
s
s
i
o
n
R
e
v
i
e
w

e
x
i
s
t
i
n
g

t
r
a
i
n
i
n
g

c
u
r
r
i
c
u
l
u
m
,

i
n
c
l
u
d
i
n
g

c
o
u
r
s
e
-
M
i
s
s
i
o
n

s
p
e
c
i
f
i
c
F
o
w
l
k
e
s
,

L
a
n
e
,
r
e
q
u
i
r
e
m
e
n
t
m
a
s
t
e
r

m
a
t
e
r
i
a
l

l
i
s
t
s
,

i
n
s
t
r
u
c
t
o
r

g
u
i
d
e
s
,

s
t
a
n
d
a
r
d

o
p
e
r
a
t
i
n
g
c
o
n
t
e
x
t
/
e
x
a
m
p
l
e
s
S
a
l
a
s
,

F
r
a
n
z
,

&
p
r
o
c
e
d
u
r
e
s

(
S
O
P
s
)
;

i
n
t
e
r
v
i
e
w

a
v
i
a
t
i
o
n

s
u
b
j
e
c
t

m
a
t
t
e
r

e
x
p
e
r
t
s
-
G
e
n
e
r
a
l

u
n
d
e
r
s
t
a
n
d
i
n
g

o
f

O
s
e
r

(
1
9
9
4
)
;

(
S
M
E
s
)
;

o
b
s
e
r
v
e

c
r
e
w
s

p
e
r
f
o
r
m
i
n
g

m
i
s
s
i
o
n
s
;

r
e
v
i
e
w

r
e
l
e
v
a
n
t
c
o
o
r
d
i
n
a
t
i
o
n

d
e
m
a
n
d
s

w
i
t
h
i
n
H
a
r
t
e
l
,

S
m
i
t
h
,

&
m
i
s
h
a
p
/
a
c
c
i
d
e
n
t

r
e
p
o
r
t
s
.
t
h
e

t
a
s
k
P
r
i
n
c
e

(
1
9
9
1
)
;
P
r
i
n
c
e

&

S
a
l
a
s
(
1
9
8
9
)
2
.

A
s
s
e
s
s

t
e
a
m

t
r
a
i
n
i
n
g

n
e
e
d
s

a
n
d
U
s
e

s
a
m
e

d
a
t
a

s
o
u
r
c
e
s

a
s

i
n

S
t
e
p

1
,

w
i
t
h

e
m
p
h
a
s
i
s

o
n
-
C
o
o
r
d
i
n
a
t
i
o
n

d
e
m
a
n
d

a
n
a
l
y
s
e
s
B
o
w
e
r
s
,

M
o
r
g
a
n
,
c
o
o
r
d
i
n
a
t
i
o
n

d
e
m
a
n
d
i
d
e
n
t
i
f
y
i
n
g

d
e
f
i
c
i
e
n
c
i
e
s

i
n

e
x
i
s
t
i
n
g

t
e
a
m

t
r
a
i
n
i
n
g

a
n
d

-
L
i
s
t

o
f

t
a
s
k
s

r
e
q
u
i
r
i
n
g
&

S
a
l
a
s

(
1
9
9
1
)
;
s
p
e
c
i
f
y
i
n
g

a
l
l

t
a
s
k
s

r
e
q
u
i
r
e
d

t
o

p
e
r
f
o
r
m

m
i
s
s
i
o
n
s

t
h
a
t
c
o
o
r
d
i
n
a
t
i
o
n
B
o
w
e
r
s
,

M
o
r
g
a
n
,
i
n
v
o
l
v
e

a

t
e
a
m
w
o
r
k

e
l
e
m
e
n
t
.
S
a
l
a
s
,

&

P
r
i
n
c
e
(
1
9
9
3
)
3
.

I
d
e
n
t
i
f
y

t
e
a
m
w
o
r
k

c
o
m
p
e
t
e
n
c
i
e
s
L
i
n
k

t
e
a
m

t
r
a
i
n
i
n
g

n
e
e
d
s

t
o

a

t
h
e
o
r
y

o
f

t
e
a
m

p
e
r
f
o
r
m
a
n
c
e
-
S
e
t

o
f

k
n
o
w
l
e
d
g
e
,

s
k
i
l
l
s
,

a
n
d
C
a
n
n
o
n
-
B
o
w
e
r
s
a
n
d

k
n
o
w
l
e
d
g
e
,

s
k
i
l
l
s
,

a
n
d

t
h
a
t

a
l
l
o
w
s

d
e
l
i
n
e
a
t
i
o
n

o
f

c
o
m
p
e
t
e
n
c
i
e
s

(
o
u
r

e
m
p
h
a
s
i
s
a
t
t
i
t
u
d
e
s
&

S
a
l
a
s

(
1
9
9
7
)
;
a
t
t
i
t
u
d
e
s

(
K
S
A
s
)
i
n
i
t
i
a
l
l
y

w
a
s

o
n

s
k
i
l
l
s
)

r
e
q
u
i
r
e
d

t
o

p
e
r
f
o
r
m

e
a
c
h

o
f

t
h
e

t
e
a
m
C
a
n
n
o
n
-
B
o
w
e
r
s
,
t
a
s
k
s

i
d
e
n
t
i
f
i
e
d

i
n

S
t
e
p

2
.
T
a
n
n
e
n
b
a
u
m
,
S
a
l
a
s
,

&

V
o
l
p
e
(
1
9
9
5
)
;

P
r
i
n
c
e

&
S
a
l
a
s

(
1
9
9
3
)
;
S
t
o
u
t
,

S
a
l
a
s
,

&

F
o
w
l
k
e
s

(
1
9
9
7
)
4
.

D
e
t
e
r
m
i
n
e

t
e
a
m

t
r
a
i
n
i
n
g

F
o
r

e
a
c
h

t
e
a
m
w
o
r
k

K
S
A
,

d
e
v
e
l
o
p

a

t
r
a
i
n
i
n
g

o
b
j
e
c
t
i
v
e

t
h
a
t
-
L
i
s
t

o
f

t
a
r
g
e
t
s

f
o
r

t
r
a
i
n
i
n
g
S
a
l
a
s

&

C
a
n
n
o
n
-
o
b
j
e
c
t
i
v
e
s
c
a
n

b
e

e
m
p
i
r
i
c
a
l
l
y

e
v
a
l
u
a
t
e
d

t
o

d
e
t
e
r
m
i
n
e

w
h
e
t
h
e
r

o
r

n
o
t

i
t
-
F
u
l
l

l
i
s
t

o
f

g
e
n
e
r
i
c

a
n
d

t
a
s
k
-
B
o
w
e
r
s

(
1
9
9
7
)
;
w
a
s

a
c
c
o
m
p
l
i
s
h
e
d
.
s
p
e
c
i
f
i
c

t
r
a
i
n
i
n
g

o
b
j
e
c
t
i
v
e
s
S
t
o
u
t
,

S
a
l
a
s
,

&
K
r
a
i
g
e
r

(
1
9
9
7
)
;
S
w
e
z
e
y
,

L
l
a
n
e
r
a
s
,
P
r
i
n
c
e
,

&

S
a
l
a
s
(
1
9
9
1
)
169
5
.

D
e
t
e
r
m
i
n
e

i
n
s
t
r
u
c
t
i
o
n
a
l

d
e
l
i
v
e
r
y
T
h
e

m
e
t
h
o
d

f
o
r

a
c
c
o
m
p
l
i
s
h
i
n
g

t
h
e

i
n
s
t
r
u
c
t
i
o
n

s
h
o
u
l
d

b
e

-
L
e
c
t
u
r
e
s
B
a
k
e
r
,

P
r
i
n
c
e
,

m
e
t
h
o
d
s
p
e
c
i
f
i
e
d

(
e
.
g
.
,

i
n
f
o
r
m
a
t
i
o
n
,

d
e
m
o
n
s
t
r
a
t
i
o
n
,

o
r

p
r
a
c
t
i
c
e

a
n
d
-
V
i
d
e
o
s
S
h
r
e
s
t
h
a
,

O
s
e
r
,
f
e
e
d
b
a
c
k
,

o
r

a
l
l
)

i
n

t
h
i
s

s
t
e
p
.

C
o
n
s
i
d
e
r
a
t
i
o
n

s
h
o
u
l
d

b
e

g
i
v
e
n
-
R
o
l
e
-
p
l
a
y

e
x
e
r
c
i
s
e
s
&

S
a
l
a
s

(
1
9
9
3
)
;
t
o

c
o
s
t
s

a
n
d

a
v
a
i
l
a
b
i
l
i
t
y

o
f

s
i
m
u
l
a
t
o
r
s
.
-
P
C
-
b
a
s
e
d

m
e
t
h
o
d
o
l
o
g
i
e
s
B
e
a
r
d
,

S
a
l
a
s
,

&
-
H
i
g
h
-
f
i
d
e
l
i
t
y

s
i
m
u
l
a
t
o
r
s
P
r
i
n
c
e

(
1
9
9
5
)
-
A
c
c
i
d
e
n
t

r
e
v
i
e
w
s

a
s

a
n
a
l
y
s
i
s
-
T
r
a
i
n
i
n
g

c
u
r
r
i
c
u
l
u
m
6
.

D
e
s
i
g
n

s
c
e
n
a
r
i
o

e
x
e
r
c
i
s
e
s

a
n
d

D
e
s
i
g
n

s
c
e
n
a
r
i
o
s

o
r

e
x
e
r
c
i
s
e
s

i
n

w
h
i
c
h

e
v
e
n
t
s

a
r
e

-
V
a
l
i
d

r
e
a
l
i
s
t
i
c

s
c
e
n
a
r
i
o
(
s
)
,
D
w
y
e
r
,

F
o
w
l
k
e
s
,
c
r
e
a
t
e

o
p
p
o
r
t
u
n
i
t
i
e
s

f
o
r

p
r
a
c
t
i
c
e
e
m
b
e
d
d
e
d

t
o

p
r
o
v
i
d
e

t
r
a
i
n
e
e
s

a
n

o
p
p
o
r
t
u
n
i
t
y

t
o

i
n
c
l
u
d
i
n
g

a
l
l

r
e
l
e
v
a
n
t

p
e
r
i
p
h
e
r
a
l
O
s
e
r
,

S
a
l
a
s
,

&
d
e
m
o
n
s
t
r
a
t
e


e
a
c
h

o
f

t
h
e

r
e
q
u
i
r
e
d

K
S
A
s

i
d
e
n
t
i
f
i
e
d

i
n

s
u
p
p
o
r
t
L
a
n
e

(
1
9
9
7
)
;

o
b
j
e
c
t
i
v
e
s

i
n

w
h
i
c
h

a
c
c
o
m
p
l
i
s
h
m
e
n
t

r
e
q
u
i
r
e
s

p
r
a
c
t
i
c
e

F
o
w
l
k
e
s
,

D
w
y
e
r
,

a
n
d

f
e
e
d
b
a
c
k
.
O
s
e
r
,

&

S
a
l
a
s
(
1
9
9
8
)
;

P
r
i
n
c
e
,
O
s
e
r
,

S
a
l
a
s
,

&
W
o
o
d
r
u
f
f

(
1
9
9
3
)
7
.

D
e
v
e
l
o
p

p
e
r
f
o
r
m
a
n
c
e

I
n

c
o
n
j
u
n
c
t
i
o
n

w
i
t
h

s
c
e
n
a
r
i
o

d
e
s
i
g
n
,

d
e
v
e
l
o
p

m
e
a
s
u
r
e
s

t
h
a
t
-
B
e
h
a
v
i
o
r
a
l
l
y
-
b
a
s
e
d

c
h
e
c
k
l
i
s
t
s
B
a
k
e
r

&

S
a
l
a
s
a
s
s
e
s
s
m
e
n
t
/
m
e
a
s
u
r
e
m
e
n
t

t
o
o
l
s
c
a
n

r
e
l
i
a
b
l
y

a
s
s
e
s
s

w
h
e
t
h
e
r

e
a
c
h

o
f

t
h
e

K
S
A
s

w
a
s
-
S
u
b
j
e
c
t
i
v
e

e
v
a
l
u
a
t
i
o
n

f
o
r
m
s

(
1
9
9
2
)
;

B
r
a
n
n
i
c
k
,
d
e
m
o
n
s
t
r
a
t
e
d

a
t

a
n

o
b
s
e
r
v
a
b
l
e

b
e
h
a
v
i
o
r
a
l

l
e
v
e
l
.
-
O
u
t
c
o
m
e

m
e
t
r
i
c
s

a
n
d

c
r
i
t
e
r
i
a
S
a
l
a
s
,

&

P
r
i
n
c
e
(
1
9
9
7
)
;

F
o
w
l
k
e
s
e
t

a
l
.

(
1
9
9
4
)
8
.

D
e
s
i
g
n

a
n
d

t
a
i
l
o
r

t
o
o
l
s

f
o
r

D
e
s
i
g
n

o
r

t
a
i
l
o
r

(
o
r

b
o
t
h
)

m
e
a
s
u
r
e
m
e
n
t

t
o
o
l
s

f
o
r

u
s
e

i
n

-
I
n
s
t
r
u
c
t
o
r

t
r
a
i
n
i
n
g
S
a
l
a
s

&

C
a
n
n
o
n
-
f
e
e
d
b
a
c
k
d
e
b
r
i
e
f
,

i
n

w
h
i
c
h

t
r
a
i
n
e
e
s

a
r
e

m
a
d
e

a
w
a
r
e

o
f

t
h
o
s
e

-
D
e
b
r
i
e
f
i
n
g

c
h
e
c
k
l
i
s
t
s

a
n
d
B
o
w
e
r
s

(
1
9
9
7
)
;
r
e
q
u
i
r
e
d

t
e
a
m

b
e
h
a
v
i
o
r
s

t
h
a
t

t
h
e
y

d
i
d

a
n
d

t
h
o
s
e

t
h
a
t

t
h
e
y

g
u
i
d
e
s
S
m
i
t
h
-
J
e
n
t
s
c
h
,
d
i
d

n
o
t

p
e
r
f
o
r
m

s
u
c
c
e
s
s
f
u
l
l
y
.

T
h
i
s

t
o
o
l

s
h
o
u
l
d

a
l
s
o

h
e
l
p

Z
e
i
s
i
g
,

A
c
t
o
n
,

&

i
n
s
t
r
u
c
t
o
r
s

d
i
a
g
n
o
s
e

t
h
e

c
a
u
s
e
s

o
f

p
o
o
r

p
e
r
f
o
r
m
a
n
c
e

a
n
d
M
c
P
h
e
r
s
o
n
p
r
o
v
i
d
e

g
u
i
d
a
n
c
e

f
o
r

s
p
e
c
i
f
i
c

i
m
p
r
o
v
e
m
e
n
t

i
n

f
u
t
u
r
e
(
1
9
9
8
)
o
p
e
r
a
t
i
o
n
s
.
9
.

E
v
a
l
u
a
t
e

t
h
e

e
x
t
e
n
t

o
f

i
m
p
r
o
v
e
d
D
e
s
i
g
n

e
x
p
e
r
i
m
e
n
t
s

t
o

a
s
s
e
s
s

t
h
e

e
f
f
e
c
t
i
v
e
n
e
s
s

o
f

t
h
e
-
R
e
a
c
t
i
o
n

d
a
t
a
F
o
w
l
k
e
s

e
t

a
l
.
t
e
a
m
w
o
r
k

i
n

t
h
e

c
o
c
k
p
i
t
t
r
a
i
n
i
n
g
.

B
e
c
a
u
s
e

o
f

o
p
e
r
a
t
i
o
n
a
l

c
o
n
s
t
r
a
i
n
t
s
,

q
u
a
s
i
-
-
L
e
a
r
n
i
n
g

d
a
t
a
(
1
9
9
4
)
;

S
a
l
a
s

e
t
r
e
s
e
a
r
c
h

m
e
t
h
o
d
s

m
a
y

n
e
e
d

t
o

b
e

a
p
p
l
i
e
d
.
-
K
n
o
w
l
e
d
g
e
-
a
c
q
u
i
s
i
t
i
o
n

d
a
t
a
a
l
.

(
1
9
9
8
)
;

S
t
o
u
t
,
-
T
r
a
n
s
f
e
r

d
a
t
a
S
a
l
a
s
,

&

F
o
w
l
k
e
s
-
P
e
r
f
o
r
m
a
n
c
e

d
a
t
a
(
1
9
9
7
)
170 March 1999 – Human Factors
theories, methods, principles, approaches, and
content available to shape the design and deliv-
ery of CRM training. Much has been done and
much remains to be done. There is little doubt
that our colleagues performing research in deci-
sion making (Orasanu, 1993), situation aware-
ness (Endsley, 1995), leadership (Pettit &
Dunlap, 1997), and other topics will contribute
useful elements to CRM training as well. In fact,
our own interests have now turned toward
training the cognitive components of teamwork
in the cockpit (Salas, Bowers, & Cannon-
Bowers, 1995; Salas, Prince, Baker, & Shrestha,
1995; Shrestha, Prince, Baker, & Salas, 1995;
Stout, Cannon-Bowers, & Salas, 1996). We
believe that by exploiting advances in training
technology and methods – and, perhaps more
important, adopting a systematic methodology
for developing training – teamwork in the cock-
pit will improve. It is up to all of us – scientists
and practitioners – to see that the potential ben-
efits of CRM training are realized in the aviation
community.
ACKNOWLEDGMENTS
The views expressed here are those of the
authors and do not reflect the official position
of the organization with which the authors are
affiliated. Several colleagues throughout the
years have contributed to this program of
research: David Baker, Rebecca Beard,
Maureen Bergondy, Mike Brannick, Jim
Driskell, Jennifer Fowlkes, Florian Jentsch,
Danielle Merket, Dana Milanovich, Ben
Morgan, Jr., Elizabeth Muñiz, Jerry Owens,
Ashley Prince, Lisa Shrestha, Bob Swezey,
Kimberly Smith-Jentsch, Mike Lilienthal, and
Scott Tannenbaum. We appreciate their efforts
in making this program of research offer solu-
tions to the aviation community. We would
also like to thank the 1400 or so aviators who
have donated their time over the years to help
us conduct this research.
REFERENCES
Baker, D. P., Prince, C., Shrestha, L., Oser, R. L., & Salas, E.
(1993). Aviation computer games for crew resource manage-
ment training. International Journal of Aviation Psychology,
3(2), 143–156.
Baker, D. P., & Salas, E. (1992). Principles for measuring team-
work skills. Human Factors, 34, 469–475.
Beard, R. L., Salas, E., & Prince, C. (1995). Enhancing transfer of
training: Using role-play to foster teamwork in the cockpit.
International Journal of Aviation Psychology, 5(2), 131–143.
Bowers, C. A., Baker, D. P., & Salas, E. (1994). Measuring the
importance of teamwork: The reliability and validity of
job/task analysis indices for team-training design. Military
Psychology, 6, 205–214.
Bowers, C. A., Deaton, J., Oser, R. L., Prince, C., & Kolb, M.
(1995). Impact of automation on aircrew communication and
decision-making performance. International Journal of
Aviation Psychology, 5(2), 145–168.
Bowers, C. A., Morgan, B. B., Jr., & Salas, E. (1991). The assessment
of aircrew coordination demand for helicopter flight require-
ments. In Proceedings of the 6th International Symposium on
Aviation Psychology (pp. 308–313). Mahwah, NJ: Erlbaum.
Bowers, C. A., Morgan, B. B., Jr., Salas, E., & Prince, C. (1993).
Assessment of coordination demand for aircrew coordination
training. Military Psychology, 5, 95–112.
Bowers, C. A., Thornton, C., Braun, C. C., & Salas, E. (1998).
Automation, task difficulty, and aircrew performance. Military
Psychology, 10, 259–274.
Brannick, M. T., Prince, C., Salas, E., & Stout, R. J. (1997).
Development and evaluation of a team training tool.
Unpublished manuscript.
Brannick, M. T., Salas, E., & Prince, C. (Eds.). (1997). Team per-
formance assessment and measurement: Theory, methods,
and applications. Mahwah, NJ: Erlbaum.
Byrnes, R. E., & Black, R. (1993). Developing and implementing
CRM programs: The Delta experience. In E. L. Wiener, B. G.
Kanki, & R. L. Helmreich (Eds.), Cockpit resource manage-
ment (pp. 421–443). San Diego: Academic.
Cannon-Bowers, J. A., Prince, C., Salas, E., Owens, J. M., Morgan,
B. B., Jr., & Gonos, G. H. (1989). Determining aircrew coor-
dination effectiveness. In Proceedings of the 11th Inter-
service/Industry Training Systems Conference (pp. 128–136).
Arlington, VA: National Defense Industrial Association.
Cannon-Bowers, J. A., & Salas, E. (1997). Teamwork competen-
cies: The intersection of team member knowledge, skills, and
attitudes. In H. F. O’Neil (Ed.), Workforce readiness:
Competencies and assessment. Mahwah, NJ: Erlbaum.
Cannon-Bowers, J. A., & Salas, E. (1998). Individual and team
decision making under stress: Theoretical underpinnings. In
J. A. Cannon-Bowers & E. Salas (Eds.), Making decisions
under stress: Implications for individual and team training (pp.
17–38). Washington, DC: American Psychological Association.
Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Converse,
S. A. (1991). Toward an integration of training theory and
technique. Human Factors, 33, 281–292.
Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Volpe, C. E.
(1995). Defining team competencies and establishing team
training requirements. In R. Guzzo & E. Salas (Eds.), Team
effectiveness and decision-making in organizations (pp.
333–380). San Francisco: Jossey-Bass.
Caro, P. W. (1988). Flight training and simulation. In E. L. Wiener
& D. C. Nagel (Eds.), Human factors in aviation (pp.
229–261). San Diego: Academic.
Chidester, T. R., & Foushee, H. C. (1988). Leader personality and
crew effectiveness: Factors influencing performance in full-mission
air transport simulation. In Proceedings of the 66th Meeting of
the Aerospace Medical Panel on Human Stress Situations in
Aerospace Operations (pp. 7-1–7-9). The Hague, Netherlands:
Advisory Group for Aerospace Research and Development.
Dwyer, D. J., Fowlkes, J. E., Oser, R. L., Salas, E., & Lane, N. E.
(1997). Team performance measurement in distributed envi-
ronments: The TARGETs methodology. In M. T. Brannick, E.
Salas, & C. Prince (Eds.), Team performance assessment and
measurement: Theory, methods, and applications (pp.
137–153). Mahwah, NJ: Erlbaum.
Endsley, M. R. (1995). Measurement of situation awareness in
dynamic systems. Human Factors, 37, 65–84.
Ford, J. K., Kozlowski, S. W. J., Kraiger, K., Salas, E., & Teachout, M. S.
(Eds.). (1997). Improving training effectiveness in work orga-
nizations. Mahwah, NJ: Erlbaum.
Foushee, H. C. (1984). Dyads and triads at 35,000 feet: Factors
affecting group process and aircrew performance. American
Psychologist, 39, 885–893.
CRM TRAINING 171
Foushee, H. C., & Helmreich, R. L. (1988). Group interaction and
flight crew performance. In E. L. Wiener & D. C. Nagel
(Eds.), Human factors in aviation (pp. 189–227). San Diego:
Academic.
Fowlkes, J. E., Dwyer, D. J., Oser, R. L., & Salas, E. (1998).
Event-based approach to training. International Journal of
Aviation Psychology, 8(3), 209–221.
Fowlkes, J. E., Lane, N. E., Salas, E., Franz, T., & Oser, R. L.
(1994). Improving the measurement of team performance:
The TARGETs methodology. Military Psychology, 6, 47–61.
Hartel, C., Smith, K. A., & Prince, C. (1991). Defining aircrew
coordination: Searching mishaps for meaning. Paper present-
ed at the 6th International Symposium on Aviation
Psychology, Columbus, OH.
Helmreich, R. L., & Foushee, H. C. (1993). Why crew resource
management? Empirical and theoretical bases of human fac-
tors training in aviation. In E. L. Wiener, B. G. Kanki, &
R. L. Helmreich (Eds.), Cockpit resource management (pp.
3–45). San Diego: Academic.
Helmreich, R. L., & Wilhelm, J. A. (1991). Outcomes of crew
resource management training. International Journal of
Aviation Psychology, 1(4), 287–300.
Jentsch, F. (1997). Meta-cognitive training for junior team mem-
bers: Solving the copilots catch-22. Unpublished doctoral dis-
sertation, University of Central Florida, Orlando.
Jentsch, F., & Bowers, C. A. (1998). Evidence of validity of low-
fidelity simulations in studying aircrew coordination.
International Journal of Aviation Psychology, 8(3), 243–260.
Johnston, J. H., Cannon-Bowers, J. A., & Smith-Jentsch, K. A.
(1995). Event-based performance measurement system for
shipboard command teams. In Proceedings of the 1st
International Symposium on Command and Control Research
and Technology (pp. 274–276). Washington, DC: Department
of Defense.
Kolb, D. A. (1984). Experiential learning: Experience as the
source of learning and development. Englewood Cliffs, NJ:
Prentice-Hall.
Lauber, J. K. (1984). Resource management in the cockpit. Air
Line Pilot, 53, 20–23.
Leedom, D. K., & Simon, R. (1995). Improving team coordina-
tion: A case for behavior based training. Military Psychology,
7, 109–122.
Marrow, A. J. (1969). The practical theorist: The life and work of
Kurt Lewin. New York: Basic Books.
McIntyre, R. M., & Salas, E. (1995). Measuring and managing for
team performance: Emerging principles from complex envi-
ronments. In R. Guzzo & E. Salas (Eds.), Team effectiveness
and decision making in organizations (pp. 149–203). San
Francisco: Jossey-Bass.
Morgan, B. B., Jr., Salas, E., & Glickman, A. S. (1994). An analy-
sis of team evolution and maturation. Journal of General
Psychology, 120, 277–291.
Orasanu, J. L. (1993). Decision-making in the cockpit. In
E. L. Wiener, B. G. Kanki, & R. L. Helmreich (Eds.), Cockpit
resource management (pp. 137–172). San Diego: Academic.
Oser, R. L., Dwyer, D. J., & Folkes, J. E. (1995). Team perfor-
mance in multi-service distributed interactive simulation exer-
cises: Initial results. In Proceedings of the 17th
Interservice/Industry Training Systems and Education
Conference (pp. 163–171). Arlington, VA: National Defense
Industrial Organization.
Pettit, M. A., & Dunlap, J. H. (1997). Cockpit leadership and fol-
lowership skills: Theoretical perspectives and training guide-
lines (Tech. Report). Washington, DC: Federal Aeronautics
Administration.
Prince, A., Brannick, M. T., Prince, C., & Salas, E. (1997). The
measurement of team process behaviors in the cockpit:
Lessons learned. In M. T. Brannick, E. Salas, & C. Prince
(Eds.), Team performance assessment and measurement (pp.
289–310). Mahwah, NJ: Erlbaum.
Prince, C., Oser, R., Salas, E., & Woodruff, W. (1993). Increasing
hits and reducing misses in CRM/LOS scenarios: Guidelines
for simulator scenario development. International Journal of
Aviation Psychology, 3(1), 69–82.
Prince, C., & Salas, E. (1989). Aircrew performance:
Coordination and skill development. In D. E. Daniel, E. Salas,
& D. M. Kotick (Eds.), Independent research and indepen-
dent exploratory development (IR/IED) programs: Annual
report for fiscal year 1988 (NTSC Tech. Report 89-009).
Orlando, FL: Naval Training Systems Center.
Prince, C., & Salas, E. (1993). Training and research for team-
work in the military aircrew. In E. L. Wiener, B. G. Kanki, &
R. L. Helmreich (Eds.), Cockpit resource management (pp.
337–366). San Diego: Academic.
Ryle, G. (1949). The concept of mind. Chicago: University of
Chicago Press.
Salas, E., Bowers, C. A., & Cannon-Bowers, J. A. (1995). Military
team research: Ten years of progress. Military Psychology, 7,
55–75.
Salas, E., & Cannon-Bowers, J. A. (1997). Methods, tools, and
strategies for team training. In M. A. Quiñones &
A. Ehrenstein (Eds.), Training for a rapidly changing work-
place: Applications of psychological research (pp. 249–279).
Washington, DC: American Psychological Association.
Salas, E., & Cannon-Bowers, J. A. (in press). The anatomy of team
training. In L. Tobias & D. Fletcher (Eds.), Handbook on
research in training. New York: Macmillan.
Salas, E., Cannon-Bowers, J. A., & Blickensderfer, E. L. (1997).
Enhancing reciprocity between training theory and practice:
Principles, guidelines, and specifications. In J. K. Ford,
S. W. J. Kozlowski, K. Kraiger, E. Salas, & M. S. Teachout
(Eds.), Improving training effectiveness in work organizations
(pp. 291–322). Mahwah, NJ: Erlbaum.
Salas, E., Cannon-Bowers, J. A., & Johnston, J. H. (1997). How
can you turn a team of experts into an expert team? Emerging
training strategies (pp. 359–370). In C. Zsambok & G. Klein
(Eds.), Naturalistic decision making. Mahwah, NJ: Erlbaum.
Salas, E., Dickinson, T. L., Converse, S. A., & Tannenbaum, S. I.
(1992). Toward an understanding of team performance and
training. In R. W. Swezey & E. Salas (Eds.), Teams: Their
training and performance (pp. 3–29). Norwood, NJ: Ablex.
Salas, E., Fowlkes, J. E., Stout, R. J., Milanovich, D., & Prince, C.
(in press). Does CRM training improve teamwork skills in the
cockpit?: Two evaluation studies. Human Factors.
Salas, E., Prince, C., Baker, D. P., & Shrestha, L. (1995). Situation
awareness in team performance: Implications for measure-
ment and training. Human Factors, 37, 123–136.
Shrestha, L. B., Prince, C., Baker, D. P., & Salas, E. (1995).
Understanding situation awareness: Concepts, methods, and
training. In W. B. Rouse (Ed.), Human/technology interaction
in complex systems (Vol. 7, pp. 45–83). Greenwich, CT: JAI
Press.
Smith-Jentsch, K. A., Salas, E., & Baker, D. (1996). Training team
performance-related assertiveness. Personnel Psychology, 49,
909–936.
Smith-Jentsch, K., Zeisig, R. L., Acton, B., & McPherson, J.
(1998). Team dimensional training: A strategy for guided team
self-correction. In J. A. Cannon-Bowers & E. Salas (Eds.),
Making decisions under stress: Implications for individual and
team training (pp. 271–297). Washington, DC: American
Psychological Association.
Stone, R. B., & Babcock, G. L. (1988). Airline pilots’ perspective.
In E. L. Wiener & D. C. Nagel (Eds.), Human factors in avia-
tion (pp. 529–560). San Diego: Academic.
Stout, R. J., Cannon-Bowers, J. A., & Salas, E. (1996). The role of
shared mental models in developing team situational aware-
ness: Implications for training. Training Research Journal, 2,
85–116.
Stout, R. J., Salas, E., & Fowlkes, J. E. (1997). Enhancing team-
work in complex environments through team training. Group
Dynamics: Theory, Research, & Practice, 1, 169–182.
Stout, R. J., Salas, E., & Kraiger, K. (1997). The role of trainee
knowledge structures in aviation team environments.
International Journal of Aviation Psychology, 7(3), 235–250.
Swezey, R., Llaneras, R., Prince, C., & Salas, E. (1991).
Instructional strategy for aircrew coordination training. In
Proceedings of the 6th International Symposium on Aviation
Psychology (pp. 302–307). Mahwah, NJ: Erlbaum.
Swezey, R. W., & Salas, E. (1992). Guidelines for use in team-
training development. In R. W. Swezey & E. Salas (Eds.),
Teams: Their training and performance (pp. 219–245).
Norwood, NJ: Ablex.
172 March 1999 – Human Factors
Tannenbaum, S. I., Beard, R. L., & Salas, E. (1992). Team build-
ing and its influence on team effectiveness: An examination of
conceptual and empirical developments. In K. Kelley (Ed.),
Issue, theory, and research in industrial/organizational psy-
chology (pp. 117–153). Amsterdam: Elsevier.
Thornton, C., Braun, C. C., Bowers, C. A., & Morgan, B. B., Jr.
(1992). Automation effects in the cockpit: A low-fidelity
investigation. In Proceedings of the Human Factors and
Ergonomics Society 36th Annual Meeting (pp. 30–34). Santa
Monica, CA: Human Factors and Ergonomics Society.
Wiener, E. L., Kanki, B. G., & Helmreich, R. L. (Eds.). (1993).
Cockpit resource management. San Diego: Academic.
Yamamori, H., & Mito, T. (1993). Keeping CRM is keeping the flight
safe. In E. L. Wiener, B. G. Kanki, & R. L. Helmreich, Cockpit
resource management (pp. 399–420). San Diego: Academic.
Eduardo Salas is a senior research psychologist and
head of the Training Technology Development
branch of the Naval Air Warfare Center Training
Systems Division (NAWCTSD). He received his
Ph.D. in industrial/organizational psychology from
Old Dominion University, Norfolk, Virginia in 1984.
Carolyn Prince is a former research psychologist at
NAWCTSD. She holds a Ph.D. in industrial/organi-
zational psychology from the University of South
Florida, Tampa, which she received in 1984.
Clint A. Bowers is an associate professor of psychology
and director of the Team Performance Laboratory
at the University of Central Florida. He received his
Ph.D. in psychology from the University of South
Florida in 1987.
Renée J. Stout is a research psychologist who works
in the Aviation Team Training Lab at NAWCTSD.
She received her Ph.D. in human factors psychology
from the University of Central Florida in 1994.
Randall L. Oser is a research psychologist at NAW-
CTSD. He received his M.S. degree in industrial/
organizational psychology from the University of
Central Florida in 1990.
Janis A. Cannon-Bowers is a research psychologist
at NAWCTSD. She received her Ph.D. in 1988
from the University of South Florida in industrial/
organizational psychology.
Date received: November 5, 1997
Date accepted: July 13, 1998

Sponsor Documents

Or use your account on DocShare.tips

Hide

Forgot your password?

Or register your new account on DocShare.tips

Hide

Lost your password? Please enter your email address. You will receive a link to create a new password.

Back to log-in

Close