2/21/15
RESEARCH
METHODS
1
Steps
of
scientific
methodology
Folk
psychology
vs.
Psychological
Science
! Everybody
has
ideas
about
how
they
think
and
behave
and
how
other
people
think
and
behave
! Scientific
psychology
sometimes
supports
and
sometimes
contradicts
these
ideas
by
Gaye
Soley
email:
[email protected]
• Psychology
is
the
scientific
study
of
behavior
and
mental
processes.
1.
Describing
behavior
• The
same
methods
that
are
used
for
biology,
chemistry,
and
physics
can
be
applied
to
study
behavior,
thought,
emotion,
and
physiological
processes.
2.
Predicting
behavior
2.
Predicting
behavior
1
2/21/15
2.
Predicting
behavior
3.
Explaining
Behavior
“It
is
not
what
a
man
of
science
believes
that
distinguishes
him,
but
how
and
why
he
believes
it.”
Women
of
science,
too!!
Bertand
Russell
Scientific
inquiry:
! A
systematic
method
of
collecting,
analyzing,
and
interpreting
data.
1.
Systematic
Empiricism
2.
!
Public
Verification
3.
Solvable
Problems
! Used
to
describe,
predict,
control,
and
explain
why
particular
behaviors
happen.
!
1.
2.
3.
4.
5.
6.
7.
8.
Empirical
approach
Observations
Questions
Hypotheses
Experiments
Analyses
Conclusions
Replication
Science
vs.
pseudoscience
–
unverifiable
observations
Are
there
angels?
1.
Empirical
Approach
!
An
evidence-‐based
approach
that
relies
on
direct
observation
and
experimentation
in
the
acquisition
of
new
knowledge.
!
In
the
empirical
approach,
scientific
decisions
are
made
based
on
the
data
derived
from
direct
observation
and
experimentation.
2
2/21/15
2.
Observations
1.
2.
!
!
Being
aware
of
the
world
around
us
Making
careful
measurements.
Observations
of
the
world
around
us
often
give
rise
to
the
questions
that
are
addressed
through
scientific
research.
E.g.,
observation:
Those
friends
who
are
into
sports,
and
who
do
regular
physical
exercise
seem
happier
than
those
who
do
not
exercise
3.
Questions
!
Depending
on
the
type
of
research
you
are
conducting,
your
questions
are
differently
formulated
3.
Questions
!
After
getting
a
research
idea,
perhaps
from
making
observations
of
the
world
around
us,
the
next
step
in
the
research
process
involves
translating
that
research
idea
into
an
answerable
question.
!
It
would
be
a
frustrating
and
ultimately
unrewarding
endeavor
to
attempt
to
answer
an
unanswerable
research
question
through
scientific
investigation.
!
E.g.,
Could
there
be
a
relationship
between
regular
exercise
and
happiness?
Types
of
research
1.
Basic
vs.
applied
! Applied
research
uses
the
data
directly
for
real
! What
learning
styles
do
1st
year
psychology
students
world
application
use?
! Basic
research
focuses
on
testing
theories
! Which
learning
style,
typically
used
by
1st
year
psychology
students,
is
the
most
effective
in
their
performance?
Types
of
research
1. Basic
vs.
applied
2.
Descriptive
vs.
explanatory
! Descriptive
research,
is
used
to
describe
characteristics
of
a
population
or
a
phenomenon
being
studied
! Explanatory
research
implies
that
the
research
in
question
is
intended
to
explain,
rather
than
simply
to
describe,
the
phenomena
studied.
Types
of
research
1. Basic
vs.
applied
2.
3.
Descriptive
vs.
explanatory
Quantitative
vs.
qualitative
! Qualitative
research
gathers
information
that
is
not
in
numerical
form.
3
2/21/15
Types
of
research
A
researcher
wonders
how
color
combinations
make
people
feel…
! Descriptive
or
explanatory?
4.
Hypothesis:
a
testable
prediction
!
!
!
!
4.
Hypothesis:
a
testable
prediction
!
!
!
Two
types
of
hypotheses:
! the
null
hypothesis
! and
the
alternate
(or
experimental)
hypothesis.
The
null
hypothesis
always
predicts
that
there
will
be
no
differences
between
the
groups
being
studied.
By
contrast,
the
alternate
hypothesis
predicts
that
there
will
be
a
difference
between
the
groups.
5.
Experiments
!
After
articulating
the
hypothesis,
the
next
step
involves
actually
conducting
the
experiment
(or
research
study).
!
A
key
aspect
of
conducting
a
research
study
is
measuring
the
phenomenon
of
interest
in
an
accurate
and
reliable
manner.
Hypotheses
can
take
various
forms,
depending
on
the
question
being
asked
and
the
type
of
study
being
conducted.
A
key
feature
of
all
hypotheses
is
that
each
must
make
a
prediction.
E.g.,
(everything
else
being
equal),
people
who
exercise
regularly
will
be
on
average
happier
than
those
who
do
not
exercise
regularly
These
predictions
are
then
tested
by
gathering
and
analyzing
data,
and
the
hypotheses
can
either
be
supported
or
refuted
on
the
basis
of
the
data.
4.
Hypothesis:
a
testable
prediction
People
who
exercise
regularly
will
be
on
average
happier
than
those
who
do
not
exercise
regularly
Null
hypothesis:
There
won’t
be
a
difference
between
exercise
group
and
no-‐exercise
group
in
happiness
levels
Alternate
hypothesis:
regular
exercise
group
will
be
happier
than
no-‐
exercise
group
5.
Experiments
!
Accuracy
refers
to
whether
the
measurement
is
correct,
whereas
reliability
refers
to
whether
the
measurement
is
consistent.
4
2/21/15
6.
Analyses
7.
Conclusions
! The
type
of
statistical
techniques
used
by
a
! After
analyzing
the
data
and
determining
! In
short,
statistics
help
researchers
minimize
! For
example,
if
the
researcher
rejected
the
8.
Replication
! Induction
(from
facts
to
hypothesis):
researcher
depends
on
the
design
of
the
study,
the
type
of
data
being
gathered,
and
the
questions
being
asked.
the
likelihood
of
reaching
an
erroneous
conclusion
about
the
relationship
between
the
variables
being
studied.
! Conducting
the
same
research
study
a
second
time
with
another
group
of
participants
to
see
whether
the
same
results
are
obtained.
whether
to
reject
the
null
hypothesis,
the
researcher
is
now
in
a
position
to
draw
some
conclusions
about
the
results
of
the
study.
null
hypothesis,
the
researcher
can
conclude
that
the
phenomenon
being
studied
had
a
statistically
significant
effect.
abstracting
a
hypothesis
from
a
collection
of
facts
! The
same
researcher
may
attempt
to
replicate
previously
obtained
results,
or
perhaps
other
researchers
may
undertake
that
task.
! Deduction
(from
theory
to
hypothesis):
a
process
of
reasoning
from
a
general
proposition
(the
theory)
to
specific
implications
of
that
proposition
(the
hypotheses).
Theory:
a
hypothetical
account
of
how
and
why
a
phenomenon
occurs
Hypothesis:
a
testable
prediction
made
by
a
theory
Heat
is
the
result
of
molecular
motion
Slowing
molecules
should
make
an
object
cooler
5
2/21/15
Theory:
a
hypothetical
account
of
how
and
why
a
phenomenon
occurs
Hypothesis:
a
testable
prediction
made
by
a
theory
Theory
of
Diffusion
of
Responsibility:
People
feel
less
responsibility
for
their
own
actions
when
they
are
in
a
group
than
when
they
are
alone
Theory
of
Diffusion
of
Responsibility:
People
feel
less
responsibility
for
their
own
actions
when
they
are
in
a
group
than
when
they
are
alone
People
are
less
likely
to
call
the
police
in
an
emergency
in
a
group
than
when
they
are
alone
People
are
less
likely
to
call
the
police
in
an
emergency
in
a
group
than
when
they
are
alone
Kitty
Genovese
! How
do
we
decide
which
theory
is
good?
1.
Parsimony
2.
Precision
3.
Generativity
4.
Testability
&
Falsifiability
2.
Precision
! Can
we
agree
on
the
theory’s
predictions?
1.
Parsimony
(Occam's
razor)
! Explain
reality
by
simplification
!
If
exceptional
assumptions
are
required,
the
theory
is
not
parsimonious
! Conceptual
definition
–
much
like
a
definition
that
one
would
find
in
a
dictionary
! Sometimes
too
fuzzy
for
precise
scientific
communication
! A
good
theory
should
have
key
terms
and
concepts
that
are
operationally
defined
! Operational
definition
–
specifies
precisely
how
a
concept
is
measured
or
manipulated
in
a
particular
study
6
2/21/15
▪ Example:
A
researcher
is
investigating
the
effects
of
regular
exercise
on
happiness.
! Example:
Babies’
ability
to
pay
attention
decreases
as
they
become
more
hungry
▪ Decides
to
use
a
ten-‐question
happiness
scale
to
measure
positive
outlook
in
participants
(operational
definition
of
happiness
in
this
case
is
a
given
subject’s
score
on
the
test)
! Hunger
(conceptual
definition):
having
a
desire
for
food
! Hunger
(operational
definition):
being
deprived
of
food
for
12
hours
▪
What
else
needs
to
be
operationally
defined?
3.
Generativity
4.
Testability
and
falsifiability
!
Does
this
formulation
of
the
theory
allow
hypotheses
to
be
formulated?
Does
it
stimulate
controversy,
disbelief,
and
resistance?
Does
stimulate
thinking,
and
offer
tentative
explanations
that
account
for
phenomena?
!
!
!
Empirical
data
to
confirm
or
refute
claims
“No
human
lives
forever”
-‐
not
falsibiable
(i.e.,
one
needs
to
observe
a
human
living
forever
to
falsify
this
claim)
! “All
humans
live
forever”
–
falisifiable
(i.e.,
presentation
of
just
one
dead
human
could
prove
the
statement
wrong)
!
4.
Testability
and
falsifiability
!
!
! The
logical
impossibility
of
proof
Empirical
data
to
confirm
or
refute
claims
Example:
Freud’s
psychoanalytic
theory:
repression
! The
practical
impossibility
of
disproof
Sigmund
Freud
1856-‐1939
7
2/21/15
The
logical
impossibility
of
proof
! We
cannot
logically
prove
a
theory
by
affirming
hypotheses
that
are
derived
from
it.
! We
often
talk
of
theories
being
confirmed
or
supported
by
research
findings.
Practical
Impossibility
of
Disproof
! Failing
to
find
research
support
for
a
hypothesis
does
not
necessarily
imply
that
the
theory
is
incorrect
because
many
things
can
lead
to
a
failure
to
obtain
supportive
data
! File-‐drawer
problem
! We
cannot
logically
prove
a
theory
by
affirming
hypotheses
that
are
derived
from
it.
! Theory:
tiny
green
men
live
under
people’s
skin.
! Hypothesis:
If
Theory
X
is
true,
people
who
are
stabbed
will
feel
pain
because
the
little
green
men
will
be
hurt.
! A
researcher
stabs
50
people
and
finds
that
they
all
feel
pain.
! Data
may
fail
to
support
our
hypotheses
for
reasons
that
have
noting
to
do
with
the
validity
of
a
theory.
! Failure
to
find
empirical
support
for
a
hypothesis
can
be
due
to
several
factors.
! What
is
truth
..?
-‐
Can’t
prove
or
disprove,
so
what?
!
Just
because
a
theory
makes
it
through
all
scientific
research
steps,
this
doesn’t
mean
it
is
‘true’
in
the
common
usage
of
the
term.
! The
findings
of
a
particular
study
are
true
to
some
extent,
but
by
definition
a
theory
is
an
approximation
to
the
truth.
8
2/21/15
RESEARCH
METHODS
2
Types
of
measures
Strategies
of
Behavioral
Research
! Psychological
measures
can
be
roughly
divided
into
the
categories
below:
1. Observational
2. Physiological
/
neuroscientific
measures
3. Self-‐report
measures
4. Archival
methods
by
Gaye
Soley
email:
[email protected]
! A
researcher
who
is
conducting
a
study
on
shyness
may:
! Observe
participants’
behavior
(observational
measure)
! Measure
their
physiological
responses
during
a
social
interaction
(physiological
measure)
! Ask
them
to
answer
questions
on
a
questionnaire
(self-‐report
measure)
! Content-‐analyze
the
entries
in
the
diaries
(archival
measure)
3
decisions
to
be
made:
! Involve
the
direct
observation
of
behavior
! Can
be
used
to
measure
anything
an
organism
does
that
can
be
observed
! Reaction
time,
whether
a
rat
presses
a
lever,
eye
contact
during
conversation,
aggressive
behavior…
! Naturalistic
observation
! Involves
the
observation
of
ongoing
behavior
as
it
! i-‐
Will
the
observation
occur
in
a
natural
or
contrived
setting?
(i.e.
natural
or
arranged
settings
for
observation)
! ii-‐
Will
the
participants
know
they
are
being
observed?
! iii-‐
How
will
participants’
behavior
be
recorded?
occurs
naturally
with
no
intrusion
or
intervention
by
the
researcher
! Parent-‐child
interactions
on
the
playground
! Tool
use
by
elephants
! Aggression
among
monkeys
9
2/21/15
Participant
Observation
! Researchers
might
pose
as
cult
members,
homeless
people,
devil
worshipers,
salespeople,
gang
members
etc..
! Contrived
observation
! Participants
know
they
are
being
observed
but
the
observers
are
usually
concealed
(e.g.,
behind
a
one-‐way
mirror)
! Losing
objectivity
! The
risk
of
influencing
the
behavior
of
the
individuals
being
studied
Contrived
Observation
(cont.):
! In
other
settings,
researchers
use
contrived
observation
in
“real
world”
! Researchers
set
up
situations
outside
of
the
laboratory
to
observe
people’s
reactions.
! Researchers
were
interested
in
factors
that
affect
helping
so
they
staged
an
emergency
on
a
NYC
subway.
! The
individuals
being
studied
know
that
the
researcher
is
observing
the
behavior
(undisguised
observation)
! People
often
do
not
respond
naturally
when
they
know
they
are
being
scrutinized
(studied
closely)
"
reactivity
!
Disguised
Observation
! When
researchers
are
concerned
about
reactivity,
they
may
conceal
the
fact
that
they
are
observing
and
recording
participants’
behavior
! May
violate
right
of
informed
consent
! Potential
violation
of
privacy
–
ethical
issues
10
2/21/15
! As
long
as
the
behaviors
under
observation
occur
in
public
and
the
researcher
does
not
unnecessarily
inconvenience
or
upset
the
participants,
the
ethical
considerations
are
small
! However,
if
the
behaviors
are
not
public
or
the
researcher
intrudes
uninvited
into
participants’
everyday
lives,
then
disguised
observation
is
problematic
! E.g.
What
parts
of
a
textbook
students
consider
important
could
be
examined
by
looking
at
the
sections
that
they
underlined
or
highlighted
! E.g.
Inferring
the
most
popular
exhibits
in
a
museum
from
where
the
floor
tiles
are
worn
down
Examples:
! Temporal
Measures:
Latency
and
duration
! Checklist:
Recording
whether
a
behavior
happened
or
not
! Rating
Scales:
Measures
quality
or
intensity
of
a
behavior
Ways
to
minimize
reactivity:
! Partial
concealment
–
allow
participants
to
know
that
they
are
being
observed
but
do
not
tell
them
what
specific
aspects
of
behavior
are
being
observed
! Unobtrusive
measures
–
use
indirect
measures
that
can
be
taken
without
the
participants
knowing
they
are
being
studied
! Simple
and
easily
recorded
behaviors:
! Number
of
times
a
pigeon
pecks
a
key
! The
number
of
M&Ms
eaten
by
participant
(to
look
at
social
influences
on
eating)
! In
other
cases,
behavior
are
more
complex
! E.g.
Embarrassment,
group
discussion,
negotiations..
! Reaction
time:
time
that
elapses
between
the
presentation
of
a
stimulus
and
the
participant’s
response
(such
as
pressing
a
key).
! The
longer
the
reaction
time,
the
more
internal
processing
must
be
occurring.
11
2/21/15
Implicit
Association
Test
–
IAT
Observational
Rating
Scales:
! E.g.
A
developmental
psychologist
may
want
to
know
how
intensely
the
baby
cried
during
the
experiment
! Inter-‐rater
Reliability:
Degree
to
which
the
observations
of
two
or
more
independent
raters
or
observers
agree.
! Researchers
must
have
clear
and
precise
operational
definitions
for
all
behaviors
that
will
be
observed
and
recorded.
! Raters
should
practice
using
the
coding
system
by
comparing
and
discussing
their
practice
ratings.
! To
study
the
relationship
between
bodily
processes
and
behavior
! Involves
the
use
of
specialized
equipment
to
measure
! Examples:
heart
rate,
brain
activity,
hormonal
changes…
Examples:
!
Measures
of
Neural
Electrical
Activity
Neuroimaging
! Measures
of
Autonomic
Nervous
System
Activity
! Blood
and
Saliva
Assays
!
!
12
2/21/15
! E.g.,
researchers
who
study
sleep,
dreaming,
and
other
states
of
consciousness
use
the
electroencephalogram
(EEG)
to
measure
brain
waves
(different
patterns
of
firing
of
nerve
cells
for
sleep,
dreaming..)
! Allows
researcher
to
see
activity
occurring
within
the
brain.
! E.g.,
fMRI
(functional
magnetic
resonance
imaging):
measures
relative
amount
of
oxygenated
blood
in
a
particular
region
is
associated
with
higher
neural
activity
in
that
part
of
the
brain.
! E.g.
A
pulse
plethysmogram
measures
pulse
rates
by
measuring
the
diameter
of
your
finger.
! Can
be
used
to
measure
anxiety.
! E.g.
certain
hormones
such
as
adrenalin
and
! Questionnaires
require
participants
to
! E.g.,
testosterone
hormone
is
related
to
! Interviews
involve
the
participant
responding
cortisol
are
released
in
response
to
stress.
activity
level
and
aggression.
respond
to
written
statements
or
questions.
orally
to
questions
13
2/21/15
Likert-‐type
scales
! Series
of
statements,
each
of
which
expresses
an
attitude
or
behavior
that
can
vary
in
frequency
or
intensity
! Require
graded
responses
to
each
item
! 1-‐-‐-‐-‐-‐-‐-‐3-‐-‐-‐-‐-‐-‐-‐5
! The
sum
of
the
item
credits
represents
an
individual’s
attitude
www.trackyourhappiness.org
! Social
desirability
response
bias
! Participants
often
worry
about
how
the
researchers
will
think
of
them
! Existing
data,
not
collected
for
the
study
! Court
records,
personal
letters,
newspapers,
census
data
etc…
! Descriptive
Research
! Correlational
Research
! Quasi-‐experimental
Research
! Experimental
Research
More
information
about
causation
! Describes
the
behavior,
thoughts,
or
feelings
of
a
particular
group
of
individuals
! opinion
poll,
census
! prevalence
or
symptoms
of
a
disorder
! developmental
studies
▪ Changes
in
behavior
due
to
age
(cross-‐sectional
vs.
longitudinal)
14
2/21/15
! Longitudinal
design
! Test
the
same
individuals
at
intervals
as
they
grow
! Cross-‐sectional
design
! Test
individuals
of
different
ages
! Surveys
▪ How prevalent it is to have a opposite-sex
roommate at each class (freshmen,
sophomore, junior, etc..) among Bogazici
students?
! Representative
Sample
–
a
sample
from
which
we
can
draw
accurate,
unbiased
estimates
of
the
characteristics
of
the
larger
population
! Surveys
! most common type of descriptive research.
! Involve observation of all of a population, or a
representative subset, at one specific point in
time.
! Psychologists may inquire about people’s
attitudes, lifestyles, behaviors, and problems
! Sampling
is
the
process
by
which
a
researcher
selects
a
sample
of
participants
for
a
study
from
the
population
of
interest.
! Simple
random
sample
–
every
possible
sample
of
the
desired
size
has
the
same
chance
of
being
selected
from
the
population
! Probability
sampling
vs
non-‐probability
sampling
15
2/21/15
! Researchers
do
not
know
the
probability
that
a
particular
case
will
be
chosen
for
the
sample
! Most
research
involves
this
type
of
sampling
! Convenience
Sampling
–
use
whatever
participants
are
readily
available
! E.g.,
convenience
sample
of
Psychology
101
students.
!
Tend
to
be
more
intelligent
than
the
general
population.
!
Tend
to
come
from
middle
or
upper
class
backgrounds.
!
Tend
to
hold
slightly
more
liberal
attitudes
than
the
population
!
Students
who
volunteer
tend
to
be
more
unconventional,
more
self-‐confident,
more
extraverted
and
higher
in
need
for
achievement
! Typicality
may
not
matter!
! Typicality
can
be
investigated
! Typicality
may
be
a
reasonable
assumption
Typical
person
Typical
subject
! Descriptive
Research
! Correlational
Research
! Experimental
Research
More
information
about
causation
! Tries
to
find
associations
and
relationships
between
naturally
occurring
variables
! Are
people
who
score
high
on
anxiety
measures
also
tend
to
score
high
on
depression
measures?
! Is
I.Q.
related
to
reaction
time?
16
2/21/15
! Positive
correlation
!
Two
variables
move
(or
vary)
in
the
same
direction
v1
v2
v1
v2
!
Negative
correlation
! Two
variables
move
(or
vary)
in
the
opposite
direction—
either
up
or
down
v1
v2
scatter
plot
Correlation
between
watching
TV
and
violent
actions
1)
Watching
TV
makes
you
act
more
violent.
Watching
TV
Acting
violent
r
ranges
from
-‐1.00
to
+1.00
2)
Acting
aggressively
makes
you
want
to
watch
more
violent
TV.
3)
A
“third
variable”
influences
both
variables,
causing
them
to
be
correlated
Watching
TV
Watching
TV
Acting
violent
Acting
violent
Living
in
a
violent
family
17
2/21/15
! Descriptive
Research
! Correlational
Research
! Experimental
Research
More
information
about
causation
“Correlation
does
not
imply
causation”
! Independent
variable
–
what
the
researcher
manipulates
! An
experiment
must
be
conducted
to
see
whether
or
not
one
variable
may
be
causing
another
variable.
! Dependent
variable
–
the
response
being
measured
in
a
study,
typically
a
measure
of
participants’
thoughts,
feelings,
behavior,
or
physiological
reactions
! e.g.
if
we
want
to
know
if
a
new
drug
improves
schizophrenic
symptoms
A
well-‐designed
experiment
has
three
essential
properties:
1. Experimental
&
control
groups
! Drug:
independent
variable
! Schizophrenic
symptoms:
dependent
variable
2.
Randomization
3.
Double-‐blind
procedure
18
2/21/15
! Experimental
group
–
participants
in
an
experiment
who
receive
a
nonzero
level
of
the
independent
variable
! E.g.,
the
group
that
receives
the
drug
! Control
group
–
participants
in
an
experiment
who
receive
a
zero
level
of
the
independent
variable
(or
the
absence
of
the
variable
of
interest)
A
well-‐designed
experiment
has
three
essential
properties:
1. Experimental
&
control
groups
2.
Randomization
3.
Double-‐blind
procedure
! The
group
that
doesn’t
receive
drug
! The
IV
is
whether
the
patients
get
the
drug
or
not;
the
DV
is
how
severe
their
symptoms
are
!
It
is
important
that
patients
are
randomly
assigned
to
conditions
–
have
equal
chance
of
being
assigned
to
the
groups
! Internal
validity
is
the
degree
to
which
a
researcher
draws
accurate
conclusions
about
the
effects
of
the
independent
variable
on
a
dependent
variable.
A
well-‐designed
experiment
has
three
essential
properties:
1. Experimental
&
control
groups
2.
Randomization
3.
Double-‐blind
procedure
!
Experimenter
expectancy
effect
occurs
when
a
researcher’s
expectations
about
the
outcome
of
a
study
influences
participants’
reactions.
!
Maze-‐bright
rats
Maze-‐dull
rats
!
19
2/21/15
Demand
characteristics
occur
when
aspects
of
a
study
indicate
to
participants
how
they
should
respond.
! Clever
Hans
!
!
To
avoid
these,
researchers
use
double-‐blind
procedures
in
which
neither
the
participants
nor
the
experimenter
who
interacts
with
them
know
which
condition
the
participant
is
in.
! External
Validity
is
the
degree
to
which
the
results
obtained
in
one
study
can
be
replicated
or
generalized
to
other
samples,
research
settings,
and
procedures
Typical
situation
! Typicality
may
not
matter!
! Typicality
can
be
investigated
! Typicality
may
be
a
reasonable
assumption
Typical
experiment
! Slides
will
be
posted
on
CIMS
! Exam
questions
will
be
from
the
lectures
as
well
as
the
textbook.
20