Mobile robot

Published on May 2016 | Categories: Types, School Work | Downloads: 15 | Comments: 0 | Views: 217
of 11
Download PDF   Embed   Report

Comments

Content


Mobile Robot Aided Silhouette
Imaging and Robust Body Pose
Recognition for Elderly-Fall
Detection
Tong Liu
1
and Jun Liu
2,
1
Department of Electronic Science, Huizhou University, Guangdong 516001, China
2
College of Physics & Electronic Information Engineering, Wenzhou University, Wenzhou, 325035, China

[email protected]
Abstract This article introduces a mobile infrared
silhouette imaging and sparse representation-based
pose recognition for building an elderly-fall detection
system. The proposed imaging paradigm exploits the
novel use of the pyroelectric infrared (PIR) sensor in
pursuit of body silhouette imaging. A mobile robot
carrying a vertical column of multi-PIR detectors is
organized for the silhouette acquisition. Then we express
the fall detection problem in silhouette image-based pose
recognition. For the pose recognition, we use a robust
sparse representation-based method for fall detection.
The normal and fall poses are sparsely represented
in the basis space spanned by the combinations of a
pose training template and an error template. The
1
norm minimizations with linear programming (LP) and
orthogonal matching pursuit (OMP) are used for finding
the sparsest solution, and the entity with the largest
amplitude encodes the class of the testing sample. The
application of the proposed sensing paradigm to fall
detection is addressed in the context of three scenarios,
including: ideal non-obstruction, simulated random pixel
obstruction and simulated random block obstruction.
Experimental studies are conducted to validate the
effectiveness of the proposed method for nursing and
homeland healthcare.
Keywords Elderly-fall Detection, Healthcare, Pyroelectric
Infrared Sensor, Mobile Robot Aided Silhouette Imaging,
Sparse Representation
1. Introduction
In recent years and for the foreseeable future, in many
countries an ageing society is increasingly obvious
due to better quality of life and a lower birth rate. The
proportion of the worldwide population over 65 years
is growing [1] and increasing numbers of elderly people
not only need advanced medical technologies for the
treatment of disease, but also more healthcare services for
independent living and a better quality of life. However,
the decreasing numbers of nursing professionals and in
some countries the decline in the working-age population
will cause a serious imbalance when looking to offer
enough healthcare services for elderly people. Therefore,
the ageing problem has motivated much research on
automated and reliable healthcare systems.
Falling is common among elderly people and a major
hindrance to daily living, especially independent living.
According to the research in reference [2], approximately
one-third of those over 65 years old fall each year and
Tong Liu and Jun Liu: Mobile Robot Aided Silhouette Imaging
and Robust Body Pose Recognition for Elderly-fall Detection
1
ARTICLE
Int J Adv Robot Syst, 2014, 11:42 | doi: 10.5772/57318
1 Department of Electronic Science, Huizhou University, Guangdong, China
2 College of Physics & Electronic Information Engineering, Wenzhou University, Wenzhou, China
* Corresponding author E-mail: [email protected]
Received 20 Feb 2013; Accepted 24 Oct 2013
DOI: 10.5772/57318
∂ 2014 The Author(s). Licensee InTech. This is an open access article distributed under the terms of the Creative
Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,
distribution, and reproduction in any medium, provided the original work is properly cited.
Tong Liu
1
and Jun Liu
2,*
Mobile Robot Aided Silhouette Imaging
and Robust Body Pose Recognition
for Elderly-fall Detection
Regular Paper
International Journal of Advanced Robotic Systems
half of them are repeat fallers. Falls often lead to dramatic
physiological injuries and psychological stress. The
elderly may remain on the floor for a long time leading
to life-threatening situations, while the fear of falling can
result in decreased activity, isolation and further functional
decline. Therefore, falls should be detected as early as
possible to reduce the risk of morbidity-mortality [3].
With regard to detecting falls effectively and quickly,
there has been a recent surge of interest into automated
fall detection and the detections of other abnormal
behaviours of elderly people. These methods can be
divided into two categories according to the use of the
sensors, including: wearable sensors and vision sensors.
Wearable sensors are usually mounted on the human
body and able to achieve accurate fall detection, such
as acceleration [4–6], gyroscopes [7] and wireless button
alarm [8]. However, these solutions break the both
physical and psychological feelings of the elderly people.
Moreover, if the elderly person forgets to wear it, the
alarm system would not work. Therefore, automatic fall
detection using a non-intrusive method is an essential
sensing for complementary under certain circumstances.
Vision sensors can realize the healthcare objectives
in a non-intrusive fashion. Several authors have
explored commercial cameras to capture video in home
environments and extracted the object feature for fall
detection [9, 10]. Although the existing methods have
obtained accurate fall detection, precise body poses
extraction and robustness against changing lighting are
challenging issues in the computer vision community.
Body poses extraction may be corrupted by the clustered
clothes and shadows. In addition, camera-based methods
intrude into the privacy of the elderly. Several alternative
solutions have been proposed in order to analyse human
motion using thermal cameras [11–13]. The human body
is considered to be a natural emitter of infrared rays.
Normally, the body temperature is different from that of
its surroundings. This leads to easy motion detection from
the background regardless of lighting conditions and the
colours of the human surfaces and surroundings. This
sensing pattern can extract meaningful information for
human motion directly. However, thermal cameras are
expensive and information processing is still difficult to
deal with.
To overcome the above limitations of data acquisition,
we propose a mobile infrared silhouette sensing with
a pyroelectric infrared (PIR) sensor array for elderly
pose acquisition and robust fall detection via sparse
representation. Thanks to the well-established studies of
the intelligent mobile robots for home care [14, 15] and the
PIR sensor-based wireless networks [16, 17] for multiple
human tracking, we make an assumption that the mobile
service robot is able to detect where a person has been
lying for a long time and move close to the incident
region. The robot also has the function of patrolling the
interested region periodically. Another assumption is
made based on the fact that elderly people usually do
not move after a fall, therefore a relatively static pose
silhouette would provide enough information for fall
detection. We have designed a sensor array consisting
of a single vertical column of PIR detectors for capturing
human thermal radiation and use a mobile service robot
for implementing silhouette imaging. The mobile robot
undertakes rotary-scanning and the pose of the human
body can be recorded as a crude binary silhouette. Fall
detection is cast as image-based object recognition.
For the data processing, we use a sparse
representation-based method for fall recognition. In
general, simple aspect-ratio-based shape analysis for
fall poses detection is able to satisfy the requirements.
However, this method is suitable for applications in an
ideal environment with no obstructions. In reality, the
object recognition will encounter numerous corruptions,
such as furniture obstructions in real home environments.
This will render the simple aspect-ratio-based shape
recognition useless. For this reason, we propose the sparse
representation-based robust pose recognition, which is
mainly motivated by the recent sparse representation
and its application for robust face recognition and
target tracking [18, 19]. The normal and fall poses are
sparsely represented in the basis space spanned by the
combination of a pose training template and an error
template. The sparsest approximation is computed based
on the
1
-minimization using linear programming (LP)
and orthogonal matching pursuit (OMP), and the
candidate with the maximum amplitude is classified as
the predefined pose. We test this classification method in
three scenarios: ideal non-obstruction, simulated random
pixel obstruction and simulated randomblock obstruction.
Figure 1 presents the schematic diagram of the
system. The fall detection system integrates a mobile
infrared silhouette imaging sensor and a sparse
representation-based pose recognition algorithm.
Mobile Robot Aided
PIR Sensor Array
InIrared Silhouette
Imaging
Data Acquisition Data Transmission Data Processing
Short-time Energy
Based State
Signal Extraction
Radio
Sparse Representation
Based Pose Recognition
Normal Activities

Transmitter
Receiver
Scene
Figure 1. Schematic diagram of the proposed system
Int J Adv Robot Syst, 2014, 11:42 | doi: 10.5772/57318 2
The experiments are conducted to demonstrate the
effectiveness of the described sensing and robust pose
recognition in elderly-fall detection. If a user’s behaviour
is detected as an abnormal event, the alarm will be
activated as soon as possible. This succinct sensing
pattern and robust recognition algorithms make it flexible
for elderly and disabled people to continue to live a free
and independent life in their own house while receiving
reliable safety assistance.
The rest of this article is organized as follows. Section 2 we
give a brief review of PIR sensing and silhouette imaging,
then present the mobile infrared silhouette imaging.
Section 3 describes the sparse representation-based robust
pose recognition. Section 4 presents the experimental
details and results. The summary and conclusions of the
article are given in Section 5.
2. Mobile Robot Aided Infrared Silhouette Sensing
2.1. Related Work
Increasing attention has focused on PIR-based motion
capturing patterns for human presence detection. The
PIR detector has several promising advantages: it is
able to convert the incident thermal radiation into
an electrical signal, and it responds to radiation with
wavelengths ranging from 8µm to 14µm which just
corresponds to the typical thermal radiation emitted
from the human body [20]; the cost of a commercially
available sensor is extremely low; the power consumption
is low and suitable for wireless networks and mobile
agent application. Because of these advantages, the PIR
detector has been developed in lightweight biometric
detection [21, 22], human identification [23–25] and
multiple human tracking [16, 17].
A recent surge of interest has focused on the silhouette
imaging sensor for the applications of electronic fence.
The silhouette sensor belongs to crude imaging devices
and captures a pixelated silhouette of the monitored
target directly. Sartain firstly introduced the concept of
the silhouette imaging sensor and discussed a variety
of approaches to realize a silhouette sensor [26]. The
crude silhouettes generated from a sparse array of sensors
would offer sufficient information for the classification
task and the classification algorithms were tractable
without complex image processing. Russomanno et al.
designed a sparse array of sensors with the near infrared
(IR) transmitters and receivers for the border, perimeter
and other intelligent electronic fence applications [27, 28].
However, the proposed sensor belongs to an active version
and it is difficult to deploy them in a nursing home. Thus,
the passive sensing pattern is a more attractive option.
Jacobs et al. introduced the concept of passive PIR
sensor-based silhouette imaging and gave a simulation
based on thermal infrared video sequences [29]. Willianm
et al. presented a pyroelectric linear array-based silhouette
sensor for distinguishing humans from animals, but they
did not discuss the application of the analysis of human
poses [30]. It should be noted that the field of view (FOV)
of the above-mentioned solutions is fixed and it difficult to
extend using sensor networks or mobile agents for home
healthcare situations.
Our sensing device is motivated by the above silhouette
imaging. A crude silhouette of the human body preserves
sufficient information for distinguishing between normal
and abnormal poses. Considering the fact that elderly
people usually do not move after a fall incident, a mobile
silhouette imaging device is more preferable for sensing a
relatively static object. Thus, we embed linear multi-cell
PIR detectors on a mobile robot for capturing the pose of
the human body. Then the fall detection problem is cast as
image-based object recognition.
2.2. Sensing Model
Figure 2 shows the sensing model of a PIR detector. The
human body is a natural infrared radiation source and
makes exchanges with the surroundings. Thus, the PIR
detector collects the incident thermal radiation. This
will make changes in the temperature on the pyroelectric
material and this will be converted into an electrical
output. The pyroelectric sensing model can be briefly
represented using a form of reference structure [31]:
M(r
m
, t) = H(t) ∗

V(r
m
, r
s
)S(r
s
, t) (1)
where H(t) = [dP
s
/dT] · [dT/dt] is the impulse response
function of the PIR detector, T the temperature, t the
time tag and P
s
the polarization per unit volume [20].
The quantity P = dP
s
/dT is known as the pyroelectric
coefficient and related to the pyroelectric materials, so
the H(t) = P · [dT/dt] is the rate of temperature
change. In particular, the stationary human body does not
trigger the detector and the PIR detector only responds
to human movement without considering the body’s
clothing textures. For a relatively static body, the body
information can be obtained using a mobile infrared
scanning. S and m denote the radiation state vector
and the measurement vector respectively. V(r
m
, r
s
) is the
visibility function, which is “1” when r
s
is visible to the
detector at r
m
, otherwise is “0”.
In this article, the commercially available pyroelectric
detectors D205b [32] are employed for sensing changes of
the thermal radiation in the object space. The object space
is defined as the collection of the thermal radiation fields
in a human body. Fresnel lenses are used to bridge the
FOV of PIR detector matching to the motion sensing space.
Due to the special property of the PIR just responding
to the thermal radiation emitted from the human body,
Ground
Fresnel Lens
PIR Detector
Measurement Output
Vector M(rm,t)
Object Space
Vector S(rs ,t)
Visibility Function V(rm,rs)
Scanning
Direction
Mobile Sensing
Region
30
60
90
120
150
180 0
2 0 -1 -2 1
Scanning Angle
Sensor's Response |Voltage|
Polar Plot of The
Scanning Output
Figure 2. Mobile pyroelectric infrared sensing model
Tong Liu and Jun Liu: Mobile Robot Aided Silhouette Imaging
and Robust Body Pose Recognition for Elderly-fall Detection
3
the intrusion of visible illumination can be removed.
When a sensing region moves across the object space
continuously, the detector would give a large voltage
output corresponding to the human body, as shown in
Figure 2.
Once the voltage outputs are collected, they will be
transmitted to the data processing centre wirelessly. At the
data processing centre, users can determine the presence
of the human body via the short-time energy method.
Figure 3 shows a typical response of a PIR detector when
scanning across a human body and explains the short-time
energy method for transforming the raw analogue signals
into an ”ON" or ”OFF" state signal. First, the collected
signal from a PIR detector is normalized by removing
its direct-current component. Second, we calculate the
squared absolute value of the normalized signal and
enframe them into overlapping frames. Third, the energy
signal is obtained by accumulating the enframed signal
in each column and a predefined threshold is used for
determining the current state.
50 100 150
-1
-0.5
0
0.5
1
Number of Sample
V
o
l
t
a
g
e

[
V
]
50 100 150
-0.5
0
0.5
Number of Sample
V
o
l
t
a
g
e

[
V
]
Number of Enframed Sample
50 100 150
5
10
15
20
50 100 150
0
0.5
1
1.5
2
2.5
3
3.5
4
Number of Sample
E
n
e
r
g
y

A
m
p
l
i
t
u
d
e
50 100 150
-1
-0.5
0
0.5
1
1.5
Number of Sample
V
o
l
t
a
g
e

[
V
]
Normalized Signal Squared Value of The
Normalized Signal
Enframed Signal
Threshold Value and Energy Signal
Across Each Column
State Signal
30
60
90
120
150
180 0
2 0 -1 -2 1
Scanning Angle
Sensor's Response |Voltage|
Polar Plot of The Signal
by Rotary-scanning
State Signal
Threshold
Energy Signal
Figure 3. Flowchart of the short-time energy method for
transforming the analogue signals into binary state signal
2.3. Mobile Silhouette Sensing
Figure 4 and Figure 5 show the proposed silhouette
sensor consisting of a single vertical column of multi-cell
PIR detectors, which is attached to an intelligent mobile
robot. The sensor array organizes a vertical column of 20
PIR detectors to capture the human pose information at
different heights. The lowest-cell detector is 6cm above
the ground, and the pitch between any two separated
detectors is 6cm. Therefore, the sensor array is able to
acquire a crude image in the region with a total height of
120cm. The optical axis of each detector is perpendicular
to the longitudinal axis of the object. For better resolution,
both the horizontal and vertical FOVs of each detector are
10
o
.
The user tracking method used by the mobile robot is
based on the wireless distributed PIR sensors [16, 17].
The FOVs of distributed sensors are cooperatively coded
to support multiple human tracking and identification.
The user’s location and biometric information are able to
transmit to the robot via data centre wirelessly. Then the
robot chooses an interesting target and moves close to
him/her for scanning imaging.
Mobile Robot
Detector 20
Detector 1
Height
120cm
Pitch 6cm
FOV 10
o
Scanning
Direction
Distance
1.5m
.

.

.

.

.

.

Cell 1
Cell 10
Cell 20
Batch State Signal
Transformation
Number oI Frame
S
e
n
s
o
r
C
h
a
n
n
e
l
20 40 60 80 100 120 140
5
10
15
20
Binary Silhouette Image
  
Outputs of Detectors
Figure 4. Presentation of the imaging sensor array consisting of
a single column of PIR detectors installed on a mobile robot
Figure 5. Prototype of the mobile infrared silhouette sensor
Under the assumption of knowing the location where
a human lays for a long time, the intelligent mobile
robot moves near to the interest region and performs a
patrol task autonomously. With the help of the mobile
robot, the PIR sensor array will undertake scanning using
self-rotation. In our experiments, the rotary speed of
the robot is approximately 9 degrees per second. The
distance between the object and the sensor is 1.5m. For
the signal sampling and wireless communication, we
use a ultra-low power consumption micro-controller
CC2430 from Texas Instruments. The micro-controller
samples the PIR sensor’s signal at a rate of 10Hz, and
then transmits the signal to the data centre wirelessly. The
wireless communication in this system is followed by the
Zigbee (802.15.4) protocol, which has a low data rate and
low power consumption compared with other wireless
protocols.
The time spent on a complete scan is approximately
20 seconds. This time is controlled by the maximum
Int J Adv Robot Syst, 2014, 11:42 | doi: 10.5772/57318 4
rotation speed of the mobile robot, which can be improved
by using a more flexible and quicker robot. If the user
disappears from the scanning region, the robot will query
this with the data centre to confirm the location of the user.
If the user still remains within the scanned region and
the robot is not able to capture the body image, we infer
that access to body sensing is not feasible. Scanning at
another position or using other devices could compensate
for this disadvantage. If the user moves away from the
scanned region, the robot will restart the tracking program.
To confirm the effectiveness of the proposed sensing
method, we collected some silhouettes based on typical
normal poses and falls in the laboratory as experimental
samples for reference. According to research on sedentary
living among elderly people [33, 34], they are more
accustomed to less frequent and low intensity activities.
Older adults often spent much time standing or sitting
to work, eat, read or socialize. These samples include
the three most common categories: standing, sitting and
fall on the ground. Figure 6 presents the experimental
scenarios and the acquired silhouettes. All pose silhouettes
are normalized as a constant dimension of 20 × 150 and
the body is located at the centre in the horizontal direction.
After the rotating scanning, the pixelated silhouette
images can be recorded. However, the raw silhouettes
contain distortion as shown in the column in Figure 6.(b).
We use a median filter with size of 3 × 3 for preprocessing
the raw silhouettes. It should also be noticed that the
binary silhouettes protect the privacy of elderly people.
Number of Frame
S
e
n
s
o
r

C
h
a
n
n
e
l
20 40 60 80 100 120 140
5
10
15
20
  
Number of Frame
S
e
n
s
o
r

C
h
a
n
n
e
l
20 40 60 80 100 120 140
5
10
15
20
Number of Frame
S
e
n
s
o
r

C
h
a
n
n
e
l
20 40 60 80 100 120 140
5
10
15
20
Number of Frame
S
e
n
s
o
r

C
h
a
n
n
e
l
20 40 60 80 100 120 140
5
10
15
20
Number of Frame
S
e
n
s
o
r

C
h
a
n
n
e
l
20 40 60 80 100 120 140
5
10
15
20
Number of Frame
S
e
n
s
o
r

C
h
a
n
n
e
l
20 40 60 80 100 120 140
5
10
15
20
Figure 6. Illustration of the typical experimental scenario and
the acquired silhouettes. (a) Scenarios with standing pose, sitting
pose and fall pose. (b) Corresponding silhouettes generated by
the mobile infrared silhouette sensor. (c) Refined silhouettes via
median filter.
3. Sparse Representation-based Robust Pose Recognition
Recent developments in sparse presentation-based
classification reveal that a test sample is able to be
linearly represented using an overcomplete dictionary
whose base elements are the combination of training
templates and error-compensation templates [18, 19]. If
the data processing centre captures sufficient samples
for reference, it is able to represent the test image with
a sparse coefficient spanned on the template of the
same class. Following this framework, we exploit the
sparse presentation-based classification to perform binary
silhouettes-based pose recognition.
In the proposed fall detection system, there are two
categories of poses including: normal pose and fall pose.
We use the labelled training samples to build the template
matrix. For all sample images, we first arrange each
of them as a vector m ∈ R
M
(M = d
v
× d
h
), M is the
dimension of the sample, while d
v
and d
h
are the vertical
and horizontal dimension of the raw image. Given n
F
and
n
N
training samples from the fall poses and normal poses,
we reshape them as columns of two template matrixes
T
F
= [t
F,1
, · · · , t
F,n
F
] and T
N
= [t
N,1
, · · · , t
N,n
N
], t ∈ R
M
.
The columns of T
F
and T
N
are assigned to the fall pose
and normal pose respectively.
Assuming the data centre collected enough training
samples of the ith pose image T
i
= [t
i,1
, · · · , t
i,n
i
],
i ∈ {F, N}, a same class of the test sample is able to be
approximated in linear form using:
m ≈ α
i,1
t
i,1
+ α
i,2
t
i,2
+ · · · + α
i,n
i
t
i,n
i
(2)
where the α
i,n
i
is a regression coefficient. Since the class
of the ith test sample is unknown for the data processing
centre, it is necessary to build a global training template by
concatenating the fall and normal pose template as:
T = [T
F
, T
N
] = [t
F,1
, · · · , t
F,n
F
, t
N,1
, · · · , t
N,n
N
] (3)
Then, the linear representation of mcan be modified as:
m = Tw (4)
here w = [0, · · · , 0, α
i,1
, · · · , α
i,n
i
, 0, · · · , 0, ]
T
∈ R
(n
F
+n
N
)
is regarded as a sparse coefficient vector with nonzero
entries of the ith class. Thus, the nonzero entries of the
vector encode the identity of the testing sample m. We can
solve the equation m = Tw to determine the class of the
pose image.
In many practical home scenarios, the silhouette image
m may be partially obstructed by unpredictable factors.
Usually, a random pixel obstruction would happen with
hardware errors or noise, while a block obstruction is
caused by the user’s furniture. Considering these cases,
the above regression model should be modified
m = Tw+ (5)
where is the errors vector and we assume that only
a small fraction of the entries is nonzero. The nonzero
entries of the are denoted as the errors or obstruction
in the image m. The errors vector may have arbitrary
magnitude and the number of the nonzero entries is
unknown, they cannot be ignored. To increase the
robust property of the constructive regression, we can
add an identity matrix I ∈ R
M×M
as the errors basis
Tong Liu and Jun Liu: Mobile Robot Aided Silhouette Imaging
and Robust Body Pose Recognition for Elderly-fall Detection
5
for approximating the nonzero error entries. The sparse
representation can be rewritten as
m = [T, I]

w
e

= Φw
e
(6)
where I = [i
1
, i
2
, · · · , i
d
], i
i
∈ R
M
is the vector with the
only unit entry, and e ∈ R
M
is the error coefficient vector.
Then Φ = [T, I] ∈ R
M×(M+n
F
+n
N
)
, the m = Φw
e
is an
underdetermined equation. The equation does not have
a unique solution usually for the extended w
e
. However,
we previously make an assumption that the entries of the
extended w
e
= [w, e] are sparse. We will cast the problem
of find sparsest solution as an optimization task.
The problem of find sparsest solution for the equation m =
Φw
e
is able to transform to the following optimization
task:
ˆ w
e
= arg min w
e

0
subject to Φw
e
= m (7)
where ·
0
is the l
0
norm of a vector, and w
e

0
denotes
the number of entries in w
e
. In general, solving the l
0
norm
minimization needs exhaustive search and the existence
of the unique sparsest solution should meet certain
conditions. This problem is regarded as NP-hard and
has combinational computational complexity. However,
recent development in the compressive sensing theory
reveals that the l
0
optimization can be replaced with the
l
1
norm minimization [35, 36]. The l
1
norm optimization is
formally defined as follows:
ˆ w
e
= arg min w
e

1
subject to Φw
e
= m (8)
where ·
1
is defined as w
e

1
= Σ
M+n
F
+n
N
n=1
|w
e
(n)|.
This problem is a convex optimization problem and there
are sophisticated methods with polynomial computational
complexity which can be used to solve it. There are two
representative algorithms for solving the sparse recovery.
The first method is based on the convex optimization and
the problem can be solved via LP [35, 36]. The second
method is the greedy algorithm and the problem can
be solved via sequentially investigating the support of
recovered signal [37]. OMP is a widely used method in
the greedy algorithm family due to its simplicity and
good performance. In this article, we first used CVX for
solving the
1
norm optimization, a package for specifying
and solving convex programs [38]. Then we find the
sparse support of the recovered signal using the normal
OMP [37].
For a newly obtained silhouette image, we first compute
its sparsest solution following (8). Generally, nonzero
entries in the ˆ w
e
should be associated with a single pose
class. However, the noise of the errors and obstructions
may lead to small nonzero entries associated with other
classes and error templates. Therefore, we assign the
silhouette image m to the pose class according to the
largest entry in the recovered ˆ w
e
. The algorithm1 below
gives the pose recognition framework. Figure 7 illustrates
the overview recognition framework, the sparse recovery
is based on LP. If the largest nonzero entry is associated
with the error template, we will reject making a decision
and other sensing methods may be needed for analysing
the pose of the human body.
4. Experimental Details and Results
4.1. Experimental Setup
The experimental acquisition of normal and fall poses is
performed with the involvement of 10 volunteers, two
female subjects and eight male subjects. All volunteers are
at normal heights, ranging from 160cm to 180cm. The data
acquisition process is based on a relatively frontal capture.
For each category of activity, the participants are required
to perform a self-selected pose and strategy. For each kind
of pose, we use the proposed sensor array to scan 6 times
under the predefined rate. Thus, there are 60 samples for
standing, 60 samples for sitting and 60 samples for fall
poses.
Experimental data are divided into two sets: the training
templates and testing samples. At the initialization
stage, we randomly select 30 samples from each pose
for building training templates. Therefore, there are
30 columns for representing fall pose and 60 columns
for representing normal pose. Hence, the extended
template matrix Φ has size 3000 × 3090. The remaining
samples are used for testing the recognition method.
The following average results are computed based on 10
times cross-validation. All the recognition experiments
are run on an Intel Pentium4 2.8GHz computer under
the Matlab implementation. The average time spent on
OMP recovery is 0.5738s with maximum 0.6125s, while
the average time for LP is 1.8618s with maximum 2.0672s.
4.2. Recognition without Obstruction
We first test the proposed method for the pose recognition
without obstruction. Figure 8 illustrates the representative
results of algorithm1, the sparse recovery is based on LP.
Figure 8.(b) gives the sparse coefficients spanned on the
Algorithm 1 Framework of the pose recognition
1: Acquiring the infrared silhouette image using the mobile PIR sensor.
2: Input: A matrix of training samples T = [T
F
, T
N
] = [t
F,1
, · · · , t
F,n
F
, t
N,1
, · · · , t
N,n
N
], a test sample m.
3: Extending the template matrix with Φ = [T, I] ∈ R
M×(M+n
F
+n
N
)
.
4: Normalizing each column of the Φ to have unit l
2
norm.
5: Solving the l
1
norm minimization problem:
ˆ w
e
= [ ˆ w, ˆ e] = arg min w
e

1
subject to Φw
e
= m,
6: return identity(m) = arg max( ˆ w
e
).
Int J Adv Robot Syst, 2014, 11:42 | doi: 10.5772/57318 6
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20 20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20 20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
 
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
10 20 30 40 50 60 70 80 90
0
50
100
150


Fall pose entries
Normal pose entries
Sparse coefficients
Largest entry
500 1000 1500 2000 2500 3000
0
50
100
150


Error entries
Test Image  Extended Template 
Extended CoeIIicients 

Training Template  Error Template  Error coeIIicients  Identity coeIIicients 
 
 


















Number oI Frame
S
e
n
s
o
r

C
h
a
n
n
e
l
Index oI CoeIIicients Index oI CoeIIicients
A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




Figure 7. Illustration of the sparse representation-based pose recognition
training template, while Figure 8.(c) shows the error
coefficients spanned on the error template. It can be
seen that the blue entries of the spare coefficients are
corresponding to the true pose class and have larger
amplitudes than those in error coefficients. The largest
amplitude in the estimated candidates is always assigned
to the true class. The red or green circle in the Figure 8.(b)
indicates the determined pose class. The proposed fall
detection method achieves a 100% recognition rate based
on both LP and OMP recovery.
4.3. Recognition with Random Pixel Obstruction
Considering the possible errors caused by PIR sensor and
the noise associated with the mobile robot or sensing
surrounding, we simulate this situation using the random
pixel obstruction on the silhouette images at various
levels, from 10% to 50%. The obstruction operation
is executed by the ’bit-or’ operation of the raw images
with a random pixel obstruction. Figure 9 illustrates the
representative results of algorithm1 with 20% obstruction,
the sparse recovery is based on LP. Figure 9.(c) shows the
images with the random pixel obstruction. Figure 9.(d)
shows the amplitude of the coefficients on the training
template, while Figure 9.(e) shows the amplitude of the
coefficients on the error template. It can be seen that
the blue entries are corresponding to the true pose class,
while a limited number of error coefficients are activated.
In these representative examples, the estimated global
candidates are sparse and have the largest amplitude
at the associated class. The red or green circle in the
Figure 9.(d) is chosen as the determined pose class. In
this test, the sparse representation-based pose recognition
method is able to detect the fall pose and normal pose
correctly with serious random noise. Table 1 exhibits the
average recognition rate at various levels of random pixel
obstruction. The proposed fall detection method achieves
similar performance based on both LP and OMP recovery.
4.4. Recognition with Random Block Obstruction
For simulating a realistic scene containing furniture
obstructions, we create random block obstructions on
the silhouette images at various levels, from 10%
to 50%. The obstruction operation is executed by
masking the raw images with random rectangle block
obstructions. Figure 10 illustrates the representative
results of algorithm1 with 20% obstruction, the sparse
recovery is based on LP. Figure 10.(a) shows the raw
images and Figure 10.(b) simulates the mask with the
random block obstructions. Figure 10.(c) is the simulated
silhouette image using the ’bit-or’ operation between
Figure 10.(a) and Figure 10.(b). It can be seen that the
blue entries are associated with the true pose class. In
these representative examples, the estimated candidates
are sparse and have the largest amplitude at the true pose
class. The red or green circle with the largest amplitude in
the Figure 10.(d) indicates the determined pose class. In
this test, the sparse representation-based pose recognition
method is able to detect the fall pose and normal pose
correctly with serious block obstructions. Table 2 exhibits
the average recognition rate at various levels of random
block obstruction. The proposed fall detection method
achieves similar performance based on both LP and OMP
recovery.
5. Discussions and Conclusions
In this article, we integrate an effective infrared sensing
method and robust pose recognition algorithm for
elderly-fall detection. For the data acquisition, we use a
single column of PIR detectors to implement the infrared
silhouette imaging. A mobile robot agent is employed
for aiding the mobile silhouette imaging. Fall detection
is cast as binary silhouette-based pose recognition. The
candidate pose is represented as a linear combination
of training templates and error templates. A good pose
is able to be approximated by the training template,
which will lead to a sparse solution. Although some error
coefficients will be activated in the simulated practical
scenarios, the combined coefficients are still sparse. The

1
norm minimizations using LP and OMP are used for
finding the sparsest solution, and the entity with the
largest amplitude indicates the class of the testing sample.
From the experimental results, both algorithms have
similar performance, but the OMP method will take less
computational time compared with the LP method. In
some limited circumstances, the OMP is a better choice.
However, there are some limitations to our system.
First, the data acquisition process is based on a relatively
frontal capture. For each category of activity, the
participants were required to perform a self-selected pose
and strategy. In the case that a frontal acquisition is not
Tong Liu and Jun Liu: Mobile Robot Aided Silhouette Imaging
and Robust Body Pose Recognition for Elderly-fall Detection
7
0 10 20 30 40 50 60 70 80 90
-40
-20
0
20
40
60
80
100
120
140


Sparse coeIIicients
Largest entry
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
0
50
100
150


Error entries
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
0
50
100
150


Error entries
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
0
50
100
150


Error entries
Index oI CoeIIicients
A
m
p
l
i
t
u
d
e




Index oI CoeIIicients
A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




Index oI CoeIIicients Index oI CoeIIicients
Index oI CoeIIicients Index oI CoeIIicients
0 10 20 30 40 50 60 70 80 90
-40
-20
0
20
40
60
80
100
120
140


Sparse coeIIicients
Largest entry
0 10 20 30 40 50 60 70 80 90
-40
-20
0
20
40
60
80
100
120
140


Sparse coeIIicients
Largest entry
Figure 8. Illustration of the recognition without obstruction. (a) The test silhouette images.(b) Estimated spare coefficients ˆ w.
(c) Estimated error coefficients ˆ e.
Recovered by LP Recovered by OMP
Percent obstructed(%) 10 20 30 40 50 10 20 30 40 50
Fall recognition(%) 100 100 98.33 96.67 55 100 100 98.16 95.97 57
Fall detected as normal pose(%) 0 0 0 0 0 0 0 0 0 0
Rejection(%) 0 0 1.67 3.33 45 0 0 1.84 4.03 43
Normal pose recognition(%) 100 100 100 98.33 57.5 100 100 99.65 98.14 56.5
Normal pose detected as fall(%) 0 0 0 0 0 0 0 0 0 0
Rejection(%) 0 0 0 1.67 42.5 0 0 0.35 1.86 43.5
Table 1. Recognition rate with random pixel obstruction
Recovered by LP Recovered by OMP
Percent obstructed(%) 10 20 30 40 50 10 20 30 40 50
Fall recognition(%) 100 100 98.33 95 88.33 100 99.04 96.76 93.5 86.45
Fall detected as normal pose(%) 0 0 0 0 0 0 0 0 0 0
Rejection(%) 0 0 0 0 0 0 0.96 3.24 6.5 13.55
Normal pose recognition(%) 100 99.17 97.5 93.33 79.17 100 98.86 97.1 92.86 76.65
Normal pose detected as fall(%) 0 0 0.83 2.5 5 0 0 0.84 3.3 6.5
Rejection(%) 0 0.83 1.67 4.17 15.83 0 1.14 2.06 3.84 16.85
Table 2. Recognition rate with random block obstructions
Int J Adv Robot Syst, 2014, 11:42 | doi: 10.5772/57318 8
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
-5
0
5
10
15
20
25
30
35
40
45
50


Error entries
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
0
20
40
60
80
100
120
140
160
180
200


Error entries
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
-5
0
5
10
15
20


Error entries
(a) (b) (c) (d) (e)
Index oI CoeIIicients
A
m
p
l
i
t
u
d
e




Index oI CoeIIicients
A
m
p
l
i
t
u
d
e




Index oI CoeIIicients
A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




Index oI CoeIIicients
Index oI CoeIIicients
Index oI CoeIIicients
0 10 20 30 40 50 60 70 80 90
-50
0
50
100
150
200


Sparse coeIIicients
Largest entry
0 10 20 30 40 50 60 70 80 90
-5
0
5
10
15
20


Sparse coeIIicients
Largest entry
0 10 20 30 40 50 60 70 80 90
-10
0
10
20
30
40
50


Sparse coeIIicients
Largest entry
Figure 9. Illustration of the recognition with random pixel obstruction. (a) The raw test silhouette images. (b) The masks used
for simulating random pixel obstruction. (c) The silhouette images with random pixel obstruction. (d) Estimated spare coefficients
ˆ w. (e) Estimated error coefficients ˆ e.
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
0
10
20
30
40
50
60
70
80
90
100


Error entries
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
0
10
20
30
40
50
60
70
80
90
100


Error entries
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
20 40 60 80 100 120 140
2
4
6
8
10
12
14
16
18
20
500 1000 1500 2000 2500 3000
0
50
100
150


Error entries
(a) (b) (c) (d) (e)
Index oI CoeIIicients Index oI CoeIIicients
Index oI CoeIIicients
Index oI CoeIIicients
Index oI CoeIIicients
Index oI CoeIIicients
A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




A
m
p
l
i
t
u
d
e




0 10 20 30 40 50 60 70 80 90
-20
0
20
40
60
80
100


Sparse coeIIicients
Largest entry
0 10 20 30 40 50 60 70 80 90
-50
0
50
100
150


Sparse coeIIicients
Largest entry
0 10 20 30 40 50 60 70 80 90
-20
0
20
40
60
80
100


Sparse coeIIicients
Largest entry
Figure 10. Illustration of the recognition with random block obstructions. (a) The raw test silhouette images. (b) The masks used for
simulating random block obstructions. (c) The silhouette images with block obstructions. (d) Estimated spare coefficients ˆ w. (e) Estimated
error coefficients ˆ e.
possible, the posture may compromise the structure of the
data acquired, which is a limitation of this fall detection
method. If the robot has difficulty accessing the frontal
position in real application scenarios, we can deploy a
similarly-structured PIR array on the ground horizontally
to acquire a tangent silhouette, or a help button to assist
the fall detection. Second, in practical usage, if the
service environment contains certain heating sources at
Tong Liu and Jun Liu: Mobile Robot Aided Silhouette Imaging
and Robust Body Pose Recognition for Elderly-fall Detection
9
body temperature, they will interfere with the silhouette
imaging process. Therefore, the proposed system will
have a better practical performance in more controlled
environments such as nursing homes.
While the proposed methods do not help to prevent
falls or decrease the number of falls occurring in the home,
they may provide a sense of comfort and reassurance to the
elderly. If an emergency occurred, immediate assistance
and care would be available to them. The proposed
sensing model is not only advantageous in providing a
low-cost, non-invasive motion sensing method, which
will not interfere with the lighting condition, it may also
become a ubiquitous agent for healthcare applications.
6. Acknowledgements
The authors would like to thank the anonymous reviewers
for their constructive comments and suggestions. They
also wish to thank all the staff of the Information
Processing & Human-Robot Systems lab in Sun Yat-sen
University for their aid in conducting the measurement
experiments. This work is partly supported by
the National Natural Science Foundation of LiaoNing
Province (grant no.2013020008) and the National Natural
Science Foundation of China (grant no.61074167).
7. References
[1] World Health Day 2012 - Good health adds life to years
(2012). World Health Organization.
[2] Chan BK, Marshall LM, Winters KM, Faulkner KA,
Schwartz AV, and Orwoll ES (2007) Incident fall risk
and physical activity and physical performance among
older men. American Journal of Epidemiology. 165(6):
696–703.
[3] Noury N, Fleury A, Rumeau P, Bourke AK, Laighin
GO, Rialle V, and Lundy JE (2007) Fall detection -
principles and methods. Proceedings of 29th Annual
International Conference of the IEEE Engineering in
Medicine and Biology Society. pp.1663 –1666.
[4] Karantonis DM, Narayanan MR, Mathie M, Lovell
NH, and Celler BG (2006) Implementation of a
real-time human movement classifier using a
triaxial accelerometer for ambulatory monitoring.
IEEE Transactions on Information Technology in
Biomedicine. 10(1): 156–167.
[5] Lai CF, Huang YM, Park JH, and Chao HC (2010)
Adaptive body posture analysis for elderly-falling
detection with multisensors. IEEE Intelligent Systems.
25(2):20 –30.
[6] Bourke AK, O’Brien JV, and Lyons GM (2007)
Evaluation of a threshold-based tri-axial accelerometer
fall detection algorithm. Gait & Posture. 26(2): 194 –
199.
[7] Bourke AK, Lyons GM (2008) A threshold-based
fall-detection algorithm using a bi-axial gyroscope
sensor. Medical Engineering & Physics. 30(1): 84 – 90.
[8] Hori T, Nishida Y, Aizawa H, Murakami S, and
Mizoguchi H (2004) Sensor network for supporting
elderly care home. Proceedings of IEEE Sensors. 2:575
–578.
[9] Lee YS, Chung WY (2008) Novel video sensor based
fall detection of the elderly using double-difference
image and temporal template. Sensor Letters. 6(2):
352–357.
[10] Lee YS, Chung WY (2011) Vision sensor based fall
incident detection of elderly persons in real-time
healthcare surveillance system. Sensor Letters. 9(1):
162–169.
[11] Han J, Bhanu B (2005) Human activity recognition
in thermal infrared imagery. Proceedings of the 2005
IEEE Computer Society Conference on Computer
Vision and Pattern Recognition- Workshops. 3,pp.17.
[12] Ming D, Xue Z, Meng L, Wan B, Hu Y, and Luk
KDK (2009) Identification of humans using infrared
gait recognition. Proceedings of the 2009 IEEE
International Conference on Virtual Environments,
Human-Computer Interfaces and Measurement
Systems. pp.319–322.
[13] Kim D, Lee S, and Paik J (2009) Active shape
model-based gait recognition using infrared images.
Signal Processing, Image Processing and Pattern
Recognition. 61:275–281.
[14] Harmo P, Taipalus T, Knuuttila J, Vallet J, and Halme
A (2005). Needs and solutions - home automation
and service robots for the elderly and disabled.
Proceedings of International Conference on Intelligent
Robots and Systems. pp.3201 – 3206.
[15] Fong T, Nourbakhsh I, and Dautenhahn K (2003)
A survey of socially interactive robots. Robotics and
Autonomous Systems. 42: 143 – 166.
[16] Hao Q, Hu F, and Xiao Y (2009) Multiple human
tracking and identification with wireless distributed
pyroelectric sensor systems. IEEE Systems Journal.
3(4): 428 –439.
[17] Hao Q, Brady DJ, Guenther BD, Burchett JB, Shankar
M, and Feller S (2006) Human tracking with wireless
distributed pyroelectric sensors. IEEE Sensors Journal.
6(6): 1683 –1696.
[18] Wright J, Yang AY, Ganesh A, Sastry SS, and
Ma Y (2009) Robust face recognition via sparse
representation. IEEE Transactions on Pattern Analysis
and Machine Intelligence. 31(2): 210 –227.
[19] Mei X, Ling H (2011) Robust visual tracking
and vehicle classification via sparse representation.
IEEE Transactions on Pattern Analysis and Machine
Intelligence. 33(11): 2259-2272.
[20] Hossain A ,Rashid MH (1991) Pyroelectric detectors
and their applications. IEEE Transactions on Industry
Applications. 27(5): 824 –829.
[21] Burchett J, Shankar M, Hamza AB, Guenther BD,
Pitsianis N, and Brady DJ (2006) Lightweight biometric
detection system for human classification using
pyroelectric infrared detectors. Applied Optics. 45(13):
3031–3037.
[22] Liu T and Liu J (2012) Feature-specific biometric
sensing using ceiling view based pyroelectric infrared
sensors. EURASIP Journal on Advances in Signal
Processing. 2012:206.
[23] Fang JS, Hao Q, Brady DJ, Guenther BD, and
Hsu KY (2006) Real-time human identification using
a pyroelectric infrared detector array and hidden
Markov models. Optics Express. 14(15): 6643–6658.
Int J Adv Robot Syst, 2014, 11:42 | doi: 10.5772/57318 10
[24] Fang JS, Hao Q, Brady DJ, Guenther BD, and Hsu
KY (2007) A pyroelectric infrared biometric system for
real-time walker recognition by use of a maximum
likelihood principal components estimation (mlpce)
method. Optics Express. 15(6): 3271–3284.
[25] Hosokawa T, Kudo M, Nonaka H, and Toyama J
(2009) Soft authentication using an infrared ceiling
sensor network. Pattern Analysis and Applications.
12(3): 237–249.
[26] Sartain RB (2008) Profiling sensor for ISR
applications. SPIE Proceedings. 6963: 69630Q.
[27] Russomanno DJ, Chari S, Jacobs EL, and Halford C
(2010) Near-IR sparse detector sensor for intelligent
electronic fence applications. IEEE Sensors Journal.
10(6): 1106 –1107.
[28] Russomanno DJ, Chari S, and Halford C(2008) Sparse
detector imaging sensor with two-class silhouette
classification. Sensors. 8(12): 7996-8015.
[29] Jacobs EL, Chari S, Halford C, and McClellan H(2009)
Pyroelectric sensors and classification algorithms for
border / perimeter security. SPIE Proceedings. 7481:
7481P.
[30] White III WE, Brown JB, Chari S, and Jacobs EL (2010)
Real-time assessment of a linear pyroelectric sensor
array for object classification. SPIE Proceedings. 7834:
783403.
[31] Brady DJ, Pitsianis NP, and Sun X (2004) Reference
structure tomography. Journal of the Optical Society of
America A. 21(7): 1140–1147.
[32] http://pirsensor.bloombiz.com, 2011.
[33] Lord S, Chastin SFM, McInnes L, Little L, Briggs
P, and Rochester L (2011) Exploring patterns
of daily physical and sedentary behaviour in
community-dwelling older adults. Age and Ageing.
40(2): 205–210.
[34] Seguin R, LaMonte M, Tinker L, Liu J, Woods N,
Michael YL, Bushnell C, and LaCroix AZ (2012)
Sedentary behavior and physical function decline in
older women: Findings from the women’s health
initiative. Journal of Aging Research. Article ID
271589.
[35] Donoho DL (2006) Compressed sensing. IEEE
Transactions on Information Theory. 52(4): 1289 –1306.
[36] Candes EJ, Wakin MB (2008) An introduction
to compressive sampling. IEEE Signal Processing
Magazine. 25(2): 21 –30.
[37] Tropp JA, Gilbert AC (2007) Signal recovery from
random measurements via orthogonal matching
pursuit. IEEE Transactions on Information Theory.
53(12): 4655–4666.
[38] Grant M, Boyd S (2011) CVX: Matlab software
for disciplined convex programming, version 1.21.
Available: http://cvxr.com/cvx.
Tong Liu and Jun Liu: Mobile Robot Aided Silhouette Imaging
and Robust Body Pose Recognition for Elderly-fall Detection
11

Sponsor Documents

Or use your account on DocShare.tips

Hide

Forgot your password?

Or register your new account on DocShare.tips

Hide

Lost your password? Please enter your email address. You will receive a link to create a new password.

Back to log-in

Close