Sumith Document
Content
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
1. INTRODUCTION
The ever increasing numbers of traffic accidents all over the world are due to diminished driver’s vigilance level. Drivers with a diminished vigilance level suffer from a marked decline in their perception; recognition and vehicle control abilities & therefore pose a serious danger to their own lives and the lives of the other people. For this reason, developing systems that actively monitors the driver’s level of vigilance and alerting the driver of any insecure driving condition is essential for accident prevention. Many efforts have been reported in the literature for developing an active safety system for reducing the number of automobiles accidents due to reduced vigilance. Drowsiness in drivers can be generally divided into the following categories: Sensing of physiological characteristics. 1. Sensing of driver operation 2. Sensing of vehicle response. 3. Monitoring the response of driver. Among these methods, the techniques based on human physiological phenomena are the most accurate. This technique is implemented in two ways: Measuring changes in physiological signals, such as brain waves, heart rate, and eye blinking. And measuring physical changes such as sagging posture, leaning of the driver’s head and the open/closed states of the eyes. The first technique, while most accurate, is not realistic, since sensing electrodes would have to be attached directly on to the driver’s body, and hence be annoying and distracting to the driver. In addition, long time driving would result in perspiration on the sensors, diminishing their ability to monitor accurately. The second technique is wellsuited for real world driving conditions since it can be non-intrusive by using video cameras to detect changes. Driver operation and vehicle behavior can be implemented by monitoring the steering wheel movement, accelerator or brake patterns, vehicle speed, lateral acceleration, and lateral displacement. These too are nonintrusive ways of detecting drowsiness, but are limited to vehicle type and driver condition. The final technique for detecting drowsiness is by monitoring the response of the driver. This involves periodically requesting the driver to send a response to KCEA ECE Page 1
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
the system to indicate alertness. The problem with this technique is that it will eventually become tiresome and annoying to the driver. The propose system based on eyes closer count & yawning count of the driver. By monitoring the eyes and mouth, it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident. The eye blink frequency increases beyond the normal rate in the fatigued state. In addition, micro sleeps that are the short periods of sleep lasting 3 to 4 seconds are the good indicator of the fatigued state, but it is difficult to predict the driver fatigue accurately or reliably based only on single driver behavior. Additionally, the changes in a driver’s performance are more complicated and not reliable so in this system second parameter is also considered which a yawning count is. In order to detect fatigue probability the facial expression parameters must be extracted first.
KCEA
ECE
Page 2
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
2. WORKING PRINCIPLE
Concept: Sleep related accidents tend to be more severe, possibly because of the higher speeds involved and because the driver is unable to take any avoiding action, or even brake, prior to the collision. Horne describes typical sleep related accidents as ones where the driver runs off the road or collides with another vehicle or an object, without any sign of hard braking before the impact. In 2002, the National Highway Traffic Safety Administration (NHTSA) estimated that 35 percent of all traffic deaths occurred in crashes in which at least one driver or no occupant had a BAC(Blood Alcohol Content) of 0.08 percent or more and that any alcohol was present in 41 percent of all fatal crashes in 2002.Such statistics are sometimes cited as proof that a third to half of all fatal crashes are caused by "drunk driving" and that none of the crashes that involve alcohol would occur if the alcohol were not present. But this is incorrect and misleading because alcohol is only one of several factors that contribute to crashes involving drinking drivers. Furthermore, some fatally injured people in alcohol-related crashes are pedestrians with positive BACs, and these fatalities still would occur even if every driver were sober. Distracted driving is a top danger behind the wheel. In fact, about eight out of 10 crashes involve some sort of driver inattention within three seconds of that crash. We've all seen it and likely even done it, driving distracted includes anything from talking on the phone, to messing with your music, to attending to your children or even pets. All of these actions can lead to serious consequences. Martha Meade with AAA Mid-Atlantic says, "People are dying because of a simple missed phone call, a dropped toy or some other event that is completely not important." Possible techniques for detecting drowsiness in drivers can be generally divided into the following categories: sensing of physiological characteristics, sensing of driver operation, sensing of vehicle response, monitoring the response of driver. Monitoring Physiological Characteristics Among these methods, the techniques that are best, based on accuracy are the ones based on human physiological phenomena. This technique is implemented in two ways: measuring changes in physiological signals, such as brain waves, heart rate, and eye blinking; and measuring physical changes such as sagging posture, leaning of the driver’s head and the KCEA ECE Page 3
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
open/closed states of the eyes. The first technique, while most accurate, is not realistic, since sensing electrodes would have to be attached directly onto the driver’s body, and hence be annoying and distracting to the driver. In addition, long time driving would result in perspiration on the sensors, diminishing their ability to monitor accurately. The second technique is well suited for real world driving conditions since it can be non-intrusive by using optical sensors of video cameras to detect changes. Other Methods Driver operation and vehicle behavior can be implemented by monitoring the steering wheel movement, accelerator or brake patterns, vehicle speed, lateral acceleration, and lateral displacement. These too are non-intrusive ways of detecting drowsiness, but are limited to vehicle type and driver conditions. The final technique for detecting drowsiness is by monitoring the response of the driver. This involves periodically requesting the driver to send a response to the system to indicate alertness. The problem with this technique is that it will eventually become tiresome and annoying to the driver. EYE BLINK DETECTION It is necessary in our working to find the blinking of eye, since it is used to drive the device and to operate events. So blink detection has to be done, for which we can avail readily available blink detectors in market (Catalog No. 9008 of Enable devices) or we can incorporate it with a special instruction written in image processing that, if there is no pupil found for the certain period of pre-determined i.e. time greater than the human eye blinking time then consider an event called “blink”, for which the set of operations will be followed. Here, in this case we need to set time as 1 second or above it, as “blink event” is different from “normal eye blinking”. We need to perform testing for only blink event estimation, and not to find normal eye blinking.
KCEA
ECE
Page 4
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig. 1. Module for eye blinks detection
KCEA
ECE
Page 5
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig. 2. Shows The Different Blinks
KCEA
ECE
Page 6
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig 1 shows the setup of IR sensors & camera module that is to be used by the driver for Eye blink detection. Fig 2. Shows the different blink events which differ from normal blinking of eye using cumulative index (CI) & Mutual index(MI) which is obtained at receiver of IR sensor in terms of Current & voltage and plotted on graph. The signal can be smoothened using above graph to avoid unnecessary blinking event other than effective blinking event. MOVEMENT DETECTION Head movement detection is done through single step Accelerometer eg: ADXL330 which measures 3-axis detection. It consists of angle based accelerometer (ACC) input to simulate accurate head movement. Angle based approach does not require any pattern matching algorithms. ACC input signal is smoothened first to ignore any unwanted movement. Angle based model is believed to effective by researchers, which is debatable considering practical use of a single ACC sensor on head.
Fig. 3. Sample tilt angle versus time plots for (a) Left (b) Right turns
KCEA
ECE
Page 7
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig 3. Shows the angle versus time plot for the movement detection in the two direction. The results obtained are test results for only one direction to smoothened the signal to avoid unwanted detection in movement other than effective movement. Definition of Tilt Angle This system uses head movements as the sole input method; more precisely head’s tilt angles are used. Head tilt angles define how much the head is rotated along an axis. There are three possible head tilt movements, which are shown in Figure 4, and they are defined as: 1. Pitch, the vertical head rotation movement (as in looking up or down) 2. Roll, the head rotation that occurs when tilting head towards the shoulders Yaw, the horizontal head rotation movement (as in looking to left or right).
Fig. 4. Three possible head tilt movements
KCEA
ECE
Page 8
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
For the movement analysis, it is needed to somehow translate the tilt angle data to displacement of mouse cursor that is calculating new head position. There are two main methods when calculating the new head cursor position: 1. Absolute mapping in which every tilt angle corresponds to a position on screen. 2. Relative mapping in which every tilt angle corresponds to a head displacement amount (step size) and this amount is summed by the coordinates of the head’s old position, to calculate new position.
Fig. 5. Accelerometer Placement. Fig. 5 shows the real time placement of the accelerometer on the driver head for the 3axis detection of the head movement while driving to monitor fatigue by converting the angle based input to voltage output (in milli volts) for accurate detection.
KCEA
ECE
Page 9
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
3. BLOCK DIAGRAM
KCEA
ECE
Page 10
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
4. HARDWARE DESCRIPTION
4.1 IR Sensing Circuit:
Infrared transmitter is one type of LED which emits infrared rays generally called as IR Transmitter. Similarly IR Receiver is used to receive the IR rays transmitted by the IR transmitter. One important point is both IR transmitter and receiver should be placed straight line to each other. The transmitted signal is given to IR transmitter whenever the signal is high, the IR transmitter LED is conducting it passes the IR rays to the receiver. The IR receiver is connected with comparator. The comparator is constructed with LM 358 operational amplifier. In the
KCEA
ECE
Page 11
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
comparator circuit the reference voltage is given to inverting input terminal. The non inverting input terminal is connected IR receiver. When interrupt the IR rays between the IR transmitter and receiver, the IR receiver is not conducting. So the comparator non inverting input terminal voltage is higher then inverting input. Now the comparator output is in the range of +5V. This voltage is given to microcontroller or PC and led so led will glow. When IR transmitter passes the rays to receiver, the IR receiver is conducting due to that non inverting input voltage is lower than inverting input. Now the comparator output is GND so the output is given to microcontroller or PC. This circuit is mainly used to for counting application, intruder detector etc. 4.2 Buzzer: A buzzer or beeper is a signaling device, usually electronic, typically used in automobiles, household appliances such as a microwave oven, or game shows. It most commonly consists of a number of switches or sensors connected to a control unit that determines if and which button was pushed or a preset time has lapsed, and usually illuminates a light on the appropriate button or control panel, and sounds a warning in the form of a continuous or intermittent buzzing or beeping sound. Initially this device was based on an electromechanical system which was identical to an electric bell without the metal gong (which makes the ringing noise). Often these units were anchored to a wall or ceiling and used the ceiling or wall as a sounding board. Another implementation with some AC-connected devices was to implement a circuit to make the AC current into a noise loud enough to drive a loudspeaker and hook this circuit up to a cheap 8-ohm speaker. Nowadays, it is more popular to use a ceramic-based piezoelectric sounder like a Sonalert which makes a high-pitched tone. Usually these were hooked up to "driver" circuits which varied the pitch of the sound or pulsed the sound on and off.
KCEA
ECE
Page 12
Accident Prevention Using Eye Blinking and Head Movement LM358 Comparator: Description:
09B41A0494
The LM358 consist of two independent, high gain, internally frequency compensated operational amplifiers which were designed specifically to operate from a single power supply over a wide range of voltage. Operation from split power supplies is also possible and the low power supply current drain is independent of the magnitude of the power supply voltage. Application areas include transducer amplifier, DC gain blocks and all the conventional OPAMP circuits which now can be easily implemented in single power supply systems. Features: • Internally Frequency Compensated for Unity Gain • Large DC Voltage Gain: 100dB • Wide Power Supply Range: LM358 3V~32V (or ±1.5V~ 16V) • Input Common Mode Voltage Range Includes Ground • Large Output Voltage Swing: 0V DC to Vcc -1.5V DC • Power Drain Suitable for Battery Operation. 4.3 Alcohol sensor: Description: This alcohol sensor is suitable for detecting alcohol concentration on your breath, just like your common breathalyzer. It has a high sensitivity and fast response time. Sensor provides an analog resistive output based on alcohol concentration. The drive circuit is very simple, all it needs is one resistor. A simple interface could be a 0-3.3V ADC. Character: High sensitivity Fast response and resume Long life and low cost KCEA ECE Page 13
Accident Prevention Using Eye Blinking and Head Movement Mini Size Configuration:
09B41A0494
Sensitivity Characteristics:
KCEA
ECE
Page 14
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig.2 is the test circuit. You could get resistance change from voltage change on fixed or adjustable load resistance. Normally, it will take several minutes preheating for sensor enter into stable working after electrified; or you could give 2.2±0.2V high voltage for 5-10secs before test, which make sensor easily stable. 4.4 Relay circuit: A relay is an electrically operated switch. Many relays use an electromagnet to operate a switching mechanism mechanically, but other operating principles are also used. Relays are used where it is necessary to control a circuit by a low-power signal (with complete electrical isolation between control and controlled circuits), or where several circuits must be controlled by one signal. The first relays were used in long distance telegraph circuits, repeating the signal coming in from one circuit and re-transmitting it to another. Relays were used extensively in telephone exchanges and early computers to perform logical operations. A type of relay that can handle the high power required to directly control an electric motor or other loads is called a contactor. Solid-state relays control power circuits with no moving parts, instead using a semiconductor device to perform switching. Relays with calibrated operating characteristics and sometimes multiple operating coils are used to protect electrical circuits from overload or faults; in modern electric power systems these functions are performed by digital instruments still called "protective relays".
KCEA
ECE
Page 15
Accident Prevention Using Eye Blinking and Head Movement 4.5 GSM Module:
09B41A0494
GSM (Global System for Mobile communication) is a digital mobile telephony system that is widely used in Europe and other parts of the world. GSM uses a variation of time division multiple access (TDMA) and is the most widely used of the three digital wireless telephony technologies (TDMA, GSM, and CDMA). GSM digitizes and compresses data, then sends it down a channel with two other streams of user data, each in its own time slot. It operates at either the 900 MHz or 1800 MHz frequency band. .
Modulation: Modulation is a form of change process where we change the input information into a suitable format for the transmission medium. We also changed the information by demodulating the signal at the receiving end. The GSM uses Gaussian Minimum Shift Keying (GMSK) modulation method.
KCEA
ECE
Page 16
Accident Prevention Using Eye Blinking and Head Movement Access Methods:
09B41A0494
Because radio spectrum is a limited resource shared by all users, a method must be devised to divide up the bandwidth among as many users as possible. GSM chose a combination of TDMA/FDMA as its method. The FDMA part involves the division by frequency of the total 25 MHz bandwidth into 124 carrier frequencies of 200 kHz bandwidth. One or more carrier frequencies are then assigned to each BS. Each of these carrier frequencies is then divided in time, using a TDMA scheme, into eight time slots. One time slot is used for transmission by the mobile and one for reception. They are separated in time so that the mobile unit does not receive and transmit at the same time. Transmission Rate: The total symbol rate for GSM at 1 bit per symbol in GMSK produces 270.833 K symbols/second. The gross transmission rate of the time slot is 22.8 Kbps. GSM is a digital system with an over-the-air bit rate of 270 kbps. Frequency Band: The uplink frequency range specified for GSM is 933 - 960 MHz (basic 900 MHz band only). The downlink frequency band 890 - 915 MHz (basic 900 MHz band only). Channel Spacing: This indicates separation between adjacent carrier frequencies. In GSM, this is 200 kHz. Speech Coding: GSM uses linear predictive coding (LPC). The purpose of LPC is to reduce the bit rate. The LPC provides parameters for a filter that mimics the vocal tract. The signal passes through this filter, leaving behind a residual signal. Speech is encoded at 13 kbps.
KCEA
ECE
Page 17
Accident Prevention Using Eye Blinking and Head Movement Duplex Distance:
09B41A0494
The duplex distance is 80 MHz. Duplex distance is the distance between the uplink and downlink frequencies. A channel has two frequencies, 80 MHz apart. Misc: Frame duration: 4.615 mS Duplex Technique: Frequency Division Duplexing (FDD) access mode previously known as WCDMA. Speech channels per RF channel 4.6 GPS Module: The Global Positioning System (GPS) is a technical marvel made possible by a group of satellites in earth orbit that transmit precise signals, allowing GPS receivers to calculate and display accurate location, speed, and time information to the user. By capturing the signals from three or more satellites (among a constellation of 31 satellites available), GPS receivers are able to triangulate data and pinpoint your location. With the addition of computing power, and data stored in memory such as road maps, points of interest, topographic information, and much more, GPS receivers are able to convert location, speed, and time information into a useful display format GPS was originally created by the United States Department of Defense (DOD) as a military application. The system has been active since the early 1980s, but began to become useful to civilians in the late 1990s. Consumer GPS has since become a multi-billion dollar industry with a wide array of products, services, and Internetbased utilities. GPS works accurately in all weather conditions, day or night, around the clock, and around the globe. There is no subscription fee for use of GPS signals. GPS signals may be blocked by dense forest, canyon walls, or skyscrapers, and they don’t penetrate indoor spaces well, so some locations may not permit accurate GPS navigation. KCEA ECE Page 18
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
GPS receivers are generally accurate within 15 meters, and newer models that use Wide Area Augmentation System (WAAS) signals are accurate within three meters. While the U.S. owned and operated GPS is currently the only active system, five other satellite-based global navigation systems are being developed by individual nations and by multi-nation consortiums.
Fig: Flow diagram of Tracing of location of vehicle
KCEA
ECE
Page 19
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
5. RESULTS
Some of the results of eye blink detector through graph are given below:
Fig 7 Show Eye Blink Fig 7 shows the saturation & excursion effects occurred before and after the blinking of eye to smoothen the signal.
Fig.8. Eyelid Closure Occurrences Fig.8. shows the “effective blinking event” for which lid closure is set 40% of closing of eye & above which if eye lid closes the event is occurred. Fig shows the lid closure % versus KCEA ECE Page 20
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Time before/after accident meanwhile which the time is used to prevent the accident by using various techniques for eg: Buzzing the Alarm, Making Fake call on drivers mobile, etc using various self developed algorithms.
Fig. 9. Accelerometer detection using reference angle
KCEA
ECE
Page 21
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
7. ADVANTAGES & DISADVANTAGES Advantages:
Component establishes interface with other drivers very easily. Life of the driver can be saved by locking the ignition system of the car. Traffic management can be maintained by reducing accidents and traffic jams can be avoided. Using GPS & GSM exact location of the Car can be traced on MAP.
Disadvantages:
KCEA
ECE
Page 22
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
8. APPLICATIONS
Automobiles. Security Guard Cabins. Operators at nuclear power plants where continuous monitoring is necessary. Pilots of airplane. Military application where high intensity monitoring of soldier is needed.
KCEA
ECE
Page 23
Accident Prevention Using Eye Blinking and Head Movement 9. FUTURE SCOPE
09B41A0494
This system only looks at the number of consecutive frames where the eyes are closed. At that point it may be too late to issue the warning. By studying eye movement patterns, it is possible to find a method to generate the warning sooner. Using 3D images is another possibility in finding the eyes. The eyes are the deepest part of a 3D image, and this maybe a more robust way of localizing the eyes. Instead of alarm we can use Automatic Braking System which will reduce the speed of the car. We can automatically park the car by first using Automatic braking system, which will slow down the car and simultaneously will turn on the parking lights of the car and will detect the parking space and will automatically park the car preventing from accident. Using Pressure sensor on the steering alarm or Automatic braking System can be set in case of drowsiness. By using wire-less technology such as Car Talk2000 If the driver gets a heart attack or he is drunk it will send signals to vehicles.
KCEA
ECE
Page 24
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
8. CONCLUSION
Eye based control will be the future of all types of device control, thus making the operation so comfortable and much easier with less human presence. Several risk operations can be easily performed with this type of application and further research and study on these areas will create a new trend of interacting with machines. Hence, a system to monitor fatigue by detecting eye blink & head movement was developed using self developed algorithms.
KCEA
ECE
Page 25
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
9. REFERENCES
1. Real-Time Non - intrusive Monitoring and Prediction of Driver Fatigue by Qiang Ji, Zhiwei Zhu, and Peilin Lan, IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 53, NO. 4, JULY 2004 2. Boston University Computer Science Technical Report No.2005-12 Real Time Eye Tracking and Blink Detection with USB Cameras Michael Chau and Margrit Betke, Computer Science Department Boston University Boston, MA 02215, USA { mikechau, [email protected]} May 12, 2005 3. IJCSNS International Journal of Computer Science and Network Security, VOL.9 No.3, March 2009, A Neuro-Genetic System Design for Monitoring Driver’s Fatigue N.G.Narole , Reserch Scholar,G.H.Raisoni College of engineering, Nagpur, Dr.P.R.Bajaj, Principal, G.H.Raisoni College of Engineering, Nagpur. 4. “Embedded Systems” by Raj Kamal, 2nd Edition, TMH. 5. Andrew sloss “ Arm System Developer guide”, Elsevier 6. Frank Vahid, “ Embedded system design “ , PHI 7. www.indiastudychannel.com/.../132270-3137-Eyeblink-report-Copy... 8. http://www.utwente.nl/gw/cpe/medewerkers_cpe/Verwey/Publicaties%20Prof.%20Dr.%20Ing.% 20Willem%20Verwey/2000_verwey_zaidel_pid.pdf 9. http://ntl.bts.gov/lib/16000/16600/16694/PB2000104521.pdf 10. http://ec.europa.eu/information_society/activities/intelligentcar/docs/icar/intelligent_car.pdf 11. http://www.waset.org/journals/waset/v49/v49-163.pdf
KCEA
ECE
Page 26
09B41A0494
1. INTRODUCTION
The ever increasing numbers of traffic accidents all over the world are due to diminished driver’s vigilance level. Drivers with a diminished vigilance level suffer from a marked decline in their perception; recognition and vehicle control abilities & therefore pose a serious danger to their own lives and the lives of the other people. For this reason, developing systems that actively monitors the driver’s level of vigilance and alerting the driver of any insecure driving condition is essential for accident prevention. Many efforts have been reported in the literature for developing an active safety system for reducing the number of automobiles accidents due to reduced vigilance. Drowsiness in drivers can be generally divided into the following categories: Sensing of physiological characteristics. 1. Sensing of driver operation 2. Sensing of vehicle response. 3. Monitoring the response of driver. Among these methods, the techniques based on human physiological phenomena are the most accurate. This technique is implemented in two ways: Measuring changes in physiological signals, such as brain waves, heart rate, and eye blinking. And measuring physical changes such as sagging posture, leaning of the driver’s head and the open/closed states of the eyes. The first technique, while most accurate, is not realistic, since sensing electrodes would have to be attached directly on to the driver’s body, and hence be annoying and distracting to the driver. In addition, long time driving would result in perspiration on the sensors, diminishing their ability to monitor accurately. The second technique is wellsuited for real world driving conditions since it can be non-intrusive by using video cameras to detect changes. Driver operation and vehicle behavior can be implemented by monitoring the steering wheel movement, accelerator or brake patterns, vehicle speed, lateral acceleration, and lateral displacement. These too are nonintrusive ways of detecting drowsiness, but are limited to vehicle type and driver condition. The final technique for detecting drowsiness is by monitoring the response of the driver. This involves periodically requesting the driver to send a response to KCEA ECE Page 1
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
the system to indicate alertness. The problem with this technique is that it will eventually become tiresome and annoying to the driver. The propose system based on eyes closer count & yawning count of the driver. By monitoring the eyes and mouth, it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident. The eye blink frequency increases beyond the normal rate in the fatigued state. In addition, micro sleeps that are the short periods of sleep lasting 3 to 4 seconds are the good indicator of the fatigued state, but it is difficult to predict the driver fatigue accurately or reliably based only on single driver behavior. Additionally, the changes in a driver’s performance are more complicated and not reliable so in this system second parameter is also considered which a yawning count is. In order to detect fatigue probability the facial expression parameters must be extracted first.
KCEA
ECE
Page 2
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
2. WORKING PRINCIPLE
Concept: Sleep related accidents tend to be more severe, possibly because of the higher speeds involved and because the driver is unable to take any avoiding action, or even brake, prior to the collision. Horne describes typical sleep related accidents as ones where the driver runs off the road or collides with another vehicle or an object, without any sign of hard braking before the impact. In 2002, the National Highway Traffic Safety Administration (NHTSA) estimated that 35 percent of all traffic deaths occurred in crashes in which at least one driver or no occupant had a BAC(Blood Alcohol Content) of 0.08 percent or more and that any alcohol was present in 41 percent of all fatal crashes in 2002.Such statistics are sometimes cited as proof that a third to half of all fatal crashes are caused by "drunk driving" and that none of the crashes that involve alcohol would occur if the alcohol were not present. But this is incorrect and misleading because alcohol is only one of several factors that contribute to crashes involving drinking drivers. Furthermore, some fatally injured people in alcohol-related crashes are pedestrians with positive BACs, and these fatalities still would occur even if every driver were sober. Distracted driving is a top danger behind the wheel. In fact, about eight out of 10 crashes involve some sort of driver inattention within three seconds of that crash. We've all seen it and likely even done it, driving distracted includes anything from talking on the phone, to messing with your music, to attending to your children or even pets. All of these actions can lead to serious consequences. Martha Meade with AAA Mid-Atlantic says, "People are dying because of a simple missed phone call, a dropped toy or some other event that is completely not important." Possible techniques for detecting drowsiness in drivers can be generally divided into the following categories: sensing of physiological characteristics, sensing of driver operation, sensing of vehicle response, monitoring the response of driver. Monitoring Physiological Characteristics Among these methods, the techniques that are best, based on accuracy are the ones based on human physiological phenomena. This technique is implemented in two ways: measuring changes in physiological signals, such as brain waves, heart rate, and eye blinking; and measuring physical changes such as sagging posture, leaning of the driver’s head and the KCEA ECE Page 3
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
open/closed states of the eyes. The first technique, while most accurate, is not realistic, since sensing electrodes would have to be attached directly onto the driver’s body, and hence be annoying and distracting to the driver. In addition, long time driving would result in perspiration on the sensors, diminishing their ability to monitor accurately. The second technique is well suited for real world driving conditions since it can be non-intrusive by using optical sensors of video cameras to detect changes. Other Methods Driver operation and vehicle behavior can be implemented by monitoring the steering wheel movement, accelerator or brake patterns, vehicle speed, lateral acceleration, and lateral displacement. These too are non-intrusive ways of detecting drowsiness, but are limited to vehicle type and driver conditions. The final technique for detecting drowsiness is by monitoring the response of the driver. This involves periodically requesting the driver to send a response to the system to indicate alertness. The problem with this technique is that it will eventually become tiresome and annoying to the driver. EYE BLINK DETECTION It is necessary in our working to find the blinking of eye, since it is used to drive the device and to operate events. So blink detection has to be done, for which we can avail readily available blink detectors in market (Catalog No. 9008 of Enable devices) or we can incorporate it with a special instruction written in image processing that, if there is no pupil found for the certain period of pre-determined i.e. time greater than the human eye blinking time then consider an event called “blink”, for which the set of operations will be followed. Here, in this case we need to set time as 1 second or above it, as “blink event” is different from “normal eye blinking”. We need to perform testing for only blink event estimation, and not to find normal eye blinking.
KCEA
ECE
Page 4
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig. 1. Module for eye blinks detection
KCEA
ECE
Page 5
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig. 2. Shows The Different Blinks
KCEA
ECE
Page 6
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig 1 shows the setup of IR sensors & camera module that is to be used by the driver for Eye blink detection. Fig 2. Shows the different blink events which differ from normal blinking of eye using cumulative index (CI) & Mutual index(MI) which is obtained at receiver of IR sensor in terms of Current & voltage and plotted on graph. The signal can be smoothened using above graph to avoid unnecessary blinking event other than effective blinking event. MOVEMENT DETECTION Head movement detection is done through single step Accelerometer eg: ADXL330 which measures 3-axis detection. It consists of angle based accelerometer (ACC) input to simulate accurate head movement. Angle based approach does not require any pattern matching algorithms. ACC input signal is smoothened first to ignore any unwanted movement. Angle based model is believed to effective by researchers, which is debatable considering practical use of a single ACC sensor on head.
Fig. 3. Sample tilt angle versus time plots for (a) Left (b) Right turns
KCEA
ECE
Page 7
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig 3. Shows the angle versus time plot for the movement detection in the two direction. The results obtained are test results for only one direction to smoothened the signal to avoid unwanted detection in movement other than effective movement. Definition of Tilt Angle This system uses head movements as the sole input method; more precisely head’s tilt angles are used. Head tilt angles define how much the head is rotated along an axis. There are three possible head tilt movements, which are shown in Figure 4, and they are defined as: 1. Pitch, the vertical head rotation movement (as in looking up or down) 2. Roll, the head rotation that occurs when tilting head towards the shoulders Yaw, the horizontal head rotation movement (as in looking to left or right).
Fig. 4. Three possible head tilt movements
KCEA
ECE
Page 8
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
For the movement analysis, it is needed to somehow translate the tilt angle data to displacement of mouse cursor that is calculating new head position. There are two main methods when calculating the new head cursor position: 1. Absolute mapping in which every tilt angle corresponds to a position on screen. 2. Relative mapping in which every tilt angle corresponds to a head displacement amount (step size) and this amount is summed by the coordinates of the head’s old position, to calculate new position.
Fig. 5. Accelerometer Placement. Fig. 5 shows the real time placement of the accelerometer on the driver head for the 3axis detection of the head movement while driving to monitor fatigue by converting the angle based input to voltage output (in milli volts) for accurate detection.
KCEA
ECE
Page 9
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
3. BLOCK DIAGRAM
KCEA
ECE
Page 10
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
4. HARDWARE DESCRIPTION
4.1 IR Sensing Circuit:
Infrared transmitter is one type of LED which emits infrared rays generally called as IR Transmitter. Similarly IR Receiver is used to receive the IR rays transmitted by the IR transmitter. One important point is both IR transmitter and receiver should be placed straight line to each other. The transmitted signal is given to IR transmitter whenever the signal is high, the IR transmitter LED is conducting it passes the IR rays to the receiver. The IR receiver is connected with comparator. The comparator is constructed with LM 358 operational amplifier. In the
KCEA
ECE
Page 11
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
comparator circuit the reference voltage is given to inverting input terminal. The non inverting input terminal is connected IR receiver. When interrupt the IR rays between the IR transmitter and receiver, the IR receiver is not conducting. So the comparator non inverting input terminal voltage is higher then inverting input. Now the comparator output is in the range of +5V. This voltage is given to microcontroller or PC and led so led will glow. When IR transmitter passes the rays to receiver, the IR receiver is conducting due to that non inverting input voltage is lower than inverting input. Now the comparator output is GND so the output is given to microcontroller or PC. This circuit is mainly used to for counting application, intruder detector etc. 4.2 Buzzer: A buzzer or beeper is a signaling device, usually electronic, typically used in automobiles, household appliances such as a microwave oven, or game shows. It most commonly consists of a number of switches or sensors connected to a control unit that determines if and which button was pushed or a preset time has lapsed, and usually illuminates a light on the appropriate button or control panel, and sounds a warning in the form of a continuous or intermittent buzzing or beeping sound. Initially this device was based on an electromechanical system which was identical to an electric bell without the metal gong (which makes the ringing noise). Often these units were anchored to a wall or ceiling and used the ceiling or wall as a sounding board. Another implementation with some AC-connected devices was to implement a circuit to make the AC current into a noise loud enough to drive a loudspeaker and hook this circuit up to a cheap 8-ohm speaker. Nowadays, it is more popular to use a ceramic-based piezoelectric sounder like a Sonalert which makes a high-pitched tone. Usually these were hooked up to "driver" circuits which varied the pitch of the sound or pulsed the sound on and off.
KCEA
ECE
Page 12
Accident Prevention Using Eye Blinking and Head Movement LM358 Comparator: Description:
09B41A0494
The LM358 consist of two independent, high gain, internally frequency compensated operational amplifiers which were designed specifically to operate from a single power supply over a wide range of voltage. Operation from split power supplies is also possible and the low power supply current drain is independent of the magnitude of the power supply voltage. Application areas include transducer amplifier, DC gain blocks and all the conventional OPAMP circuits which now can be easily implemented in single power supply systems. Features: • Internally Frequency Compensated for Unity Gain • Large DC Voltage Gain: 100dB • Wide Power Supply Range: LM358 3V~32V (or ±1.5V~ 16V) • Input Common Mode Voltage Range Includes Ground • Large Output Voltage Swing: 0V DC to Vcc -1.5V DC • Power Drain Suitable for Battery Operation. 4.3 Alcohol sensor: Description: This alcohol sensor is suitable for detecting alcohol concentration on your breath, just like your common breathalyzer. It has a high sensitivity and fast response time. Sensor provides an analog resistive output based on alcohol concentration. The drive circuit is very simple, all it needs is one resistor. A simple interface could be a 0-3.3V ADC. Character: High sensitivity Fast response and resume Long life and low cost KCEA ECE Page 13
Accident Prevention Using Eye Blinking and Head Movement Mini Size Configuration:
09B41A0494
Sensitivity Characteristics:
KCEA
ECE
Page 14
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Fig.2 is the test circuit. You could get resistance change from voltage change on fixed or adjustable load resistance. Normally, it will take several minutes preheating for sensor enter into stable working after electrified; or you could give 2.2±0.2V high voltage for 5-10secs before test, which make sensor easily stable. 4.4 Relay circuit: A relay is an electrically operated switch. Many relays use an electromagnet to operate a switching mechanism mechanically, but other operating principles are also used. Relays are used where it is necessary to control a circuit by a low-power signal (with complete electrical isolation between control and controlled circuits), or where several circuits must be controlled by one signal. The first relays were used in long distance telegraph circuits, repeating the signal coming in from one circuit and re-transmitting it to another. Relays were used extensively in telephone exchanges and early computers to perform logical operations. A type of relay that can handle the high power required to directly control an electric motor or other loads is called a contactor. Solid-state relays control power circuits with no moving parts, instead using a semiconductor device to perform switching. Relays with calibrated operating characteristics and sometimes multiple operating coils are used to protect electrical circuits from overload or faults; in modern electric power systems these functions are performed by digital instruments still called "protective relays".
KCEA
ECE
Page 15
Accident Prevention Using Eye Blinking and Head Movement 4.5 GSM Module:
09B41A0494
GSM (Global System for Mobile communication) is a digital mobile telephony system that is widely used in Europe and other parts of the world. GSM uses a variation of time division multiple access (TDMA) and is the most widely used of the three digital wireless telephony technologies (TDMA, GSM, and CDMA). GSM digitizes and compresses data, then sends it down a channel with two other streams of user data, each in its own time slot. It operates at either the 900 MHz or 1800 MHz frequency band. .
Modulation: Modulation is a form of change process where we change the input information into a suitable format for the transmission medium. We also changed the information by demodulating the signal at the receiving end. The GSM uses Gaussian Minimum Shift Keying (GMSK) modulation method.
KCEA
ECE
Page 16
Accident Prevention Using Eye Blinking and Head Movement Access Methods:
09B41A0494
Because radio spectrum is a limited resource shared by all users, a method must be devised to divide up the bandwidth among as many users as possible. GSM chose a combination of TDMA/FDMA as its method. The FDMA part involves the division by frequency of the total 25 MHz bandwidth into 124 carrier frequencies of 200 kHz bandwidth. One or more carrier frequencies are then assigned to each BS. Each of these carrier frequencies is then divided in time, using a TDMA scheme, into eight time slots. One time slot is used for transmission by the mobile and one for reception. They are separated in time so that the mobile unit does not receive and transmit at the same time. Transmission Rate: The total symbol rate for GSM at 1 bit per symbol in GMSK produces 270.833 K symbols/second. The gross transmission rate of the time slot is 22.8 Kbps. GSM is a digital system with an over-the-air bit rate of 270 kbps. Frequency Band: The uplink frequency range specified for GSM is 933 - 960 MHz (basic 900 MHz band only). The downlink frequency band 890 - 915 MHz (basic 900 MHz band only). Channel Spacing: This indicates separation between adjacent carrier frequencies. In GSM, this is 200 kHz. Speech Coding: GSM uses linear predictive coding (LPC). The purpose of LPC is to reduce the bit rate. The LPC provides parameters for a filter that mimics the vocal tract. The signal passes through this filter, leaving behind a residual signal. Speech is encoded at 13 kbps.
KCEA
ECE
Page 17
Accident Prevention Using Eye Blinking and Head Movement Duplex Distance:
09B41A0494
The duplex distance is 80 MHz. Duplex distance is the distance between the uplink and downlink frequencies. A channel has two frequencies, 80 MHz apart. Misc: Frame duration: 4.615 mS Duplex Technique: Frequency Division Duplexing (FDD) access mode previously known as WCDMA. Speech channels per RF channel 4.6 GPS Module: The Global Positioning System (GPS) is a technical marvel made possible by a group of satellites in earth orbit that transmit precise signals, allowing GPS receivers to calculate and display accurate location, speed, and time information to the user. By capturing the signals from three or more satellites (among a constellation of 31 satellites available), GPS receivers are able to triangulate data and pinpoint your location. With the addition of computing power, and data stored in memory such as road maps, points of interest, topographic information, and much more, GPS receivers are able to convert location, speed, and time information into a useful display format GPS was originally created by the United States Department of Defense (DOD) as a military application. The system has been active since the early 1980s, but began to become useful to civilians in the late 1990s. Consumer GPS has since become a multi-billion dollar industry with a wide array of products, services, and Internetbased utilities. GPS works accurately in all weather conditions, day or night, around the clock, and around the globe. There is no subscription fee for use of GPS signals. GPS signals may be blocked by dense forest, canyon walls, or skyscrapers, and they don’t penetrate indoor spaces well, so some locations may not permit accurate GPS navigation. KCEA ECE Page 18
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
GPS receivers are generally accurate within 15 meters, and newer models that use Wide Area Augmentation System (WAAS) signals are accurate within three meters. While the U.S. owned and operated GPS is currently the only active system, five other satellite-based global navigation systems are being developed by individual nations and by multi-nation consortiums.
Fig: Flow diagram of Tracing of location of vehicle
KCEA
ECE
Page 19
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
5. RESULTS
Some of the results of eye blink detector through graph are given below:
Fig 7 Show Eye Blink Fig 7 shows the saturation & excursion effects occurred before and after the blinking of eye to smoothen the signal.
Fig.8. Eyelid Closure Occurrences Fig.8. shows the “effective blinking event” for which lid closure is set 40% of closing of eye & above which if eye lid closes the event is occurred. Fig shows the lid closure % versus KCEA ECE Page 20
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
Time before/after accident meanwhile which the time is used to prevent the accident by using various techniques for eg: Buzzing the Alarm, Making Fake call on drivers mobile, etc using various self developed algorithms.
Fig. 9. Accelerometer detection using reference angle
KCEA
ECE
Page 21
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
7. ADVANTAGES & DISADVANTAGES Advantages:
Component establishes interface with other drivers very easily. Life of the driver can be saved by locking the ignition system of the car. Traffic management can be maintained by reducing accidents and traffic jams can be avoided. Using GPS & GSM exact location of the Car can be traced on MAP.
Disadvantages:
KCEA
ECE
Page 22
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
8. APPLICATIONS
Automobiles. Security Guard Cabins. Operators at nuclear power plants where continuous monitoring is necessary. Pilots of airplane. Military application where high intensity monitoring of soldier is needed.
KCEA
ECE
Page 23
Accident Prevention Using Eye Blinking and Head Movement 9. FUTURE SCOPE
09B41A0494
This system only looks at the number of consecutive frames where the eyes are closed. At that point it may be too late to issue the warning. By studying eye movement patterns, it is possible to find a method to generate the warning sooner. Using 3D images is another possibility in finding the eyes. The eyes are the deepest part of a 3D image, and this maybe a more robust way of localizing the eyes. Instead of alarm we can use Automatic Braking System which will reduce the speed of the car. We can automatically park the car by first using Automatic braking system, which will slow down the car and simultaneously will turn on the parking lights of the car and will detect the parking space and will automatically park the car preventing from accident. Using Pressure sensor on the steering alarm or Automatic braking System can be set in case of drowsiness. By using wire-less technology such as Car Talk2000 If the driver gets a heart attack or he is drunk it will send signals to vehicles.
KCEA
ECE
Page 24
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
8. CONCLUSION
Eye based control will be the future of all types of device control, thus making the operation so comfortable and much easier with less human presence. Several risk operations can be easily performed with this type of application and further research and study on these areas will create a new trend of interacting with machines. Hence, a system to monitor fatigue by detecting eye blink & head movement was developed using self developed algorithms.
KCEA
ECE
Page 25
Accident Prevention Using Eye Blinking and Head Movement
09B41A0494
9. REFERENCES
1. Real-Time Non - intrusive Monitoring and Prediction of Driver Fatigue by Qiang Ji, Zhiwei Zhu, and Peilin Lan, IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 53, NO. 4, JULY 2004 2. Boston University Computer Science Technical Report No.2005-12 Real Time Eye Tracking and Blink Detection with USB Cameras Michael Chau and Margrit Betke, Computer Science Department Boston University Boston, MA 02215, USA { mikechau, [email protected]} May 12, 2005 3. IJCSNS International Journal of Computer Science and Network Security, VOL.9 No.3, March 2009, A Neuro-Genetic System Design for Monitoring Driver’s Fatigue N.G.Narole , Reserch Scholar,G.H.Raisoni College of engineering, Nagpur, Dr.P.R.Bajaj, Principal, G.H.Raisoni College of Engineering, Nagpur. 4. “Embedded Systems” by Raj Kamal, 2nd Edition, TMH. 5. Andrew sloss “ Arm System Developer guide”, Elsevier 6. Frank Vahid, “ Embedded system design “ , PHI 7. www.indiastudychannel.com/.../132270-3137-Eyeblink-report-Copy... 8. http://www.utwente.nl/gw/cpe/medewerkers_cpe/Verwey/Publicaties%20Prof.%20Dr.%20Ing.% 20Willem%20Verwey/2000_verwey_zaidel_pid.pdf 9. http://ntl.bts.gov/lib/16000/16600/16694/PB2000104521.pdf 10. http://ec.europa.eu/information_society/activities/intelligentcar/docs/icar/intelligent_car.pdf 11. http://www.waset.org/journals/waset/v49/v49-163.pdf
KCEA
ECE
Page 26
