Evaluating HealthCare Service(rev).pptx
Content
Evaluating HealthCare Service
Public Health Department, Medical Faculty Brawijaya University
Problem Definition
Solution Generation
Intermediate & Outcome assesment
Impact evaluation
Resource Mobilisation
Impact Assesment
Implementation
Where is the place of Evaluation ?
Evaluation Benefit
The Impact of Evaluation on Finance
Cost Impact Process of care Positive Negative
Hospital Wide Nosocomial Infection Rate
High Risk environment (cardiac surgery) Operating Room Intensive Care Unit
No Infection Short LOS
Infection Long LOS
No Complication Short LOS No Complication Minimal Use
Complication Long LOS Reoperation High Consumption
Medication Delivery
Minimal Use
High Consumption
LOS : Length of Stay
The Impact of Evaluation on Health Status
70 Million Americans Benefit from Quality Measurement
96% of heart attack victims were prescribed beta-blocker treatment in 2005, up from 62% in 1996* 77.7% of children enrolled in private health plans received all recommended immunizations, up 5% from 72.5% in 2004* Evidence-based guidelines from the American College of Cardiology and the American Heart Association have reduced mortality among patients who have had a heart attack
* National Committee for Quality Assurance in US
Evidence Based Medicine (EBM)
(Example of Global Evaluation on Medical Treatment)
“The integration of individual clinical expertise with the best available clinical evidence from systematic research.”
David L Sackett, W Scott Richardson, William Rosenberg, R Brian Haynes Evidence Based Medicine--How to Practice and Teach EBM, 1996
Uses of “EBM”
Use of empirically-verified treatments in the care of patients Incorporation of research results into the process of care Ability to critically appraise research results
Motivation: EBM “Successes”
Theophylline and asthma
◦ We were doing the wrong thing
Littenberg, 1988
Beta blockers and Myocardial Infarcts
◦ We weren’t doing the right thing
Yusuf, 1987
Levels of Evidences
(I-1) a well done systematic review of 2 or more RCTs (I-2) a RCT (II-1) a cohort study
(II-2) a case-control study (II-3) a dramatic uncontrolled experiment (III) respected authorities, expert committees, etc.. (IV) ...someone once told me....
Note : RCT = Randomized Control Trial
What to measure (evaluate)
If We can not measure, we can not manage
Who Is the Healthiest One?
Health Model
Determinants of Health
Environment Physical Social Human Biology Life Style/Behavior Risk Factors Health Care System
Health Status
Subjective Health Mortality Morbidity Injury MCH/Others
Consequences of Health
Quality of Life Health Service Utilization Use of Medication Others
How To Measure Health? (Community Approach)
Mortality-based measures Crude mortality Age-specific mortality Age adjusted mortality Leading causes of deaths Life expectancy Years of potential life lost (YPLL)
How To Measure Health?
Morbidity and injury Risk behavioral factors Social and cultural influence Genetic factors
What to Measure ? (1)
Leadership defines priorities by answering such questions as these :
◦ What aspects of the organization are critical to its succeed ◦ What expenses are most and least profitable ◦ How can exelent patient outcomes be achieved efficiently and aconomically ◦ What variables influence patient satisfaction ?
What to Measure ? (2)
Developing a Performance Improvement Plan (Remember PDCA Cycle)
◦ ◦ ◦ ◦ Plan Do Check Act
What to Measure ? (3)
Monitoring Variation from the standard BOR,LOS,BTO,GDR,NDR
The concept of Quality in Health Care (HealthCare Organization
Approach)
Structure (input), process, outcome Quality Domains: effectiveness, responsiveness …. Quality of services versus quality of systems Health care helps individuals and populations to realize their potential health Quality relates to compliance of practices with evidence Quality relates to health outcomes
Source: Arah OA, et al. A conceptural framework for the OECD Health Care Quality Indicators Project. Int J Quality Health Care. 2006; Sep 18; Suppl.1:5-13.
The Key of Evaluation is Data
Data Sources
Administrative Data
Demographics, Diagnosis, Treatment, Medication, Laboratory, Mortality/morbidity, etc
Data Warehouse
(Storing and Organize Data)
Data Analyses
Primary Data
Adverse outcomes, inpatient indicators, infection control, APACHE for ICU,
Medical records,etc
* APACHE : Acute Physiology, age, and Chronic Health Evaluation)
“Regular” data Collection
Outcome Measures Implementation Measures Initial Indicator Set
Breast Cancer Survival Cervical Cancer Survival Colorectal Cancer Survival Asthma mortality rate AMI 30-day case fatality rate Stroke 30-day case fatality rate Incidence of Vaccine Preventable Diseases Mammography Screening Cervical Cancer Screening Coverage for basic vaccination Waiting time for femur fracture surgery Influenza vaccination for adults over 65 Smoking rates
Additional Indicator Set (in 2006 collection)
Lower extremity amputation rates for diabetics Transfusion reaction rate Postoperative hip fracture rate Uncontrolled diabetes admission rate Adult asthma admission rate Hypertension admission rate Annual eye examination for diabetics
Benign Prostate Hypertrophy
Health Status Indicator:
◦ Functional status: dysuria…
Health Determinants:
◦ Age ◦ Sex ◦ Other risk factor
Impact of BPH or treatment of BPH:
◦ Sexual dysfunction
BPH-community
Health Status:
◦ Morbidity: incidence or prevalence ◦ Disability
Health Determinants:
◦ Age specific morbidity
Health Impact
Hospital Care Evaluation for BPH
Input:
◦ Characteristic of patient ◦ Provider skill ◦ Standard of facility
Process:
◦ Waiting time in outpatient service ◦ Operation waiting time ◦ Readmission to OT
Output:
◦ Healing time
Impact:
◦ Quality of life including sexual life
Example : ICU Data Collected by System Hospitals
Indicators Number of admissions ICU LOS Number of mortalities Number of ventilated patients Number of readmission within 72 hours Number of low-risk patients (based on APCHE score) 2003 2004 2005
Example : Calculating the Impact of Sedation on Patient Falls
Opportunity Event
Quality Measure = Event / Oppurtunity = Numerator (N) / Denominator (D) N = Number of Sedated Patients who Fell D = Total Number of Patient Falls
“Medicine used to be simple, ineffective and relatively safe. Now it is complex, effective and potentially dangerous.”
Sir Cyril Chantler, former Dean Guy’s, King and St. Thomas’s Medical and Dental School, Lancet 1999
Public Health Department, Medical Faculty Brawijaya University
Problem Definition
Solution Generation
Intermediate & Outcome assesment
Impact evaluation
Resource Mobilisation
Impact Assesment
Implementation
Where is the place of Evaluation ?
Evaluation Benefit
The Impact of Evaluation on Finance
Cost Impact Process of care Positive Negative
Hospital Wide Nosocomial Infection Rate
High Risk environment (cardiac surgery) Operating Room Intensive Care Unit
No Infection Short LOS
Infection Long LOS
No Complication Short LOS No Complication Minimal Use
Complication Long LOS Reoperation High Consumption
Medication Delivery
Minimal Use
High Consumption
LOS : Length of Stay
The Impact of Evaluation on Health Status
70 Million Americans Benefit from Quality Measurement
96% of heart attack victims were prescribed beta-blocker treatment in 2005, up from 62% in 1996* 77.7% of children enrolled in private health plans received all recommended immunizations, up 5% from 72.5% in 2004* Evidence-based guidelines from the American College of Cardiology and the American Heart Association have reduced mortality among patients who have had a heart attack
* National Committee for Quality Assurance in US
Evidence Based Medicine (EBM)
(Example of Global Evaluation on Medical Treatment)
“The integration of individual clinical expertise with the best available clinical evidence from systematic research.”
David L Sackett, W Scott Richardson, William Rosenberg, R Brian Haynes Evidence Based Medicine--How to Practice and Teach EBM, 1996
Uses of “EBM”
Use of empirically-verified treatments in the care of patients Incorporation of research results into the process of care Ability to critically appraise research results
Motivation: EBM “Successes”
Theophylline and asthma
◦ We were doing the wrong thing
Littenberg, 1988
Beta blockers and Myocardial Infarcts
◦ We weren’t doing the right thing
Yusuf, 1987
Levels of Evidences
(I-1) a well done systematic review of 2 or more RCTs (I-2) a RCT (II-1) a cohort study
(II-2) a case-control study (II-3) a dramatic uncontrolled experiment (III) respected authorities, expert committees, etc.. (IV) ...someone once told me....
Note : RCT = Randomized Control Trial
What to measure (evaluate)
If We can not measure, we can not manage
Who Is the Healthiest One?
Health Model
Determinants of Health
Environment Physical Social Human Biology Life Style/Behavior Risk Factors Health Care System
Health Status
Subjective Health Mortality Morbidity Injury MCH/Others
Consequences of Health
Quality of Life Health Service Utilization Use of Medication Others
How To Measure Health? (Community Approach)
Mortality-based measures Crude mortality Age-specific mortality Age adjusted mortality Leading causes of deaths Life expectancy Years of potential life lost (YPLL)
How To Measure Health?
Morbidity and injury Risk behavioral factors Social and cultural influence Genetic factors
What to Measure ? (1)
Leadership defines priorities by answering such questions as these :
◦ What aspects of the organization are critical to its succeed ◦ What expenses are most and least profitable ◦ How can exelent patient outcomes be achieved efficiently and aconomically ◦ What variables influence patient satisfaction ?
What to Measure ? (2)
Developing a Performance Improvement Plan (Remember PDCA Cycle)
◦ ◦ ◦ ◦ Plan Do Check Act
What to Measure ? (3)
Monitoring Variation from the standard BOR,LOS,BTO,GDR,NDR
The concept of Quality in Health Care (HealthCare Organization
Approach)
Structure (input), process, outcome Quality Domains: effectiveness, responsiveness …. Quality of services versus quality of systems Health care helps individuals and populations to realize their potential health Quality relates to compliance of practices with evidence Quality relates to health outcomes
Source: Arah OA, et al. A conceptural framework for the OECD Health Care Quality Indicators Project. Int J Quality Health Care. 2006; Sep 18; Suppl.1:5-13.
The Key of Evaluation is Data
Data Sources
Administrative Data
Demographics, Diagnosis, Treatment, Medication, Laboratory, Mortality/morbidity, etc
Data Warehouse
(Storing and Organize Data)
Data Analyses
Primary Data
Adverse outcomes, inpatient indicators, infection control, APACHE for ICU,
Medical records,etc
* APACHE : Acute Physiology, age, and Chronic Health Evaluation)
“Regular” data Collection
Outcome Measures Implementation Measures Initial Indicator Set
Breast Cancer Survival Cervical Cancer Survival Colorectal Cancer Survival Asthma mortality rate AMI 30-day case fatality rate Stroke 30-day case fatality rate Incidence of Vaccine Preventable Diseases Mammography Screening Cervical Cancer Screening Coverage for basic vaccination Waiting time for femur fracture surgery Influenza vaccination for adults over 65 Smoking rates
Additional Indicator Set (in 2006 collection)
Lower extremity amputation rates for diabetics Transfusion reaction rate Postoperative hip fracture rate Uncontrolled diabetes admission rate Adult asthma admission rate Hypertension admission rate Annual eye examination for diabetics
Benign Prostate Hypertrophy
Health Status Indicator:
◦ Functional status: dysuria…
Health Determinants:
◦ Age ◦ Sex ◦ Other risk factor
Impact of BPH or treatment of BPH:
◦ Sexual dysfunction
BPH-community
Health Status:
◦ Morbidity: incidence or prevalence ◦ Disability
Health Determinants:
◦ Age specific morbidity
Health Impact
Hospital Care Evaluation for BPH
Input:
◦ Characteristic of patient ◦ Provider skill ◦ Standard of facility
Process:
◦ Waiting time in outpatient service ◦ Operation waiting time ◦ Readmission to OT
Output:
◦ Healing time
Impact:
◦ Quality of life including sexual life
Example : ICU Data Collected by System Hospitals
Indicators Number of admissions ICU LOS Number of mortalities Number of ventilated patients Number of readmission within 72 hours Number of low-risk patients (based on APCHE score) 2003 2004 2005
Example : Calculating the Impact of Sedation on Patient Falls
Opportunity Event
Quality Measure = Event / Oppurtunity = Numerator (N) / Denominator (D) N = Number of Sedated Patients who Fell D = Total Number of Patient Falls
“Medicine used to be simple, ineffective and relatively safe. Now it is complex, effective and potentially dangerous.”
Sir Cyril Chantler, former Dean Guy’s, King and St. Thomas’s Medical and Dental School, Lancet 1999
