Nursing 2
Content
Clinical Effectiveness in Nursing (2005) 9, 84–87
Clinical
Effectiveness
in Nursing
http://intl.elsevierhealth.com/journals/cein
A measurement issue in the assessment of social dysfunction and dysphoria in the third trimester of pregnancy
Wan Yim Ip a, Colin R. Martin
a
b,*
Faculty of Medicine, The Nethersole School of Nursing, Chinese University of Hong Kong, Esther Lee Building, Chung Chi College, Shatin, New Territories, Hong Kong SAR, China b Department of Mental Health and Learning Disability, University of Sheffield, UK
KEYWORDS
GHQ-12; Pregnancy; Psychometrics; Structural equation modeling
Summary Objective: Screening for psychological disturbance during pregnancy is clinically desirable, however the current battery of brief screening instruments have limited utility in this group. A short-version of the General Health Questionnaire (GHQ), the GHQ-8 comprising two sub-scales has recently been developed. The current investigation sought to determine if common covariates (age and level of education) impacted on item responsiveness within the two GHQ-8 sub-scales. Design: A cross sectional design was used with GHQ-8 item scores, age and level of education as dependent variables evaluated within a structural equation model. Setting: Women in the third trimester of pregnancy recruited from one obstetric out-patient clinic of a public teaching hospital in Hong Kong. Main outcome measure: The 8-item GHQ-8 [Kalliath, T.J., O’Driscoll, M.P., Brough, P., 2004. A confirmatory factor analysis of the General Health Questionnaire-12. Stress and Health 20, 11–20]. Results: The GHQ-8 was shown to have good factorial validity, however, one of the GHQ-8 items in the social dysfunction sub-scale was found to be differentially influenced by age. Conclusions: The GHQ-8 may have to be revised, or age-normed cut-off scores developed, in order to prevent screening accuracy of a clinically-applied version of the instrument being compromised by the age of the individual being screened. c 2006 Elsevier Ltd. All rights reserved.
* Corresponding author. Address: Department of Mental Health and Learning Disability, University of Sheffield, Humphry Davy House, Manvers S63 7ER, UK. Tel.: 0114 222 9943; fax: 0114 222 9601. E-mail address: C.R.Martin@sheffield.ac.uk (C.R. Martin).
Introduction
Identification of psychological distress during the course of pregnancy is an area of important clinical
1361-9004/$ - see front matter c 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.cein.2005.11.001
A measurement issue in the assessment of social dysfunction and dysphoria concern (Jomeen and Martin, 2004). Unfortunately, a number of standard screening instruments appear to be unsuitable for use during pregnancy (Karimova and Martin, 2003; Martin and Jomeen, 2003; Martin, 2005). An investigation of the 12-item General Health Questionnaire (GHQ-12; Goldberg and Williams, 1988) suggested this instrument may be unsuitable for screening use in this group due to problems with alternative scoring systems (Martin and Jomeen, 2003). However, a study by Martin and Newell (2005) has revealed that the alternative scoring systems used with the GHQ-12 are generally concordant. Kalliath et al. (2004) has convincingly demonstrated that the GHQ-12 can be reduced still further to an 8-item measure (GHQ-8) with the removal of redundant items. The GHQ-8 comprises two subscales, psychosocial dysfunction and anxiety/ depression each comprising four items. Kalliath et al. (2004) reported impressive psychometric qualities for the GHQ-8 suggesting this instrument may be suitable as a generic and accurate screening measure. One issue with a new screening tool relating to both validity and utility is the way the instrument responds to heterogeneity within the clinical group of interest. Ideally, an instrument should be invariant within the group in order that cut-points for case detection are both consistent and reliable, thus occurrence of false positives and highly undesirable false negatives are reduced. Given that the GHQ-8 represents a new version or revision of the GHQ-12, cut-scores are not currently available.1 However, the opportunity to explore psychometric issues of the impact of common covariates such as age and level of education in terms of possible confound is presented by the development of this tool. This facilitates identification of items within the measure which perform inconsistently and thus reveals the presence of a potential confounding factor. Cautions can thus be identified with the instrument when applied to particular sub-groups within a general clinical classification. This study seeks to examine the impact of two common covariates on the psychometric performance of the GHQ-8. The present study addresses the following research questions: 1. Does age have a significant impact on the psychometric properties of the GHQ-8? 2. Does level of education have a significant impact on the psychometric properties of the GHQ-8?
85
Method
154 nulliparous women recruited from one public hospital participated in the current investigation. Eligible subjects were at their third trimester of pregnancy and be able to read Chinese. The mean age of women providing complete data was 28.21 years (SD = 5.08). The mean gestational age at recruitment was 33.79 weeks (SD = 1.64). Level of education was measured on a three-point scale (secondary, higher secondary, tertiary). Age and level of education was chosen as the key covariates because these will be common to any clinical group where this instrument is used as a screening tool. Participants were administered the Chinese version of the GHQ-12 (Pan and Goldberg, 1990) and the GHQ-8 items were extracted for analysis based on the two sub-scale model of Kalliath et al. (2004). The study was approved by the Clinical Research Ethics Committees of the study hospital and The Chinese University of Hong Kong. All participants were volunteers in the study and gave written informed consent prior to enrollment. None of the participants were paid for participation in the investigation. Evaluation of the performance of sub-scales and sub-scale items in relation to the co-variates (age and level of education) was conducted using structural equation modeling (SEM).
Results
Mean scores and standard deviations of participants ratings on the social dysfunction sub-scale was 0.47 (SD = 0.86), and on the anxiety/depression sub-scale was 1.53 (SD = 1.45). The SEM model was analysed using Mplus version 3 (Muthe ´n and Muthe ´n, 1998-2004). Evaluation of the two-factor GHQ-8 revealed adequate fit, v2 = 41.92(df=31), p = 0.09, comparative fit index (CFI) = 0.95, and root mean squared error of approximation (RMSEA) = 0.05. However, investigation of modification indices revealed that model fit could be improved by specifying a direct regression of GHQ item-4 ‘felt capable of making decisions about things?’2 on to the covariate age. The resulting modified model fit, v2 = 37.16(df=30), p = 0.17, CFI = 0.97, and RMSEA = 0.04, was found to provide a statistically significantly fit to the data compared to the original model, v2 ðdiffÞ ¼ 4:76ðdf¼1Þ , p < 0.05. The structural model is shown in Fig. 1.
1 The GHQ-12 is scored so that high scores are associated with psychopathology and cross the threshold cut-score for a ‘case’.
2 Item numbers consistent with that of the GHQ-12 are used to allow comparison with that measure at the individual item level.
86
GHQ 4
0.01 0.13 0.24 0.05
W.Y. Ip, C.R. Martin
Error 0.09
GHQ 7 GHQ 8
Error 0.07
Age
0.02
Social 0.12 Dysfunction
0.15
Error 0.06
GHQ 12
0.64 0.51
Error 0.07
GHQ 6
0.23 0.29 0.26 0.18
Error 0.16
Education
Anxiety/ Depression
GHQ 9
0.39 0.30
Error 0.18
GHQ 10 GHQ 11
Error 0.08
Error 0.09
Figure 1 Two-factor 8-item model based on Kalliath et al. (2004). Estimates are standardised. Dashed line represents modification to model.
Discussion
The findings from the current study have a number of implications for the potential use of this instrument during pregnancy. The descriptive statistics reveal low mean sub-scale scores which would suggest that if the GHQ-8 was used in case detection, an alternative scoring system to the standard GHQ method (Goldberg and Williams, 1988) would be preferred. A suitable alternative scoring system, the likert system is however available (Goldberg and Williams, 1988) and would increase the range of scores and consequently the discriminate ability of a cut-score. The SEM findings are also illuminating as to the possible influence of covariates on this potential screening measure. A significant improvement in model fit was found by specifying that the GHQ-4 item is regressed directly on to the age covariate. This finding shows that increasing age is associated with a reduction in the score of this item. Consequently, for a given threshold score, the effect here is for comparatively older women to have a lower overall score due to differential deflation of the GHQ-4. This may be interpreted in two ways, for a given cut-off score comparatively older women who may be borderline cases within the group will be more likely to be undetected as cases as they are less likely to reach threshold. Alternatively, setting a more conservative threshold to improve case detection rates of older women is likely to be accompanied with an increased false-positive rate in younger women in this clinical group. Since the SEM reveals that only
the GHQ-4 item is influenced by age, either revision of this item may be desirable or age-normed cut-scores developed for the GHQ-8. The current study has demonstrated that inspite of good evidence for factorial validity in a measure, the performance of individual items within the scale may be significantly affected by general socio-demographic attributes. This is of particular importance in the context of clinical screening instruments since variant items may reduce the case detection accuracy of the tool. Health professionals using screening measures routinely may wish to consider salient aspects of the clinical group which may impact on the efficacy of such screening tools. Nurses are in a very good position to administer screening instruments, score them and integrate the findings into a clinical-decision making framework. However, as evidenced-based practitioners, nurses should be aware that screening instruments are not infaliable in terms of case detection accuracy, and that other important characteristics (age in the case of the current study) of the clinical group have a salient influence on the screening measure used. Awareness of these factors is very useful in both administration and interpretation of screening scores and in reminding health professionals that case positive/negative status determined by a questionnaire score is additional information in forming case management strategies; however the nurses clinical insight and experience retain a significant and central role in the identification of psychological disturbance during pregnancy.
A measurement issue in the assessment of social dysfunction and dysphoria
87
References
Goldberg, D.P., Williams, 1988. A User’s Guide to the General Health Questionnaire. NFER-Nelson, Basingstoke. Jomeen, J., Martin, C.R., 2004. Is the Hospital Anxiety and Depression Scale (HADS) a reliable screening tool in early pregnancy? Psychology and Health 19, 787–800. Kalliath, T.J., O’Driscoll, M.P., Brough, P., 2004. A confirmatory factor analysis of the General Health Questionnaire-12. Stress and Health 20, 11–20. Karimova, G., Martin, C.R., 2003. A psychometric evaluation of the Hospital Anxiety and Depression Scale during pregnancy. Psychology, Health and Medicine 8, 89–103. Martin, C.R., 2005. What does the Hospital Anxiety and Depression Scale (HADS) really measure in liaison
psychiatry settings? Current Psychiatry Reviews 1, 69– 73. Martin, C.R., Jomeen, J., 2003. Is the 12-item General Health Questionnaire (GHQ-12) confounded by scoring method during pregnancy and following birth? Journal of Reproductive and Infant Psychology 21, 267–278. Martin, C.R., Newell, R.J., 2005. Is the 12-item General Health Questionnaire (GHQ-12) confounded by scoring method in individuals with facial disfigurement? Psychology and Health 20, 651–659. Muthe ´n, L.K., Muthe ´n, B.O., 1998-2004. Mplus Users Guide, third ed. Muthe ´n and Muthe ´n, Los Angeles, C.A. Pan, P.C., Goldberg, D.P., 1990. A comparison of the validity of GHQ-12 and CHQ-12 in Chinese primary care patients in Manchester. Psychological Medicine 20, 931– 940.
Clinical
Effectiveness
in Nursing
http://intl.elsevierhealth.com/journals/cein
A measurement issue in the assessment of social dysfunction and dysphoria in the third trimester of pregnancy
Wan Yim Ip a, Colin R. Martin
a
b,*
Faculty of Medicine, The Nethersole School of Nursing, Chinese University of Hong Kong, Esther Lee Building, Chung Chi College, Shatin, New Territories, Hong Kong SAR, China b Department of Mental Health and Learning Disability, University of Sheffield, UK
KEYWORDS
GHQ-12; Pregnancy; Psychometrics; Structural equation modeling
Summary Objective: Screening for psychological disturbance during pregnancy is clinically desirable, however the current battery of brief screening instruments have limited utility in this group. A short-version of the General Health Questionnaire (GHQ), the GHQ-8 comprising two sub-scales has recently been developed. The current investigation sought to determine if common covariates (age and level of education) impacted on item responsiveness within the two GHQ-8 sub-scales. Design: A cross sectional design was used with GHQ-8 item scores, age and level of education as dependent variables evaluated within a structural equation model. Setting: Women in the third trimester of pregnancy recruited from one obstetric out-patient clinic of a public teaching hospital in Hong Kong. Main outcome measure: The 8-item GHQ-8 [Kalliath, T.J., O’Driscoll, M.P., Brough, P., 2004. A confirmatory factor analysis of the General Health Questionnaire-12. Stress and Health 20, 11–20]. Results: The GHQ-8 was shown to have good factorial validity, however, one of the GHQ-8 items in the social dysfunction sub-scale was found to be differentially influenced by age. Conclusions: The GHQ-8 may have to be revised, or age-normed cut-off scores developed, in order to prevent screening accuracy of a clinically-applied version of the instrument being compromised by the age of the individual being screened. c 2006 Elsevier Ltd. All rights reserved.
* Corresponding author. Address: Department of Mental Health and Learning Disability, University of Sheffield, Humphry Davy House, Manvers S63 7ER, UK. Tel.: 0114 222 9943; fax: 0114 222 9601. E-mail address: C.R.Martin@sheffield.ac.uk (C.R. Martin).
Introduction
Identification of psychological distress during the course of pregnancy is an area of important clinical
1361-9004/$ - see front matter c 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.cein.2005.11.001
A measurement issue in the assessment of social dysfunction and dysphoria concern (Jomeen and Martin, 2004). Unfortunately, a number of standard screening instruments appear to be unsuitable for use during pregnancy (Karimova and Martin, 2003; Martin and Jomeen, 2003; Martin, 2005). An investigation of the 12-item General Health Questionnaire (GHQ-12; Goldberg and Williams, 1988) suggested this instrument may be unsuitable for screening use in this group due to problems with alternative scoring systems (Martin and Jomeen, 2003). However, a study by Martin and Newell (2005) has revealed that the alternative scoring systems used with the GHQ-12 are generally concordant. Kalliath et al. (2004) has convincingly demonstrated that the GHQ-12 can be reduced still further to an 8-item measure (GHQ-8) with the removal of redundant items. The GHQ-8 comprises two subscales, psychosocial dysfunction and anxiety/ depression each comprising four items. Kalliath et al. (2004) reported impressive psychometric qualities for the GHQ-8 suggesting this instrument may be suitable as a generic and accurate screening measure. One issue with a new screening tool relating to both validity and utility is the way the instrument responds to heterogeneity within the clinical group of interest. Ideally, an instrument should be invariant within the group in order that cut-points for case detection are both consistent and reliable, thus occurrence of false positives and highly undesirable false negatives are reduced. Given that the GHQ-8 represents a new version or revision of the GHQ-12, cut-scores are not currently available.1 However, the opportunity to explore psychometric issues of the impact of common covariates such as age and level of education in terms of possible confound is presented by the development of this tool. This facilitates identification of items within the measure which perform inconsistently and thus reveals the presence of a potential confounding factor. Cautions can thus be identified with the instrument when applied to particular sub-groups within a general clinical classification. This study seeks to examine the impact of two common covariates on the psychometric performance of the GHQ-8. The present study addresses the following research questions: 1. Does age have a significant impact on the psychometric properties of the GHQ-8? 2. Does level of education have a significant impact on the psychometric properties of the GHQ-8?
85
Method
154 nulliparous women recruited from one public hospital participated in the current investigation. Eligible subjects were at their third trimester of pregnancy and be able to read Chinese. The mean age of women providing complete data was 28.21 years (SD = 5.08). The mean gestational age at recruitment was 33.79 weeks (SD = 1.64). Level of education was measured on a three-point scale (secondary, higher secondary, tertiary). Age and level of education was chosen as the key covariates because these will be common to any clinical group where this instrument is used as a screening tool. Participants were administered the Chinese version of the GHQ-12 (Pan and Goldberg, 1990) and the GHQ-8 items were extracted for analysis based on the two sub-scale model of Kalliath et al. (2004). The study was approved by the Clinical Research Ethics Committees of the study hospital and The Chinese University of Hong Kong. All participants were volunteers in the study and gave written informed consent prior to enrollment. None of the participants were paid for participation in the investigation. Evaluation of the performance of sub-scales and sub-scale items in relation to the co-variates (age and level of education) was conducted using structural equation modeling (SEM).
Results
Mean scores and standard deviations of participants ratings on the social dysfunction sub-scale was 0.47 (SD = 0.86), and on the anxiety/depression sub-scale was 1.53 (SD = 1.45). The SEM model was analysed using Mplus version 3 (Muthe ´n and Muthe ´n, 1998-2004). Evaluation of the two-factor GHQ-8 revealed adequate fit, v2 = 41.92(df=31), p = 0.09, comparative fit index (CFI) = 0.95, and root mean squared error of approximation (RMSEA) = 0.05. However, investigation of modification indices revealed that model fit could be improved by specifying a direct regression of GHQ item-4 ‘felt capable of making decisions about things?’2 on to the covariate age. The resulting modified model fit, v2 = 37.16(df=30), p = 0.17, CFI = 0.97, and RMSEA = 0.04, was found to provide a statistically significantly fit to the data compared to the original model, v2 ðdiffÞ ¼ 4:76ðdf¼1Þ , p < 0.05. The structural model is shown in Fig. 1.
1 The GHQ-12 is scored so that high scores are associated with psychopathology and cross the threshold cut-score for a ‘case’.
2 Item numbers consistent with that of the GHQ-12 are used to allow comparison with that measure at the individual item level.
86
GHQ 4
0.01 0.13 0.24 0.05
W.Y. Ip, C.R. Martin
Error 0.09
GHQ 7 GHQ 8
Error 0.07
Age
0.02
Social 0.12 Dysfunction
0.15
Error 0.06
GHQ 12
0.64 0.51
Error 0.07
GHQ 6
0.23 0.29 0.26 0.18
Error 0.16
Education
Anxiety/ Depression
GHQ 9
0.39 0.30
Error 0.18
GHQ 10 GHQ 11
Error 0.08
Error 0.09
Figure 1 Two-factor 8-item model based on Kalliath et al. (2004). Estimates are standardised. Dashed line represents modification to model.
Discussion
The findings from the current study have a number of implications for the potential use of this instrument during pregnancy. The descriptive statistics reveal low mean sub-scale scores which would suggest that if the GHQ-8 was used in case detection, an alternative scoring system to the standard GHQ method (Goldberg and Williams, 1988) would be preferred. A suitable alternative scoring system, the likert system is however available (Goldberg and Williams, 1988) and would increase the range of scores and consequently the discriminate ability of a cut-score. The SEM findings are also illuminating as to the possible influence of covariates on this potential screening measure. A significant improvement in model fit was found by specifying that the GHQ-4 item is regressed directly on to the age covariate. This finding shows that increasing age is associated with a reduction in the score of this item. Consequently, for a given threshold score, the effect here is for comparatively older women to have a lower overall score due to differential deflation of the GHQ-4. This may be interpreted in two ways, for a given cut-off score comparatively older women who may be borderline cases within the group will be more likely to be undetected as cases as they are less likely to reach threshold. Alternatively, setting a more conservative threshold to improve case detection rates of older women is likely to be accompanied with an increased false-positive rate in younger women in this clinical group. Since the SEM reveals that only
the GHQ-4 item is influenced by age, either revision of this item may be desirable or age-normed cut-scores developed for the GHQ-8. The current study has demonstrated that inspite of good evidence for factorial validity in a measure, the performance of individual items within the scale may be significantly affected by general socio-demographic attributes. This is of particular importance in the context of clinical screening instruments since variant items may reduce the case detection accuracy of the tool. Health professionals using screening measures routinely may wish to consider salient aspects of the clinical group which may impact on the efficacy of such screening tools. Nurses are in a very good position to administer screening instruments, score them and integrate the findings into a clinical-decision making framework. However, as evidenced-based practitioners, nurses should be aware that screening instruments are not infaliable in terms of case detection accuracy, and that other important characteristics (age in the case of the current study) of the clinical group have a salient influence on the screening measure used. Awareness of these factors is very useful in both administration and interpretation of screening scores and in reminding health professionals that case positive/negative status determined by a questionnaire score is additional information in forming case management strategies; however the nurses clinical insight and experience retain a significant and central role in the identification of psychological disturbance during pregnancy.
A measurement issue in the assessment of social dysfunction and dysphoria
87
References
Goldberg, D.P., Williams, 1988. A User’s Guide to the General Health Questionnaire. NFER-Nelson, Basingstoke. Jomeen, J., Martin, C.R., 2004. Is the Hospital Anxiety and Depression Scale (HADS) a reliable screening tool in early pregnancy? Psychology and Health 19, 787–800. Kalliath, T.J., O’Driscoll, M.P., Brough, P., 2004. A confirmatory factor analysis of the General Health Questionnaire-12. Stress and Health 20, 11–20. Karimova, G., Martin, C.R., 2003. A psychometric evaluation of the Hospital Anxiety and Depression Scale during pregnancy. Psychology, Health and Medicine 8, 89–103. Martin, C.R., 2005. What does the Hospital Anxiety and Depression Scale (HADS) really measure in liaison
psychiatry settings? Current Psychiatry Reviews 1, 69– 73. Martin, C.R., Jomeen, J., 2003. Is the 12-item General Health Questionnaire (GHQ-12) confounded by scoring method during pregnancy and following birth? Journal of Reproductive and Infant Psychology 21, 267–278. Martin, C.R., Newell, R.J., 2005. Is the 12-item General Health Questionnaire (GHQ-12) confounded by scoring method in individuals with facial disfigurement? Psychology and Health 20, 651–659. Muthe ´n, L.K., Muthe ´n, B.O., 1998-2004. Mplus Users Guide, third ed. Muthe ´n and Muthe ´n, Los Angeles, C.A. Pan, P.C., Goldberg, D.P., 1990. A comparison of the validity of GHQ-12 and CHQ-12 in Chinese primary care patients in Manchester. Psychological Medicine 20, 931– 940.
