A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas A Dissertation Defense by Arthur L. Petterway
William Allan Kritsonis, PhD Dissertation Committee Member
May 04, 2007
Arthur L. Petterway
1
Committee Members
M. Paul Mehta, Ph.D. (Dissertation Chair) William Allan Kritsonis, Ph.D. (Member) Douglas Hermond, Ph.D. (Member) David Herrington, Ph.D. (Member) Camille Gibson, Ph.D. (Outside Member)
May 04, 2007 Arthur L. Petterway 2
Dissertation Defense Format
I. Theoretical Framework II. Purpose of the Study III. Quantitative Research Question IV. Null Hypotheses V. Methods: Subjects VI. Methods: Instrumentation VII. Methods: Quantitative VIII. Quantitative Findings IX. Qualitative Research Question X. Qualitative Pilot Study XI. Qualitative Findings/Review of Literature XII. Practical Recommendations XIII. Recommendations for Further Study
Arthur L. Petterway
3
Theoretical Framework A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas
Explanatory Design
Q U A N T I T A T I V E
% All 10th Grade % ELLs Students
Enrolled
Passing In a TAKS High School ELA/ Math
Student Performance
Q U A L I T A T I V E
Views/ Opinions Administrators Teachers District Personnel
Impact of Statewide Testing On ELLs
May 04, 2007
Arthur L. Petterway
4
Purpose of the Study
The purpose of this study was to determine the impact of high-stakes testing on English Language Learners.
May 04, 2007
Arthur L. Petterway
5
Quantitative Research Question 1. Is there a relationship between the percentage of English Language Learners enrolled in a school and the percentage of all students passing the 10th grade TAKS test in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006?
May 04, 2007
Arthur L. Petterway
6
Null Hypotheses
H01: There is no statistically significant relationship between the percentage of English Language Learners enrolled in a school and the percentage of all students passing the 10th grade TAKS test in English Language Arts given in 2003, 2004, 2005, and 2006.
May 04, 2007
Arthur L. Petterway
7
Null Hypotheses
H02: There is no statistically significant relationship between the percentage of English Language Learners enrolled in a school and the percentage of all students passing the 10th grade TAKS test in Mathematics given in 2003, 2004, 2005, and 2006.
May 04, 2007
Arthur L. Petterway
8
Method
Subjects of the Study • Quantitative – 173 Urban High Schools in Texas • Qualitative (N = 55)
– – – – – 6 principals 9 assistant principals 6 ESL district personnel 15 ESL certified teachers 19 Non-ESL certified teachers
May 04, 2007
Arthur L. Petterway
9
Method
Instrumentation • Quantitative data were accessed and retrieved from the TEA website regarding the major urban high schools in Texas. • Qualitative data were derived from the on-line open-ended questionnaire, focus group, and individual interviews.
May 04, 2007
Arthur L. Petterway
10
Method Quantitative
• Descriptive Statistics • Correlation Statistics – Pearson r Coefficient • Simple Regression Analysis ลท=a+b1x1
May 04, 2007 Arthur L. Petterway 11
Method Quantitative
• Predictor Variable – The percentage of English Language Learners enrolled in school.
• Outcome Variable – The percentage of all students passing the 10th grade TAKS test in English Language Arts and Mathematics.
May 04, 2007 Arthur L. Petterway 12
Major Findings: Descriptive Statistics English Language Arts
Year
2003 2004 2005 2006
Percent of ELLs Percent of All Students Enrolled th Passing 10 in Schools Grade TAKS
8.30 8.29 7.62 6.94 62.87 68.28 59.39 78.05
Arthur L. Petterway
13
Major Findings: Descriptive Statistics Mathematics
Year
2003 2004 2005 2006
Percent of ELLs Percent of All Students Enrolled th Passing 10 in Schools Grade TAKS
8.30 8.29 7.62 6.94 61.85 53.57 47.68 50.13
Arthur L. Petterway
14
Major Findings: Correlation Pearson r Coefficients
Year 2003 2004 2005 2006 ELA -0.349** -0.392** -0.297** -0.398** Mathematics -0.293** -0.351** -0.382** -0.356**
** Correlation is significant at the 0.05 level
May 04, 2007
Arthur L. Petterway
15
Qualitative Research Question
• What are the anticipated and observed consequences of the statewide testing specifically, TAKS, on ELLs, ESL curriculum, and instruction as viewed by certified ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel?
May 04, 2007 Arthur L. Petterway 16
Pilot Study
Qualitative
• Developed an open-ended online questionnaire
The major question was explored using the following six probes:
1. Why is TAKS given as a statewide test? 2. What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?) 3. What problems have occurred related to or because of TAKS? 4. What changes were caused by this statewide testing? 5. What are your recommendations to improve this statewide testing? 6. What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?
May 04, 2007
Arthur L. Petterway
18
Major Findings-Probe: 1. Why is TAKS given as a statewide test?
• TAKS is given as a tool to gauge knowledge in the core areas. • TAKS is considered as a means to determine the school’s status (Exemplary, etc.). • TAKS is a means to assess the state curriculum or standards. • This statewide test is mandated by law and is aligned with NCLB.
May 04, 2007 Arthur L. Petterway 19
Major Findings-Review of Literature: 1.Why is TAKS given as a statewide
test? • Abrams & Madaus (2003)-Today’s widespread implementation of standardsbased reform and the federal government’s commitment to test-based accountability ensure that testing will remain a central issue in education for the foreseeable future. Test results can provide useful information about student progress toward meeting curricular standards.
May 04, 2007 Arthur L. Petterway 20
Major Findings-Review of Literature: 1.Why is TAKS given as a statewide
test?
• Madaus & O’Dwyer (1999)- Current emphasis on testing as a tool of education reform continues a long tradition of using tests to change pedagogical priorities and practices.
May 04, 2007
Arthur L. Petterway
21
Major Findings-Probe: 2. What are the intended consequences of this statewide testing?
• Statewide testing is intended to eventually result in ELLs performing as well as the rest of the students. • ELLs can improve academically and eventually join the mainstream. • TAKS is a requirement for graduation.
May 04, 2007
Arthur L. Petterway
22
Major Findings-Review of Literature:
2. What are the intended consequences of this statewide testing?
• Abedi (2003)- In a positive light, valid assessment may provide diagnostic information that can inform instruction and classification.
May 04, 2007
Arthur L. Petterway
23
Major Findings-Review of Literature: 2. What are the intended consequences of this statewide testing?
• Harlow & Jones (2003)- We also need to recognize that when test scores are linked to high-stakes consequences, they can weaken the learning experiences of students, transform teaching into test preparation, and taint the test itself so that it no longer measures what it was intended to measure.
May 04, 2007
Arthur L. Petterway
24
Major Findings-Probe: 3. What problems have occurred related to or because of TAKS?
• • • •
Higher dropout rate Decrease in graduation rate for ELLs Lower self-esteem of ELLs. Dismal or failing performance of ELLs
May 04, 2007
Arthur L. Petterway
25
Major Findings-Review of Literature: 3. What problems have occurred related to or because of TAKS?
• Anderson (2004)- For English Language Learners, the additional requirements of an exit examination could increase dropout rates. • Rothstein (2002)- Dropout rates are climbing throughout the United States and many researchers hold high-stakes testing at least partly to blame.
May 04, 2007
Arthur L. Petterway
26
Major Findings-Review of Literature: 3. What problems have occurred related to or because of TAKS?
• Jacob (2001)- Some researchers found that dropout rates were 4 to 6 percent higher in schools with high school graduation examinations. Another study reported that students in the bottom quintile in states with high-stakes testing were 25% more likely to drop out of high school than were their peers in states without high-stakes testing.
May 04, 2007
Arthur L. Petterway
27
Major Findings-Probe : 4. What changes were caused by this statewide testing? • Schools experienced the negative reality that there is a high failure rate among ELLs. • Low self-esteem because of low scores and tests caused frustration and exasperation on the part of ELLs. • Pressure on the school and ELLs. • Emphasis is placed on test performance.
• ELLs need more time to learn English
May 04, 2007 Arthur L. Petterway 28
Major Findings-Review of Literature: 4. What changes were caused by this statewide testing?
• Anderson (2004)- Labeling schools can have an impact on teacher and student morale. • Flores & Clark (2003)- Teachers have also reported that the high-stakes nature of some assessments can have a negative impact on student morale.
May 04, 2007
Arthur L. Petterway
29
Major Findings-Review of Literature: 4. What changes were caused by this statewide testing?
• Hood (2003)- Although some teachers have reported that their English language learners can reach the high standards set for them, they may need more time than other students. • Lane & Stone (2002)- Certainly, poor test scores or poorly explained assessment systems can result in decreased student motivation.
May 04, 2007
Arthur L. Petterway
30
Major Findings-Probe : 5. What are your recommendations to improve this statewide testing?
• Deferment of the test, possibly a different but fair test. • Better assistance from the school to provide for teacher collaboration. • Modifications in teaching strategies. • Improved and paced curriculum for ELLs.
May 04, 2007
Arthur L. Petterway
31
Major Findings-Review of Literature: 5. What are your recommendations to improve this statewide testing?
• Anderson (2004)- Positive consequences that were identified included more teacher collaboration, changes in curriculum and instruction, better alignment between ESL and content area curricula and more focus on reading and writing. Negative consequences included student and teacher frustration, more teaching to the test occurring, and a narrowed curriculum.
May 04, 2007
Arthur L. Petterway
32
Major Findings-Probe:
6. What needs to be done for the ELLs to improve their performance in general and specifically for this
statewide test?
• Specific interventions for ELLs. • Quality instruction. • Develop a more intensive English program. • Support from home. • Provide a more meaningful tutoring program in school.
May 04, 2007 Arthur L. Petterway 33
Major Findings-Review of Literature:
6. What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?
• Flores and Clark (2003) found that teachers were not against accountability and viewed it as distinct from statewide testing, but also thought that an overemphasis on testing resulted in unbalanced curriculum and inappropriate instructional decisions.
May 04, 2007
Arthur L. Petterway
34
Major Findings-Review of Literature: 6. What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test?
• Popham (2003)- In order for teachers to make specific changes to instruction, the assessments needs to be clear as to what skills are being assessed.
May 04, 2007
Arthur L. Petterway
35
Practical Recommendations
• School leaders should consider that the performance of schools on high stakes testing is affected by size and the proportion of ELLs taking the test. • School leaders should consider extended deferment of standardized tests administered in English to ELLs.
May 04, 2007
Arthur L. Petterway
36
Practical Recommendations
• School leaders should consider taking a more active role with LPAC in the monitoring of ELLs in schools. • School leaders should implement specific action plans to improve situation of ELLs.
May 04, 2007
Arthur L. Petterway
37
Recommendations for Further Study
• A study could be conducted to explore additional supports needed to ensure ELLs to pass high stakes testing. • A study could be conducted to identify what data are needed to make fair decisions about ELLs.
May 04, 2007
Arthur L. Petterway
38
Recommendations for Further Study
• A study could be conducted to determine reasons why ELLs scored lowest among student groups on TAKS. • A study could be conducted to Explore different approaches on campuses in dealing with ELLs in terms of curriculum and instruction.
May 04, 2007
Arthur L. Petterway
39
Recommendations for Further Study
• A study could be conducted to compare the performance of ELLs vs. non-ELLs based on different objectives of TAKS. • A study could be conducted to understand the impact of high stakes testing as viewed by parents and students.
May 04, 2007
Arthur L. Petterway
40
Recommendations for Further Study
• A study could be conducted to Explore different instruments to measure performance of ELLs. • A study could be conducted to determine whether the provision of English Language instruction to parents of ELLS would have a significant effect on student achievement on high stakes testing.
May 04, 2007
Arthur L. Petterway
41
A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas
Explanatory Design
Q U A N T I T A T I V E
% All 10th Grade % ELLs Students
Enrolled
Passing In a TAKS High School ELA/ Math
Student Performance
Q U A L I T A T I V E
Views/ Opinions Administrators Teachers District Personnel