Participatory Learning and
Assessment in Competency-Based
Contexts
Daniel T. Hickey
Associate Professor of Learning Sciences
Center for Research on Learning and Technology
Indiana University
Research Context
2009 in Sakai
@IU
2012 in
CourseBuilder
BOOC
2013 in Canvas
@ IU & IUHS
4
Use public contexts give
meaning to knowledge tools
Reward productive
disciplinary engagement
Grade artifacts through local
reflections
Let individuals assess their
understanding privately
Measure aggregated
achievement discreetly
5
James
Greeno
• Ed Psych 540: Learning &
Cognition in Education
• Challenging to Teach
– Diverse experience & ambitions
among students
– Ambitious but busy instructor!
– Started adjuncts in 2011
• Typical Accountability
– Use textbook item bank
– Include a pretest
• Online in Sakai in 2009
– Canvas in 2015
Gina Howard
Andi Rehak
Joshua Quick
The P540 Team
Lauren Smith
Xinyi Shen
7
Firat Soylu
• Educational Assessment BOOC
– P507 Assessment in Schools embedded for credit
• Offered to 500 students for free in 2013 with a grant
from Google using CourseBuilder
– 460 registered, 160 started, 60 finished, 9 in P507
• Offered a smaller streamlined version in 2014
– Added videos and quizzed and automated key features
– 60 started, 22 completed, 12 in P507
Use “public” contexts to give
meaning to knowledge tools
• Problematize knowledge from
learners’ perspective
– Publically open up the discipline
• Learners define (and redefine) a
personally-meaningful context
– Can be disciplinary problem, project,
investigation, etc.
– Embodies their experience, interests,
& aspirations
11
Rogers
Hall
15
16
20
Reward productive
disciplinary engagement
• DE involves declarative knowledge and
cultural practices
• PDE connects knowledge and practices,
makes connections, brings in resources,
etc.
• Support PDE so it can be rewarded
Randi
Engle
– Comment directly & publically on wikifolios
– Work in the open and encourage “lurking”
– No grades or mandatory posts
25
28
29
30
Grade artifacts through
“local” reflections
• Wikifolios generate lots of text and
comments
– Grading artifacts and comments
undermines participation
• Grade reflections on PDE.
– Critical engagement (suitability of context
for learning concepts)
– Collaborative engagement (what you
learned from others)
– Consequential engagement
(consequences of new knowledge)
31
Melissa
Gresalfi
33
Let individuals assess
understanding privately
• Offer ungraded open-ended
items and performance tasks
• Cover all of the big ideas in the
assignment
• Use to re-engage rather than
remediate.
– Items themselves have little formative
value beyond the correct answer
Pamela
Moss
34
35
Measure achievement
discreetly
• Standards-oriented tests
– Measure gains using pretests, compare
instruction, document improvement
– Primary student function is motivating
prior engagement
• Refine tests (but don’t teach to them)
– Use the item analysis routines in your LMS
• Protect test security
– Withhold item-level feedback
• Randomly selected timed items
– Proctor or use unsearchable items
Me!
Results:
Overall Public Engagement in P540
Average Words per Wikifolio & Words per Groupwiki
5000
4000
3000
Wikifolios
2000
Groupwikis
1000
0
1
2
3
4
5 6&7 8
9
10 11 12 13 14 15
• Wikifolios averaged 1569 words (144-3291)
• Groupwikis averaged 3477 words (2705-4630)
Course Evaluation Results
Campus-Wide Items
Overall, I would rate the quality of this
COURSE as outstanding.
Overall, I would rate this INSTRUCTOR as
outstanding.
My instructor treats students with
RESPECT
My instructor is WELL-PREPARED for class
meetings.
0
(11/15, 5-point Likert, SD to SA)
The instructor used the available TECHNOLOGY well.
I would take ANOTHER ONLINE COURSE from the
School of Education
The level of INTERACTION in this distance course
was sufficient.
The ONLINE FORMAT was well suited for this course.
The online technology used in this course was
RELIABLE
This distance education format helped me feel IN
CONTROL of my own learning
0
(11/15, 5-point Likert, SD to SA)
1
14 Other Sections
2
3
This Course
4
5
Issues with Semi-Formal (Cohorted)
Course Format
Assumes regular instructor effort
Is not what online learners expect
Not flexible
Can’t get behind
Led many students to drop out of the BOOC
Led many IUHS students to stay with
distance ed courses
Moving to Self-Paced Courses
Not simple to get interaction
Need to find like-minded peers
Need to find peers working on same unit
Need a large number of learners
Currently creating a self-paced BOOC
New participant list shows current learners
New wikifolio feature allowing archiving
Learners can look at archived work
Currently creating self-paced IUHS courses
Modules with exams and Canvabadge
Moving to Open Learning Resources
Need to automate many features
Automated registration and homepage
Real-time participatory feedback
Exam scoring and badges issuer
Relying extensively on open ed resources
Use ranking feature with OERs
Pending NSF proposals for IUHS and IU
Questions?
Thank you very much!
More info at www.RemediatingAssessment.blogspot.com
Hickey, D. T. (in press). Situative approaches to aligning summative, formative, and
transformative assessment functions. Assessment in Education: Principles,
Policies, and Practices.
Hickey, D. T., & Rehak, A. (2013). Wikifolios and participatory assessment for
engagement, understanding, and achievement in online courses. Journal of
Educational Media and Hypermedia, 22 (4), 229-263.
Hickey, D. T., & Zuiker, S. J. (2012). Multi-level assessment for discourse,
understanding, and achievement in innovative learning contexts. The Journal of
the Learning Sciences, 22, (4) 1-65
Hickey, D. T., Kelly, T. A, & Shen, X. (2014). Small to big before massive: Scaling up
participatory learning and assessment. Proceedings of the Fourth International
Conference on Learning Analytics and Knowledge, Indianapolis, IN (pp. 93-97)