Assessment Evidence
There are five essential characteristics of effective evidence of student learning: direct, valid and reliable, aligned, embedded, and fair.
Direct
Assessment of student learning requires direct, rather than indirect, evidence. To make supportable claims about student learning and program effectiveness, gather and analyze multiple lines of direct evidence.
Direct evidence of student learning is data from which it is possible to observe students’ demonstrations of knowledge and / or skills. In other words, direct evidence of learning reveals what students can demonstrate, as well as the degree to which students have moved toward mastery-level attainment of faculty-identified expectations.
Why does this matter? Direct evidence allows you to assess student learning, while indirect evidence allows you to understand students’ experiences in your program.
Sources of direct evidence of learning
Gather direct evidence from required courses in which students are expected to demonstrate mastery-level attainment of program learning outcomes. Meaningful and useful evidence of student learning for assessment purposes should be drawn from senior-level capstone or culminating courses; senior design project presentations; and theses -- if all students in the program are required to complete a thesis. Other examples of direct evidence that can be drawn from required courses include: Artifacts; samples of student work; instructor evaluations of real-time performances/presentations (assessed with an analytic rubric), and/or final portfolios showcasing a collection of student work.
Other sources of direct evidence, which can be gathered from required courses throughout the program include:
- Scores from constructed-responses (aka multiple choice) exams or quizzes aligned to specific PLOs
- Open-responses from exams or quizzes aligned to specific PLOs
- Lab reports, problem sets, oral presentations, and/or simulations
- Case study write-ups, lab reports, papers
- Oral presentations / debates assessed in real-time with an analytic rubric
- Critiques, reviews of journal articles, problem sets, product reviews
Indirect evidence refers to data from which it is possible to make inferences about students’ learning, rather than observing their learning directly. Also known as proxy evidence, indirect evidence alone is insufficient to support conclusions about or recommendations for program effectiveness or student learning. Some sources of indirect evidence include self-assessment, surveys, and/or learning-related feedback (during or after the term).
NOTE: Evaluations of teaching are not appropriate for inclusion of student learning outcomes assessment.
Valid and Reliable
Evidence is valid when it captures what you think it captures. For example, if you want to assess quantitative reasoning, valid evidence would mean that you use student work that actually demonstrates quantitative reasoning, rather than basic recall of mathematical formulas.
Reliable evidence will capture what it should even at different times and/or among different raters or observers of student work. For example, the student work that demonstrates quantitative reasoning should be able to do so across different sections of the same course.
Aligned
Aligned evidence generally means that the evidence is both valid – it captures what you want to observe – and also aligned with the PLOs and curriculum. In other words: The knowledge and/or skill you want to observe is aligned with the PLOs and curriculum, and the assignment/activity actually elicits demonstration of that knowledge and/or skill. At the course-level, assignments, learning activities, and assessment efforts should be aligned to each other and to course learning outcomes.
Embedded
Embedded evidence is direct and derived from work that students are already producing and that instructors are already assigning in their courses. Inventing new assignments for assessment purposes is generally not necessary. Instead, gather work (or parts thereof) that students are already doing and from multiple courses. At the course-level, gather evidence from more than one assignment that aligns to at least one course learning outcome.
Fair
The final characteristic of effective evidence is that it is fair. In addition to being aligned, to be considered fair the evidence should be sufficiently varied in type, (i.e., drawn from different types of assignments / assessment activities). Fair evidence is also drawn from transparent assignments, meaning that students understand the purpose of the assignment; the tasks required to complete it; and the criteria by which their work will be assessed and/or graded.
To learn more about assessment evidence, go to the:
Program Assessment Resource Kit
UC Davis login credentials required.