What's the difference between evidence and data?
One of the first steps in developing a useful plan for collecting evidence of student learning is to discern between often conflated terms: evidence and data. Following are definitions of evidence, data, data generation, and data collection.
Evidence derives from the assignments and activities that prompt students to demonstrate the learning outcome and performance indicators.
Data does not mean the student work itself, but rather the scores or ratings of the work. The scores are generated by following an analytic rubric which articulates what different levels of performance would look like.
Scores - Keep in mind that the data for the inquiry will only include the subset of scores that align to the performance indicators, not necessarily the total score on the assignment.
Data generation happens when instructors or graders score student work as part of the normal functioning of the course.
Data collection refers to the compiling of all rubric-aligned scores into a central location. We recommend that this happen after the conclusion of a term.
Data Collection Planning
Assessment is a team sport. Make sure you and your colleagues stay connected through a data collection plan. It doesn't need to be complex. Create a shared document to keep track of who is doing what. The plan should also include decisions about the data identified for the inquiry.
Characteristics of appropriate evidence
As a reminder, effective evidence for student learning outcomes assessment is:
- Direct: demonstrations of students’ learning (vs. experiences)
- Valid/Reliable: captures the desired knowledge/skill every time
- Aligned: knowledge/skill is aligned to PLO and curriculum; assignment elicits demonstration of the skill
- Embedded: drawn from what students are already doing/producing in the program
- Fair: aligned with PLOs, sufficiently varied kinds of demonstration, and transparent assignment purpose and expectations
Data for inquiry project
- Scores / ratings on (parts of) aligned assignments from required courses that align to a given PLO. NOTE: Your data do not include: artifacts of student work; scores for every assignment in the course(s); assignment or course grades.
- Generated by instructors, TAs, graders, etc. during a course, as part of normal operations; stored securely after term ends - ideally in Canvas.
- Selected to answer a specific question (e.g., What proportion of students in the major are proficiently attaining PLOs?)
Data Collection
Remember: relevant evidence of student learning will lead to relevant data. Effective data collection begins with evidence of student learning that is aligned with PLO(s) / curriculum; sufficiently varied; and transparent.
Data should be:
- Representative. Include the entire student population in a program.
- Generate data for ALL students, not a select few (e.g., honors students; control group).
- Confidential. Use student IDs only.
- No names. In addition, remove identifying information (e.g., section, TA, assignment groups / teams). Note that data that are anonymous (rather than confidential) can't be disaggregated.
- Private. Behind every data point is a real person, as such, data should be secured and protected.
- Develop a privacy plan that specifies where the data will be secured; who will have access; and when the data will be deleted / destroyed after the inquiry project.
To learn more about data collection, go to the:
Program Assessment Resource Kit
UC Davis login credentials required.