Learning Outcomes
Learning outcomes (LOs) are short statements that describe the knowledge and skills students are expected to demonstrate as the result of a learning opportunity. LOs describe what students should be able to demonstrate, not what the program offers or instructor teaches / delivers. Learning outcomes need to be specific and clear enough to provide information that is useful for assessment inquiry and curricular improvement–applicable at course–and program-level.
Course learning outcomes (CLOs) derive directly from one or more Program Learning Outcomes (PLOs).
The process of developing learning outcomes invites faculty to reflect on, and then translate, their embodied disciplinary knowledge into clear expectations for students in their majors / courses.
We wouldn't want to leave something as important as student learning and success to guesswork.
Learning outcomes need to be transparent and ubiquitous.
Not surprisingly, students benefit when learning outcomes are clear, but transparency of student learning goals is especially beneficial for first-generation students, students from historically marginalized communities, and transfer students (Winkelmes, 2013; Balloo, Evans, Hughes, Zhu, and Winstone, 2018). Fair assessment practices demand that, in addition to making expectations for student learning clear and known, we must ensure that the primary intended users of those outcomes, i.e., students themselves, are able to make sense of them (Montenegro & Jankowski, 2020).
To learn more about program learning outcomes, go to the:
Program Assessment Resource Kit.
UC Davis login credentials required.
Make Learning Outcomes Visible: Rubrics
Performance Indicators
Performance indicators (PIs) are distinct from LOs in that they describe the concrete, tangible, and/or observable (sub)skills and/or discrete concepts that are entailed within a LO. They are constituent knowledge and skills required to demonstrate attainment of the LO (in other words, PIs are the aspects of the performance that will be assessed).
To assess a learning outcome, it’s necessary to divide it into its performance indicators (PIs), which are the discrete and observable constituent features. Deconstructing the LO into pieces supports the process of discerning the criteria for assessing students’ performance aligned to the outcome. For example, consider this fictional Program Learning Outcome:
Generate ethnographic field notes to aid in analyses and interpretations of field research observations.
It would be difficult (and inadvisable) to assess this LO as is and/or from a single assignment (i.e., writing notes during a field research exercise) without breaking it into observable demonstrations that capture the nuance of the outcome. Following is an example of how the example LO above is split into three PIs.
To demonstrate attainment of this learning outcomes, students should be able to:
- Differentiate between empirical observation and ethnographic interpretation;
- Record subject interactions and utterances with accuracy; and
- Present observational evidence consistent with research ethics and integrity.
As the examples above show, PIs start with an observable verb followed by a key concept or skill specific to the discipline. Some performance indicators will also include a prepositional phrase, or additional features and/or characteristics that signal mastery.
Performance indicators matter because they clearly communicate expectations and eliminate ambiguity about what students should be able to demonstrate. NOTE: None of us can guarantee what another person will do, which is why the language we use is students should be able …
Finally, some people refer to PIs as criteria: They are the same concept with different language.
Performance Levels
Performance levels (PLs) indicate the progressive degrees of mastery expected for each PI (e.g., they are the components of a ratings “scale”). The “scale” of performance levels might range from proficient at one end to initial at the other end of the scale (interim levels: approaching / emerging). Some faculty believe that “proficient” is not a high enough standard for UC Davis students. As undergraduates, our students should be held accountable to outcomes that are appropriate in complexity and scope.
NOTE: Performance levels should not be converted to letter grades (e.g., A, B, C, D). The purpose of PLs is to describe degrees of students’ attainment of the performance indicators, not generate grades.
Performance Level Descriptors
Performance level descriptors (PLDs) are the characteristics associated with each performance level. They describe what a performance / demonstration must include to be identified as proficient, emerging, and/or initial (or whatever PLs you use.) The descriptions should describe the performance, not the student. For example, a clear and fair description would read: Includes required components, rather than didn’t include required components. This distinction may seem minor, but it’s important. The process of assessing student learning outcomes focuses on the work that students produce, not students themselves.
PIs + PLs + PLDs = rubric
Although you may use different terminology, the combination of Performance Indicators, Performance Levels, and Performance Level Descriptors are the constituent parts of an analytic rubric.
In the left column are the PIs (sometimes called "traits" or "criteria") that will be used to assess students’ work.
Along the top are the PLs -- words (not numbers) indicating the progressive degrees of attainment.
Within the cells of the table are the PLDs, which describe the characteristics that are expected at each level.
Rubrics are vital to ensuring your assessment practices are fair and transparent. Consistently using a rubric to assess students’ work is fair. Sharing the rubric with students with the assignment guidelines and making sure they understand the expectations and how they’ll be assessed is a transparent practice.
To learn more about building clear and effective rubrics with performance indicators, performance levels, and performance level descriptors, go to the:
Program Assessment Resource Kit.
UC Davis login credentials required.
Curriculum Analysis and Mapping
We “need to consider how curricula are structured to advance student learning and how courses within curricula are linked to help students make connections, transfer learning, and reach the goals we set for them in learning outcomes” (Jankowski & Marshall, 2017, p. 27).
The purpose of analyzing and mapping the curriculum is to ensure alignment.
Alignment
We cannot hold students responsible for demonstrating mastery of learning outcomes unless we provide a sufficient number of sufficiently-scaffolded opportunities for them to be introduced to, practice, and demonstrate mastery of those outcomes. This is true at the course- and program-level. An aligned curriculum is characterized by direct and intentionally transparent relationships between Program Learning Outcome (PLOs); Course Learning Outcomes (CLOs); learning and practice activities; and strategies for assessing of learning.
When and how often does your program provide students opportunities to demonstrate their learning?
Curriculum analysis
Curriculum analysis is an intentional, iterative, and collaborative examination of the degree to which the program’s curriculum provides sufficient and sufficiently-scaffolded opportunities for students to learn, develop, and demonstrate their mastery-level attainment of PLOs.
- Sufficient in number: the program’s curriculum includes multiple opportunities (i.e., more than two required courses) for students to learn, develop, and demonstrate their mastery-level attainment of the knowledge / skills associated with each PLO.
- Sufficiently scaffolded: the program includes opportunities for increasing levels of engagement with each PLO through which students learn, develop, and demonstrate their mastery-level attainment of the knowledge / skills associated with each PLO.
Levels of engagement
It is necessary to identify the degree to which each required course provides opportunities for students to engage with the (sub)skill/knowledge for each PLO. We refer to those levels I, R, and M. They reflect successive levels of engagement that support students' development across the program.
- In the Introduce level (indicated with an I), the course introduces one or more PLO and includes opportunities for students to acquire introductory knowledge and skills.
- Courses designated as Reinforce (indicated by R), provide opportunities for students to practice and reinforce their knowledge and skills related to the PLO(s).
- Courses that provide opportunities for students to demonstrate mastery-level attainment of knowledge and skills related to the PLO(s) are labeled with an M.
Curriculum mapping
“Mapping is a strategy for visualizing the areas where we think learning is happening as it relates to specific learning outcomes” (NILOA, 2018, p. 7).
The process of curriculum analysis yields a curriculum map, which visually represents the opportunities the program provides for students to engage with skill(s)/knowledge related to each PLO. Strong curriculum maps must include: All of the PLOs; Required courses offered by the department / program; and Degrees of engagement with each PLO. (See image below.)
NOTE: Establishing a strong curriculum map takes considerable knowledge of the required courses in your program. That's why we recommend that the curriculum analysis process include myriad perspectives from faculty and instructional personnel, program staff, staff advisors, and most importantly(!) students. Together, these viewpoints will help ensure the curriculum analysis is accurate. It’s vital that the map accurately represents the current curriculum, not an aspirational or potential sequence of courses. Ironically, a strong map can reveal misalignment and gaps in the curriculum. That gives the program a place to start inquiring about how best to scaffold student learning.
To learn more about analyzing and mapping curriculum, go to the:
Program Assessment Resource Kit.
UC Davis login credentials required.)
Why does this matter?
When was the last time you thought about what it was like to be a novice in your field?
As experts, many faculty demonstrate their disciplinary mastery “so automatically and instinctively that they are no longer consciously aware of what they know or do” (Ambrose et al., 2010, p. 97).
Unfortunately, the implicit nature of faculty expertise doesn't always translate directly to clear goals for student learning. Instead, sometimes the very thing that qualifies someone as an expert–unconscious competence–can lead to disciplinary blind spots which may create unintended obstacles for students.
The iterative process of reflecting on your expectations for student performance may help bring your implicit expectations about disciplinary knowledge, skills, and abilities to an explicit level.