"Not everything that counts can be counted, and not everything that can be counted counts." - William Bruce Cameron (and on a sign hanging in Albert Einstein's office)
"What is water?" said one fish to the other, illustrating that "the most obvious, important realities are often the ones that are hardest to see and talk about" (David Foster Wallace). One of these realities is that we teach competencies that can be easily tested and quantified rather than what is most important. This reality may seem obvious, but why do we keep doing it? If we strive to develop student agency, can we do a better job at taking agency ourselves for what we deem important?
How we can get into trouble
What leads us to focus on competencies that can be easily quantified and overlook other critical competencies such as a growth mindset, self-management, initiative, communication, collaboration, and lifelong learning, is the tendency in our modern world to manage by the numbers, which is not a bad thing, but can lead to unintended consequences if we don't examine our actions. A need to measure and quantify progress drives policy-makers, administrators and teachers to focus on easy-to-quantify competencies, since it's the easiest path to management by the numbers. At an educator level, we also must teach knowledge and skills that can be assessed so that we know where students are in their development and what they need next. This often leads us to easy-to-quantify competencies that can be assessed through highly structured tests that provide quantifiable measures. Unfortunately, we might then overlook the less quantifiable skills that students also need, and that we can also assess.
Counting vs. assessing
It's hard to teach something that can't be assessed. How do we know whether what we're doing is working? How do we know where students are in their understanding and what they need next?
Fortunately, hard-to-measure 21st century competencies can be assessed, and are assessed all the time. In our workplaces, we have a sense of our colleagues' collaboration skills, communication skills, creativity, self-management, mindset, initiative, self-growth, vision, and many other important competencies. Yet we don't go around saying "Kathy, I need to assess your collaboration skills today, can you please do this performance assessment?" We gain a much better sense of Kathy's collaboration skills, or any other hard-to-measure competency, by working with her than through a pre-packaged or highly structured test that results in a quantifiable measure. By working with Kathy, we gain valuable information that allows us to share thoughts on where she is, how she is most helpful to the team, and what she could do to further increase her contributions and influence, which helps her continue to develop. If we want to prepare students to thrive in work and in life, why not assess and develop these critical competencies as is done in the real world?
If students are engaged in learning activities, which leads to deeper learning, teachers and fellow students can assess these competencies all the time. The teachers we train at Mindset Works understand what a growth mindset is and how it manifests itself, which allows them to assess their own mindset as well as that of their students, their families, and their colleagues, as they go about living, learning, and working together. Being able to assess these competencies in daily life enables us to self-monitor, help one another build awareness and grow, and respond to moments of challenge.
While some 21st century competencies are hard to measure through highly structured, summative assessments, skilled educators can assess these competencies formatively to inform instruction, influence learner mindsets, and prepare their students for the 21st century.
We must also be able to assess whether the systems we put in place are working as a whole. How do we know, at a systems level, whether our efforts to develop hard-to-measure 21st century competencies are actually successful?
Highly structured assessments are becoming more powerful. A good overview of leading-edge research on assessing hard-to-measure competencies is Measuring 21st Century Competencies by Jim Soland, Laura Hamilton, and Brian Stecher of RAND. There is exciting work being done. Even at the leading edge, however, these assessments don't yet come near to being able to measure all the competencies that count.
We can't wait decades or more for traditional assessments to reliably measure competencies like motivation, self-management, and learning-orientation. Fortunately, there are ways, today, to assess whether our systems are developing these competencies in students.
In the report Creating Systems of Assessment for Deeper Learning, David Conley and Linda Darling-Hammond describe ways that we accomplish this goal, involving a continuum of assessments for different purposes. For example, for competencies that can be measured through pencil-and-paper tests, we could use such tests. For competencies that could be measured through performance assessments or by having students interact with virtual agents, we could use those. For harder-to-measure competencies, such as creativity, mindset, communication with peers, learning orientation, and use of learning strategies, assessments could involve real human beings assessing students' abilities and dispositions. These assessments could involve authentic projects, rubrics, or portfolio-like collections of assignments, which could be integrated into projects that schools use today. They could involve interactions with real people and revisions of work, as is part of performance in the real world.
Some of these assessments may be more expensive to administer, so rather than administering them to all students, we could administer them to a small, randomized sample of students or schools. That would allow us to determine to what extent the system is working and under what conditions, so that we could adjust accordingly where needed. Teachers could assess all students formatively, while external summative systems assess randomized samples. Colleges and workplaces could also invest more resources into assessing individual students by looking at their projects and portfolios, speaking with them, and having them perform authentic activities. Through these systems, we would, as a society, better assess the human qualities that matter most for personal and professional success.
Doing what counts
Traditional education policy has not encouraged schools to develop students to be effective learners. Instead, standards have focused on teaching students capabilities that are easy to measure through tests, be it facts, procedures, or higher order thinking skills (all of which are important but not sufficient). Yet, in this complex, specialty-driven, and fast-changing world, little is more important than developing students as effective and motivated lifelong learners. And as students develop as learners, they can also learn facts, procedures, higher order thinking skills, and 21st century skills that will prepare them to engage in lifelong creation, contribution, and growth.
Now that we can design more sophisticated assessments that measure some higher order thinking skills, we are expanding the set of competencies that we're teaching beyond facts and procedures, yet continuing to limit our focus to what can be quantified. Often we don't realize we're doing this, because we're like fish in water.
Can we encourage what counts, whether or not it can be easily counted? If policy doesn't shift fast enough, will we lead our districts, schools, and classrooms to make it happen? As we transition to deeper learning environments, such as project-based and competency-based education, will we seize the moment, capitalize on this transition period, and design systems that do what counts?
Conley, D.T. & Darling-Hammond, L. (2013). Creating systems of assessment for deeper learning. Stanford, CA: Stanford Center for Opportunity Policy in Education. Available at https://edpolicy.stanford.edu/sites/default/files/publications/creating-systems-assessment-deeper-learning_0.pdf
Soland, Jim, Laura S. Hamilton, and Brian M. Stecher (2013), Measuring 21st-Century Competencies: Guidance for Educators. Santa Monica, Calif.: RAND Corporation. Available at http://www.rand.org/pubs/external_publications/EP50463.html
This article was adapted from a blog post by Eduardo originally published by CompetencyWorks.org called Counted or not, doing what counts in competency-based education. It has been adjusted for a broader audience.