UD professor serves as policy expert on national report about equitable educational assessments

As the end of the 2020-21 school year approaches, the debate about educational assessments is gaining renewed momentum. While educators and policymakers may hold differing views about how to proceed with testing, they all recognize that equity lies at the heart of this issue.

Some argue that educational assessments provide valuable information for understanding how the pandemic has affected learning, especially among students from underrepresented backgrounds. Others argue that these tests, especially in today’s context, will not accurately reflect learning among students who need the most support.

Kenneth Shores, assistant professor in the University of Delaware School of Education, served as an educational policy expert for the National Academy of Education’s report “Educational Assessments in the COVID-19 Era and Beyond,” which synthesizes the thinking of a group of scholars, policy leaders, and educators about this issue.

In an interview with UDaily, Shores shares highlights from the report, including recommendations for developing equitable assessment systems.

Q: This report is grounded in a debate about whether to delay or move forward with educational assessments during this pandemic. Could you describe this debate for the non-expert?

Shores: One of the key ideas from the report is that tests have multiple uses. We use them to describe how much learning a student has made in the course of a year; we use them to hold teachers accountable for student learning; we use them to determine which students are eligible for special education or for gifted and talented classes; we use them in some cases for grade-level promotion; and, at the state and national levels, we use them to monitor how well different populations are doing in terms of their learning. The critical questions are which of these uses are essential and whether the test results for these uses will still be valid and reliable measures, given the challenges due to COVID-19.

So I think the debate is too narrowly construed if we only put it in terms of delaying or moving forward. The question that takes up a lot of attention is whether or not to conduct end-of-year assessments — recently the Biden administration has said that states should continue with these — but we should first consider what uses these tests will serve and, then, whether schools will be able to administer them in contexts that will generate valid results.

Q: In preparation for this report, you served as one of the scholars, policy leaders, and educators for an online discussion of “how” and “why” to proceed with testing. Given the range of participants, did anything from this conversation surprise you?

Shores: One thing that did surprise me was how much the conversation focused on end-of-year standardized assessments and not on the assessments that were already in-use and under-scrutinized. In my own work, I’ve documented the very large racial disparities in access to advanced course work, such as Advanced Placement (AP) classes and gifted and talented programs. There has been very little attention to the assessments that are used to make these decisions and whether those assessments are less valid during the pandemic.

Specifically, think about a student with limited internet access who should be receiving special education services. Are they receiving the assessments they need to initiate the individualized education plan (IEP) they qualify for, and would the assessment provide an accurate result? Many assessments are already happening or have happened with little scrutiny, and the consequences for those assessments are in some ways more meaningful for students. End-of-year assessments rarely impact a student directly, whereas one’s performance on an Algebra I test or a diagnostic assessment for AP classes, special education, or gifted and talented programs can determine one’s opportunity set.

I’d like to see more attention paid to those assessments and how much more inequity has occurred as a result. I called attention to this issue, which you can see in the report under the heading “Examine the Equity Concerns Inherent in Other Assessments.”

Q: As your answer suggests, issues of equity lie at the center of this report. Could you describe how testing during the pandemic affects students from underrepresented backgrounds and underserved communities?

Shores: Much of the justification for assessment is to understand whether student subgroups are performing as well as other groups and whether their learning growth is comparable. I think one real concern here is that the pandemic is going to exacerbate many inequalities in opportunities for learning that predated it. Thus, testing provides an opportunity to document whether that is indeed the case, and whether inequalities in student academic outcomes have increased.

Realistically, we don’t need a test to know that students who have had two working parents, a sick parent, or who have unreliable internet access are going to be falling further behind. We then should ask whether documenting that learning loss in the form of an assessment is worth the costs and whether we can document it at all. The costs are very real: many students have had precious few hours of in-school learning. How many hours should we then spend having them take tests in isolation?

The question of validity is also important: if students are taking assessments in non-traditional settings, will the scores be valid? If only a subset of students are taking the assessments, does their performance generalize to the non-test takers?

Q: Given the conditions in which students are now learning, what are some factors that we must consider when interpreting data from assessments?

Shores: The big danger that people are worried about is attributing learning growth to a particular teacher in the form of accountability. These concerns were certainly present before the pandemic, but now I think it would be a huge mistake to hold teachers accountable for these test scores. First, the test scores themselves will have validity problems: many students are not showing up in the testing records, and the environments in which some assessments are being distributed are far from ideal. Second, teachers have a limited capacity to control the learning opportunities of students that are not physically present in their classrooms, so tying accountability to those teachers would be unfair.

On top of that, it is possible that an end-of-year assessment would only capture the students who have been able to maintain their learning, thus missing the students who need the most support. Thus, we have to be very careful here in the interpretation of these scores and ask ourselves whether or not we need them.

Q: The report offers many suggestions for developing equitable and appropriate assessments, including a balanced assessment system. Could you describe what this suggestion means?

Shores: Our overall suggestion is to provide a richer and more comprehensive description of student learning. Part of this means that we move away from the deficit lens, which emphasizes where a student fails to “measure up.” Instead, we can recognize that student learning is always happening and that tests can provide information to improve instructional practice and student learning.

Second, we want to make sure that the assessments are tied to the instructional content teachers actually deliver. We also recognize that a single assessment is not going to capture the full range of student learning, so we want to include multiple ways to measure learning, which can include test scores but also student portfolios, for example. The report references a comprehensive assessment system, used in New Mexico, that provides assessment tools in multiple formats (paper and digital) with flexible administration so that teachers can adapt to remote, hybrid, or in-person learning.

Finally, since all of these assessment systems are focused on outcomes, it’s important to also document processes, which we referred to as “Opportunities to Learn” (OTL). OTL characterizes the experiences students had during the year, so it is focused on process and not just outcomes. It’s really important that we understand what the learning experiences of students were like this year, so that we can properly contextualize our interpretation of the assessment data and provide compensatory instruction to students when possible.

Article by Jessica Henderson.

Photo by Kathy F. Atkinson

Read this article on UDaily.