Wilson and Beard study use of writing software in elementary classrooms

Over the past five years, Joshua Wilson, assistant professor in the University of Delaware’s School of Education (SOE) has partnered with Gaysha Beard, supervisor of English language arts at Red Clay School District, to study the effects of automated computer software on elementary school students’ writing.

Wilson’s work studies the use of Project Essay Grade Writing (PEG Writing) software in the teaching and learning of elementary and middle school students. The software, based on work by education researcher Ellis B. Page and sold by Measurement Incorporated, evaluates more than 500 text-level characteristics to deliver immediate feedback on aspects of writing quality like idea development, organization, style, word choice, sentence structure, spelling, and grammar.

While other researchers have tested the reliability of automated scoring software, Wilson’s work is the first to consider how such software could be used in conjunction with classroom instruction. With the help of this software, for example, teachers are able to spend more time addressing higher-level components of their students’ writing, such as critical thinking, content, and idea development, rather than lower-level writing skills like capitalization or spelling.

Collaborative partnership with the Red Clay School District is also integral to Wilson’s work. With SOE professor Charles MacArthur, Wilson and Beard received a two-year, nearly $400,000 grant from the Institute of Education Sciences (IES) in 2017. This grant allowed them to develop a research-practice partnership and study whether the PEG Writing software contributed to growth in the writing of about 3,000 Red Clay students in grades three through five.

“To me, partnership work is about developing a shared plan. The research questions, research design, data collection methods, and eventual publications are all developed collaboratively as opposed to a traditional research study that is conceptualized by the researcher who hopes to ‘sell’ their idea to convince districts and schools to participate,” said Wilson. “In our partnership work, we have made it a goal to enact data collection methods that are sustainable from a district perspective. This has meant identifying ways that the district can support data collection, rather than only the researcher. For the researcher, this means sacrificing a bit of control, but the trade-off, I think, is worth it, knowing that what you’re doing is sustainable.”

Now in the final year of their IES grant, Wilson and Beard are in the process of analyzing the data from the previous year of research. After controlling for student and school-level variables, they have found that students whose teachers used PEG Writing to help them revise their writing showed greater growth in writing quality than students whose teachers did not.

“While the study was not designed to test whether using PEG caused this growth in writing quality, it is encouraging that teachers used PEG in ways that are positively associated with writing outcomes. Even if these teachers provided better writing instruction overall, using PEG was part of that overall package of good instruction,” said Wilson. “In fact, that’s exactly what we would hope for: automated writing evaluation programs are a tool to support good instruction; they are not a substitute.”

The partnership has also been instrumental in Red Clay’s decision to expand the use of PEG Writing to grades three to six districtwide.

Given these promising results, Wilson was surprised to find that teachers did not use PEG Writing as much as he thought they would, even after demonstrating an understanding of the program and how it might be used with their students.

He realized that, even if PEG Writing makes writing instruction easier for teachers and leads to student improvement, the software cannot change a curriculum with little writing in it or adjust a school schedule with minimal time set aside for writing.

“Once teachers understand the technical side of things, like how to use the program, the key is to help them learn how to integrate the program into their instruction and how their instruction may change as a result of using the system,” said Wilson. “When the project started, we were very focused on PEG—what teachers thought of it and how it might support writing outcomes—but now we’ve shifted our focus to examine the match between the system and the curriculum and instruction, a more global or environmental view.”

As a next step in this project, Wilson and Beard hope to develop a comprehensive professional development program aimed at training teachers to integrate PEG with evidence-based writing instruction.

With a team of UD College of Education and Human Development faculty and colleagues from Measurement Incorporated and Colonial School District, they plan to submit an IES grant proposal centered on integrating PEG Writing with instruction and how instruction would change after introducing the software.

Article by Jessica Henderson.

Photo courtesy of the UD College of Education and Human Development.