SBIR-STTR Award

In this project, the research team will develop a prototype of an artificial intelligence and performance-based reading assessment for students in grades three to twelve. The prototype will integrate existing content and develop new content on a range of reading skills, including comprehension, critical thinking, and perspective taking. The components of the prototype will include a student interface for taking the scenario-based assessment, a teacher dashboard for managing assessments, and a backend database for storing assessments. In the Phase I pilot study with 50 students and four grade six educators, the researchers will examine whether the prototype functions as planned, whether students are able engage in performance-based tasks while reading, and if teachers perceive that the reports generated by the prototype are accurate for each student and facilitate making informed instructional decisions to promote student growth.

Purpose: In this project, the research team will develop a scenario-based reading assessment that employs machine learning to personalize student learning. Students who fail to achieve proficiency in reading are at increased risk for dropping out of school. New forms of assessment can be used to measure student performance in real-time and to provide insights to educators to inform instruction. Project Activities: In the Phase I project in 2021, the team developed a prototype which included scenario-based reading assessments for students in Grades 3 to 12 that employ machine learning to personalize the user experience and a teacher dashboard that provides insights to inform instruction on the assessments. The Phase I pilot study with 50 students and four Grade 6 educators demonstrated that the prototype functioned as planned, that students were able to engage in performance-based reading tasks, and that teachers perceived that the reports on student performance were accurate and could be used to provide instruction. In the Phase II project, the team will fully develop the product, including finalizing the content of the assessments, optimizing the user-experience and implementation supports, developing additional scenario-based assessments, and validating the machine learning algorithm to personalize the assessments based on students' level of proficiency. Iterative refinements will be conducted with feedback from educators and students at major production milestones until the product is fully functional. After development concludes, researchers will conduct a pilot study to assess the feasibility and usability, fidelity of implementation, and the promise of the product to assess and improve reading. The team will collect data from 240 students, including 80 each from elementary, middle, and high schools. Half of the sample will be randomly assigned to use the product and the other half will use business-as-usual reading assessment activities. Researchers will compare pre- and post-scores for students' scores on standardized reading measures, including the STAR, iReady, and a New York State standardized test. Researchers will gather cost information using the "ingredients method" and will include all expenditures on things such as personnel, facilities, equipment, materials, and training. Product: The team will fully develop Capti Assess, a scenario-based reading assessment that employs machine learning to personalize assessment for students in Grades 3 to 12. At baseline, the product will evaluate reading skills using a 10-minute screenerthat evaluates comprehension, critical thinking, and perspective taking, and following a machine learning algorithm will route students to content aligned to their reading level. The content of the assessments are from existing evidence-based programs, including the ETS ReadBasixTM (formerly RISE), and the Global, Integrated Scenario-Based Assessment (GISA), which was developed and evaluated through the IES Reading for Understanding Research initiative.