SBIR-STTR Award

This Small Business Technology Transfer Phase 1 research project will develop and implement an on-line performance-based assessment system for quantifying scientific problem solving effectiveness where quantitative measures can be normalized across problem solving tasks allowing longitudinal comparisons to be made across individuals, classes, schools and science domains. The research project will derive a metric that combine estimates of the quality of a strategy that are derived from artificial neural network analysis with the strategic outcomes on problem solving tasks to develop a value-based metric of the problem solving process. This metric will also provide a strong measure of the value of the strategy employed to document the validity, utility, and generality of this assessment measure using an existing dataset of over 200,000 problem solving performances that span grade levels from middle school through the university. Such an extensible formative, summative and programmatic assessment system of learning will have broad relevance for helping teachers to teach, students to learn, and administrators to make informed data-driven decisions through the continual, and real-time formative evaluation of students' problem solving progress, a dimension not frequently or rigorously assessed in today's classrooms, yet a critical component of 21st century skills. This system will impact all levels of science education and would allow cumulative comparisons of problem solving across science domains, classrooms, teachers and school systems thus helping to re-think the ways scientific problem solving is systemically assessed and how the impact of teaching these skills becomes quantified. The development of a commercialization pathway for this assessment tool will be facilitated by the involvement of the American Chemical Society Examinations Institute, the Higher Education Press in Beijing, and the Alternative Education Program of a local school district

This Small Business Innovation Research (SBIR) Phase II research project will investigate methodologies required to scale and disseminate an online performance-based assessment system for quantifying the scientific problem solving skills of middle school students. This Phase II research will be based on the Phase I results which identified the technical, logistical and professional development challenges that influence the rapid calculation, aggregation and real-time, online, reporting of problem solving assessment data to diverse educational stakeholders. The research will first design and implement an Online Analytical Processing (OLAP) model for data analysis and reporting and incorporate these designs into a system scale-up plan to flexibly accommodate the 10-20 fold increase in users indicated by our commercialization plan. A central component of this development will be a data warehouse that will be instrumented allowing the analysis of how teachers access the performance data, which will be linked to a digital dashboard which will provide teachers with an easy, and highly visual access to multi-dimensional assessments of their students and comparison classrooms. Additionally, this information will be used to develop new forms of professional development to support teachers in the better use of the data available. The impact of this extensible formative, summative and programmatic assessment system of learning will have broad relevance for helping teachers to teach, students to learn, and administrators to make informed data-driven decisions through the continual, and real-time formative evaluation of a student's problem solving progress, a dimension not frequently or rigorously assessed in today's classrooms, yet a critical component of 21st century skills. The outcomes of this project should have widespread utility at all levels of science education and should allow cumulative comparisons of problem solving across science domains, classrooms, teachers and school systems thus helping to re-think the ways scientific problem solving is systemically assessed and how the impact of teaching these skills becomes quantified