Research questions:
- How can exposure to science, technology, engineering, and math (STEM) content and practices be measured?
- How can existing measures of instructional practice be used to measure students’ exposure to STEM content and practices?
- What new technology-based approaches could be adapted to create measures of STEM content and practices?
In 2013, the National Research Council identified 14 indicators for tracking the nation’s progress toward improving science, technology, engineering, and mathematics (STEM) education in the United States. This report focuses on indicator 5, classroom coverage of content and practices in the Common Core State Standards for mathematics and the Next Generation Science Standards.
One of the primary limitations of most existing methods is their focus on what the teacher does rather than what students experience. This focus on the teacher can be particularly limiting in classrooms in which different students are engaged in different learning activities simultaneously. These within-classroom differences can result from efforts to differentiate instruction to meet the needs of individual students and are likely to be especially prevalent in classrooms that rely on technology-based, personalized-learning approaches. This report describes the rationale for examining new approaches to measuring students’ exposure to standard-aligned content and practices, summarizes what is known about currently available measures, and explores innovative approaches that might be adopted to create new measures.
Key Findings
Surveys Appear to Be the Most Plausible Method for Measuring Exposure to STEM Content and Practices Across the United States at This Time
- Surveys are relatively inexpensive and can be deployed fairly easily for large-scale data collection.
Technology-Based Learning Systems, Particularly Simulations, Have Potential to Support Future STEM Indicator Measurement Efforts
- New applications of instructional technology offer the possibility of novel data-collection approaches that could gather detailed information on what students are doing and how much time they spend doing it.
The Use of Technology-Based Learning Systems for Large-Scale Measurement Is Likely to Be Limited Because of Variability Across Schools in the Computer-Based Tools and Other Instructional Materials That Are Adopted and Used
- One major factor that will hinder large-scale adoption is the substantial diversity of software products used across schools, along with inconsistency in how data are collected and how the software reports them.
STEM Practices, Such as Those Identified in the Common Core State Standards for Math and the Next Generation Science Standards, Are Enacted Differently in Different Content Areas
- For a large-scale indicator system, measures of these practices are likely to require sampling across classrooms or schools to capture different content areas without overburdening respondents.
Some Opportunities to Engage in STEM Content and Practices Occur Outside of Traditional Courses
- Out-of-school opportunities to engage in STEM learning are common, particularly in such areas as robotics.
Recommendations
- Create a working group to inform indicator development.
- Use multiple measures to collect evidence related to indicator 5.
- Begin by building on existing data-collection tools and systems.
- Design the measures to support longitudinal comparisons.
- Consider incorporating measures of student knowledge into the broader indicator system.
- Avoid attaching stakes to the measures.
- Continue to conduct research on STEM teaching and learning to inform future indicator efforts.
Read full report here.
Source: Laura S. Hamilton, Brian M. Stecher, Kun Yuan of the RAND Corporation. Image courtesy: Inside Higher Ed.