INTRODUCTION

Despite the diversity of the U.S. population overall, our nation continues to see high rates of gender and racial/ethnic inequality in science, technology, engineering, and mathematics (STEM) fields, with the composition of STEM college majors and the STEM workforce more broadly dominated by White males (e.g., National Science Board, 2016; National Science Foundation, 2017). For example, Hispanics/Latino and Black students, who represent a combined 26% of the U.S. population (21 or older), account for only 15% of STEM college degree holders and 11% of the STEM workforce (National Science Board, 2016). Similarly, women represent only 29% of total workers in STEM fields, although this number varies greatly across fields (with women well-represented in social and biological/life sciences occupations) (National Science Board, 2016). These inequalities, if not addressed, pose critical problems for the country in that a continuing lack of ethnic and gender diversity in these high-income STEM fields has the potential to exacerbate existing financial and cultural stratifications of traditionally disadvantaged groups (Weis, Eisenhart, Cipollone, Stich, Nikischer, Hanson, Leibrandt, Allenm, and Dominguez, 2015).

In response, educators and policymakers alike are working to increase and broaden participation in STEM education (National Research Council, 2011). Over the past several years, inclusive STEM schools have emerged as one strategy for broadening the pipeline to STEM careers with providing high-quality and engaging STEM education for all students, but particularly for currently underrepresented students in those fields (Peters-Burton, Lynch, Behrend, and Means, 2014). As such, the goals of inclusive STEM schools include increasing students’ STEM achievement and their attitudes toward and interest in STEM. While some studies have begun to examine inclusive STEM school student outcomes (e.g., Gnagey and Lavertu, 2016; Means, Wang, Wei, Lynch, Peters, Young, and Allen, 2017), there is currently only limited evidence of the specific mechanisms by which these schools may be working to promote student outcomes and to reduce academic achievement and interest gaps between different groups of students. The current study contributes to the existing literature on this topic by examining whether strategies used in inclusive STEM schools predict science attitude and achievement outcomes, and whether outcomes differ by student gender identity or race/ethnicity. By doing so, we begin to gain a clearer picture of not only if, but how STEM schools may be reducing long-standing gaps in STEM education, in turn potentially contributing to greater equality in the workforce and society over time.

STEM Participation – Inequities and the Importance of Attitudes and Interest

Despite overall improvements in standardized test score performance over the past 25 years, Black and Hispanic/Latino students continue to have lower levels of general academic achievement compared to White students (National Center for Education Statistics, 2013). Gaps are large in STEM disciplines, with recent National Assessment of Educational Progress (NAEP) data showing substantial differences between White and both Black and Hispanic/Latino students in science (NAEP, 2015a, 2015b). Gender differences also exist, with only 19% of female students scoring at or above proficient, compared to 25% of males. Recent Trends in International Mathematics and Science Study (TIMSS) data indicate that the gender gap has steadily narrowed over the past 20 years; however, for example, 8^th grade boys continue to have significantly higher science achievement scores than girls (Martin, Mullis, Foy, and Hooper, 2016). While achievement scores represent only one measure of success, these numbers are important, as ‘test score disparities in elementary and secondary school are highly predictive of corresponding disparities in subsequent labor market outcomes’ (Reardon, Robinson, and Weathers, 2014: 18).

Moreover, female and racial/ethnic minority students who are high-achieving in STEM courses during high school are still less likely to pursue STEM majors in college compared to males and White students, regardless of previous achievement level (U.S. Congress Joint Economic Committee, 2012). This finding suggests that something other than achievement may influence students to stay in-or drop out of STEM fields. Indeed, evidence has shown that students’ attitudes and interests play a role. As such, we focus specifically on students’ intrinsic motivation for science, science ability beliefs, and interest in future STEM careers.

We define intrinsic motivation similar to Ryan and Deci (2000): finding something inherently interesting or enjoyable. Intrinsic motivation is activity-specific, meaning that one might be intrinsically motivated to do one thing, but not another. Additionally, it has been found to result in high-quality learning, but may be impacted positively, or negatively, by parent and/or teacher practices (Ryan and Deci, 2000). Ability beliefs, which include concepts such as confidence, self-efficacy, and self-concept, ‘are defined as the individual’s perception of his or her current competence at a given activity’ (Wigfield and Eccles, 2000: 70). Beliefs about abilities play a large part in various motivation theories, including the expectancy-value theory (an individual’s expectations for outcomes and how they value outcomes motivates their behaviors and performance when seeking such outcomes; Wigfield and Eccles, 2000); social cognitive theory (learning occurs across diverse social contexts and interpersonal dynamics; Bandura, 1997); attribution theory (the way that individuals view and interpret events contributes to their behaviors and thought processes; Weiner, 1985); and self-determination theory (individuals’ intrinsic motivation aside from external influence; Ryan and Deci , 2000).” Unsurprisingly, one’s abilities are measured in different ways, depending on the researchers’ interest in particular aspects of perceived ability (Wigfield and Eccles, 2000). In this study, we focus on student’s confidence in their abilities in science classes. Generally, beliefs about ability have been found to be positively related to meaningful cognitive engagement in tasks and involvement in academic work as well as effort, persistence, and academic achievement (Greene and Miller, 1996). In the literature, interest may refer to situational interest, a more temporary state induced by particular features of an environment or activity, or to individual interest, ‘a relatively stable evaluative orientation towards certain domains’ (Eccles and Wigfield, 2002:114). In the current study we focus on the latter, with the domain being future STEM education and career paths. While intrinsic motivation causes a person to act, interest influences the direction in which he will do so (Fortus, 2014).

High school students’ interest in science has been found to be more strongly related to pursuing a STEM major than STEM course enrollment or academic achievement (Maltese and Tai, 2009). Previous research also suggests that students who have greater confidence in their abilities in STEM, higher levels of motivation, and more positive attitudes towards STEM are more likely to be interested in and to pursue STEM-related coursework and careers (Mau, 2003; Pajares, 2005; Stevens, Olivarez, Lan, and Tallent-Runnels, 2004; Wang, 2013). These findings underscore the influential role that these attitudes can have on students’ pursuit of and persistence in STEM majors and/or careers. Thus, shaping positive attitudes and interests related to STEM subjects may be especially critical when it comes to closing gender and race/ethnicity gaps in STEM participation.

Gender and race gaps in such attitudes are less clear-cut than those observed in academic achievement. For instance, Black students have been found to have lower beliefs in their science abilities compared to White students (Britner and Pajares, 2001), and similarly, females are generally less confident in their science abilities than males (Wigfield and Eccles, 2002). Others, however, have found that Black and Hispanic/Latino students were as likely as White students to report an intention to declare a STEM major (Anderson and Kim, 2006; Hurtado, Eagan, and Chang, 2010). Moreover, studies exploring STEM attitudes have found complex relationships and confounding effects of gender and race. For example, the results of one study indicated that Black males were as likely as White males to report high levels of science self-concept, as well as interest in a science career. White females, Black females, and Hispanic/Latino students of both genders, however, were found to report significantly lower scores than their White male counterparts on both measures (Riegle-Crumb, Moore, and Ramos-Wada, 2011). In a second study, Black female students reported more positive attitudes towards science than White females (Hanson, 2006), whereas other research suggests that Black and Hispanic/Latino females were doubly disadvantaged given that their science achievement test scores as well as their attitudes toward the subject fell far below White males’ scores (Riegle-Crumb, Moore, and Ramos-Wada, 2011). Adding to this, Else-Quest and colleagues (2013) found that males reported a higher self-concept in math than females; however, females rated science as having higher value than males. Taken together, these findings paint a nuanced, yet unclear picture of the relationships between students’ gender and racial/ethnic identities and their STEM-related attitudes.

More research investigating diverse populations of students’ STEM attitudes, interest, and academic achievement, as well as strategies that may promote equality across these groups, is needed. To that end, the current study examined students enrolled in inclusive STEM high schools and associations between specific strategies employed in STEM schools and students’ beliefs in their science abilities, intrinsic motivation for science, and interest in future STEM careers, as well as their academic achievement. Importantly, this study also investigated whether such associations differed for males and females, and for students of different races/ethnicities.

History and Goals of Inclusive STEM Schools

Highly competitive mathematics- and science-focused high schools, which admit students based on academic achievement, have been part of the American education system for many years (Subotnik, Tai, and Almarode, 2011). However, the federal government has more recently increased efforts to improve access to high-quality K-12 STEM education for all students with different backgrounds, regardless of prior academic achievement (White House Office of Science and Technology Policy, 2015). As such, inclusive STEM schools with open-admissions processes have emerged as one alternative to schools with the more traditionally-rigorous academic enrollment requirements (Peters-Burton et al., 2014). These schools allow for greater numbers of students—particularly students from diverse racial, ethnic, and socioeconomic backgrounds—to engage with STEM curricula.

Previous research suggests that inclusive STEM schools have goals for students that differ from many comprehensive, or traditional, high schools (LaForce et al., 2016). These goals include growing student interest and improving academic achievement in STEM-related disciplines, bolstering students’ confidence in their abilities to perform in STEM fields, and increasing the number and diversity of students who pursue STEM majors and/or careers (Unfried, Faber, and Wiebe, 2014). Additionally, inclusive STEM schools put more focus on 21^st century skills such as problem-solving, critical thinking, technology use, and teamwork (LaForce et al., 2016), that are also critical for success in the workplace (Burrus, Jackson, Xi, and Steinberg, 2013). This type of STEM skill-development, along with disciplinary content, is important not only for employment, but also for enabling citizens to make informed and reasoned social and political decisions, as well as personal choices about health, the environment, and other consequential issues (Committee on STEM Education National Science and Technology Council, 2013).

However, there is limited empirical evidence regarding the efficacy and effectiveness of inclusive STEM schools. Moreover, the results of research that has examined the impact of STEM school attendance on students’ achievement outcomes has been mixed (Bicer, Navruz, Capraro, Capraro, Oner, and Boedeker, 2015; Hansen, 2013; Means, Wang, Young, Peters, and Lynch, 2016). For example, research focusing exclusively on STEM schools in Ohio indicated positive effects in science achievement at some of the schools, but found negative effects on academic achievement across subjects (including science) at others (Gnagey and Lavertu, 2016). Means, Wang, Young, Peters and Lynch (2016) found that STEM school attendance had a positive effect on grade point average (GPA), though not on ACT scores, whereas Hansen (2013) found no relationship between STEM school participation and science or mathematics outcomes. One reason for these varied findings may be that, by design, large-scale policy research studies examine inclusive STEM schools as a whole—that is, as one uniform entity—without examining what is happening within the schools themselves that might contribute to effects seen or differences that are observed. Inclusive STEM schools are complex innovations that employ many types of strategies to promote student success (LaForce et al., 2016), and this ‘black box’ approach leaves researchers unable to target specific improvements or explain variation in findings. In order to fully understand whether inclusive STEM schools may indeed be an effective means to reduce inequalities in student STEM outcomes—from short-term ones such as attitudes towards STEM to longer-term outcomes such as participation in STEM majors and/or careers—it is important to consider the specific teaching and learning experiences enacted in these schools.

What STEM Schools Do

Researchers have begun to articulate the common strategies of inclusive STEM schools by looking at existing literature and resources (e.g., Peters-Burton, Lynch, Behrend, and Means, 2014), and by working with STEM school leadership to understand and synthesize the work they do (LaForce et al., 2016). The latter was the approach taken in the current study, where partnerships with school leaders from inclusive STEM high schools across the U.S. led to the creation of an overarching theoretical model for STEM schools. This particular framework articulates eight common themes or goals of inclusive STEM schools—referred to as the 8 Elements—that describe the structures enacted in the schools and behaviors engaged in by school staff and students.

The 8 Elements are: 1) Problem-Based Learning (PBL); 2) Rigorous Learning; 3) Personalization of Learning; 4) Career, Technology, and Life Skills; 5) School Community and Belonging; 6) External Community; 7) Staff Foundations; and 8) Essential Factors¹ (LaForce et al., 2016). Each Element is composed of a number of strategies representing the concrete ways that members of inclusive STEM schools and their communities work to reach the schools’ goals; the Elements serve as organizers for understanding the complex innovation of STEM schools, so that we may begin to unpack exactly how these schools function to improve student outcomes. In this exploratory study, we begin this work by focusing on the first five Elements, which are the frameworks’ student-facing Elements (i.e., those that directly address the schools’ goals for students).

In our prior work (LaForce et al., 2016), we identified how each Element manifests operationally in inclusive STEM schools. For example, Personalization of Learning may occur through strategies such as student autonomy in the classroom and/or having a dedicated Advisory class. Each student-facing Element is made up of between 10 and 17 strategies; thus, inclusion in this study was determined by three criteria: a) association with one or more student-facing Element(s), as noted above, b) measurement viability (described below under Measures), and c) importance to practitioners. The last criterion was determined by identifying the strategies most frequently emphasized in open-ended interview data by inclusive STEM school leaders (LaForce et al., 2016). For example, problem solving projects (PBL Element) and staff supporting the needs of the whole student (including academic, social, and emotional needs; School Community and Belonging Element) were the most frequently emphasized student-facing strategies described by a sample of school leaders as essential to the schools’ missions and goals. The strategies examined, organized by their respective Element(s), and the percent of the sample of school leaders who emphasized each strategy are listed in Table 1. (Note that some strategies are associated with multiple Elements, indicating that these strategies serve multiple goals for students.)

The Current Study

The current study investigated whether implementation of strategies used by inclusive STEM schools predicts students’ STEM-related outcomes and academic achievement, and whether these associations differ by student gender identity and race/ethnicity. To do so, we asked the following research questions to begin to understand the specific ways in which inclusive STEM schools might contribute to improved outcomes for students:

1. How do student-facing strategies used by inclusive STEM schools relate to students’ science attitudes, interest in STEM, and overall achievement?

2. How do these relationships vary for students of different gender identities and races/ethnicities?

This study moves beyond existing research by looking inside the ‘black box’ of STEM schools to uncover findings about the specific strategies that may lead to desired student outcomes, and whether these strategies have the potential to decrease existing interest and achievement gaps for populations currently underrepresented in STEM fields.

METHODS

This paper describes two analyses that were a part of a larger study of inclusive STEM high schools known as “STEM School Study” (NSF# 1238552; see LaForce et al., 2016 for a review). Analysis 1 examines problem-solving projects, supportive relationships, and student culture. Measures for these variables were given to the entire sample of students, as problem-solving projects, supportive relationships, and student culture are variables that students may experience across multiple classes as well as in non-academic settings. Analysis 2 examines student cooperation and teamwork, student autonomy, interdisciplinary connections, cognitively demanding work, and technology use. These measures were administered within a class-specific frame (described below). Because the outcome measures for this study focus on science, we selected students who reported on these measures for their science class, only.

Sample

School identification and sample. Researchers worked with 20 inclusive STEM high schools from seven states over the course of the larger project. Educators and students in these schools were invited to participate in a variety of data collection activities including school leader interviews; teacher interviews, focus groups, and questionnaires; student interviews and questionnaires; and classroom observations. To select schools for participation, we identified seven states with established state-level STEM networks to connect with, as these networks facilitated our ability to recruit schools. Network contacts provided the research team with the names and contacts for a number of schools in their respective states. The final set of schools included four schools each from Ohio, Texas, and Washington, three from California, two from both North Carolina and Tennessee, and one from New York.² While the schools varied in their models and missions, and were founded and developed independently of one another, all schools in our sample had significant overlap in terms of the strategies they employed, as represented in the 8 Elements framework (LaForce et al., 2016).

The current analyses draw on student questionnaire data from the final year of data collection (2015), and included 17 of the original 20 schools (due to two schools dropping out of the study after Year One and an additional school having an insufficient sample of students participating in the final year’s questionnaire).

Analysis 1. Participants in this analysis were 9^th to 12^th grade students (33% 9^th, 28% 10^th, and 25% 11^th, and 14% 12^th graders). The analytic sample consisted of 2,943 students (50% female) who were racially/ethnically diverse (50% White, 36% Hispanic/Latino, and 13% Black).³ Of the 17 schools, 12 were populated predominantly by students identifying as White; the remaining five schools were populated predominantly by students who identified as Hispanic/Latino. Seven of the schools were suburban, six were urban, and four were rural. The number of respondents in each school ranged from 44 to 619 students.

In addition, a subsample of students with school-provided GPA data (N = 944; 39% 9^th, 34% 10^th, 20% 11^th, and 7% 12^th graders) were examined to explore associations between strategies used by inclusive STEM schools and students’ cumulative GPAs. We also explored whether such associations varied based on student gender identity and race/ethnicity. This sample included relatively equal numbers of female (53%) and male students and was racially/ethnicity diverse (46% White, 36% Hispanic/Latino, and 18% Black).

Analysis 2. This analysis looked at a sub-sample of 433 9^th to 12^th students (24% 9^th, 39% 10^th, 21% 11^th, and 16% 12^th graders) reporting on classroom-specific strategies in their science classes. On the questionnaire, students were asked to report on the frequency of instructional practices experienced in a specific class (i.e., the class they attended on Monday mornings at 10am). Proportions of students identifying as male/female, as well as race/ethnicity proportions in this sample were similar to Analysis 1 (48% female; 50% White, 38% Hispanic/Latino, and 12.2% Black). We did not examine GPA in this analysis due to insufficient sample size.

Measures⁴

Outcome variables for all studies (four measures, and all of the items in each measure scored using a six-point Likert scale with higher scores indicating positivity in the attribute being measured (e.g., 1 = strongly disagree, and 6 = strongly agree)):

1. Science intrinsic motivation: Four items (α = 0.96) derived from the Attitude toward Science in School Assessment (Germann, 1988) were used to assess students’ motivation in science (e.g., I find science very interesting).

2. Science ability beliefs: Four items (α = 0.92) were used to assess students’ beliefs about their science abilities (e.g., I have the skills and ability to learn about science). Items were adapted from the Student Self-Report of Academic Self-Efficacy (Hoover-Dempsey and Sandler, 2005; Patrick, Ryan, and Kaplan 2007).

3. Interest in future STEM careers: Four items (α = 0.97) developed for the current study were used to assess students’ interest in pursuing STEM courses in college and in future careers (e.g., I see myself pursuing a career in STEM).

4. Cumulative GPAs (Analysis 1 only): Students’ cumulative GPAs across all academic subjects was provided by the school districts. Cumulative GPAs was standardized using z-scores across all schools in the sample due to different scales used when reporting students’ cumulative GPAs.

Demographic and control variables. Students were asked to indicate race/ethnicity, gender identity, grade, and school type (dummy-coded variables: Predominantly Hispanic/Latino schools and Predominantly White schools. No schools in the sample were predominantly African-American).

This study also included one response bias control variable:

General school intrinsic motivation: Four items (α = 0.86) were used to examine students’ motivation toward schoolwork and school in general (e.g., I enjoy doing my schoolwork). A six-point Likert scale was used (e.g., 1 = strongly disagree, 6 = strongly agree), with higher scores indicating positivity in the attribute being measured. This variable was included to account for response bias amongst students (i.e., students who typically rate all variables high or low).

Key Variables. As noted, measures were selected for their prominence in school leaders’ descriptions of their respective STEM school models. In addition, measures were also required to meet viability criteria. To maximize power, variability, and reliability, we included only items with a Likert-type scale of 5 response options or more, and a Cronbach’s alpha of 0.75 or higher.

Moreover, it is beyond the scope of this paper to thoroughly summarize the literature on each of the strategies outlined here. Instead, we provide a brief summary of each element, describe how it was measured, and share additional resources providing support for each strategy. While evidence of the efficacy of several of these strategies exists, they are relatively understudied in inclusive STEM school contexts. It is important to keep in mind that the goal of this study is to better understand how such strategies work alongside many other strategies in the context of the inclusive STEM schools.

In Analysis 1, key variables were included as (Items in each measure were scored on a six-point Likert scale (e.g., 1 = Never, 6 = Always)):

1. Problem-solving projects measure. A key strategy for PBL is problem-solving projects, which we define as curriculum which includes projects, often interdisciplinary, focused on solving an authentic problem (LaForce et al., 2016). Problem-solving projects may occur within one class, across several classes, or as part of a dedicated timeframe during the school year where students work solely on a specific project (LaForce, Noble, and Blackwell, 2017; for a review, see Savery and Duffy, 1995). Twelve items (α = 0.96) derived from previous research (Munshi, El Zayat, and Dolmans, 2008; Savery, 2006) were adapted for use with high school-age students and used to capture the frequency with which students reported experiencing various aspects of PBL at their school (e.g., How often do you engage in PBL activities that are interesting and relevant to your lives?).

2. Supportive relationships measure. Supportive relationships posit that students feel that their emotional and academic needs are supported by adults at their school (e.g., teachers, leadership, and other school staff). Previous research has found associations between supportive student-teacher relationships and student engagement, and academic achievement (Roorda, Koomen, Spilt, and Oort, 2011; Wu, Hughes, and Kwok, 2010). Six items (α = 0.96) developed for the study were used to assess the extent to which students felt supported by teachers and school leaders (e.g., Adults understand students’ lives outside of school and take them into account when teaching).

3. Student culture measure. Student culture, including mutual trust and respect, adherence to a code of behavior and values, and contribution to school-wide decision-making, plays a role in students’ ability to function at optimal levels needed to achieve academic success in STEM disciplines (Wang, Haertel, and Walberg, 1997). An exploratory factor analysis of three separate measures supported a one-factor solution for this 10-item measure (α = 0.94). Items assessed students’ value systems within their school (e.g., Students always demonstrate good school citizenship), respectful behaviors (e.g., Even when they disagree, students at this school still treat each other with respect), and decision-making opportunities (e.g., Students at this school participate in important, whole-school decisions).

In Analysis 2, five science classroom-specific measures were included as (All items used for each measure were scored using a six-point Likert scale (e.g., 1 = No classes, 6 = More than once in every class session):

1. Autonomy: Being autonomous is defined as students showing independence in and ownership of their learning, setting their own goals for learning, and making choices about how to accomplish those goals. Autonomy has been linked to student engagement (Reeve, Jang, Carrell, Jeon, and Barch, 2004), as well as interest and perceived competence in science (Black and Deci, 2000). Four items (α = 0.79) were used to assess student autonomy (e.g., I set my own goals for learning).

2. Cognitively demanding work: Engaging in cognitively-demanding work refers to students’ ability to use critical thinking and processing skills, to make alternative arguments or explanations, to make predictions, to interpret experiences, to analyze data, to explain reasoning, and to support conclusions with evidence. These types of skills are important for scientific reasoning and effective problem solving in academic, social, and personal settings (Halpern, 1999; Shakirova, 2007). Four items (α = 0.87) were used to explore students’ engagement in cognitively demanding work (e.g., I supported my conclusions with evidence).

3. Interdisciplinary content: Making interdisciplinary connections refers to students’ possessing the ability to identify the ways in which academic disciplines are interrelated, reinforced, and complement one another. The interdisciplinary nature of learning and knowledge, and the importance of students’ understanding of this and their ability to make cross-disciplinary connections, is well-recognized (NGSS Lead States, 2013; Partnership for 21^st Century Learning, 2015). Three items (α = 0.88) were used to examine the frequency with which content taught in science classes was transferable to other subjects (e.g., I used information or content from another subject in an assignment or activity for this class).

4. Technology use: Using technology in an appropriate manner refers not only to students’ use of technology in the classroom, but also to their use of emerging and innovative technologies for meaningful learning. Given the ubiquity of technology in daily life and modern workplaces, incorporating its use in the classroom and promoting students’ digital literacy and technical abilities is critical (U.S. Department of Education, 2017). Three items (α = 0.84) were used to assess the use of innovative technology in learning (e.g., I had all of the technological resources needed to complete assignments/reach goals).

5. Cooperation and teamwork: The ability to collaborate and work as part of a team is considered by inclusive STEM schools to be a critical workplace and 21^st century skill for all students (Partnership for 21^st Century Learning, 2015). Four items (α = 0.85) were used to assess the degree of teamwork among students (e.g., Another student helped me with an assignment or problem that I was struggling with).

Procedure

Teachers at sample schools administered the online student questionnaire between February and May, 2015, during the school day. The questionnaire took approximately 25 minutes for students to complete and addressed a range of school experiences and attitudes.

Analytic Strategy

Multiple regression in the Statistical Package for the Social Sciences © (SPSS) Version 24 was conducted to examine associations between strategies used by inclusive STEM schools and STEM-related attitudes and achievement outcomes. Separate linear regression models were used to investigate students’ science intrinsic motivation, science ability beliefs, interest in future STEM careers, and cumulative GPAs, which served as the dependent variables. In these regression models, main effects were entered in blocks as follows: 1) Student demographics, including gender identity (male served as the reference group), race/ethnicity (White served as the reference group), and grade level (9^th grade as the reference group); 2) students’ general motivation toward school; 3) Student experiences with problem-solving projects, student culture, and supportive relationships, or classroom-specific science strategies. Interaction terms (e.g., problem solving project by gender) were added to the regression models to investigate whether associations between the strategies used and attitudinal and academic achievement outcomes varied across different groups of students. To create the interaction terms for these analyses, all continuous main effect variables were first centered by subtracting the sample mean to create a new mean of zero. Three-level moderator variables, specifically race/ethnicity, were dummy coded, such that White students served as the reference group. Interaction terms were then created by multiplying the centered continuous main effect variables with each of the moderator variables (Frazier, Tix, and Barron, 2004). The Interaction program for Windows (Soper, 2006-2013) was used to depict significant interaction terms that emerged. Holm-Bonferroni sequential correction was adopted to control for Type I Error (Holm, 1979).

RESULTS

Analysis 1: Problem-Solving Projects, Student Culture, and Supportive Relationships and STEM-related Attitudes

Specific F statistics, degrees of freedom, and adjusted \(R^{2}\) for STEM-related attitudes and academic achievement (cumulative GPA) are included in Table 2 and 3.

Females rated STEM-related attitudes (i.e., science intrinsic motivation, science ability beliefs, and interest in future STEM career) significantly lower than males. Compared to White students, Black students reported significantly lower scores on science intrinsic motivation. Twelfth graders reported significantly lower scores on science intrinsic motivation and interest in future STEM careers than 9^th graders, whereas 10^th graders rated interest in future STEM careers significantly lower than 9^th graders. Higher levels of general school intrinsic motivation were significantly associated with higher scores on all three STEM-related attitudes. Higher ratings of problem-solving projects were significantly associated with higher scores on science intrinsic motivation and science ability beliefs. Significant main effects of race/ethnicity and completion of problem-solving projects on interest in future STEM careers were qualified by a significant race/ethnicity by problem-solving projects interaction. White students (\(\beta\) = .41, SE = .03, p < .001, CI95%: [.35, .47]) with higher ratings of problem-solving projects were more likely to have higher interest in future STEM careers than Hispanic/Latino students (\(\beta\) = .23, SE = .04, p < .001, CI95%: [.15, .30]). A significant interaction was also observed between supportive relationships and race/ethnicity for science ability beliefs. In this case, Hispanic/Latino students (\(\beta\) = .33, SE = .03, p < .001, CI95%: [.27, .39]) with higher ratings of supportive relationships also reported higher ratings of science ability beliefs compared to White students (\(\beta\) = .30, SE = .02, p < .001, CI95%: [.25, .35]). See Table 2 for estimates for all variables included in this regression model.

Academic Achievement (cumulative GPAs)

Females had significantly higher cumulative GPAs than males, and Black students had significantly lower cumulative GPAs compared to White students. Ninth grade students had significantly higher cumulative GPAs compared to 11th and 12th graders. Supportive relationships were associated with higher cumulative GPAs. A significant race/ethnicity by student culture interaction were found on cumulative GPAs: Hispanic students (\(\beta\) = .32, SE = .05, p < .001, CI95%: [.23, .42]) had a higher cumulative GPAs than White students (\(\beta\) = .12, SE = .04, p = .001, CI95%: [.05, .19]) (Table 2).

Analysis 2: Science-specific Classroom Measures and STEM-related Attitudes

Three separate models were conducted to examine associations between each instructional practice (i.e., student autonomy, cognitively demanding work, interdisciplinary content, technology use, and student teamwork) and science intrinsic motivation, science ability beliefs, and interest in future STEM careers. General school intrinsic motivation (response bias control) was positively associated with all three outcome variables. A significant race/ethnicity by cognitively-demanding work interaction emerged for science intrinsic motivation. Black students (\(\beta\) = .51, SE = .05, p < .001, CI95%: [.26, .76]) with higher ratings of cognitively-demanding work reported higher levels of science intrinsic motivation compared to White students (\(\beta\) = .31, SE = .05, p < .001, CI95%: [.21, .41]). See Table 3 for estimates for all variables included in this regression model.

DISCUSSION

This study explores how strategies used by inclusive STEM schools may work to improve outcomes for students, and particularly those currently underrepresented in STEM. In doing so, it is one of the first studies to demonstrate the potential value of those strategies within the context of inclusive STEM schools. Other studies examining student outcomes of STEM schools (such as Gnagey and Lavertu, 2016) do not look at student outcomes as a function of the strategies used within STEM schools; instead, they focus on comparisons between STEM and non-STEM schools. Our research complements these between-schools comparison studies by providing a deeper look within the ‘black box’ of inclusive STEM schools to explore how specific strategies may be contributing to student outcomes.

Additionally, this study contributes to our understanding of the implications of strategies used in inclusive STEM schools for students of different races/ethnicities and gender identities. Because arguably one of the central goals of inclusive STEM schools is to diversify the STEM workforce, it is imperative to understand how student outcomes manifest across a diverse body of inclusive STEM school students. Our findings suggest that while some strategies used by inclusive STEM schools are associated with positive attitudes toward science and STEM careers (and in some cases, GPA) for all or some students, these strategies are not sufficient to reduce race and gender gaps. Within inclusive STEM high schools in this study, Black and Hispanic/Latino students still fall below White students in attitudes and achievement. Girls consistently demonstrate higher GPA than males, though they continue to show less positive attitudes towards science and STEM careers. In addition, results indicate that White students receive disproportional benefits from completion of problem-solving projects compared to Hispanic/Latino students.

However, when students report strong, positive experiences with certain strategies used by inclusive STEM schools, there may be potential to narrow some gaps. For instance, Hispanic/Latino students who reported higher ratings of supportive relationships also reported significantly higher ratings of science ability beliefs compared to White students. This result suggests that supportive relationships may have the potential to boost Hispanic/Latino students’ science ability beliefs. This finding aligns with Benner et al. (2017) who found that caring environments and positive connections with educators may be particularly beneficial for ethnic minority youth. Moreover, Hispanic/Latino students who reported higher levels of student culture had higher GPAs than White students who reported the same levels of student culture. This finding also echoes previous studies that have found significant effects of supportive teacher and school staff relationships on high school students’ success (Croninger and Lee, 2001; Benner, Boyle, and Bakhtiari, 2017). In general, supportive relationships and positive student culture may enhance student happiness and/or mental health, and subsequently attitudes and achievement (Dix, Slee, Lawson, and Keeves, 2012). These effects may be particularly critical for students who are marginalized in US classrooms. Marginalized students may benefit more from relationships that help build social capital than non-marginalized students, who may, by definition, have more access to social capital (e.g., useful networks and support) in their education (Fields, 2017).

Furthermore, we examined associations between specific classroom strategies enacted in science classes with a smaller sample of students (i.e., those who reported on their science class, as opposed to reporting on another subject). Female students had significantly lower interest in future STEM careers than males, whereas Black students has higher levels of science intrinsic motivation when they reported engaging in cognitively-demanding work more often in their science classes. This finding may suggest that rigorous teaching, with an emphasis on critical thinking and reasoning skills, in science classes may provide a potential benefit for Black students. Emphasizing cognitively-demanding work in such classroom settings may provide opportunities for students to engage more deeply with science curriculum through carrying out higher levels of interpretation, flexibility, and construction (Tekkumru‐Kisa, Stein, and Schunn, 2015).

Across analyses, we often saw a negative association between students’ grade level and science attitudes, which may be linked to students becoming more in-tune with their abilities, and also, having more opportunities to explore, their interests as they progress through school. Young inclusive STEM high school students may be excited for all of what STEM and science can offer, and over time, they may realize that it is not a core interest, or that they would rather pursue other courses consistent with their academic abilities. Future research is necessary to disentangle this finding further, and to better understand age-related, developmental correlates of science attitudes.

Future Directions

This study contributes to a growing body of research exploring associations between inclusive STEM school strategies and student outcomes. While findings here suggest some preliminary evidence for the usefulness of a number of the inclusive STEM school strategies identified in the 8 Elements Framework (LaForce et al., 2016), further research is needed to better understand the impact of such strategies on diverse populations of students. For example, the data presented here are cross-sectional, collected through a correlational research design, which prevents causal inferences from being drawn. Future studies could incorporate more-rigorous (i.e., longitudinal and experimental) research designs to gather information about growth and change of students STEM attitudes and exposure to STEM school strategies. Future studies also need to continue to dig into the ‘black box’ of inclusive STEM schools to understand which strategies, and what characteristics of them, truly benefit underrepresented groups students. Finally, additional research must be done to examine success indicators of inclusive STEM schools beyond test scores and student attitudes toward science, such as postsecondary transition, persistence with STEM majors, and matriculation to STEM careers immediately after high school.

Limitations

The results of the current study must be interpreted in light of several limitations. First of all, although this study takes a critical first step in examining strategies used by inclusive STEM schools in context, statistical limitations (e.g., investigating large number of variables through a set of models) prevented us from analyzing all strategies simultaneously. It should also be noted that the use of Holm-Bonferroni correction to account for Type I Error limited the number of significant findings that were reported. Thus, larger sample sizes may be necessary to identify all possible associations between strategies used by inclusive STEM schools and student outcomes. Additionally, while we see evidence of success across several individual strategies studied here, less is known about which strategies may be most critical to student outcomes, as well as how these strategies work together within the inclusive STEM school setting. Given the complex tapestry of strategies used by inclusive STEM schools, this will be a challenge for researchers to study in the future; the strategies (and outcomes) in this study, while extensive, are by no means exhaustive of what inclusive STEM school creators and leaders consider critical. Furthermore, the use of students’ cumulative GPA rather than annual GPA is a limitation. Students’ annual GPA, more proximal to the strategies students experienced and reported on at the time of the questionnaire, would be a more ideal achievement outcome variable, rather than cumulative GPA, which is a function of several years of schooling, and thus holds a larger potential for uncontrolled variability. Finally, it is also important to note that this paper reflects implementation as measured by the student voice. We, along with others, would argue that students provide a critical voice in understanding implementation (Cook-Sather, 2006); however, it is only one form of implementation measurement, and capturing teachers’, parents’, and school district administrators’ voices will be an important future direction of this research. Finally, this study did not include the use of interaction terms to examine intersectionality between different groups (e.g., gender, and race/ethnicity) of students. Future research should investigate intersectionality within these groups of students (e.g., African American female students) in order to gain a better understanding of how dual-identities contribute to students’ science ability beliefs, intrinsic motivation, and interest in future STEM careers.

Summary Conclusions

In summary, we see the potential of inclusive STEM school strategies, as well as clear areas for future work. Many strategies used in inclusive STEM schools, when implemented well in such settings, show promise for improving student attitudes towards and interest in STEM, and for reducing some of the long-standing gender and race/ethnicity gaps seen in STEM education. However, within our sample of inclusive STEM schools, who seek in part to provide equitable STEM outcomes for all students, we still see significant and disappointing race and gender gaps in these outcomes. These findings should be considered alongside comparison studies investigating inclusive STEM schools and non-STEM schools that show, at best, varying levels of success for underrepresented students (Means et al., 2017; Gnagey and Lavertu, 2016). While STEM schools seem to be successful at providing a diverse body of students with access to an environment that promotes STEM success, access alone may not be sufficient to eliminate gaps in science attitudes and GPA across race/ethnicity and gender identity. As demonstrated in this study, when students have strong ratings of STEM school strategies – some gaps may diminish, which suggests the potential of these schools, and their associated strategies, to contribute in meaningful ways to the STEM workforce issues at hand in the U.S. Researchers, practitioners, and school district administrations striving to close long-standing gender and race/ethnicity education gaps should not overlook the importance of these implications when it comes to correcting and closing long-standing educational gaps between diverse members of our society.

1. The descriptive nature of the 8 Elements framework is critical to note. The Elements were derived directly from conversations with inclusive STEM high school leaders about what the schools do and what defines them as STEM schools; therefore, these Elements and components do not represent a set of strategies we defined or mean to prescribe for schools. Rather, the Elements tell us what established inclusive STEM schools are doing. While many of these practices are rooted in the literature and empirical evidence, it is necessary to investigate the specific strategies to understand if, in the context of STEM schools, they do indeed have the potential to improve student outcomes overall, and to reduce gaps in achievement. See LaForce et al., 2016 for more detail on each of the Elements and the process used to derive them.↩

2. Detailed school recruitment strategy is available from the authors upon request.↩

3. Across both Analyses 1 and 2, samples included students who identified as other race/ethnicities (e.g. Native American/American Indian, Mixed Race) and/or non-binary gender identities; however, the number of these students was insufficient to include as variables in these analyses.↩

4. All attitudinal measures (predictor and outcome variables) were developed and revised across the three years of questionnaire administration. The measures utilized in the present study represent final instruments administered in Year 3 (2014-2015). Further data and documentation on the validation and adaptation of these measures can be obtained from the authors upon request.↩