ECURE Impact, Cohort One

Participation Numbers

List Of ECURE Course Implementations, Cohort One.

• PHYS 1115, Survey of Physics. Darcy Baron
• GEOG 1160, Home Planet. Caitlin Lippitt
• GEOG 1160L, Home Planet Lab. Chris Duvall
• CHEM 1215L, Gen Chem 1, STEM Majors Lab. Jo Ho
• CHEM 1225L, Gen Chem 2, STEM Majors Lab. Jo Ho 
• SOC 2120, Intro to Criminal Justice System. Colin Olson
• PSYCH 2250, Brain and Behavior. Ben Clark
• BIOL 2410, Genetics. Ben Flicker (at UNM two-year branch campus)
• BIOL 2410, Genetics. Kelly Howe
• ENVS 1130, Blue Planet. Corinne Myers
• BIOL 1110L, General Biology Lab. Satya Witt
• SOC 2315, Dynamics of Prejudice. Elizabeth Korver-Glenn
• PHYS 1320L, Calculus-based Physics Lab, implementation delayed to Year Two due to Covid. Paul Schwoebel

Cohort One Impact Narrative

What is the impact on the development of the principal discipline(s) of the project?

There are two important limitations to the preliminary findings from Cohort One. First, this dataset encompasses less than 25% of our projected data. Cohorts Two through Four will add additional student enrollments that will complete our study.

Second, Cohort One was offered during the first two full semesters of the Covid pandemic. In Fall 2020 and Spring 2021, most courses at UNM were offered online, with a few exceptions offered hybrid or in person. Consequently, it will be important to compare Cohort One findings with later cohorts that rely primarily on in-person instruction (especially Cohort Three and Four) to determine if the pandemic significantly influenced ECURE student impact. We do anticipate that our findings will change as the cohorts progress.

Analysis Structure and Definitions: Cohort One Data comes from two sources: pre and post ECURE surveys; and student records in Banner. Cohort One student populations are primarily: students in ECURE courses/sections (ECURE or TREATMENT); and students not-in ECURE courses/sections who have been matched to ECURE students using demographic and academic variables (CONTROL). Matching variables include race, ethnicity, gender, age, Pell-receiving status, academic standing, and STEM-affiliation, among others. ECURE students are further subdivided into three categories: students in ECURE courses/sections with “full” research engagement level (FULL); students in ECURE courses/sections with “partial” research engagement (PARTIAL); and students in ECURE courses/sections with “preparatory” research engagement (PREP).

Survey-based data were analyzed using two approaches. First, we compared changes in student responses on the pre and post surveys (GAINS). While this approach provides the most accurate assessment of gains or losses throughout the ECURE semester, it also comes with one primarily limitation. Since response rates for CONTROL students have been lower than desired, the number of these students who have completed both the pre and the post surveys reduces our confidence level in these findings, especially after just one cohort. As a result, we also utilized an “end of term” approach (EOT). EOT allows us to compare end-of-semester perceptions, based only on student responses on the post surveys

Analysis methodology: To evaluate differences associated with either Treatment or ECURE engagement level, we used multiple linear regression for selected survey questions (Likert-scale treated as numeric) and multiple logistic regression for STEM/non-STEM major persistence, college retention, and upper-level transition success. Multiple linear regression adjusted for Gender, Ethnicity and Race, Pell Status, and Academic Standing, and stepwise model selection with Akaike Information Criterion (AIC) was used to identify the explanatory factors; the stepwise starting model is the main-effects model with the scope up to the full two-way interaction model and the minimum model of only with Treatment or ECURE engagement level.  Multiple linear regression model fit assumptions on the residuals are equal variance and normality, which are both assessed visually. Multiple logistic regression assesses model fit using a deviance lack-of-fit test. All models satisfied model assumptions prior to interpretation.

Preliminary Findings:

Impact on science literacy. Science Literacy was assessed based on a seven point scale (ranging from “very unconfident” to “very confident), on the following five questions: How unconfident or confident are you that you can... (1) use technical skills (use of tools, instruments, and/or techniques of your field of study) to do research? (2) generate a research question to answer? (3) figure out which data/observations to collect and how to collect them? (4) explain the analysis results? (5) use academic literature to guide your research? ECURE students showed GAINS in “…use academic literature to guide your research” 0.9 points higher than CONTROL. This means that, on a seven-point scale, ECURE students showed gains in their comfort levels using academic literature nearly one full step greater than CONTROL. For this same question, females showed gains 0.6 points higher than males, and non-Asian/white students showed gains 0.8 points higher than students who are Asian or white. We observed no differences in science literacy outcomes among the three ECURE engagement levels (prep, partial and full). This is a particularly rewarding finding in that ECURE fellows often talked of the challenges associated with introducing undergraduate student to academic and peer reviewed literature that is often written by PhDs for PhDs. This topic was addressed during both ECURE Summer Institutes, and was discussed often at Community of Transformation meetings and informational sessions. As a result, we will catalog the relevant approaches used by Cohort One fellows in anticipation of a publication or conference presentation submission.

Impact on research self-efficacy. Research efficacy was assessed based on four questions from the pre and post surveys. We observed somewhat conflicting results from two of these questions. On the question (1) “How much or little do you perceive yourself as a researcher right now?”, with a seven-point scale ranging from “very little” to “very much”, ECURE NonAsian/White students scored 0.60 points higher EOT than CONTROL students, which would indicate a positive impact of ECURE on those populations. However, on the question (2) “How much or little do you perceive yourself as a future researcher?”, with the same seven-point scale, ECURE students scored 0.45 points lower EOT than CONTROL, which would indicate a negative impact of ECURE. We anticipate that adding new cohorts to the analysis will help us better analyze this research self-efficacy.

Increased STEM degree persistence. For students who started the semester as non-STEM, the probability of students changing majors to STEM was virtually the same (.01 probability) for ECURE and CONTROL. However, for students who started the semester as STEM, the probability of changing to non-STEM majors for CONTROL was 0.34 compared to 0.12 for ECURE. ECURE students are approximately 1/3 as likely to switch out of STEM as CONTROL. We observed no differences the three ECURE engagement levels (prep, partial and full).

Increased next-semester retention. ECURE students were 53% more likely to return to UNM the next semester than CONTROL students [the probability of next-semester retention for ECURE students (0.676) is higher than for CONTROL students (0.441)]. We observed no differences among the three ECURE engagement levels (prep, partial and full).

Impact on Lower Division to Upper Division Transition (TRM). A TRM of “1” means that students are progressing from LD to UD as we would expect them to. A TRM of lower than “1” means that students are progressing more slowly than we would expect. ECURE students are twice as likely to be on track compared to CONTROL students, regardless of their major of STEM affiliation [Probability of having TRM of 1 for ECURE (0.587) is higher (OR=0.257) than Control (0.268)]. However, in our TRM model, there has been little time for students to change their trajectory within one or two semesters. The real test of this metric will be looking at their TRM progress after four semesters. We will conduct this analysis following the Spring 2022 semester.