Program Profile: iMentor’s College Ready Program

This mentoring program for urban high school students is designed to improve college readiness. The program is rated Ineffective. Compared with comparison students, treatment students were more likely to graduate and reported higher self-advocacy and critical thinking. These differences were statistically significant. There were no statistically significant differences in attending college, taking AP courses, task persistence, sitting in on college courses, seeking help, or growth mindset.

An Ineffective rating implies that implementing the program is unlikely to result in the intended outcome(s).

An Ineffective rating implies that implementing the program is unlikely to result in the intended outcome(s).

Program Goals/Target Population

iMentor College Ready Program is a school-based mentoring program that targets low-income urban high-school students. The goal of the program is to teach the social and emotional learning (SEL) skills required to successfully navigate college and to improve college readiness overall.

Program Activities

From 9th through 12th grade, students attend weekly in-school classes taught by iMentor staff using a “college ready” curriculum that targets specific SEL skills. Students also are matched with a mentor with whom they communicate electronically during designated class time for the duration of the program. Both the mentor and mentee are provided with prompts related to the skill being taught in the course, but students can also reach out to mentors for personal guidance and advice. Mentors are expected to respond to students’ posts or emails on a weekly basis.

Every 4 weeks, students also participate in afterschool events that mentors attend in person. These events, which usually take place at students’ schools around 6 p.m. to accommodate mentors’ work schedules, typically include guided discussions or worksheets related to the students’ course curricula. In 11th and 12th grades, events sometimes take place on local college campuses.

Key Personnel

Program mentors are college-educated persons who can commit to 4 years of mentoring and are recruited through corporate volunteer programs and general advertising. Each mentor must pass a background check and attend a 2-hour training on mentor expectations and the program model.

Program Managers (PMs) are paid staff hired by iMentor for each participating school. Some larger schools have a PM for each grade. PMs match and provide support to mentor pairs, teach the course curriculum, and coordinate program events. PMs are expected to check in with each mentor at least five times a year and to send mentors weekly group emails with updates about school and iMentor activities.

PMs are supported by Program Directors, who liaise between iMentor’s central office and school staff. Participating schools appoint a staff member, such as a guidance counselor or teacher, to serve as the iMentor point person within the school.

Study

Merrill (2020) used a quasi-experimental design to investigate the effectiveness of iMentor’s College Ready Program in a sample of about 1,600 students from two consecutive cohorts of incoming ninth graders in the 2012–13 and 2013–14 school years at eight New York City high schools. To be eligible, schools needed to serve low-income students and have a principal who was committed to the program. Compared with other New York City high schools, iMentor schools were more likely to serve Latino students and were less likely to serve White or Asian students. They also had more English language learners and students from low-income families. In addition, students in the iMentor schools scored somewhat lower on the New York state English and language arts and mathematics assessments and were more likely to be chronically absent, compared with students in other New York City high schools.

All incoming ninth graders at schools where iMentor was implemented were eligible to participate. Treatment group participants participated in the iMentor program, while comparison group students attended school as usual and had access to alternative college planning resources.

Two sets of comparison groups were used to assess student outcomes. The first group served as a comparison to assess social and emotional learning (SEL) and future planning outcomes, using annual surveys. Students who began ninth grade in the year before iMentor’s implementation served as the comparison group. The initial sample included 1,678 students who were on each school’s ninth grade roster as of Oct. 20, 2012 (for comparison schools), and Oct. 20, 2013 (for treatment schools). Of these students, 1,438 responded to the posttest survey in the spring of their senior year. At pretest, the treatment group (n = 1,177) was 45.6 percent female, 3.0 percent Asian, 32.2 percent Black, 62.2 percent Latino, 1.3 percent White, and 1.3 percent Other. Of the treatment youths, 22.9 percent exhibited chronic absenteeism in eighth grade, 35.4 percent were overage as of ninth grade (i.e., 15 years or older), 88.9 percent received free/reduced lunch, 28.3 percent were English language learners, and 21 percent received special education in eighth grade. The comparison group (n = 501) was 52.3 percent female, 47.7 percent male, 2.8 percent Asian, 34.8 percent Black, 59.1 percent Latino, 1.1 percent White, and 2.0 percent Other; 27.2 percent exhibited chronic absenteeism in eighth grade, 36.4 percent were overage as of ninth grade, 85.2 percent received free/reduced lunch in eighth grade, 22.9 percent were English language learners in eighth grade, and 21.9 percent received special education in eighth grade. There were no statistically significant differences between iMentor treatment group students and comparison group students on demographics, nor on their average eighth grade New York state math test and English and language arts test scores.

The student survey contained more 100 items, including measures of SEL and background demographics. Task persistence, defined as a student’s ability to sustain effort despite setbacks, was measured using eight items on a four-point scale, from strongly disagree (1) to strongly agree (4) [Walker and Arbreton 2004]. Critical Thinking, which measured problem-solving abilities, consisted of five items rated on a five-point scale from never (1) to always (5) [D’Zurilla and Maydeu–Olivares 1995]. Growth mindset, meaning one’s belief that intelligence and performance improve with effort, was measured using three items on a four-point scale, from strongly disagree (1) to strongly agree (4) [Richards, Ellis, and Neill 2002]. Help seeking, referring to the extent to which students will ask for help in class, was assessed with five items developed for the study on a five-point scale, from never (1) to always (5). Finally, self-advocacy, or the degree to which students promote themselves and their competencies, was measured using four items on a five-point scale, from never (1) to always (5) [Bolino and Turnley 1999].

Program effects on outcomes were assessed using a student-level fixed-effects model that controlled for school-fixed effects, baseline scores on the outcome measure, demographics, pre–high school characteristics (including being overage in ninth grade), and baseline scores on a measure of self-efficacy.

A separate comparison group was used to assess graduating with any degree and attending any college using administrative records provided by the New York Department of Education in spring 2018, the treatment group’s senior year. Sixteen comparison high schools were chosen from a pool of 356 potential matches that had a single admission method (i.e., screened or limited unscreened, not both), and were not a special education, charter, or specialized high school. Students from these schools who entered ninth grade in 2013, the same year the iMentor program was implemented, served as the comparison group. The treatment group (n = 764) was 51.2 percent female, 2.8 percent Asian, 34.8 percent Black, 59.7 percent Latino, 1.1 percent White and 2.0 percent Other. Average middle school attendance was 87.7 percent, 36.4 percent were overage as of ninth grade, 92.44 percent received free/reduced lunch, 24.9 percent were English language learners, and 17.6 percent received special education in eighth grade. The comparison group (n = 1,571) was 57.3 percent female, 42.7 percent male, 6.3 percent Asian, 28.5 percent Black, 60.6 percent Latino, 1.3 percent White, and 1.5 percent Other. Average eighth grade attendance was 88.2 percent, 31.9 percent were overage as of ninth grade, 92.3 percent received free/reduced lunch in eighth grade, 18.1 percent were English language learners in eighth grade, and 18.4 percent received special education in eighth grade. There were no statistically significant differences between the iMentor treatment group students and comparison group students in demographics or in their average eighth grade New York state math test and English test scores.

A comparative interrupted time-series analysis was used to assess program impacts on graduating with any degree (i.e., a Regents Degree, or a Local Degree that requires a lower score on the New York state Regents Exam) and attending any college (i.e., a 2- or a 4-year college), controlling for school, school year, demographics, and pre–high school characteristics. The models estimated trends in these outcomes in the years leading up to the start of iMentor and deviations from those trends after iMentor implementation began. In the years before the intervention, treatment and comparison schools did not statistically significantly differ in their proportion of “on track” students (defined as passing at least one Regents Exam and earning at least 10 course credits). During the pre-intervention period, students across the two groups of schools also did not statistically significantly differ in weighted grade-point averages, chronic absenteeism, having passed one Regents Exam, credits earned, academic credits earned, or attendance.

The study conducted subgroup analyses to investigate whether students and schools who met program participation goals differed in their outcomes from those who did not and whether students who reported feeling very close to their mentor differed in their outcomes from students who rated their relationships as less close.

The program requires a college-ready social and emotional learning curriculum developed by iMentor and background checks for volunteer mentors. The program also uses proprietary software that serves several purposes. First, the software includes an algorithm that assists Program Managers (PMs) with matching mentor–mentee pairs based on mutual interests. Second, it is used to collect program implementation data, including length of matches (mentor–mentee pairs), how often matches connect online, event attendance, and the frequency with which PMs contact matches. Third, the software tracks relationship closeness though monthly prompts to students. These data help PMs develop strategies for mentor support. Finally, the software provides a platform for mentors and mentees to communicate with one another online (Merrill 2020).

Additional Outcomes

The study by Merrill (2020) investigated program effects on several other outcomes (not considered for the CrimeSolutions review of this study) at posttest. There were no statistically significant differences between treatment and comparison group students in reports of social support, personal initiative, hope and optimism, curiosity and love of learning, researching colleges or possible career paths, developing a résumé, participating in a college campus event, visiting a college campus, completing the common application, researching a college major, taking the SAT/ACT or participating in preparation (i.e., a class, a practice test, or studying alone), filing a Free Application for Federal Student Aid (FAFSA), applying for a scholarship, submitting an SAR, comparing financial aid offers, making a college list, applying to college, school attendance, credits earned, graduating with a Regents Diploma, College Ready status, attending a 2-year college, or attending a 4-year college. However, students in the treatment group, compared with the comparison group students, were statistically significantly more likely to have compared financial aid offers and to have graduated with a local diploma.

In addition, students and schools who met program participation goals did not statistically significantly differ from those who did not meet program participation goals on any outcome other than growth mindset, which statistically significantly improved for students who fully participated in the program. Students who reported feeling very close to their mentor, as opposed to those who did not feel close to their mentor, scored statistically significantly higher on all social and emotional learning outcomes, and they were statistically significantly more likely to make a college list and compare financial aid offers.

These sources were used in the development of the program profile:

These sources were used in the development of the program profile:

Bolino, Mark C., and William H. Turnley 1999. “Measuring Impression Management in Organizations: A Scale Development Based on the Jones and Pittman Taxonomy.” Organizational Research Methods 2(2):187–206.

D’Zurilla, Thomas J., and Albert Maydeu–Olivares. 1995. “Conceptual and Methodological Issues in Social Problem-Solving Assessment.” Behavior Therapy 26(3):409–32.

Richards, Garry E., Louise A. Ellis, and James T. Neill. 2002. “The ROPELOC: Review of Personal Effectiveness and Locus of Control: A Comprehensive Instrument for Reviewing Life Effectiveness.” Sydney, Australia: Paper presented at Self-Concept Research: Driving International Research Agendas.

Walker, Karen E., and Amy J.A. Arbreton. 2004. After-School Pursuits: An Examination of Outcomes in the San Francisco Beacon Initiative. Philadelphia, Pa.: Public/Private Ventures.

Following are CrimeSolutions-rated programs that are related to this practice:

Mentoring for Youth Development

This practice provides youth with a positive and consistent adult or older youth relationship to promote healthy youth development and social functioning and to reduce risk factors. The practice is rated Effective in reducing delinquency and improving educational outcomes; Promising in improving psychological outcomes and cognitive functioning; and Ineffective in reducing substance use.

Evidence Ratings for Outcomes

	Crime & Delinquency - Multiple crime/offense types
	Education - Multiple education outcomes
	Mental Health & Behavioral Health - Psychological functioning
	Mental Health & Behavioral Health - Cognitive functioning
	Mental Health & Behavioral Health - Social functioning
	Drugs & Substance Abuse - Multiple substances

School-Based Social and Emotional Learning (SEL) Programs

Designed to foster the development of five interrelated sets of cognitive, affective, and behavioral competencies, in order to provide a foundation for better adjustment and academic performance in students, which can result in more positive social behaviors, fewer conduct problems, and less emotional distress. The practice was rated Effective in reducing students’ conduct problems and emotional stress.

Evidence Ratings for Outcomes

	Juvenile Problem & At-Risk Behaviors - Multiple juvenile problem/at-risk behaviors
	Mental Health & Behavioral Health - Internalizing behavior

Why might a practice's outcome ratings differ from the ratings of specific programs encompassed by that practice?