Program Profile: Citizen Schools Extended Learning Time Model

This is an afterschool program that prepares middle school students for academic and social success. The program is rated Ineffective. Participants showed statistically significant higher rates of attendance and a greater likelihood of being on track to graduate and passing 12th grade English/language arts (ELA) than nonparticipants. Groups did not differ in ELA or math test scores, 12th grade suspensions, passing ELA and math comprehensive tests, or on-time promotion to 12th grade.

An Ineffective rating implies that implementing the program is unlikely to result in the intended outcome(s).

An Ineffective rating implies that implementing the program is unlikely to result in the intended outcome(s).

Program Goals/Target Population

The Citizen Schools Extended Learning Time (ELT) model is an afterschool enrichment program, which is designed to prepare middle school students to achieve long-term academic, social, career, and civic success through supportive relationships with adults. Citizen Schools uses hands-on learning experiences and supportive relationships with adults to provide positive youth development opportunities and help youth successfully complete high school, attend college, and participate in and contribute to the civic and economic life of their communities. Citizen Schools partners with middle schools serving predominantly low-income, minority, and academically struggling students.

Program Components

The Citizen Schools ELT model extends the length of the student learning day 3 to 4 days a week, to offer activities in three program components: apprenticeships, academic support, and explore. Apprenticeships connect students to adult volunteers, called citizen teachers, who teach a skill or content area that they are passionate about, including robotics, mock trials, poetry, dance, and other areas. Students are assigned to their apprenticeships based on their topic preferences and the availability of citizen teachers. Students participate in four apprenticeships each year, two per semester. Each apprenticeship consists of 10, 90-minute sessions per semester that culminates in a WOW! Showcase, during which students “teach back” what they learned to friends, family, and community members.

Academic support is provided through daily, 1-hour, structured homework time and includes one-on-one goal setting and tutoring. Additionally, focused academic support (also known as the Academic League) in either English/language arts (ELA) or math is offered twice a week for between 30 and 90 minutes.

The Explore component provides additional enrichment activities using team-building exercises. While these activities may be designed for each participating school’s priorities and circumstances, they are intended to connect middle school students’ experiences with knowledge/guidance about college and careers, and how they can develop a pathway to future goals.

In addition, Citizen Schools’ grade-specific program element, the Eighth Grade Academy (8GA), consists of a capstone experience that supports students as they navigate from middle to high school, including helping them apply to and succeed in a competitive high school. Additionally, 8GA includes experiential learning activities designed to build academic and life skills and provide access to coaches, technology, internships, and other educational programs that can contribute to high school success.

At the beginning of the school year, students are assigned to a team to participate in various activities. The number of student teams at each school and the number of students in each team may vary, ranging from 4 to 10 teams per school and 7 to 25 students per team. Team assignment may be based on grade level, homeroom, or the students’ regular math and/or ELA teachers. In general, students remain on the same team for both the first and second semesters of the school year.

Key Personnel

Program components are implemented by citizen teachers, teaching fellows, and teaching associates. Citizen teachers (the supportive adults) are members of the local community recruited from business, government, the arts, and education fields. Teaching fellows and teaching associates, who provide programming and support for the extended school day, are primarily recent college graduates who have been hired as Citizen Schools’ AmeriCorps members. Teaching fellows generally work full time and are expected to serve for 2 years, and teaching associates work part time. Teaching fellows are also expected to have prior experience working with children.

Campus directors, who provide professional development, coaching, and mentoring to their teaching fellows, are generally expected to have more relevant experience, whether through Teach for America, charter schools, or international teaching experience. Additionally, because the Citizen Schools program is based in public schools, it includes regular communication with students’ school-day teachers and with parents.

Program Theory

The conceptual framework of the Citizen Schools ELT model reflects the expectation that when certain structural and institutional features and certain process and contextual features are delivered to participants with certain characteristics at a certain level of frequency and duration, participants will exhibit expected short-term and long-term outcomes (Sinclair et al. 2010). Structural and institutional features include program goals, staff characteristics (e.g., education, expertise, supports), program structure (e.g., staff–student ratios and group size), and program partners/links (e.g., relationships with parents, schools, and communities). Process and contextual features include content-based program activities (academic and non-academic skill-building activities), content delivery strategies (structured and unstructured learning opportunities that promote autonomy, choice, and leadership), and positive personal relationships (including staff–youth and peer relationships).

Study

Fountain and colleagues (2016) evaluated the effects of Citizen Schools on state ELA and math test scores among middle school students using a comparative interrupted time series. Program participants attended one of 27 schools, in seven different states, which started implementing the program in the 2010–2011 (Cohort 1), 2011–2012 (Cohort 2), 2012–2013 (Cohort 3), and 2013–2014 (Cohort 4) school years. Each program school was matched with multiple comparison schools within the same school district and with similar student demographics and achievement profiles. The authors used Mahalanobis distance-matching methods with maximum permitted difference between matches, or calipers (Rosenbaum and Rubin 1985). Further, school and district staff provided feedback about the suitability of the prospective matches regarding important contextual factors such as school status (e.g., slated for closure or restructuring) or impending implementation of other, similar expanded learning time initiatives. Following this, the top four ranked schools became the matched comparison schools. Citizen Schools and matched comparison schools were assessed for comparability on baseline scores on ELA and math tests; however, these differences were not statistically significant.

Following test administration to students in the spring, ELA and math achievement data were obtained from publicly available state education databases in the fall. Data were available for up to 5 years prior to the start of the Citizen Schools program and for up to 4 implementation years. Cohort 1 schools had data for up to 4 implementation years, Cohort 2 schools had data for 3 years, Cohort 3 schools had data for up to 2 years, and Cohort 4 schools had data for 1 year. Because achievement tests differed across states, test scores were standardized for comparability.

Analysis models adjusted for individual- and school-level covariates (i.e., race/ethnicity, English proficiency status, low-income status, and gender), used school fixed effects to adjust for school-level preexisting differences that may have affected outcomes, and controlled for baseline outcome measures.

Study

Arcaira and colleagues (2010) conducted a longitudinal evaluation of Boston Public School (BPS) students who participated in the Citizen Schools Eighth Grade Academy (8GA) between 2001 and 2006. The sample included 448 former 8GA participants in five cohorts (classes of 2001–2002 through 2005–2006) from several schools as they graduated from high school. Program participants were compared with similar nonparticipants within BPS and matched based on demographic and performance characteristics, including gender, race, grade in school, eligibility for free or reduced-price lunch, student test scores on fourth grade Massachusetts Comprehensive Assessment System (MCAS) tests in math and ELA, and school attended.

About half of the participants and matched nonparticipants were female. About three quarters (71 to 74 percent) were African American, 18 to 21 percent were Hispanic, and the remaining students were Asian, Native American, or white. Participants and matched nonparticipants scored at proficiency on their fourth grade ELA and math MCAS exams at similar rates. Participant and nonparticipant groups were not statistically different at baseline, except for 10th grade free or reduced-price lunch status.

The study authors collected the students’ final outcome measures in 12th grade, 4 years after the end of the program. These measures related to school engagement (i.e., attendance and suspension), academic achievement (i.e., course grades and standardized test scores), and progress toward and achievement of high school graduation (i.e., on-time promotion and 4-year graduation rate) . School attendance was measured as a ratio of the number of days students attended school to the number of days their schools were open. The school suspension measure assessed whether students had any suspensions during a school year. Academic performance was measured as whether students scored at the proficient or advanced level on 10th grade MCAS ELA and math tests and whether they passed their core ELA and math courses. On track to graduation was measured using a composite indicator that incorporated measures of grade promotion and final grades in core ELA and math courses. On-time promotion was measured by whether students were promoted to the next grade level within defined time periods, and 4-year graduation was measured by whether students received their diploma within 4 years of enrolling in ninth grade.

Analyses compared the rates of outcomes among participants and their matched nonparticipants using paired samples t tests and odds ratios, indicating the probability of reaching a particular outcome for both the intervention and comparison groups. Effect sizes were computed and reported as Hedge’s g or Cox index scores (ESCox).

The chief mechanism by which program components are delivered is through an embedded “second shift” of educators, funded by AmeriCorps, who are hired to support the schools’ extended day, either as teaching fellows (TFs) or teaching associates (TAs). Program delivery includes ensuring that TFs are prepared to work in campuses attended primarily by low-income students of color, from families of diverse backgrounds, who may speak different languages, and whose cultural norms may well differ from those of the TFs. Furthermore, TFs can also benefit from ongoing training and professional development opportunities, to develop their classroom management skills. Analyses of implementation data by Fountain and colleagues (2016) found that the adequacy of local (both school and district) resources, the support of school and district leadership, clear communication with stakeholders, and the integration of Citizen Schools staff into the functions of the partner school were important for smooth program launch and implementation within a school.

Although partner schools do not necessarily adhere to the same universal schedule network-wide, schools typically schedule activities across all three program components each week. Students learn about possible apprenticeship topics early each semester via an Apprenticeship Fair. They rank their top choices after hearing citizen teachers present short pitches.

Additional implementation information is available at https://www.citizenschools.org/.

Subgroup Analysis

Fountain and colleagues (2016) also evaluated the effects of the Citizen Schools Extended Learning Time model on middle school students’ academic performance by grade, implementation year, dosage, and implementation levels. Exploratory analyses found evidence of a statistically significant positive impact of Citizen Schools on math achievement in the first year of implementation, and a marginally significant positive effect on 7th grade math achievement.

Arcaira and colleagues (2010) also assessed the effects of the program on students’ selection of and persistence in top-tier high schools. They found that program participants were more than twice as likely to enroll in top-tier high schools, were more likely to persist in a top-tier high school from 9th to 11th grade and from 9th to 12th grade, and were more than three times as likely to be enrolled in and complete all 4 years of high school in a top-tier high school, compared with matched nonparticipants. They also found that former Citizen Schools participants outperformed their matches on high school graduation rates, regardless of the quality of their high schools.

These sources were used in the development of the program profile:

These sources were used in the development of the program profile:

Espino, Juan, Lara Fabiano, and Lee M. Pearson. 2004. Citizen Schools: Evidence From Two Student Cohorts on the Use of Community Resources to Promote Youth Development. Boston, Mass.: Policy Studies Associates, Inc.

Sinclair, Beth, Yvonne Woods, Andrea Palmiter, Joy Mears, and Elizabeth R. Reisner. 2010. Extending the Citizen Schools Model to Sites Across the Country: Report on the Program’s National Implementation. Boston, Mass.: Policy Studies Associates, Inc.

Rosenbaum, Paul R., and Donald B. Rubin. 1985. “Constructing a Control Group by Multivariate Matched Sampling Methods That Incorporate the Propensity Score.” The American Statistician 39(1):33–8.

Following are CrimeSolutions-rated programs that are related to this practice:

Mentoring for Youth Development

This practice provides youth with a positive and consistent adult or older youth relationship to promote healthy youth development and social functioning and to reduce risk factors. The practice is rated Effective in reducing delinquency and improving educational outcomes; Promising in improving psychological outcomes and cognitive functioning; and Ineffective in reducing substance use.

Evidence Ratings for Outcomes

	Crime & Delinquency - Multiple crime/offense types
	Education - Multiple education outcomes
	Mental Health & Behavioral Health - Psychological functioning
	Mental Health & Behavioral Health - Cognitive functioning
	Mental Health & Behavioral Health - Social functioning
	Drugs & Substance Abuse - Multiple substances

Why might a practice's outcome ratings differ from the ratings of specific programs encompassed by that practice?