Improving Results and Expanding Learning: Using Research and Evaluation to Inform Practice in New Jersey 21st Century Community Learning Centers

The 21st Century Community Learning Centers program in New Jersey is offering expanded learning opportunities to thousands of students, with significant and positive results. The program aims to assist children who attend low-performing schools in high-poverty areas to attain the skills needed to meet the state’s content standards (www.state.nj.us/education/21cclc). 


Currently, nearly 17,000 youth are participating in these important learning opportunities in 21st Century Community Learning Centers programs at more than 122 sites across the state. The state’s program goals call for a well-aligned, engaging, and individualized expansion of learning time beyond the school day that provide


• remedial education activities to increase students’ college and career readiness; 

• a broad array of creative activities (art, music, dance, recreation, and cultural activities) that complement the school day and equalize enrichment opportunities; 

• family literacy and other activities that assist families in becoming full partners in the education of their children; and 

• support services that target social, emotional, and character development to deter problem behaviors. 


Since the inception of the 21st Century Community Learning Centers program , the New Jersey Department of Education (NJDE) has promoted program quality and continuous improvement by applying promising practices described in the emerging body of research on expanded learning programs and other research on teaching and learning. New Jersey’s 21st Century Community Learning Centers program embeds many of the recommendations highlighted by the Institute of Education Sciences in a seminal publication, Structuring Out-of-School Time to Improve Academic Achievement: A Practice Guide (Beckett et al., 2009). Going beyond the federal requirements, New Jersey’s program requirements have evolved to support college and career readiness and to embed the components of successful expanded learning opportunities. Of equal importance in informing programmatic decisions is state-level information provided by the state’s program evaluator.


New Jersey significantly modified its program in 2010 to require many of the strategies that the NJDE had been promoting in recent years and that put into practice the latest research on expanded learning.


This redesign conveys the expectation that local programs would incorporate five major elements:


1. Aligning the 21st Century Community Learning Centers program with school-day learning to provide more time for youth to practice skills and expand knowledge.
The state’s 21st Century Community Learning Centers programs are expected to link their activities to the New Jersey Core Curriculum Content Standards and the Common Core State Standards. Further, they are expected to document these links with the school day through lesson plans, progress reports, and regularly scheduled meetings. Activities are designed to assist youth with the development of skills as well as content knowledge. Centers must focus on one of the following themes: science, technology, math, and engineering (STEM); civic engagement; career awareness and exploration; or visual and performing arts.


   To support the alignment with the school day, programs are expected to have regularly scheduled communication and intentional planning between school day and center staff. Each program designates a regular school-day staff person at each school site to coordinate communication with the afterschool program to help them support school needs. Afterschool program staff participate in school meetings and committees, such as professional learning communities and school improvement teams.


   Certified teachers are required for academic remediation activities in 21st Century Community Learning Centers programs. Local programs coordinate with their affiliated schools to identify appropriate staff for the center and hire classroom teachers who demonstrate success during the school day to continue to build a positive relationship. 


   21st Century Community Learning Centers programs link professional development to identified, school-based goals and learning objectives and conduct joint training for both school-day and afterschool staff on relevant topics, such as how children and youth learn and develop, how to establish appropriate learning environments, and how to deliver crosscurricular content. 


2. Developing the capacity of staff and promoting networking. New Jersey’s 21st Century Community Learning Centers programs are expected to build relationships with school day staff through joint professional development opportunities between school-day/district and center staff. This joint professional development provides a forum for staff to learn about each other’s assets while acquiring a common professional language, learning the same instructional strategies and techniques, and gaining new information about programs and approaches being implemented. 


   New Jersey’s programs are also participating in action research to self-evaluate and continuously improve their programs using research-based practices. This strategy provides the opportunity for job-embedded professional development through a community of practice. The strategy also promotes more intentional and frequent interaction between the evaluator and program staff to assess the effectiveness of the practices being implemented.


3. Maximizing student engagement and attendance. New Jersey’s programs operate at least 3 hours per day, 5 days per week, during the school year and at least 4 hours per day, 4 days per week, for 4 weeks in the summer to engage youth in additional learning opportunities and reduce summer learning loss. The required theme-based programming establishes relevance and interest through cross-content integration of information and skills. It also roots experiences in the real-world and promotes multisession involvement.


   To address the challenge of enticing youth to attend regularly, local programs are required to provide transportation, offer engaging learning experiences, create a youth-centered environment, and use guided-inquiry to increase opportunities for experiential learning, problem solving, self-direction, creativity, exploration, and expression. 


   Student voice is critical to student engagement. To include youth intentionally in the design of learning experiences that are relevant and interesting to them, programs are expected to have a student council that meets at least bimonthly. 


4. Establishing partnerships and focusing on sustainability. Each of New Jersey’s 21st Century Community Learning Centers programs is expected to create and maintain a set of partnerships that produce tangible resources that directly benefit participants. To assist programs in meeting this expectation, the centers are required to maintain a stakeholder advisory board comprised of partners, collaborators, the evaluator, parents, a youth representative, and other interested parties that meets at least quarterly. The advisory board offers guidance in the areas of program planning, implementation, evaluation, and sustainability. 


   The NJDE collaborates with NJSACC-New Jersey’s Afterschool Network to provide training and technical assistance to the 21st Century Community Learning Centers programs and other potential grantees. These professional development opportunities have focused on promoting partnership building and program sustainability, among other research-based strategies that support quality afterschool programs statewide. Together NJDE and NJSACC developed the NJ Celebrates Afterschool Toolkit to help programs conduct open house events for parents, community members, and potential partners to increase awareness of the program’s offerings and benefits. 


5. Promoting family engagement. The program’s advisory board includes parent representation. Also, programs are expected to provide parents with an opportunity to provide input on all facets of the program, inform parents of participants’ progress, and formally invite parents to attend program events. Also, local programs are required to provide adult family members of participating students with opportunities to participate in an array of literacy activities. 


Assessing Program Performance and Using Data for Continuous Quality Improvement


Using the state-level goals and objectives that are prescribed by the NJDE, local 21st Century Community Learning Centers programs establish their own performance indicators. They contract with independent evaluators to conduct local evaluations to measure progress toward the achievement of goals, objectives, and indicators. The local evaluation gauges the impact of the program on participating students and families, including student attendance, student engagement during the school day and during the afterschool program, parental involvement, and skills acquired by parents.


NJDE has contracted with American Institutes of Research to conduct a state-level evaluation of its 21st Century Community Learning Centers programs. Positive findings in the evaluation of the 2009-10 programs noted in Table 1 include the following:


• Students who attended the center for 70 days or more during the school year performed better on state assessments in mathematics compared to similar students who did not participate in the center.

• Students with higher attendance in 21st Century Community Learning Centers programs showed greater improvement in teacher-reported student motivation, attentiveness, pro-social behaviors, and homework completion/quality.

• Students who participated in the program for multiple years performed better on state assessments in reading and mathematics. 


Table 1. Positive student outcomes linked to 21st Century Community Learning Centers program participation.


Naftzger, N., Vinson, D., Manzeske, D., and Gibbs, C. (2011). New Jersey 21st Century Community Learning Centers (21st CCLC) impact report 2009–10. Naperville, IL: American Institutes for Research.


Leading Indicator System to Make Further Advancements in Quality and Achievement


One of the goals of the statewide evaluation is to provide 21st Century Community Learning Centers grantees with feedback about their performance in the areas of program design and delivery. NJDE is therefore working with American Institutes of Research on the development of a leading indicator system to enhance its understanding of the impact of the New Jersey 21st Century Community Learning Centers program. 


The focus of the leading indicator system is on quality implementation that has potential to lead to positive youth outcomes, rather than just focusing on assessing the achievement of youth outcomes after the program year is completed. This system innovation will continue to keep New Jersey’s 21st Century Community Learning Centers programs and other interested afterschool and summer learning programs in the state moving forward.


Conclusion


The 21st Century Community Learning Centers in New Jersey are providing critical learning opportunities tied to important education goals to thousands of young people across the state. Independent evaluations of the impact of local programs show they are making a positive difference in student achievement and teacher-reported student motivation, attentiveness, pro-social behaviors, and homework completion/quality. 


These improved student outcomes did not happen by accident. The NJDE, along with local school and statewide and community partners, have worked diligently on five improvement strategies. New efforts to make future advances are under way, utilizing the latest research on quality and outcome improvement.


References


Beckett, M., Borman, G., Capizzano, J., Parsley, D., Ross, S., Schirm, A., & Taylor, J. (2009).Structuring out-of-school time to improve academic achievement: A practice guide (NCEE #2009-012). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/publications/practiceguides


Naftzger, N., Vinson, D., Manzeske, D., and Gibbs, C. (2011). New Jersey 21st Century Community Learning Centers (21st CCLC) impact report 2009–10. Naperville, IL: American Institutes for Research.