Accountability

PASA employs many methods to ensure our work’s relevance and efficacy, and to measure overall performance. These methods provide various levels of accountability. 

Market Street Research

PASA’s accountability strategy began with a look at what youth and parents want from programming.

In 2004, prior to establishing the AfterZones, PASA enlisted the help of Market Street Research to take the pulse of Providence’s parents and youth. According to the 2004 study, 48% of youth reported that they are home alone after school, while only one in four participated in a formal after-school program and only 10% reported participation in programs three or more days a week. Parents reported safety as their primary concern regarding after-school programming.

In 2006, Market Street Research convened another group of students and parents to discuss barriers to participation in programming and program interests and preferences. The 2006 study found that safety was no longer an issue, having been replaced by a desire for additional academic programming and homework help. PASA began, and continues, to meet those needs through its programming.

PASA again commissioned Market Street Research in 2009 to survey more than 200 AfterZone parents and families. The overwhelmingly positive results of the 2009 study showed over 75% of AfterZone parents as very satisfied with the quality of programs, with the safety of their children in the programs, and believing that the AfterZone has a highly positive impact on aspects of student success, including intra-familial relationships, friendships with other kids, and interest in school. Read the entire report.

Tools

Based on the needs assessment, and as PASA and our provider list grew, we discovered the need for a set of quality standards. With providers, we created standards that comprise a shared definition of quality and serve as a benchmark for future providers. These standards led to a set of indicators and a quality self-assessment process, the Rhode Island Program Quality Assessment (RIPQA).

PASA also focuses on capacity building to increase providers’ ability to provide high-quality programs to youth. By providing technical support and one-on-one assistance from trained quality-assurance personnel, PASA:

  • ensures that providers offer high-quality programming
  • teaches newer providers the standards to which they should aspire
  • assists struggling providers in improving programming

Most importantly, PASA relies on positive youth development and engagement – we wouldn’t exist without it. Our practices are research driven and speak directly to the needs of youth in Providence. Learn more about our quality improvement strategy designed to ensure both participants and providers maintain the Rhode Island Quality After School standards.

Evaluation

PASA has defined a set of achievable outcomes on three levels: youth, program and system.

  • Youth: supporting students’ academic success and improving social and emotional competencies
  • Program: focusing on improvements in the quality of programming and the number of youth served
  • System: looking toward successful partnerships and leveraging funding to create a more efficient systems building effort

These outcomes must be evaluated to determine our success. Public/Private Ventures, a nonprofit organization that assesses and evaluates community programming to identify and replicate effective programs, is conducting a 40-month evaluation of the AfterZones, supported by the Wallace Foundation.

Their two-part study consists of an implementation and quality study of the structure and operations of the AfterZones model, which can be read here, and a participation and outcomes study designed to examine the patterns of youth participation in the AfterZones and the effects of participation on developmental outcomes compared with a group of non-participants. The results of the outcome study will be published in summer 2010 -- preliminary results can be viewed here.

In addition to P/PV, we use a tracking tool, YouthServices.net, to track participation and retention and survey youth four times per year to determine their satisfaction with programming and suggestions for improvement. 

Finally, PASA has been one of several cities involved in two national studies funded by the Wallace Foundation:

  • The RAND Corporation is conducting a study of the 5 Wallace Foundation-funded OST systems-building grantees to explore their efforts to build city-wide after school systems with a specific focus on their use of management information systems (MIS).  This study is complete and will be published in the fall of 2010.
  • Harvard Family Research Project and Public/Private Ventures jointly conducted a study of how six cities engage older youth in out of school time activities, looking specifically at strategies and features of associated with high levels of participation.  This study is complete and can be read here.

For more information about PASA’s evaluation or to receive specific results, please contact

Elizabeth Devaney, Deputy Director
401.490.9599 x103
edevaney@mypasa.org