Substance Education and Awareness Program Year 1 Evaluation Report
Boulder Substance
Education and Awareness
(SEA) Program
Year 1 Evaluation Report
Submitted To: Kammi Siemens, City of Boulder
December 2017
ii
Boulder Substance
Education and Awareness
(SEA) Program
Year 1 Evaluation Report
For more information, please contact:
Holen Hirsh, PhD
hhirsh@omni.org
303-839-9422 ext. 122
Michael Schiel, MS
mschiel@omni.org
303-839-9422 ext. 117
Julia Simhai, MPH
jsimhai@omni.org
303-839-9422 ext. 137
OMNI Institute
899 Logan Street, Suite 600
Denver, CO 80203
www.omni.org
3
Table of Contents
The Substance Education and Awareness Program.................................................................................. 4
Information Dissemination......................................................................................................................... 5
Boulder County Public Health: Safe Storage Campaign .................................................................. 5
Boulder Valley School District: EFFEKT ............................................................................................. 5
Education ....................................................................................................................................................... 6
Boulder Valley School District: Sources of Strength ........................................................................ 6
El Centro Amistad: Teen Outreach Program ..................................................................................... 6
YMCA: Influencer Training .................................................................................................................... 7
Boulder County Public Health: Break the Cycle................................................................................ 7
Alternative Activities ................................................................................................................................... 7
YMCA: Prosocial Activities .................................................................................................................... 7
Phoenix Multisport .................................................................................................................................. 7
Community-Based Processes .................................................................................................................... 8
Boulder County Public Health: TA & SAPST Training ...................................................................... 8
Evaluation Overview ....................................................................................................................................... 8
Evaluation Goals........................................................................................................................................... 8
Process Evaluation Questions ................................................................................................................... 9
Outcome Evaluation Questions ................................................................................................................ 9
Evaluation Activities .................................................................................................................................. 10
SEA Logic Model ........................................................................................................................................ 11
Evaluation Methods .................................................................................................................................. 12
Year 1 Results ................................................................................................................................................. 14
Working in Partnership: Successes & Challenges ............................................................................... 14
Subcontractor Findings by CSAP Strategy ........................................................................................... 16
Information Dissemination .................................................................................................................. 16
Education ................................................................................................................................................. 19
Alternative Activities ............................................................................................................................. 27
Community-Based Processes .............................................................................................................. 30
Conclusions & Next Steps ............................................................................................................................ 31
Future Process Evaluation Questions .................................................................................................... 32
Future Outcome Evaluation Questions ................................................................................................. 32
Future Evaluation Activities ..................................................................................................................... 32
Appendix A. SEA Subcontractor Survey .................................................................................................... 34
Appendix B. Proposed SEA Program Year 2 Evaluation Report Outline ............................................ 38
4
Substance Education and Awareness
Program Annual Report
The Substance Education and
Awareness Program 1
In November 2013, City of Boulder voters approved Ordinance 7916, which authorized the city
to impose an excise tax of up to ten percent and a sales and use tax of up to ten percent on
recreational marijuana sales to offset some of the indirect costs of recreational marijuana.
In June 2016, the City of Boulder launched the Substance Education and Awareness (SEA)
Program, which serves as a community-wide substance abuse prevention education initiative,
including recreational marijuana education, for children, youth and families. Funding is used by
community agencies (subcontractors) to develop a consistent message for children, youth,
families and the City of Boulder community related to the impacts of drug/alcohol use on
children and youth. This messaging, along with prevention strategies, is designed to achieve the
SEA Program goals of:
1. Widespread community distribution and awareness of information and programs
developed;
2. Shift in community perceptions of risk associated with substance use, including the
impact of drugs, alcohol, recreational marijuana, and abuse of prescription medications on
children and youth;
3. Prevent/reduce youth abuse of alcohol and recreational drugs including marijuana; and
4. Reduce accidental ingestion of marijuana and other drugs.
The SEA Program funds subcontractors to implement prevention activities that fall into four
prevention strategies, as defined by the Substance Abuse and Mental Health Services
Administration (SAMHSA) Center for Substance Abuse Prevention (CSAP). The City of Boulder
sought to subcontract with agencies within the City implementing evidence-based programs or
programs with promising practice for impacting awareness, perceptions of risk, and prevention of
substance use and abuse. These agencies and their prevention activities were identified by a
competitive Request for Proposal process. The Healthy Futures Coalition (HFC) submitted an
1 Descriptions of each SEA subcontractor program contained in this section were drawn from text on
program websites, in program material, and/or were provided to OMNI by subcontractor contacts.
5
RFP application with a set of agencies and prevention activities, in which they identified any
programs that are evidence-based or a promising practice as follows:
• Sources of Strength is included in SAMHSA’s National Registry of Evidence-Based
Programs and Practices.
• Teen Outreach Program (model program) and EFFEKT (promising program) are in the
Blueprints for Healthy Youth Development list.
• Evidence for the effectiveness of Break the Cycle has been published in peer- reviewed
journals.
The SEA Program funded the HFC proposal, including the following subcontractors and services,
which are organized below by SAMHSA CSAP strategy.
INFORMATION DISSEMINATION
This strategy provides awareness and knowledge of the nature and extent of substance use,
abuse and addiction, and their effects on individuals, families and communities as well as
available prevention programs and services. Information Dissemination is characterized by one-
way communication from the source to the audience with limited contact between the two.
Boulder County Public Health: Safe Storage Campaign
The Boulder County Public Health Communication and Marketing team provides strategic
communication guidance and solutions in partnership with Boulder County Public Health (BCPH)
programs and partners to encourage behavior and policy change that inspires a healthy, engaged
community. The team researches, designs, develops and implements relevant and culturally
competent marketing campaigns using social marketing best practices, media relations, and
traditional, digital, and social media channels. With the support of SEA Program funding, the Safe
Storage media and educational campaign for adults and retailers is designed to reduce accidental
ingestion of all substances (marijuana, alcohol, and prescription drugs) and provide information
about the effects of substance use, abuse, and addiction on individuals, families, and
communities.
Boulder Valley School District: EFFEKT
The Boulder Valley School District (BVSD) Office of Student Support provides support within the
school district to promote coordination among the many efforts that contribute to student and
staff health. SEA Program funding supports EFFEKT, a program that seeks to prevent adolescent
alcohol use by giving parents the tools to reinforce clear expectations and attitudes towards
young people's use of alcohol.
The EFFEKT program consists of three components:
• Encouraging parents to make preventing alcohol use by their children a priority
6
• Reinforcing to parents that they have a strong influence on their children's attitudes and
behaviors
• Giving parents practical advice on how they can positively influence their children's
attitudes and behaviors
Information is disseminated to parents of middle school students at the beginning of each
semester in large group meetings and by regular communications throughout the year. Families
are also regularly provided lists of organized activities taking place in the community that offer
adolescents constructive, sober ways to spend their time.
EDUCATION
This strategy involves two-way communication and is distinguished from merely disseminating
information by the fact that it is based on an interaction between the educator and the
participants. Activities under this strategy aim to affect critical life and social skills, including
decision-making, refusal skills and critical analysis (e.g., of media messages).
Boulder Valley School District: Sources of Strength
The BVSD Office of Student Support provides support within the school district to promote
coordination among the many efforts that contribute to student and staff health. SEA Program
funding supports the evidence-based prevention program, Sources of Strength (SOS). The
mission of SOS is to increase help-seeking behaviors and promote connections between peers
and caring adults. SOS uses peer leaders to enhance protective factors and leverages the power
of peer social networks to change unhealthy norms and culture, ultimately preventing suicide,
bullying, and substance abuse. This upstream model strengthens multiple sources of support
(protective factors) around young individuals so that when times get hard they have strengths to
rely on.
Sources of Strength is being implemented in five BVSD middle schools:
• Centennial Middle School
• Casey Middle School
• Manhattan Middle School for Arts and Academics
• Nevin Platt Middle School
• Southern Hills Middle School
El Centro Amistad: Teen Outreach Program
El Centro Amistad's mission is to integrate and transform the Boulder County community
through opportunities and programs for Latinos that promote education, health, and quality of
life. SEA Program funding supports implementation of the Teen Outreach Program (TOP) for
Latino boys in Boulder. TOP is an evidence-based program that empowers teens with the tools
and opportunities needed to build a foundation of healthy behaviors, life skills and a sense of
purpose, and avoid risky behaviors that can derail success.
7
YMCA: Influencer Training
The YMCA Boulder Valley Influencer Program is designed to educate adults who work and
interact with youth on how to become an "ask-able" or "safe" person to talk with. The goal of the
program is to train adults on how to effectively communicate and interact with youth to have a
positive impact on their lives. Specifically, the training focuses on topics including substance
abuse, bullying, and parental disputes, and teaches adults to comfortably m anage conversations
with youth.
Boulder County Public Health: Break the Cycle
SEA Program funding supports Break the Cycle (BTC), an evidence-based intervention designed
to prevent initiation of injection drug use in young adults. BTC is based on a peer model where
young adults (24 years and under) participate in group and individual sessions surrounding
positive health behaviors.
Trained counselors provide group and individual sessions to:
• Provide tools and skills to prevent the modelling of injection behavior in front of peers.
• Develop skills to deny request for initiating others to injection behavior.
• Provide space to empower young adults to support positive health behaviors.
ALTERNATIVE ACTIVITIES
This strategy provides the opportunity to participate in healthy, positive, and constructive
activities that exclude substance use. These activities are assumed to offset the attraction to,
and/or meet the needs filled by, alcohol and drugs, thereby reducing the likelihood of substance
use.
YMCA: Prosocial Activities
The YMCA Boulder Valley Pro-Social Activities Program was created to provide teens with a
space for healthy activities as well as the development of leadership skills with adult mentors.
The program includes Teen Night Out which is designed to give teens a safe environment in
which they can grow and thrive, as an alternative to interacting over cell phones, screens, or
engaging in unsafe activities. Teen Night Out includes team-building, ropes courses, game nights
and more. It also provides the opportunity for teens to volunteer for Days of Service. Days of
Service give teens the chance to become leaders and experience the inspiring feeling of helping
others, as well as instill the ideals of volunteerism as teens grow into young adults.
Phoenix Multisport
Phoenix Multisport (Phoenix) fosters a supportive, physically active community for individuals
who are recovering from a substance use disorder and those who choose to live sober. Through
pursuits such as climbing, hiking, running, strength training, yoga, road/mountain biking, socials
8
and other activities, Phoenix seeks to help members develop and maintain the emotional
strength they need to stay sober.
COMMUNITY-BASED PROCESSES
This strategy aims to enhance the ability of the community to more effectively provide
prevention and treatment services for alcohol and drug abuse disorders. Activities in this
strategy include organizing, planning, enhancing the efficiency and effectiveness of service
implementation, building coalitions and networking.
Boulder County Public Health: TA & SAPST Training
The Boulder County Public Health Community Substance Abuse Prevention (CSAP) Program
works to reduce and prevent substance abuse in Boulder County through individual
interventions for teens and young adults and community-level education, advocacy, and policy.
With SEA Program funding, CSAP provides technical support for subcontractors, assists with
project coordination, reinforces consistent messaging across community partners, and offers
education and training opportunities for SEA Program subcontractors. The SEA Program
supports CSAP’s dedication to community-based processes and strategies to strengthen the
capacity of the community to engage in effective prevention activities.
Evaluation Overview
In November 2016, the City of Boulder hired OMNI Institute (OMNI) to lead an evaluation of the
SEA Program. OMNI is a non-profit, social science agency that provides evaluation research,
capacity building, and technical solutions services. In December 2016, OMNI began working in
partnership with the SEA Program Steering Committee (represented by the City of Boulder and
Healthy Futures Coalition) to develop the structure and design of the evaluation.
The SEA Program Steering Committee helped to guide the development of an evaluation
research design and ensure that research questions, methods, and other aspects of the design
met the needs of subcontractors as well as the City of Boulder. The evaluation approach
undertaken by the SEA Program Steering Committee is highly collaborative and participatory.
EVALUATION GOALS
The overarching goal of the ongoing evaluation is to evaluate the effectiveness of individual and
joint efforts of funded subcontractors in addressing the SEA Program goals:
1. Widespread community distribution and awareness of information and programs
developed;
9
2. Shift in community perceptions of risk associated with substance use, including the
impact of drugs, alcohol, recreational marijuana, and abuse of prescription medications on
children and youth;
3. Prevent/reduce youth abuse of alcohol and recreational drugs including marijuana; and
4. Reduce accidental ingestion of marijuana and other drugs.
The following evaluation questions are designed to focus the evaluation on the goals listed
above. Process evaluation questions center on how the implementation of funded strategies was
carried out. Outcome evaluation questions center on measuring the effectiveness of the SEA
Program in changing attitudes, beliefs, and behaviors related to substance use.
PROCESS EVALUATION QUESTIONS
1. How many youth received direct services2 funded by the SEA Program?
2. How many youth were exposed to the community-wide messaging3 implemented for
this grant?
3. How many adults were exposed to the community-wide messaging implemented for this
grant?
4. What successes did subcontractors achieve during implementation of SEA strategies?
5. What challenges did subcontractors face during implementation of SEA strategies?
6. What did effective partnerships among subcontractors look like? What was gained?
7. What was learned by working in partnership? Were there challenges to establishing
effective partnerships among subcontractors?
OUTCOME EVALUATION QUESTIONS
1. What evidence is there that SEA strategies resulted in increases in adult risk perceptions
of youth substance use?
2. What evidence is there that SEA strategies resulted in increases in youth risk perceptions
of substance use?
3. What evidence is there that SEA strategies resulted in lower rates of youth substance use
or related consequences?
2 Direct services refer to programs that are provided to a defined group of participants, such as
educational programming or alternative pro-social activities delivered through a SEA subcontractor. All
youth who participate in or are influenced through grant-funded programming may be considered
recipients of a direct service.
3 Community-wide messaging is distributed to the whole community, for example, through an ad campaign
about safe storage of medication in the home. Those who view the ad campaign are considered to have
received community-wide messaging, but they are not considered a direct recipient of SEA funded
programming.
10
4. What evidence is there that SEA strategies resulted in lower rates of accidental ingestion
of marijuana and other drugs among youth 0-9 years old?
5. What does subcontractor program data suggest about SEA Program effectiveness?
6. What evidence is there of the community impact of SEA strategies?
EVALUATION ACTIVITIES
Cross-site evaluations typically require an initial phase of information gathering to develop an
evaluation plan that accommodates individual variations across sites (i.e., subcontractors) while
also ensuring a common framework for measuring processes and outcomes. Accordingly, OMNI’s
activities in Year 1 of the evaluation focused on documentation of funded activities undertaken
by subcontractors, measurement already in place by subcontractors, and development of initial
evaluation infrastructure to support an ongoing evaluation. In particular, evaluation
infrastructure development included a comprehensive SEA Program logic model outlining types
of data collected by each participating subcontractor and how subcontractors individual efforts
work collaboratively to impact SEA goals (see page 11). OMNI will continue to implement the
SEA Program evaluation plan in multiple phases to ensure that evaluation reflects both
participation from the City of Boulder and SEA subcontractors and standardized qualities.
To this end, during this first year of the evaluation, efforts have focused on:
• Identifying common and unique components of subcontractor implementation.
• Developing a comprehensive evaluation plan, including a logic model, for the five-year
grant period.
• Identifying indicators to track SEA progress, successes, and challenges.
• Building infrastructure and processes to assess progress on the SEA Program.
• Developing a data collection and display dashboard to track and report on SEA indicators.
• Measuring subcontractor perceptions of collaboration and shared messaging efforts.
The evaluation design is formative; that is, evaluation methods are designed to inform program
improvements throughout the implementation process rather than waiting until the program
ends to assess effectiveness. Thus, in addition to gathering qualitative and quantitative data to
address the impact of the SEA Program on its intended goals, the evaluation will also explore
successes and barriers to implementation to provide a contextualized understanding of the
findings and to develop feasible and effective recommendations to guide the program.
11
SEA LOGIC MODEL
12
EVALUATION METHODS
Data presented in this report were obtained from multiple sources and evaluation methods. With
a focus on participatory and formative evaluation techniques involving stakeholders throughout
the process, the evaluation employed a multiple-case, non-comparative design with mixed
methodology, collecting both quantitative and qualitative data through survey and interview
methods. The specific methods are outlined below.
IDENTIFY COMMON COMPONENTS OF SEA PROGRAM
IMPLEMENTATION
▪ Document review. Evaluators reviewed SEA Program agreements, including Healthy Future
Coalition’s (HFC) proposal to the City of Boulder, to gain an initial understanding of
subcontractor activities, implementation plans as they relate to SEA Program goals, and
HFC’s framework for comprehensive services.
▪ Initial interviews with all subcontractors. Evaluators interviewed the main contact(s) for each
subcontractor to gain an overview of program responsibilities, including SEA Program
activities, target population for services, and program implementation timelines.
Conversations also included a discussion of evaluation activities to date in which
subcontractors described data currently being collected and specific measures administered.
Program outcomes and data sharing needs/concerns were also discussed. An interview guide
was developed by evaluators and followed for each interview.
▪ Meeting participation and documentation. Evaluators participated in SEA Program Steering
Committee meetings, subcontractor meetings, Healthy Futures Coalition meetings, and
regular meetings with the SEA Program Manager to understand and contextualize SEA
Program implementation.
CREATE INFRASTRUCTURE TO EVALUATE AND SHARE PROGRESS ON
PROGRAM GOALS
▪ Development of logic model. Evaluators worked with the SEA Program Steering Committee
and subcontractors to develop a SEA Program logic model. The logic model outlines process
and outcome measures as they relate to the SEA Program goals. The logic model is on page
11.
▪ Development of evaluation plan. Based on the logic model, Evaluators developed a five-year
evaluation plan for the SEA Program that details key evaluation questions and activities
designed to address each question. The evaluation plan was delivered to the City of Boulder
in February 2017.
▪ Survey of subcontractor successes & challenges. OMNI developed a custom online survey to
document successes and challenges with program implementation. The survey was
administered to SEA subcontractors in May 2017 and OMNI shared a summary of findings
13
with the City of Boulder and subcontractors in June 2017. Findings from the subcontractor
survey also inform this annual report.
▪ Design of data dashboard. Evaluators designed a custom data dashboard application for the
collection of process and outcome indicators by subcontractors as well as for the display of
data collected by subcontractors. During this year, OMNI completed Phase I of dashboard
development to collect and display subcontractor process data indicators. Evaluators trained
all subcontractors in use of the data dashboard for data entry, display, and reporting. In
future years, OMNI will expand the dashboard to collect and display subcontractor outcome
data as well as relevant community-level indicators (e.g., police data, municipal court data,
accidental ingestion data, etc.). The SEA dashboard supports data sharing with the public
regarding SEA Program goals.
MEASURE PERCEPTIONS OF SHARED MESSAGING AND
COLLABORATIVE EFFECTIVENESS
▪ Survey of subcontractor collaboration. Evaluators developed a custom online survey to
surface learnings related to understanding and dissemination of the SEA shared messages as
well as collaboration among subcontractors. The survey was administered to SEA
subcontractors in May 2017 and OMNI shared a summary of findings with the City of
Boulder and subcontractors in June 2017. Findings from the subcontractor survey also
inform this annual report. A copy of the survey items is in Appendix A.
EVALUATE SEA PERFORMANCE MEASUREMENT
▪ Compilation of common outcomes. Evaluators assessed feasibility for collecting common SEA
Program outcome measures across subcontractors.
▪ Development of process & outcome metrics. Evaluators consulted individually with each
subcontractor to assist with defining process and outcome metrics relevant to their unique
programs and SEA Program outcomes. Process and outcome indicators identified by each
subcontractor informed the development of the logic model and the SEA Program
dashboard.
▪ Synthesis of annual performance measures. Evaluators systematically reviewed
subcontractor-reported performance information (e.g., numbers served and population
demographics) to report on target populations and process indicators.
▪ Review of available community data sources. Evaluators had initial conversations with
several community agencies, including Boulder County Community Justice Services, Boulder
County Public Health, and the Boulder Valley School District (regarding Healthy Kids
Colorado Survey data). These conversations assessed feasibility for collecting city-level
indicators relevant to SEA Program goals that could be publishable on the public-facing SEA
dashboard. Evaluators also reviewed data sets from the City of Boulder Police Department
and the City of Boulder Municipal Court.
14
Year 1 Results
Findings from the first year of the SEA Program evaluation are organized into two sections
below: partnership efforts and individual subcontractor efforts. Data included in these sections
focus on the activities, successes, challenges, and associated process indicators for each
subcontractor in Year 1 of SEA Program implementation. Because process measures were
established after programming was started, some subcontractors were not able to capture the
level of detail for demographics or other process measures that they will be capturing in future
years. In addition, at this early stage of the evaluation, outcome data are not available from
subcontractors to assess progress towards SEA Program goals.
Efforts in the first year of the SEA Program focused largely on subcontractors establishing
processes for implementation and documentation of indicators. As such, the Year 1 annual
evaluation report focuses on the individual efforts of each subcontractor. Future annual
evaluation reports will not provide such detailed information by subcontractor, but will aggregate
data across subcontractors to develop a picture of overall SEA Program implementation.
Year 1 of the SEA Program evaluation was designed to answer five of the seven process
questions established in the SEA Program evaluation plan:
1. What did effective partnerships among subcontractors look like? What was gained?
2. What was learned by working in partnership? Where there challenges to establishing
effective partnerships among subcontractors?
3. What successes did subcontractors achieve during implementation of SEA strategies?
4. What challenges did subcontractors face during implementation of SEA strategies?
5. How many youth received direct services funded by the SEA Program?
WORKING IN PARTNERSHIP: SUCCESSES &
CHALLENGES
Subcontractors are charged with disseminating a consistent message for children, youth, families
and the City of Boulder community related to the impacts of drug/alcohol use on children and
youth. The SEA Program shared message was developed by Healthy Futures Coalition (HFC). In
Year 1 of the program, subcontractors were trained in the shared message through HFC
meetings, subcontractor meetings, and ongoing technical assistance provided directly to
subcontractors by HFC. Results from the subcontractor survey indicated that subcontractors
know and understand the SEA Program shared message, but vary in the degree to which they
feel comfortable discussing the message within their organizations and in the community.
Year 1 Successes: Subcontractors reported feeling supported through their work with the City of
Boulder, HFC, and OMNI. In particular, subcontractors noted that partnership with HFC was
15
important to their success, as the coalition provided opportunities to share program information
and cross-promote with other organizations. Subcontractors also reported an increase in their
ability to provide community outreach as a result of the SEA Program. Individual subcontractor
program successes are discussed in the next section.
Year 1 Challenges: Two subcontractors were unable to complete Year 1 in the SEA program
(Alternatives for Youth and Phoenix Multisport; discussed individually below) due to challenges
with SEA program priorities, timelines, and recruitment/program attendance. Collaborative
initiatives across programs require alignment and shared expectations among subcontractors is
key for successful collaboration. Subcontractors reported that funding delays and understanding
expectations of participation in the SEA Program were challenges with program implementation.
Individual subcontractor program challenges are discussed in the next section.
Year 1 Partnerships: Some subcontractors developed partnerships with other subcontractors,
including a planned shared training between two organizations. Other subcontractors have yet
to partner directly with other subcontractors. Quarterly meetings were critical for
subcontractors in coming together to identify, discuss, and form meaningful partnerships with
one another. With support from HFC, subcontractors are beginning to formalize partnerships.
Subcontractors noted that continued partnerships among subcontractors will be a key factor in
the success of the SEA Program in future years.
Year 1 Process Indicators: Across all subcontractors, in Year 1 of the SEA Program 350 people
received direct services. The demographic breakdown of people served through the SEA
Program is in the table below.
Age Count %
0 – 4 0 0%
5 – 11 4 1%
12 – 14 172 49%
15 – 17 44 13%
18 – 20 5 1%
21 – 24 9 3%
25+ 0 0%
Age not Known 116 33%
Gender Count %
Male 103 29%
Female 70 20%
Transgender 2 1%
Other or Unknown 175 50%
Race Count %
American Indian/Alaskan Native 2 1%
Asian 0 0%
Black/African American 3 1%
Native Hawaiian or other
Pacific Islander 0 0%
White 46 13%
Multiracial 12 3%
Other or Unknown 287 82%
Ethnicity Count %
Hispanic or Latino/a 102 29%
Non-Hispanic and non-Latino/a 32 9%
Ethnicity not Known 216 62%
16
SUBCONTRACTOR FINDINGS BY CSAP STRATEGY
SEA subcontractors were funded to implement prevention activities that fall into four prevention
strategies, as defined by the Substance Abuse and Mental Health Services Administration
(SAMHSA) Center for Substance Abuse Prevention (CSAP). SEA subcontractors are funded in the
following prevention strategies:
• Information Dissemination: Activities or materials to provide awareness and knowledge
of substance use, abuse and addiction, as well as available prevention programs and
services
• Education: Education activities aimed to affect critical life and social skills that promote
healthy decisions and behavior
• Alternative Activities: Healthy, positive and substance-free activities designed to meet
the needs filled by alcohol and drugs
• Community-Based Processes: Organizing and planning activities or coalition work aimed
at enhancing the efficiency and effectiveness of prevention service implementation
Results presented below on individual subcontractor efforts focus on successes, challenges, and
process indicator data collected in Year 1 of the evaluation. Data presented in this section are
derived from subcontractor self-report through the annual Subcontractor Survey, as well as from
ongoing evaluation technical assistance conversations with OMNI evaluators.
Information Dissemination
BOULDER COUNTY PUBLIC HEALTH: SAFE STORAGE CAMPAIGN
With the support of SEA Program funding, the Safe Storage media and educational campaign for
adults and retailers is designed to reduce accidental ingestion of all substances (marijuana,
alcohol, and prescription drugs) and provide information about the effects of substance use,
abuse, and addiction on individuals, families, and communities.
Year 1 Successes: In the first year of the SEA Program, the Communication and Marketing team
collected in-depth information on parents’ sense of self-efficacy and perception of risk related to
safely storing prescription drugs, alcohol, and marijuana by holding focus groups with parents
who live in the City of Boulder. This qualitative effort provided data specific to Latino parents in
addition to general population data. Focus group findings informed development of a Safe
Storage campaign strategy, that was deployed in fall 2017.
Year 1 Challenges: As the Communication and Marketing team developed a campaign strategy
and determined all materials that would be included, some adjustments were made to
accommodate the limited funding available for this campaign. These adjustments included
limiting the number of dissemination channels, condensing dissemination to the “back to school”
17
time frame in early fall, and deferring the creation of a Spanish version of the web page until
other funding can be identified.
Year 1 Summary: Process measures were not tracked for the Safe Storage Campaign in the first
year of the SEA Program. A summary of the focus group process and findings, as well as steps
taken during Year 1, are included below:
The Safe Storage campaign has demonstrated through focus groups that parents want to have
good, close relationships with children and communicate their values within that context. Safe
Storage builds on that knowledge and provides a skill for parents who care about kids and their
health. The Safe Storage Campaign addresses concerns about access to substances and teaches
parents, through SEA shared messaging about adult influencers, how to talk early and often
about substances, and the importance of keeping them away from youth. The campaign was
designed based on input from parents of young children living in Boulder. It aims to:
• Help parents understand the importance of safely storing alcohol, marijuana, and
prescription drugs out of reach of their children by sharing local and state-level data.
• Arm parents with tips for storing and disposing of their substances safely, and to have
conversations about substances with their children, as well as family and friends.
• Give parents opportunities to test their knowledge through an online quiz, and share
what they’ve learned with their networks through social media.
The Safe Storage Campaign began with research, planning, and review of all related campaigns
nationally and internationally to generate a suite of messages that expanded the focus group
materials. A comprehensive focus group report enabled Boulder County Public Health (BCPH) to
develop a health communication approach that integrated substances, took a positive skills
approach, and created a campaign that felt relevant and potentially well received by our
community. A safe storage message was developed and an online quiz was created. Additionally,
an online poll was developed to parallel the broader campaign. A state and local data review was
completed via BCPH Health Planning and Evaluation. Suitability for the inclusion of this data
within the messaging campaign was determined by BCPH Communications Manager in
consultation with Community Substance Abuse Prevention Coordinator. Advertising channels
were identified with social media targeted to demographics of parents and caregivers of young
children who live or work in the City of Boulder. A web service implemented the poll and
content was finalized. All content was translated to Spanish and a Spanish language quiz is
available with a full build-out of materials currently in development. Community-wide
distribution planning began with focus on dissemination of Safe Storage messaging in schools
and the broader community.
18
BOULDER VALLEY SCHOOL DISTRICT: EFFEKT
SEA Program funding supports EFFEKT – as part of the Boulder Valley School District (BVSD)
Office of Student Support – a program that seeks to prevent adolescent alcohol use by giving
parents the tools to reinforce clear expectations and attitudes towards young people's use of
alcohol.
Year 1 Successes: The EFFEKT Program was adopted by BVSD, who connected with the program
developer in Spring 2017 to discuss implementation practices and evaluation methods. Program
implementation is planned for Fall 2018. In September 2017, BVSD participated in a two-day
training with the EFFEKT program developer and learned how to present the program to
parents. Trained presenters will roll out the program to participating middle schools, where they
will share program goals and discuss ways to communicate zero-tolerance messages about
alcohol use with their children.
Year 1 Challenges: The EFFEKT Program was originally proposed for and awarded to the
organization Alternatives for Youth (AFY), but this organization was unable to implement
programming in Year 1 due to delayed funding and conflicting timelines. While several BVSD
middle schools expressed interest in EFFEKT, AFY was unable to coordinate with schools early
enough in the summer to obtain approval from administrators and roll out programming in Fall
2016 (Year 1). In addition, AFY underwent internal changes including relocation to Northglenn
and reprioritization of programming, which ultimately left the organization unable to continue
involvement in the SEA Program.
Year 1 Process Indicators: In Year 1 of the SEA Program the EFFEKT process measures were not
tracked because EFFEKT was in planning stages with BVSD. Planning involved developing a
contract with the EFFEKT developer, determining dates for training, and identifying participating
schools. Training was scheduled and completed in September 2017.
19
Education
BOULDER VALLEY SCHOOL DISTRICT: SOURCES OF STRENGTH
SEA Program funding supports the evidence-based prevention program, Sources of Strength
(SOS), as part of the Boulder Valley School District (BVSD) Office of Student Support. The
mission of SOS is to increase help-seeking behaviors and promote connections between peers
and caring adults
Year 1 Successes: Sources of Strength was successfully implemented in all five of the identified
BVSD schools in 2017. This included recruitment and training of adult advisors and peer leaders
at each school, conducting regular peer leader meetings throughout the year, and the
development and dissemination of positive messages unique to each school. In Spring 2017,
BVSD consulted with the University of Rochester, who developed the evidence-based
evaluation of the SOS program. With the help of the OMNI evaluation team, BVSD is planning to
implement a pre-post outcome measure with peer leaders in Year 2. In addition, BVSD held an
end of year meeting with adult advisors to learn about needs at each school, and to build on
successes and learnings during Year 2 of programming.
Year 1 Challenges: SOS is an evidence-based program for suicide prevention, but has yet to be
validated as an evidence-based program specifically for substance use prevention. To address
this challenge, in 2018 BVSD plans to tailor the traditional SOS approach to the SEA Program by
instructing each school to develop at least one message each year related to substance use and
assess risk and protective factors related to substance use in their peer leader survey.
Year 1 Process Indicators: In Year 1 of the SEA Program the Sources of Strength program served
100 people. The demographic breakdown of youth served and indicators for Sources of Strength
programming are in the tables below.
Age Count %
0 – 4 0 0%
5 – 11 0 0%
12 – 14 100 100%
15 – 17 0 0%
18 – 20 0 0%
21 – 24 0 0%
25+ 0 0%
Age not Known 0 0%
Gender Count %
Male 0 0%
Female 0 0%
Transgender 0 0%
Other or Unknown 100 100%
20
Race Count %
American Indian/Alaskan Native 0 0%
Asian 0 0%
Black/African American 0 0%
Native Hawaiian or other Pacific
Islander 0 0%
White 0 0%
Multiracial 0 0%
Other or Unknown 100 100%
Ethnicity Count %
Hispanic or Latino/a 0 0%
Non-Hispanic and non-Latino/a 0 0%
Ethnicity not Known 100 100%
Other Indicators Details Count
Number of faculty outreach
presentations
The total number of SOS informational presentations to
faculty. 5
Number of faculty attending
outreach presentations
The total number of unique faculty members that
attended SOS informational presentations.
Not
tracked
in Year 1
Number of adult advisor
trainings
The total number of SOS adult advisor trainings
provided. 5
Number of adult advisors
trained
The total number of unique adult advisors who attended
SOS trainings. 22
Number of peer leader trainings The total number of SOS peer leader trainings provided. 5
Number of peer leaders trained The total number of unique peer leaders who attended
SOS trainings. 212
Number of support sessions The total number of SOS support sessions provided to
adult advisors and peer leaders 10
Number of SOS meetings held The total number of SOS meetings held (sum of
meetings held at all SOS schools) 12
Number of adult advisors The total number of unique adult advisors attending SOS
meetings (sum of unique adult advisors at all SOS
schools)
22
Number of engaged peer
leaders (attending at least 50%
of meetings)
The number of unique peer leaders who attend at least
50% of sessions. (% may be updated at later date)
Not
tracked
in Year 1
Number of messaging activities
The total number of messaging activities implemented
by SOS schools (sum of messages at all schools) 10
Number of youth targeted by
messages
The total number of youth in the target population for
messaging activities (sum of school populations) 2,935
21
EL CENTRO AMISTAD: TEEN OUTREACH PROGRAM
SEA Program funding supports implementation of the Teen Outreach Program (TOP) for Latino
boys in Boulder. TOP is an evidence-based program that empowers teens with the tools and
opportunities needed to build a foundation of healthy behaviors, life skills and a sense of
purpose, and avoid risky behaviors that can derail success.
Year 1 Successes: El Centro Amistad successfully implemented TOP with Latino boys and their
parents throughout the 2016-17 school year. Internal evaluations completed by El Centro
Amistad showed success in reducing marijuana use among participants in the school years
program. El Centro Amistad built on its success during the school year to assemble a summer
program serving over 80 participants that is partially funded through the SEA Program.
Year 1 Challenges: El Centro Amistad hopes to expand their efforts in spreading the SEA
Program shared messages in future years, now that program staff are more confident in their
internal understanding of the message. They also have goals for increased collaboration with
other subcontractors and would like to develop more partnership in future years than what they
were able to accomplish during Year 1.
Year 1 Process Indicators: In Year 1 of the SEA Program the Teen Outreach Program served 86
people. The demographic breakdown of youth served and other indicators for the Teen
Outreach Program are in the tables below.
Age Count %
0 – 4 0 0%
5 – 11 0 0%
12 – 14 42 49%
15 – 17 44 51%
18 – 20 0 0%
21 – 24 0 0%
25+ 0 0%
Age not Known 0 0%
Gender Count %
Male 62 72%
Female 22 26%
Transgender 2 2%
Other or Unknown 0 0%
Race Count %
American Indian/Alaskan Native 0 0%
Asian 0 0%
Black/African American 0 0%
Native Hawaiian or other Pacific
Islander 0 0%
White 25 29%
Multiracial 0 0%
Other or Unknown 61 71%
22
Ethnicity Count %
Hispanic or Latino/a 86 100%
Non-Hispanic and non-Latino/a 0 0%
Ethnicity not Known 0 0%
Other Indicators Details Count
Number of parent participants The number of parents who attended a parent session 37
Average youth attendance rate The average percentage of sessions that youth attended. 95%
23
YMCA: INFLUENCER TRAINING
The YMCA Boulder Valley Influencer Program is designed to educate adults who work and
interact with youth on how to become an "ask-able" or "safe" person to talk with.
Year 1 Successes: The YMCA Influencer Program curriculum builds on the SEA Program shared
message developed by Healthy Futures Coalition (HFC) by reinforcing the role of adults in the
community as positive and approachable role models for youth. The YMCA has begun
conducting internal trainings among staff, and presented its first external training to Boulder
County staff in May 2017. With the help of OMNI’s evaluation team, the YMCA has identified
training outcomes (e.g., trainees feel more comfortable discussing sensitive issues with youth;
trainees react differently when youth approach them with a problem or concern) that are being
assessed before and after trainings.
Year 1 Challenges: Although the program curriculum was developed relatively quickly, trainings
did not begin until late Spring 2017 because of delays in hiring a program coordinator. As a
result, the number of people served in Year 1 was not as high as initially planned.
Process Indicators: In Year 1 of the SEA Program the YMCA Influencer Program served 51
people. The demographic breakdown of people served and other indicators for the YMCA
Influencer Program are in the tables below.
Age Count %
0 – 4 0 100%
5 – 11 0 100%
12 – 14 0 100%
15 – 17 0 100%
18 – 20 0 100%
21 – 24 0 100%
25+ 0 100%
Age not Known 51 100%
Gender Count %
Male 0 100%
Female 0 100%
Transgender 0 100%
Other or Unknown 51 100%
Race Count %
American Indian/Alaskan Native 0 100%
Asian 0 100%
Black/African American 0 100%
Native Hawaiian or other Pacific
Islander 0 100%
White 0 100%
Multiracial 0 100%
Other or Unknown 51 100%
Ethnicity Count %
Hispanic or Latino/a 0 100%
Non-Hispanic and non-Latino/a 0 100%
Ethnicity not Known 51 100%
24
Other Indicators Details Count
Number of YMCA training
sessions
The total number of internal influencer trainings
provided to YMCA staff. 2
Number of YMCA staff trained The total number of unique YMCA staff who
receive/complete the influencer training. 51
Number of external training
sessions
The total number of external influencer trainings
provided to other organizations and non-YMCA staff. 0
Number of non-YMCA
individuals trained
The total number of unique non-YMCA individuals who
receive/complete the influencer training. 0
25
BOULDER COUNTY PUBLIC HEALTH: BREAK THE CYCLE
Break the Cycle (BTC) is an evidence-based intervention designed to prevent initiation of
injection drug use in young adults. BTC is based on a peer model where young adults (24 years
and under) participate in group and individual sessions surrounding positive health behaviors.
Year 1 Successes: BTC began program implementation in late October 2016 and developed a set
of process and outcome measures that they track. BCPH initiated the process of hiring additional
staff, who may be trained as BTC providers, to expand the number of locations where services
are offered. BTC is also exploring options for individuals who complete the intervention to grow
into a more active leadership role in the program, and act as advisors or role models for new
individuals in the program.
Year 1 Challenges: Program recruitment was a challenge early in program implementation, as
BCPH was only able to offer the program at one location. BCPH also experimented with offering
group sessions that require additional planning and coordination. For the target demographic of
this intervention, ensuring attendance at group events was a challenge, and BCPH had more
success recruiting walk-ins and providing on-the-spot trainings instead of pre-planned group
sessions.
Process Indicators: In Year 1 of the SEA Program the Break the Cycle program served 14 people.
The demographic breakdown of young adults served and indicators for Break the Cycle
programming are in the tables below.
Age Count %
0 – 4 0 0%
5 – 11 0 0%
12 – 14 0 0%
15 – 17 0 0%
18 – 20 5 36%
21 – 24 9 64%
25+ 0 0%
Age not Known 0 0%
Gender Count %
Male 9 64%
Female 5 36%
Transgender 0 0%
Other or Unknown 0 0%
Race Count %
American Indian/Alaskan Native 2 14%
Asian 0 0%
Black/African American 0 0%
Native Hawaiian or other Pacific
Islander 0 0%
White 8 57%
Multiracial 1 7%
Other or Unknown 3 21%
26
Ethnicity Count %
Hispanic or Latino/a 1 7%
Non-Hispanic and non-Latino/a 10 71%
Ethnicity not Known 3 21%
Other Indicators Details Count
Number of info cards
distributed
The total number of BTC informational cards distributed
(sum of cards distributed at all locations) 450
Number of intervention
providers
The total number of individuals providing the BTC
intervention 6
Number of registrations The total number of online & in-person registrations for
the intervention 47
Number of groups sessions The total number of group intervention sessions
provided 35
Number of intervention
participants referred to
community resources
The total number of attendees who are referred to other
community organizations or resources for additional
services 14
27
Alternative Activities
YMCA: PROSOCIAL ACTIVITIES
Supported in part by SEA funding, the YMCA Boulder Valley Pro-Social Activities Program was
created to provide teens with a space for healthy activities. Youth develop leadership skills as
well as relationships with adult mentors through a variety of activities including Teen Night Out
and Days of Service programs.
Year 1 Successes: The popularity of Teen Night Out events grew steadily during spring 2017
events. In early implementations of the program only 4-5 youth attended, but by the end of the
program year Teen Night Out events had waitlists for the program with up to 26 youth attending
events. There are several opportunities to cross-promote prosocial activities, that may include
partnership with SEA subcontractors implementing Sources of Strength, Teen Outreach
Program, and EFFEKT. The YMCA is exploring partnerships to promote Teen Night Out events
with other SEA subcontractors.
Year 1 Challenges: Initially, program outreach was a challenge, with low attendance at events.
The YMCA has been learning what marketing strategies result in increased event attendance,
and plans to continue implementing successful strategies to improve outreach in the future.
Process Indicators: In Year 1 of the SEA Program the YMCA Prosocial Activities program served
99 youth. The demographic breakdown of youth served and indicators for Prosocial
programming are in the tables below.
Age Count %
0 – 4 0 0%
5 – 11 4 4%
12 – 14 30 30%
15 – 17 0 0%
18 – 20 0 0%
21 – 24 0 0%
25+ 0 0%
Age not Known 65 66%
Gender Count %
Male 32 32%
Female 43 43%
Transgender 0 0%
Other or Unknown 24 24%
Race Count %
American Indian/Alaskan Native 0 0%
Asian 0 0%
Black/African American 3 3%
Native Hawaiian or other Pacific Islander 0 0%
White 13 13%
Multiracial 11 11%
Other or Unknown 72 73%
28
Ethnicity Count %
Hispanic or Latino/a 15 15%
Non-Hispanic and non-Latino/a 22 22%
Ethnicity not Known 62 63%
Other Indicators Details Count
Number of Teen Night Out
Events The total number of Teen Night Out events held. 8
Number of Days of Service
Events The total number of Days of Service events held. 3
Average number of events
attended The average number of events attended by youth. 3
29
PHOENIX MULTISPORT
Phoenix Multisport was funded by the SEA Program to expand their program outreach to City of
Boulder teens, promoting sober events, healthy lifestyles, and an active community environment.
Phoenix Multisport provided multiple events for youth beginning in September 2016; however,
there was very low youth attendance at most events. Despite a wide variety of outreach efforts
including presentations to high school student councils and announcing events at other programs
(Natural Highs, iThrive, TEENS), recruitment remained a challenge particularly because adults are
Phoenix Multisport’s main client population and including teens in their outreach was a new and
expanded effort. Although there were future opportunities to cross-promote events with other
SEA Program subcontractors and the Healthy Futures Coalition, Phoenix Multisport ultimately
decided to withdraw from their involvement in the SEA Program.
30
Community-Based Processes
BOULDER COUNTY PUBLIC HEALTH: TA & SAPST TRAINING
With SEA Program funding, Boulder County Public Health Community Substance Abuse
Prevention (CSAP) program provides technical support for subcontractors, assists with project
coordination, reinforces consistent messaging across community partners, and offers education
and training opportunities for SEA Program subcontractors.
Year 1 Successes: In addition to organizing a SAPST class (Substance Abuse Prevention Skills
Training) for all SEA Program collaborators during the first year, the CSAP team developed
infrastructure to coordinate subcontractor TA, document sharing, and quarterly reporting. The
Google Drive developed by the CSAP team has become a hub of resources for all subcontractors
and facilitates individualized TA sessions. In addition, the CSAP team developed a YouTube
channel for subcontractors as a centralized platform for sharing SEA relevant resources,
including a webinar on the SEA shared messages. Finally, the CSAP team facilitated quarterly
meetings for all SEA subcontractors and partner organizations, which furthered the SEA Program
mission, activities, and evaluation.
Year 1 Challenges: The CSAP team adjusted the content of the subcontractor quarterly reports
throughout Year 1 to adapt to changing needs of the SEA Program and the evaluation. The team
also experimented with various strategies to maintain regular communication with all
subcontractors. In addition, the CSAP team is working to foster more partnerships among
subcontractors and is looking to expand subcontractor partnership opportunities in in future
years.
Process Indicators: Demographic data were not collected for SAPST training participants. Other
process indicators for SAPST training are in the table below.
Other Indicators Details Count
# of cohort participants
receiving SAPST training
The number of participants who completed SAPST
training in February/March 2017 8
# of TA topics/themes covered The total number of TA topics/themes covered in
presentations and one-on-one TA sessions. 25
31
Conclusions & Next Steps
The first year of the SEA Program evaluation was designed to assess how many people received
direct services by SEA Program subcontractors, successes and challenges of subcontractor
program implementation, and learnings from subcontractor partnerships. Much of the early SEA
implementation work was focused on laying the foundation for a strong collaboration among
subcontractors and developing infrastructure for tracking progress made toward SEA Program
goals over time. In the first year of the evaluation two subcontractors elected to not continue
with the program (AFY and Phoenix Multisport) due to capacity issues with program
implementation. In addition, several of the subcontractors who will continue with the program
made changes to their data collection strategies and processes in order to build enough program
capacity to contribute to collective data collection efforts, both through tracking individuals
served through programming and beginning to collect demographic data about individuals
served. Subcontractors participating in SEA moving forward will need to continue to build and
maintain capacity for tracking program indicators and outcomes for SEA to tell the story of
program impact.
Results from the evaluation indicate that the SEA Program is making progress toward its first
goal of widespread community distribution and awareness of information and programs
developed. SEA Subcontractors provided direct services to 350 individuals in the first year of
implementation. Subcontractors’ work toward learning and disseminating the SEA Program
shared message is underway, and subcontractors are beginning to develop partnerships that will
strengthen their ability to disseminate the SEA Program shared message in the community.
Future years of data collection will allow the City of Boulder to assess the degree to which SEA-
funded subcontractors working in collaboration collectively impacts the SEA Program goals.
The SEA Program is a five-year initiative and as such the evaluation will build over time to
continue to assess indicators discussed in the current report as well as new indicators as more
data become available. Additional indicators will inform the evaluation of the remaining three
program goals:
• Shift in community perceptions of risk associated with substance use, including the
impact of drugs, alcohol, recreational marijuana, and abuse of prescription medications on
children and youth;
• Prevent/reduce youth abuse of alcohol and recreational drugs including marijuana; and
• Reduce accidental ingestion of marijuana and other drugs.
32
Evaluation questions that will be addressed in future years in addition to those tracked in Year 1
include the following:
FUTURE PROCESS EVALUATION QUESTIONS
1. How many youth were exposed to the community-wide messaging implemented for this
grant?
2. How many adults were exposed to the community-wide messaging implemented for this
grant?
FUTURE OUTCOME EVALUATION QUESTIONS
1. What evidence is there that SEA strategies resulted in increases in adult risk perceptions
of youth substance use?
2. What evidence is there that SEA strategies resulted in increases in youth risk perceptions
of substance use?
3. What evidence is there that SEA strategies resulted in lower rates of youth substance use
or related consequences?
4. What evidence is there that SEA strategies resulted in lower rates of accidental ingestion
of marijuana and other drugs among youth 0-9 years old?
5. What does subcontractor program data suggest about SEA program effectiveness?
6. What evidence is there of the community impact of SEA strategies?
FUTURE EVALUATION ACTIVITIES
To answer future evaluation questions, the following evaluation activities will be carried out in
future years of the SEA Program.
SEA Subcontractor Outcome Data Collection: To assess progress towards achievement of SEA
Program goals, OMNI will work with SEA subcontractors to define and refine SEA outcome
indicators for each program. OMNI will also partner with the SEA Program Manager and
subcontractors to generate ideas about common outcome metrics that can be established across
subcontractors. SEA outcome indicators will be built into the SEA dashboard and available for
analysis and display in future years.
Data Dashboard: In future years of the SEA Program the dashboard will be expanded to collect
and display subcontractor outcome data as well as relevant community-level indicators (e.g.,
police data, municipal court data, accidental ingestion data, etc.).
Parent Survey: To assess the reach of the SEA community-wide impact, OMNI will explore
developing and distributing a parent survey through the school system to a sample of parents in
the City of Boulder. The survey would evaluate parents’ awareness of SEA community-wide
messaging; perceptions of risk associated with substance use, including the impact of drugs,
alcohol, recreational marijuana, and abuse of prescription medications on children and youth; and
safe storage behavior.
33
Year 2 Subcontractor Survey: As in Year 1 of the program, OMNI plans to administer a survey to
subcontractors in future years to continue to assess subcontractor collaboration, the
dissemination of the SEA message and successes and challenges with implementation of SEA
Program strategies.
Community-Level Measures: In Year 1 of the evaluation, OMNI had initial conversations with
several community agencies to assess feasibility for collecting city level indicators that could be
published on the public-facing SEA dashboard. In future years, the evaluation aims to gather,
analyze and aggregate available community-level data for display on the SEA dashboard.
Year 2 Evaluation Report: The report for Year 2 of the SEA Program evaluation will be designed
to aggregate data across subcontractors to develop a comprehensive picture of overall SEA
Program implementation. The report will also draw on outcome data provided by subcontractors
in addition to process data that builds on the Year 1 evaluation. As such, the Year 2 report will be
organized to focus on progress made toward the four SEA Program goals and will focus less on
the individual implementation successes and challenges of SEA subcontractors. A proposed
outline for the Year 2 annual evaluation report is in Appendix B.
34
Appendix A. SEA Subcontractor Survey
35
36
37
38
Appendix B. Proposed SEA Program
Year 2 Evaluation Report Outline
I. The Substance Education and Awareness Program
a. Information Dissemination
i. Boulder County Public Health: Safe Storage Campaign
ii. Boulder Valley School District: EFFEKT
b. Education
i. Boulder Valley School District: Sources of Strength
ii. El Centro Amistad: Teen Outreach Program
iii. YMCA: Influencer Training
c. Alternative Activities
i. YMCA: Prosocial Activities
d. Community-Based Processes
i. Boulder County Public Health: TA & SAPST Training
II. Evaluation Overview
a. Evaluation Goals
b. Process Evaluation Questions
c. Outcome Evaluation Questions
d. Evaluation Activities
e. Evaluation Methods
III. Year 2 Results
a. Evidence of widespread community distribution and awareness of information
and programs developed
b. Evidence of shift in community perceptions of risk associated with substance use,
including the impact of drugs, alcohol, recreational marijuana, and abuse of
prescription medications on children and youth
c. Evidence of prevention/reduction of youth abuse of alcohol and recreational
drugs including marijuana
d. Evidence of reduction of accidental ingestion of marijuana and other drugs
IV. Working in Partnership: Successes & Challenges
V. Conclusions & Next Steps
VI. Future Evaluation Activities