ISEOST Information > Evaluation Plan
The ISEOST Symposium has been evaluated by Branch Associates. Assistance in these efforts was provided by Dr. Susan Kinnevy, Director of the University of Pennsylvania’s Center for Research on Youth and Social Policy (CRYSP), as well as by the OSTRC research staff. Evaluation activities gathered information in 3 key areas:
- The level of “buy-in” and reactions to the proposed professional development model.
- The identities and details of the Symposium participants (who they are; how they articulate their needs; what they currently experience in professional development; etc.)
- Recommendations for future professional development efforts in terms of impact on staff, programs, and students; in terms of increased representation of underserved students; and in terms of replication.
The evaluation was driven by the following research questions:
Level of “buy-in”
- Is the ISEOST professional development model perceived as relevant or valuable?
- What are the anticipated applications of this model to both ISE and OST programs?
- What are the anticipated support systems and challenges for this model?
Key Players
- Who are the desired or likely participants in the ISEOST professional development model?
- What is the current level of familiarity within the ISE and OST communities?
- In what types of professional development do ISE and OST staff currently participate?
- What are the professional development needs of ISE and OST staff?
- What are the professional development needs of ISE staff members interested in incorporating OST methodologies and student outcomes into ISE programs?
- What are the professional development needs of OST staff members interested in incorporating ISE curricula and student outcomes into OST programs?
- Are there other logistical steps to implementing the ISEOST professional development model?
Future Professional Development Efforts
- What will be the most effective way of identifying, recruiting and retaining participants for the ISEOST professional development model?
- What formats, or combinations of formats, will be most effective for the professional development activities?
- Who should facilitate the professional development model?
- What are the anticipated staff, program, and student outcomes of the model?
- How should the increased inclusion of underserved audiences be ensured?
- How should the professional development model be promoted, distributed, and replicated?
- How should the outcomes of this model be monitored and evaluated?
Evaluation Findings
Through the evaluation activities of the planning grant, the Project Team learned about the participants of the symposium (and thus the likely participants of a pilot ISEOST professional development model) and their reactions to the model, which will be important for sustaining the project. In addition, useful information was compiled on the advisory committee members’ attitudes about the planning process and a start was made in gathering organizational demographic information that may affect the content of the model. Due to the small sample sizes, there was not enough information on the organizational demographic differences between ISE and OST to draw conclusions.
Overall, participants expressed a high level of satisfaction with the symposium. Participants enjoyed networking with others in their field and outside their field, learned new content and gathered new insights. In particular, participants learned about the fields of ISE and OST, and learned about the proposed model, which increased their belief in the importance of the model in terms of benefiting youth. These self-reported changes in knowledge were especially true at the end of the symposium, but also true, to a lesser extent, one month after the symposium. Participants posed many questions about the model and the topics addressed during the discussion and focus groups will help refine the model as it continues to be developed.
Participants generally viewed the model as relevant, valuable, and applicable to their work and anticipated support from their organization. Importantly, many of the attendees would like to participate in future professional development activities using the model. Their contributions will be critical in sustaining the momentum for planning, developing and implementing a collaborative professional development model for staff from ISE and OST organizations.
Summary of Symposium Evaluation Findings
Complete Symposium Evaluation Findings
