Monitoring Tool
Content
Attachment 3
Florida Department of Health (DOH) Sexual Violence Prevention Program (SVPP) Programmatic Monitoring Tool
Site Visit
Education/Training #_________ Hotline #_________ Social Norms Campaign #_________ Special Project #________
Organization Name: Organization Phone Number: Organization Representative at site visit: DOH Contract Manager: Organization Program Director: Organization Mailing Address: Monitoring Date: Report Date: Follow-up needed?__ Yes __ No Date Needed: ____
2012 Monitoring Tool
1
Attachment 3
Not Met
PART I. Organization Contract Requirements Organization Contract Requirements Provide Evidence of the Following: Provide Evidence of the Following:
C. Community Outreach (continued) A. Staffing.
2. SVPP (or other) television and/or radio Public 1. Documents required for DOH contract (see Service Announcements (PSAs) are played Monitoring Tool Addendum for required list.)in the community. 2. Notification of staff changes (for staff paid by the 3. SVPP contract) posters and educational materials are distributed in the community. 3. Timesheets signed by supervisors for staff paid by 4. The Community Action Team (CAT) has had the contract. meeting(s). personnel policiesdate(s). 4. Written Provide meeting and procedures manual 5. The Community Action Team (CAT) has proposed maintained and accessible to staff. activities for each of the 6 contract that have 5. List of staff paid by thestrategic plan goals.a(Discuss) 6. The Community Action screened, & most background screening, dateTeam’s (CAT)by which significant success(s) and challenge(s). Provide short organization. synopsis. Organization Subcontracts Recordkeeping Assignments & 1. Section I, I, 1-4RequirementsSubcontracts of the 1. Contract recordsbeing met. Provider and standard contract is are kept in a secure confidential place and retained for subcontractor is held agreements/subcontracts indicate the required seven years. to the same standards as are in the DOH standard 2. The REVISE THIS? contract.provider ensures that shared costs apportioned to this contract Outreach equitable. Provide a copy of Community are fair and the cost allocation formulas. 1. If paid for by DOH contract funds, all publications, Property and Equipment media or program advertisements are pre-approved by 1. The provider does not purchase Disease Control SVPP and contain DOH and Centers for property or equipment valued at $1,000 verbiage. & Prevention (CDC) requiredor more with DOH funds. 2. Service location(s) are accessible and appear to be compliant with the Americans with Disabilities Act (ADA). 3. Service location(s) are clean, well organized, and well maintained.
Exceeded
N/A
Met
Comments
Did contract manager attend CAT meeting or meet with CAT members?
B. D.
C. E.
F. Audit Requirements
1. Expenditures of $500,000 or greater in state or federal funds are reported to DOH annually. 2. Annually, audits and/or financial reports are submitted to DOH in accordance with contract requirements. 3. Any weaknesses or reportable conditions in prior fiscal year’s audit report have been addressed. 4. Projected budgets using the SVPP template are 2012 Monitoring Tool submitted to SVPP yearly or as requested.
2
Attachment 3
2012 Monitoring Tool
3
Attachment 3
Not Met
PART I. Organization Contract Requirements Provide Evidence of the Following:
G. General Activities
1. The provider has an active Board of Directors that meets at least quarterly. 2. Data has been entered correctly into the Sexual Violence Data Registry (SVDR) and in a timely manner. 3. Communication between contract manager and organization staff is effective. 4. The provider participates in department-sponsored meetings, conferences, and conference calls. 5. The provider responds to SVPP–initiated contact(s) via e-mail, fax, and other means of communication to meet the needs of the contract(s). 6. The provider is currently on schedule to meet its required contract deliverables.
Exceeded
N/A
Met
Comments
Please Check All That Apply: Organization is a certified Sexual Violence (SV) center. Organization is a certified Domestic Violence (DV) center. All travel costs appropriated to this contract are processed in accordance with S. 112.061, F.S., and Rule 3A-42, F.A.C, Rules of the Bureau of Auditing, governing travel expenses. Does organization have a working relationship with county health department(s) (CHD)? Organization serves more than one county. Organization has a disaster plan. Organization is becoming more “eco-friendly”.
YES
NO
NA
Comments
Contact Name: List Counties: Describe how services will continue. Describe “greening” efforts.
2012 Monitoring Tool
4
Attachment 3
Primary Prevention Education & Training
Exceeded Not Met N/A Met
PART II. Education & Training Assessment Provide Evidence of the Following:
A. Service Tasks
Monthly Narrative Report is adequately completed. Timeline of Activities completed monthly. Program evaluations & data collections are used. 4. Sexual violence prevention presentations adhere to the approved topics as defined in the contract. 5. The provider obtains first name only on sign-in sheets for each presentation. 6. The “Tell Us What You Think...” form is completed by 75% or more of attendees. 7. Seventy-five percent (75%) or more of the polled attendees answered “Excellent or Good” to the question “Your Opinion of the Program Overall” on the “Tell Us What You Think...” form. 8. Pre-tests are administered at the first multi-session presentation, and post-tests are administered at the last multi-session presentation at least 75% of the time.
Comments
C. Financial Reporting
1. Quarterly expenditure reports are correct, allowable, and received by SVPP on time. 2. The percentage of funds drawn down is equal or greater than the percentage of time into the contract year.
D. Presentation Observations
1. Curriculum is age appropriate and has been approved by SVPP prior to use. 2. Supplemental materials are age appropriate and have been approved by SVPP prior to use. (attach copy) 3. The educator’s focus is on primary prevention of sexual violence. 4. The presenter demonstrates knowledge of FL laws and other aspects of sexual violence.
2012 Monitoring Tool
5
Attachment 3
PART II. Education & Training Assessment Provide Evidence of the Following:
D. Presentation Observations (continued)
5. The presenter involves the attendees with engaging activities. 6. The presenter is enthusiastic, uses movement, creative tools or other methods that enhance participation and learning. 7. The presentation delivery style is culturally and age appropriate. 8. The presenter uses effective focusing techniques if the participants become disruptive. 9. Does the educator see indications that social norms re. sexual violence are changing? Yes/No? Please explain. OBSERVED PRESENTATION: Date: Location: Audience: Name & length of video (if used) Strengths: Areas for Improvement: Quality of presentation Quality of answers to questions:
Exceeded
Not Met
N/A
Met
Comments
Excellent
Good
Poor
2012 Monitoring Tool
6
Attachment 3
Hotline
Exceeded Not Met N/A Met
PART III. Hotline Assessment Provide Evidence of the Following:
A. Hotline Service Tasks
1. Monthly Narrative Report is adequately completed. 2. Timeline of Activities completed monthly. 3. Program evaluations & data collections are used. 4. A 24-hour toll-free hotline is implemented maintained or enhanced as described by the contract. 5. A DOH approved 30-hour training or a 6-hour annual training for staff and volunteers has occurred prior to answering the hotline. 6. The Training Certification forms for Hotline Staff and Volunteers are completed and sent to DOH. 7. A signed letter submitted by the organization indicates the hotline provided 24-hour, 7-day week sexual violence information and referral coverage by trained staff or volunteers. 8. Language assistance is available to non-English speaking hotline clients & is reported to SVPP on the Hotline Year-End Summary Report 9. All hotline advertisements contain the following, “TDD users may call through Florida Relay Service at 1800-955-8771” (or 711). 10. If indicated, proof that within 72 hours of interruption in hotline service, the problem was resolved. 11. At least once each quarter the hotline is advertised and if applicable, CDC & DOH required verbiage is used. 12. Hotline is answered by a trained advocate; not a pager, answering machine, voice mail or answering service. 13. Hotline calls are answered “rape crisis hotline”, “hotline”, “helpline” or “crisis line”. 14. The hotline has at least one bypass feature to accommodate more than one call at a time.
Comments
B. Financial Reporting
1. Quarterly expenditure reports are correct, allowable, and received by SVPP on time.
2012 Monitoring Tool
7
Attachment 3
Social Norms
Exceeded Not Met N/A Met
PART IV. Social Norms Assessment Provide Evidence of the Following:
A. Service Tasks
1.Monthly Narrative Report is adequately completed. 2.Timeline of Activities completed monthly. 3.Program evaluations & data collections are used. 4.Provider reviews current rape prevention data and background information. 5.Provider conducts research by surveys, focus groups or observational studies. 6.Provider prepares and shares with SVPP the market assessment report. 7.Provider develops creative concepts and pre-approved campaign materials. 8.Provider implements social norms campaign. 9.Provider evaluates social norms campaign on an ongoing basis. 10. Provider ensures that information or research is updated with medical data reflecting the most current facts. 11. Provider submits to SVPP the results of any publication(s). 12. If used, the provider submits pre and post survey results. 13. Does the educator see indications that social norms re. sexual violence are changing? Yes/No? Please explain.
Comments
B. Financial Reporting
1. Quarterly expenditure reports are correct, allowable, and on time. 2. The percentage of funds drawn down is equal or greater than the percentage of time into the contract year.
2012 Monitoring Tool
8
Attachment 3
Special Projects
Not Met Exceeded N/A Met
PART V. Special Projects Assessment Provide Evidence of the Following:
A. Service Tasks
1. Monthly Narrative Report is adequately completed. 2. Timeline of Activities completed monthly. 3. Program evaluations & data collections are used. 4. Pre-approved methods & activities are used to educate the priority population. 5. Pre-approved pre and post survey questions are used and results are submitted to SVPP. 6. Concepts of the public health model and socio-ecological model are incorporated into the special project. 7. Information or research is updated with medical data reflecting the most current facts. 8. Results of any publication are made available to DOH. 9. The Nine Principles of Effective Prevention Programs by Nation, M., et.al. (2003). What works in prevention: Principles of Effective Prevention Programs. American Psychologist, 58, 449-456 are incorporated into the project. 10. Collaborative partners and their roles and responsibilities are reported to SVPP. 11. Does the educator see indications that social norms re. sexual violence are changing? Yes/No? Please explain.
Comments
B. Financial Reporting
1. Quarterly expenditure reports are correct, allowable, and on time. 2. The percentage of funds drawn down is equal or greater than the percentage of time into the contract year.
2012 Monitoring Tool
9
Attachment 3
Summary Report
1. Summary of observations and findings:
2. Strengths and commendations:
3. Suggestions for program improvement:
4. Supportive documentation required for contract file:
5. Corrective actions required:
6. Does organization have a need for technical assistance from SVPP?
7. Does organization have suggestions/recommendations for SVPP?
2012 Monitoring Tool
10
Florida Department of Health (DOH) Sexual Violence Prevention Program (SVPP) Programmatic Monitoring Tool
Site Visit
Education/Training #_________ Hotline #_________ Social Norms Campaign #_________ Special Project #________
Organization Name: Organization Phone Number: Organization Representative at site visit: DOH Contract Manager: Organization Program Director: Organization Mailing Address: Monitoring Date: Report Date: Follow-up needed?__ Yes __ No Date Needed: ____
2012 Monitoring Tool
1
Attachment 3
Not Met
PART I. Organization Contract Requirements Organization Contract Requirements Provide Evidence of the Following: Provide Evidence of the Following:
C. Community Outreach (continued) A. Staffing.
2. SVPP (or other) television and/or radio Public 1. Documents required for DOH contract (see Service Announcements (PSAs) are played Monitoring Tool Addendum for required list.)in the community. 2. Notification of staff changes (for staff paid by the 3. SVPP contract) posters and educational materials are distributed in the community. 3. Timesheets signed by supervisors for staff paid by 4. The Community Action Team (CAT) has had the contract. meeting(s). personnel policiesdate(s). 4. Written Provide meeting and procedures manual 5. The Community Action Team (CAT) has proposed maintained and accessible to staff. activities for each of the 6 contract that have 5. List of staff paid by thestrategic plan goals.a(Discuss) 6. The Community Action screened, & most background screening, dateTeam’s (CAT)by which significant success(s) and challenge(s). Provide short organization. synopsis. Organization Subcontracts Recordkeeping Assignments & 1. Section I, I, 1-4RequirementsSubcontracts of the 1. Contract recordsbeing met. Provider and standard contract is are kept in a secure confidential place and retained for subcontractor is held agreements/subcontracts indicate the required seven years. to the same standards as are in the DOH standard 2. The REVISE THIS? contract.provider ensures that shared costs apportioned to this contract Outreach equitable. Provide a copy of Community are fair and the cost allocation formulas. 1. If paid for by DOH contract funds, all publications, Property and Equipment media or program advertisements are pre-approved by 1. The provider does not purchase Disease Control SVPP and contain DOH and Centers for property or equipment valued at $1,000 verbiage. & Prevention (CDC) requiredor more with DOH funds. 2. Service location(s) are accessible and appear to be compliant with the Americans with Disabilities Act (ADA). 3. Service location(s) are clean, well organized, and well maintained.
Exceeded
N/A
Met
Comments
Did contract manager attend CAT meeting or meet with CAT members?
B. D.
C. E.
F. Audit Requirements
1. Expenditures of $500,000 or greater in state or federal funds are reported to DOH annually. 2. Annually, audits and/or financial reports are submitted to DOH in accordance with contract requirements. 3. Any weaknesses or reportable conditions in prior fiscal year’s audit report have been addressed. 4. Projected budgets using the SVPP template are 2012 Monitoring Tool submitted to SVPP yearly or as requested.
2
Attachment 3
2012 Monitoring Tool
3
Attachment 3
Not Met
PART I. Organization Contract Requirements Provide Evidence of the Following:
G. General Activities
1. The provider has an active Board of Directors that meets at least quarterly. 2. Data has been entered correctly into the Sexual Violence Data Registry (SVDR) and in a timely manner. 3. Communication between contract manager and organization staff is effective. 4. The provider participates in department-sponsored meetings, conferences, and conference calls. 5. The provider responds to SVPP–initiated contact(s) via e-mail, fax, and other means of communication to meet the needs of the contract(s). 6. The provider is currently on schedule to meet its required contract deliverables.
Exceeded
N/A
Met
Comments
Please Check All That Apply: Organization is a certified Sexual Violence (SV) center. Organization is a certified Domestic Violence (DV) center. All travel costs appropriated to this contract are processed in accordance with S. 112.061, F.S., and Rule 3A-42, F.A.C, Rules of the Bureau of Auditing, governing travel expenses. Does organization have a working relationship with county health department(s) (CHD)? Organization serves more than one county. Organization has a disaster plan. Organization is becoming more “eco-friendly”.
YES
NO
NA
Comments
Contact Name: List Counties: Describe how services will continue. Describe “greening” efforts.
2012 Monitoring Tool
4
Attachment 3
Primary Prevention Education & Training
Exceeded Not Met N/A Met
PART II. Education & Training Assessment Provide Evidence of the Following:
A. Service Tasks
Monthly Narrative Report is adequately completed. Timeline of Activities completed monthly. Program evaluations & data collections are used. 4. Sexual violence prevention presentations adhere to the approved topics as defined in the contract. 5. The provider obtains first name only on sign-in sheets for each presentation. 6. The “Tell Us What You Think...” form is completed by 75% or more of attendees. 7. Seventy-five percent (75%) or more of the polled attendees answered “Excellent or Good” to the question “Your Opinion of the Program Overall” on the “Tell Us What You Think...” form. 8. Pre-tests are administered at the first multi-session presentation, and post-tests are administered at the last multi-session presentation at least 75% of the time.
Comments
C. Financial Reporting
1. Quarterly expenditure reports are correct, allowable, and received by SVPP on time. 2. The percentage of funds drawn down is equal or greater than the percentage of time into the contract year.
D. Presentation Observations
1. Curriculum is age appropriate and has been approved by SVPP prior to use. 2. Supplemental materials are age appropriate and have been approved by SVPP prior to use. (attach copy) 3. The educator’s focus is on primary prevention of sexual violence. 4. The presenter demonstrates knowledge of FL laws and other aspects of sexual violence.
2012 Monitoring Tool
5
Attachment 3
PART II. Education & Training Assessment Provide Evidence of the Following:
D. Presentation Observations (continued)
5. The presenter involves the attendees with engaging activities. 6. The presenter is enthusiastic, uses movement, creative tools or other methods that enhance participation and learning. 7. The presentation delivery style is culturally and age appropriate. 8. The presenter uses effective focusing techniques if the participants become disruptive. 9. Does the educator see indications that social norms re. sexual violence are changing? Yes/No? Please explain. OBSERVED PRESENTATION: Date: Location: Audience: Name & length of video (if used) Strengths: Areas for Improvement: Quality of presentation Quality of answers to questions:
Exceeded
Not Met
N/A
Met
Comments
Excellent
Good
Poor
2012 Monitoring Tool
6
Attachment 3
Hotline
Exceeded Not Met N/A Met
PART III. Hotline Assessment Provide Evidence of the Following:
A. Hotline Service Tasks
1. Monthly Narrative Report is adequately completed. 2. Timeline of Activities completed monthly. 3. Program evaluations & data collections are used. 4. A 24-hour toll-free hotline is implemented maintained or enhanced as described by the contract. 5. A DOH approved 30-hour training or a 6-hour annual training for staff and volunteers has occurred prior to answering the hotline. 6. The Training Certification forms for Hotline Staff and Volunteers are completed and sent to DOH. 7. A signed letter submitted by the organization indicates the hotline provided 24-hour, 7-day week sexual violence information and referral coverage by trained staff or volunteers. 8. Language assistance is available to non-English speaking hotline clients & is reported to SVPP on the Hotline Year-End Summary Report 9. All hotline advertisements contain the following, “TDD users may call through Florida Relay Service at 1800-955-8771” (or 711). 10. If indicated, proof that within 72 hours of interruption in hotline service, the problem was resolved. 11. At least once each quarter the hotline is advertised and if applicable, CDC & DOH required verbiage is used. 12. Hotline is answered by a trained advocate; not a pager, answering machine, voice mail or answering service. 13. Hotline calls are answered “rape crisis hotline”, “hotline”, “helpline” or “crisis line”. 14. The hotline has at least one bypass feature to accommodate more than one call at a time.
Comments
B. Financial Reporting
1. Quarterly expenditure reports are correct, allowable, and received by SVPP on time.
2012 Monitoring Tool
7
Attachment 3
Social Norms
Exceeded Not Met N/A Met
PART IV. Social Norms Assessment Provide Evidence of the Following:
A. Service Tasks
1.Monthly Narrative Report is adequately completed. 2.Timeline of Activities completed monthly. 3.Program evaluations & data collections are used. 4.Provider reviews current rape prevention data and background information. 5.Provider conducts research by surveys, focus groups or observational studies. 6.Provider prepares and shares with SVPP the market assessment report. 7.Provider develops creative concepts and pre-approved campaign materials. 8.Provider implements social norms campaign. 9.Provider evaluates social norms campaign on an ongoing basis. 10. Provider ensures that information or research is updated with medical data reflecting the most current facts. 11. Provider submits to SVPP the results of any publication(s). 12. If used, the provider submits pre and post survey results. 13. Does the educator see indications that social norms re. sexual violence are changing? Yes/No? Please explain.
Comments
B. Financial Reporting
1. Quarterly expenditure reports are correct, allowable, and on time. 2. The percentage of funds drawn down is equal or greater than the percentage of time into the contract year.
2012 Monitoring Tool
8
Attachment 3
Special Projects
Not Met Exceeded N/A Met
PART V. Special Projects Assessment Provide Evidence of the Following:
A. Service Tasks
1. Monthly Narrative Report is adequately completed. 2. Timeline of Activities completed monthly. 3. Program evaluations & data collections are used. 4. Pre-approved methods & activities are used to educate the priority population. 5. Pre-approved pre and post survey questions are used and results are submitted to SVPP. 6. Concepts of the public health model and socio-ecological model are incorporated into the special project. 7. Information or research is updated with medical data reflecting the most current facts. 8. Results of any publication are made available to DOH. 9. The Nine Principles of Effective Prevention Programs by Nation, M., et.al. (2003). What works in prevention: Principles of Effective Prevention Programs. American Psychologist, 58, 449-456 are incorporated into the project. 10. Collaborative partners and their roles and responsibilities are reported to SVPP. 11. Does the educator see indications that social norms re. sexual violence are changing? Yes/No? Please explain.
Comments
B. Financial Reporting
1. Quarterly expenditure reports are correct, allowable, and on time. 2. The percentage of funds drawn down is equal or greater than the percentage of time into the contract year.
2012 Monitoring Tool
9
Attachment 3
Summary Report
1. Summary of observations and findings:
2. Strengths and commendations:
3. Suggestions for program improvement:
4. Supportive documentation required for contract file:
5. Corrective actions required:
6. Does organization have a need for technical assistance from SVPP?
7. Does organization have suggestions/recommendations for SVPP?
2012 Monitoring Tool
10
