What do you need to know for the Ministry of Training, Colleges and Universities (MTCU) and what does your agency itself want to know?
We know from the Ontario Transfer Payment Agreement that you have to complete the LBS Exit and Follow-Up Form. The LBS Exit and Follow-Up Form also has sections to complete at 3, 6 and 12 months after the learner exits your program.
What questions does your agency need answered?
Designing your follow-up is about asking questions, then coming up with ways to get useful answers. Take time to determine which questions are the “right” ones. Consider the “who”, “what”, “where”, “why”, “when” and “how” and develop appropriate questions. There is no point asking questions if you do not intend to react to or use the responses. Inappropriate or unrealistic questions will produce ineffective and useless answers. You should ask questions that will provide positive quantitative or qualitative results to help your program improve, grow or develop.
In order to get the best information for comparison and tallying, you should design the answers to the questions on a weighted scale (as in a scale of 1 to 5 with 1 indicating complete dissatisfaction and 5 indicating complete satisfaction)
Community Literacy of Ontario’s (CLO) Smartsteps to Organizational Excellence has samples of both learner and community partner surveys in its Program Evaluation section. These may assist you in developing a suitable survey for your agency needs. CLO’s Capacity Plus: Organizational Capacity Resource Guide for Ontario’s Community Literacy Agencies has sample customer satisfaction questions for learners, volunteers and other community stakeholders in its Customer Service Management chapter. You can download these valuable resources from CLO’s Publications webpage.
Where will you record the results?
Once you get the facts and feedback from your Follow-Up service, where will you record the results?
You will record the results of the Ministry required information on the LBS Exit and Follow-Up Form. (For convenience, you can download the LBS Exit and Follow-Up Form from the Employment Ontario Partners’ Gateway (EOPG) Tools link, Forms section as a Microsoft Word document and print it.) All the sections of the LBS Exit and Follow-Up Form are also included in the Enhanced Learner Plan, which you can download for each learner from the Employment Ontario Information System – Case Management System (EOIS-CaMS). Each learner file must include one of these forms.
It is up to you to decide where you will keep the follow-up results that are specific to your agency. Sheets kept in learner files are not so easy to refer to when it comes time to complete your program’s evaluation. Many programs use a spreadsheet or database to compile all or some of the results. Others keep all the results in one file, while still others use tracking forms for recording information to which they will respond later.
Imagine Canada’s Project Evaluation Guide for Nonprofit Organizations has a section on Managing Data Collection that you may find helpful.
Where will you use the results?
Where you use the results of the follow-up facts and feedback depends on the type of information you have collected. Some results will be positive, while others may require changes or call for improvement. Imagine Canada’s Project Evaluation Guide for Nonprofit Organizations suggests ways you can use the results of project evaluation. Since a good part of the Follow-Up service is project evaluation, we have selected ways from Imagine Canada’s list that are transferable to LBS service provider options:
- identify ways to improve or shift your project activities
- facilitate changes in the project plan
- prepare project reports (e.g., mid-term reports, final reports)
- inform internal and external stakeholders about the project
- plan for the sustainability of the project
- learn more about the environment in which the project is being or has been carried out
- learn more about the target population of the project
- present the worth and value of the project to stakeholders and the public (e.g., 98% of learners who use our programs are extremely satisfied with our programming)
- compare projects to plan for their futures
- make evidence-based organizational decisions
- demonstrate your organization’s ability to perform evaluations
- demonstrate your organization’s accountability concerns for implementing its plans, pursuing its goals and measuring its outcomes
Information generated in MTCU’s Report 60D can be used for outreach purposes. All of the exit and follow-up information can be used to create an infographic, similar to one done by the Labour Market Group. Infographics can easily be displayed in a rack card size / postcard size.
When will you do the follow-up? An evaluation plan is not just a single event. It should involve a number of inputs that take place over an extended length of time. Your plan may be for three months, a year or be part of a three or five-year strategic plan. When you do the various activities in your plan will depend on why you are doing them, who is doing them, with whom and what they are doing. You may decide to survey community partners once a year, carry out random interviews with active learners at a different time in the year, and complete exit and post-exit interviews with learners monthly.
To meet MTCU requirements, you must complete a learner follow-up interview at 3, 6 and 12 months after learners exit your program. Some service providers attend to these interviews all at once on a monthly or weekly basis, as they become due, while others start the process on the day, 3, 6 and 12 months after the learners exit.
So, how do programs remember when it is time for these follow-up interviews? While some set up reminders in electronic calendar software or track them on spreadsheets, most programs use the reports and reminder systems generated by the Employment Ontario Information System-Case Management System (e.g., print the “Pending Reviews”page from EIOS-CaMS and use that as a tangible reminder of who needs to be contacted). Some programs also find it helpful to pick a specific date in the month and complete the follow-ups around the 15th of the month. .