David looks at the importance of careers programme development in light of the new Education Inspection Framework
In its latest framework for inspection Ofsted has organised its judgements about the quality of education provided around three themes: intent; implementation; and impact. In broad terms the relationship between these three perspectives can be described as follows:
what the school or college states it plans to provide for its pupils and students
the delivery of the programme by teachers, tutors and others
the achievements and learning outcomes that pupils and students gain from the provision and its delivery.
Applying these headings to the careers programme in a school or college the intent is set out in the policy statement, strategic plan and detailed schemes of work and programmes of activity; the implementation covers the delivery of all the elements of the programme, including work in the classroom, other in-school and in-college activities, experiences of the workplace, activities based in colleges, universities and apprenticeship providers and personal guidance interviews; the impact is the learning that pupils and students gain from the programme and its various components.
The careers leader’s role is not only to plan and manage the design and delivery of the programme, but also to keep it under review and to evaluate it. Indeed, Gatsby Benchmark 1 requires the careers programme to be “regularly evaluated with feedback from pupils, parents, teachers and employers”. In this short article I have attempted to offer some clarification about the differences and relationships between review and evaluation.
Too often the words monitoring, review and evaluation are used interchangeably or elided into one continuous process, whereas in fact they represent different levels of activity. Monitoring simply refers to checking whether or not something that was planned actually took place. Reviewing extends beyond this and involves asking questions about how it went. Together monitoring and review become relevant when looking at intent and implementation. They involve actions such as looking at what actually happened and then discussing what went well, and what didn’t go so well, with school and college staff, with external partners such as employers, FE and HE representatives and careers advisers and with pupils, students and possibly their parents as well.
Evaluation, however, involves more than examining the delivery: it has to extend to impact and asking questions about what pupils and students gained, or did not gain, from the programme as implemented. Therefore evaluation cannot be done without assessing what pupils and students have learned, and this, in turn, cannot be done properly without referring back to the intended learning outcomes.
The CDI’s Certificate in Careers Leadership, which is being used to accredit several of the courses that form part of the Careers Leader Training programme, includes sessions on how to monitor, review and evaluate your careers programmes and align this to Ofsted’s Education Inspection Framework (EIF).
Careers leaders have access to several frameworks and tools to assist these different, but related, levels of activity.
The Gatsby Benchmarks set out a framework of inputs which can be used as basis for identifying in part the intent of the programme and then as a framework for reviewing the content of the programme. What the Gatsby framework doesn’t include however is any indication of what the intended outcomes should be, i.e. what knowledge, understanding and skills should pupils and students gain through the programme: for help with that aspect of planning careers leaders need to turn to the CDI’s Framework for Careers, Employability and Enterprise Education. Therefore the Benchmarks themselves can only be used as a basis for reviewing the programme, asking such questions as what have we got in place and how it is going, and not for evaluating impact. The original Compass tool, now referred to as Compass Classic, is based on the Benchmarks and provides an online facility for schools and colleges to self-review their careers programmes.
The Quality in Careers Standard offers another means of reviewing a school or college careers programme. The assessment criteria have been fully aligned to the Benchmarks but the Standard differs from Compass in that the process involves firstly self-review but then an external assessment from a visiting assessor. The criteria also extend beyond the Benchmarks in a few places, particularly with regard to having in place a planned programme of careers education in the curriculum referenced to a recognised framework of learning outcomes. The Standard itself is still largely a tool for review not evaluation, although it does require schools and colleges to have in place processes for assessing pupils’ and students’ learning and for evaluating the programme.
To date there are very few readily available tools to assist evaluation but The Careers & Enterprise Company has produced a Future Skills Questionnaire which schools and colleges can use with pupils and students to begin to assess their learning from the careers provision. The Skills audit is still in development and has yet to cover the full range of career learning outcomes set out in the CDI’s Framework. In time the intention is to make the questionnaire available through the careers leaders’ dashboard, Compass+. Of the three level of activity – monitoring, review and evaluation – it is the last one that is least well developed, not least because of the difficulties in finding effective practical approaches to assessing pupils’ and students’ learning from careers activities. Nevertheless it is a challenge we need to face if we are to make a successful case for more support for this aspect of education. We need to be able to answer the question – how do you know if the activities are making a difference?
David Andrews is an independent consultant, a former policy adviser to the CDI and co-author of The Careers Leader Handbook. He has recently published a second edition of his book on Careers Education in Schools.