Adventures with Machine Learning – Part 1

Last year, I wrote this post which shared how we had developed some analytics tools and data dashboards across Northampton Primary Multi Academy Trust. This was part of ongoing work in our trust to get better at using data we already have to describe, predict and ultimately intervene in the way we operate.

Click the picture to read about how we’ve used data dashboards to share information from standardised tests.

Since this post, we’ve had a lot of interest in how we’ve been developing analytics tools in the trust and so this is the first in a series of three blogs which I’ll interview the brains behind the analytics, self-confessed data geek, Matt Woodruff.

Matt Woodruff: The data man…

 

Matt is the founder and ‘Chief Data Scientist’ at Coscole Ltd. (now a part of Groupcall Ltd.) and I’ve been working with him for the last 3 years on this project within our trust.

 

 

One of the things I’ve learned from Matt is the power of predictive analytics. I believe that too much time is spent looking at pupil data as a rear view mirror in schools. This is often driven by a need to ‘know your data’ for accountability purposes rather than to help you think about what data can tell us about the future. If we think about the publication lag of documents like ASP (previously RaiseOnline), it’s crazy to suggest leaders should wait until November to find out about what happened in the past to a group of children who have already left the school. I think there’s more we could do to analyse the information we have about future cohorts of Year 6 to help adapt and tailor their provision whilst we still have time.

An example of how Matt’s brain works is our exchange over the title for this blog. I went for, ‘Matt and Tom’s Excellent (Analytics) Adventure’ whilst Matt suggested the catchy ‘A ‘Small Data’ Predictive Experiment using Machine Learning – Can MAT pupil level data generate reliable predictions for outcomes or identify pupils ‘at risk’?’.

Matt & Tom’s Analytics Adventure…

In this first interview, we go back about 3 years to a point in time where we were fumbling around in the dark for answers to a life without levels.

Me: Matt – when we first sat down with you, we had a ideas session where we outlined our vision of trying to bring together many pieces of pupil data in one place. What were your initial thoughts when you looked at the sea of post it notes which represented the many different pieces of data we wanted to bring together?

Matt: This is taking me back some time! It was at this stage that Coscole was finishing its direct commission with another MAT where we had spent three years building an approach to personalised learning, and in aggregating and visualising data.  During this time I’d worked with multiple stakeholders from head office staff and directors of education, with data managers and school leaders as well as some pilot projects with teachers, students and parents.  What was refreshing about beginning to work with NPAT is that you had, even back then, a good understanding of where there was challenge, and therefore where the opportunity was to improve: you were not setting out to boil the ocean; to have analytics be all things to all people.

I think like the mantra that ‘Exams are necessary, but not sufficient’ you imparted a view to me that said ‘Teacher Assessment is necessary but not sufficient’ in relation to understanding the whole profile of the pupil.  You had already engaged with cognitive ability testing and understanding pupil attitudes to self and school, and were bringing in external assessment as standardised scores to provide a more rounded profile.

Of course it is natural at that stage you are inclined to pull your hair out – a growing MAT, albeit with a single MIS provider, with a growing need to make more effective use of data and to put it at the finger tips in an easily digestible manner to those that need it most – when your data comes from different providers, in different forms, at different times.

The ‘sea of post-it’ notes therefore represented the fact that as a Trust you had already embarked on the journey of understanding that there was a challenge in making effective use of data and saw the opportunity in the potential impact to improving outcomes if you could do it right.  More than this – you’d moved down the road in determining exactly what you thought it important to capture, but also some things that you may have been doing by rote up until that point that you reassessed and decided it was not as important as you once thought it might be.

Me: There was a lot of planning and work that went on behind the scenes before we ever got close to inputting data. Aligning the MIS databases for across the trust was a big job and we spent a lot of time cleaning up our data as a trust which was an important but time consuming stage. Is this a normal part of the process with all schools you work with?

Matt: Getting to effective analytics is a journey.  Everyone knows the adage ‘Garbage in Garbage out’, and it becomes particularly true when you compound the issue by aggregating ‘garbage’ across schools.  Once again though this is not about boiling the ocean.  Do you have to align everything across an MIS?  Absolutely not.  Should you seek alignment over time, in the things that really matter? Absolutely yes.  You can do this in a way that does not undermine the context of different schools – thats vital.  It is also not about a doctrine of ‘top down’.  In my mind it’s about identifying good practice around data, and making sure all schools understand the importance of that.  Good practices with data lead to much less wasted time further down the chain, and not only wasted time but the impact of not really being sure about your data providence.  The typical reaction to seeing numbers, metrics, percentages is that we believe them.  In too few cases are the underpinning assumptions challenged – “how was this data derived”, “what moderation is in place across schools to ensure that an apple in one is an apple in another”.

Yes, technically, the MIS databases were aligned in so much as NPAT standardised on naming conventions for Aspects.  Yes we put in place data extraction technology and we warehouse that data and layer education modelling on top (the calculations that do your %GLD, Combined Y1/2 Phonics etc).  That’s business as usual really.  The fun starts again with the people and process elements. As soon as you visualise data in a more effective way, and don’t forget we’re not inventing new data here – we are just taking data you already have available – you instantly see gaps.  You instantly notice things that aren’t right.  And when I say ‘you’ I mean from CEO down.  That can be a scary place for some because we lift up all the rocks.  I think that’s great, because this is absolutely not about blame for a legacy of data whose quality can be improved, it’s about finally having access to the tools to quickly spot variations and to scaffold the people and the processes to ensure data is reliable.

There is no one I have worked with that has not done something different in their schools after joining and visualising data, and that’s a great thing.

Me: We had several U-turns and changes throughout the process as we switched our position on the types of assessment data and teacher assessment descriptors. How were you able to manage these changing demands from a technical perspective?

Matt: You’d started with fairly granular objectives in teacher assessment if I remember rightly: levels and sub-levels, or steps and stages, or milestones and smaller objectives. The change specifically did not provide too much of a technical challenge around how we got the data, but I think we found it a particularly challenging time to understand the way in which you wanted to visualise it – how leaders and staff would need to see that in the most digestible form.  We use both a flexible visualisation approach with the Trust with Microsoft Power BI, as well as our own Apps built for Office 365.  PowerBI is naturally easier and quicker for us to adapt than the code in our Apps, but by the same stretch our Apps can provide a more effective interface at times for staff.

The biggest issue though is both from a technical and a data perspective.  We lose consistency, and history.  For me a major incentive for a mature approach to data and analytics is having access to this history so we can build trend analysis and forecasting.  Every time we decide to do something different it makes that more difficult.  In this case with NPAT those decisions and changes were actually dealt with fairly early on and we’ve collectively been consistent since.

Me : After a year or so of being to analyse pupil information, we then started the conversation around how we could use technology to start to predict future attainment. You introduced me to the concept of Machine Learning. Can you explain (to a non-data specialist) how Machine Learning works?

Matt: There is a lot of hype around Artificial Intelligence (AI) and Machine Learning (ML) right now.  Three years ago everything was Big Data, in much the same way.  In many respects there is absolutely nothing new about ML, its been an active research area since the 1950’s and arguably in different forms well before that.  Today, ML is a subset of the domain of AI and deals with the ability of computers to learn from data.  It is technology that is now prevalent in just about every other aspect of our lives – from blocking spam in emails, to recommending products on Amazon, films on Netflix, and of course most recently in developing self-driving cars.

ML is itself subdivided into:

  • ways that we set out specific parameters for the computer and where we know what we are looking for (supervised learning),
  • where we want the computer to look at unstructured data and classify it itself (unsupervised learning), or
  • where we set up a computer to learn through its own exploration – most famously used with Google’s AlphaGo team beating the world champion (reinforcement learning).

These developments have brought ML more recently into the mainstream.  Tools are widely available to utilise ML models both with open source approaches, with Microsoft, or in combination.  In fact, Microsoft have recently announced the fact that they are integrating ML/AI approaches with PowerBI, which is really exciting.  EdTech companies like CenturyTech integrate ML in their adaptive learning routines.

In these ways, the technology already exists to make what we do in education faster, better, and deliver more impact.  We spend 95% of our time looking backwards – what has just happened – whether that is last year or the last ‘data drop’.  This is ‘descriptive analytics’  – we are simply describing the things that have happened.  Other industries have already moved into ‘predictive analytics’ – using data to predict what is likely to happen in the future.  Where we can get to beyond that is with ‘prescriptive analytics’ – if we know what is likely to happen in the future what should we be doing now either to mitigate risk or extend opportunity?  The potential for a learning system that provides effective and efficient decision support for the human in an education context is vast.

This isn’t about ML being able to take peoples jobs; that in five years we’ll have robot teachers.  This is about us leveraging what computers do far better than us, in order that we focus our human intelligence on the things that the machines will never be able to do (there is some ongoing debate on when the singularity might occur, but I think that’s beyond this blog…).

Me: There came a point where I asked you if you thought you would be able to predict our SATs results last year and you went off and did something clever on your laptop and came back with some results. What did you do?

Yes, we got to that point where we knew that we had good consistent data, and conceivably enough to do something meaningful from a predictive point of view.  We tend to like lots of data for reliability, and when you boil it down a Year 6 cohort is not a lot of kids, even over 7 schools.  However, we were at the point 18 months in where we had one years of historical data on the same basis as the current Y6 cohort.  The same ‘schema’ if you like, of the things that we thought might matter in predicting outcomes.

Truth be told this was new to me.  We’d been active in development at Coscole around the AI stack being released by Microsoft and also at that time had been engaged in another predictive proof of concept with Microsoft and one of the largest MATs around Progress 8.  Looping back to my mantra of not boiling the ocean, I thought why not try something – it’ll either work or not work.  If it works, that’d be pretty cool.  If it doesn’t work, we’d be interested in why not.

The easy bit was actually the ‘data wrangling’.  This is normally the bit of any data scientists life that consumes 60-80% of their time – finding data, cleaning data, putting it in a form that can actually be consumed by something and then do something useful…  The joy for me is that we’ve done most of that: we have all your data warehoused, clean, ready to go.

I set out then to run a very simple experiment.  This type of work is not new, lots of providers do it with their own data and I’m sure in more advanced ways, FFT, Rising Stars, CenturyTech etc – but for me it was a validation of others results and I was personally interested in the correlations with the MAT data.  The question I was interested in was “Can we use school owned, in year data, from different data sources, for reliable prediction?” .  If so, the follow up would be more important – “How should this impact current data collection practices to save time for staff, and to highlight interventions early?”.  I also had an academic interest in a methods comparison study in the context of my PhD.

The experiment was a straightforward linear regression model trained on your prior year Year 6 MAT data including a selection of pupil characteristics and principally their standardised scores in Reading and Maths.  I completed this in both Python as an open source approach as well as in Microsoft AzureML.  I then used this model to run against your (then) current Year 6 cohort to predict their Reading and Maths test outcomes.

The results were interesting – in one way or the other I’d described earlier. Or both – pretty cool.

In Part 2, I’ll be asking Matt to explain how accurate his predictions were when we opened the envelopes* on results day in July 2018 and what the implications of this are for adopting predictive analytics for outcomes or identifying those at risk in the future.

*We didn’t really open any envelopes on results day. It was a downloadable csv. at the much more civil time of 8am this year rather than waiting till midnight.

Rethinking Accountability Part 3 – School Improvement Processes

This is the third in a series of five posts about the issue of accountability in schools with some thoughts on how we can create healthier accountability for schools and teachers. Much of the content I am sharing in these posts is taken from my book, Wholesome Leadership. Other resources have either been created kindly by Oliver Caviglioli or are those which have been developed by the leadership team at Simon de Senlis Primary School or within Northampton Primary Academy Trust.

Rethinking Accountability

Part 1 – Introduction

Part 2 – A culture of healthy accountability

Part 3 – School improvement and monitoring processes (you’re reading this now!)

Part 4 – Assessing like a consultant doctor

Part 5 – Managing external accountability

In the first post, I laid out the following four key challenges for us as school leaders to focus on in schools in pursuit of a healthier accountability system which I believe is at the heart of making teaching a more effective and sustainable profession.

  1. Take the fear out of the challenge, creating the right conditions in our school for healthy accountability.
  2. Develop effective processes for school improvement, monitoring & evaluation that are used consistently as part of an ongoing cycle of improvement.
  3. Think really carefully about the purpose and processes of assessment and improve both its use and the quality of conversations that surround it.
  4. Manage external accountability with confidence, staying in control of inspections and audits and not allowing them to influence school improvement negatively.

This post focuses on the second challenge: the specific processes we use in schools as part of our quality assurance, accountability or school improvement cycles.

Informal first

A starting point for keeping our finger on the pulse of a school is (obviously?) to spend as much time as possible visiting classrooms. This means avoiding getting comfortable on a chair somewhere in an office or being drawn into time-heavy paperwork and meetings and getting out among the action. Visiting lessons, walking the corridors, lunch halls and playgrounds is the best way I know to remain in touch with what’s really going on in a school. Teaching (although not every Head has to be a current teacher), talking to parents on the gate, flicking through the books of the children you meet on your travels and playing with children in reception are all important parts of the job and help us gain an important wider perspective of school life. This informal approach has always been my favourite part of leadership and probably suits my naturally restlessness and impatient urges to walk around rather than sit still.

A school improvement schedule

Alongside an consistent informal presence, there has to be some kind of structured school improvement cycle  that takes place throughout the year. Without some kind of rigour or systematic approach to school improvement, things can go wrong in a school for the following reasons (and probably more):

  • Leaders become too preoccupied with operational and day to day issues and lose sight of the longer-term strategic work.
  • Regular checks or reviews get missed and so things stop happening that should be.
  • The right people don’t sit down regularly enough to evaluate findings from monitoring and agree next steps.
  • Steps for improvement are identified, but don’t become reality due to a lack of follow-up and follow-through activity.

In order to ensure a coordinated approach, most schools have some kind of schedule such as the one below which lays out different activities such as assessment, monitoring and evaluation. This is important in giving clarity to staff and leaders as to when different activities are happening and to ensure that the different processes flow from one to another. NB – I call these school improvement processes (because I think that’s what they are) but these are also commonly known as Quality Assurance, Monitoring and Accountability. Tomato, tomato.

A school improvement schedule. Be clear about what’s happening and when and avoid congestion and overload…

Now this is the dangerous bit because all that stuff I wrote in Part 2 about building a High Challenge; Low Threat culture in the school can quickly be undone if we go about monitoring in a clumsy or heavy-handed way. Trust takes time to build up but can be lost in seconds.

With this in mind, whatever school improvement or monitoring processes we carry out, it’s the way we do them that matters most.

Do no harm…

Here are ten things I try and think about when carrying out any school improvement/QA activity to try and help build a ‘High Challenge; Low Fear’ culture.

  • First, do no harm: Keep in mind the unintended consequences on workload, motivation and wellbeing that monitoring and evaluation processes can cause. Plan and execute them carefully to avoid any counterproductive effect.
  • Plan and communicate well: Logistics are always a challenge for monitoring processes, as there are often several people who need to be involved and cover implications. Let everyone know what is happening in good time so that they can plan around it. Last-minute arrangements can cause frustration and add unnecessary stress to the process. If staff need to share planning or children’s books, or present data, make sure the details of this are clarified in good time.
  • Execute well: Whether it’s a learning walk, book look or classroom observation, carry out the process carefully and diligently, ensuring that your ‘bedside manner’ keeps people at ease. Make arrangements for others to be free to pick up any other urgent issues in school so you don’t get distracted or pulled away; it is frustrating for teachers who will have put in a lot of preparation if those carrying out the monitoring get distracted by other issues.
  • Do ‘with’, not ‘to’: Avoid falling into the trap of acting like an inspector and involve teachers and children in whatever process you are carrying out, where possible. A book look is far more effective when talking to children and teachers about what you see on the pages and a planning scrutiny is almost pointless unless it is part of a conversation with the teachers who have written it. Try to capture the staff and student voice within any monitoring processes.
  • Avoid judging: In my experience, a lot of the discussion that takes place on these processes still relates to Ofsted grades, with people keen to hang their hat on a particular peg such as ‘Requires Improvement’ or ‘Outstanding’. This type of discussion is pointless and avoids the more difficult thinking process of identifying the most useful types of development.
  • Give good feedback: Give kind, specific and useful oral feedback immediately after the activity and provide written notes within a week. This feedback should identify time-bound actions (5 minutes? 5 days? 5 weeks?) so it is clear which are urgent issues and which are more medium term. The feedback should also identify a time and date to meet again to review.
  • Follow-up and follow-through: This is an essential part of the process, but one which is often neglected due to time pressures and leaders being distracted by other challenges. If actions have been agreed, they must be executed, and it is the job of leaders to follow up the conversations and next steps. Never assume that anything has changed unless you have seen it with your own eyes. It’s not enough to think that something has been done; we have to know it has been done.
  • Keep workload lean: Avoid the temptation to create lengthy documentation from different monitoring activities to try to prove that leadership happened. It is a common trap to spend your days monitoring and then evenings and weekends typing up wordy documentation to capture it all. Keep follow-up documents short and focus on clarity rather than an exhaustive list of everything you found out; otherwise, the main messages get lost in the waffle. If you feel that you are writing documentation for Ofsted or governors, remember that the only real evidence that matters is whether improvement happens in the classroom. Schools don’t get better when leaders are writing documents.
  • Evaluate properly: This part of the process is another that is often missed, but is incredibly important. There is no point finding out information from across a school unless you then sit down and reflect strategically on what it means. Once the processes are over, put a bit of space between them and then sit down and ask some good evaluative questions. What were the main messages from this monitoring? What patterns can you notice? Do the things that need improving relate to people or processes? What is our next move?
  • Repeat: Once you have processes established, keep repeating the cycles to get improvement and avoid tinkering with them too much. Once embedded, they cause less stress for everyone.

And now onto some specific processes themselves…

School Improvement Processes

The following four School Improvement Processes are shared here in WalkThru format, created by Oliver Caviglioli and adapted from Wholesome Leadership.

  1. How to carry out a Review Morning
  2. How to carry out a Learning Walk
  3. How to carry out a Progress Meeting
  4. How to carry out an Appraisal Meeting

The purpose of these WalkThrus is to try and ‘demystify’ the things school leaders do by making visible some of the common school improvement routines and processes they carry out. I have found, through working with Oliver, that these visual aids are helpful in simplifying otherwise complex processes.

Feel free to use, adapt around your own school, subject and context.

How to carry out a Review Morning

Review Mornings: Click the image to download a pdf poster of this WalkThru

Over the last three years, we have developed termly review mornings as a way of taking a more in-depth look at the learning taking place across a year group. Rather than a series of individual observations or occasional book looks, we tend to spend most of a morning in a single year group observing, reflecting and exploring the issues. Being released from the shackles of grading lessons is empowering for both teachers and leaders, and this type of process can switch the focus from one of ‘judging’ to one of supporting improvement.

  • A couple of weeks before the review morning, send around a short survey asking staff for their opinions on what is working well in the year group at the moment, any areas that they feel might be weaker and any specific challenges or children that they would welcome support with.
  • Read the feedback and do your homework on the issues or children that have been flagged up. If concerns were raised about the behaviour of individuals, do some background checks on them. If a particular subject was flagged as a weakness, talk to the subject leader about any involvement they have had. Analyse what any available data says about this issue too.
  • Try to put people at ease. A morning can feel like a long time for staff to have leaders snooping in and around their classrooms, so it is important to try to make it feel more like a joint process of investigation rather than being judged.
  • Have access to pupil data during the morning. It is really useful to be able to check out assessment information around groups or individuals that you might notice while in the classroom.
  • Give some short general feedback to all staff involved (including support staff) at the end of the morning and then follow up with teachers individually that evening if possible. Do not give Ofsted-style grades as part of this feedback; it should focus only on helpful developmental advice and a ‘next steps’ discussion.

Click this link for an individual post of the Review Morning WalkThru

Learning Walks

Learning Walks: Click this image to download a pdf poster of this WalkThru

Learning walks are more focused than the kind of unplanned classroom visits described above, but should still have a relatively informal feel. They should have one or two areas of specific focus, such as classroom behaviour routines, quality of classroom discussion or effective direction of support staff. These could either be a follow-up from previous development work or be identified through other monitoring or data analysis.

  • Identify your area of focus and communicate a few days in advance, either by email or at a briefing, that you will carry out a learning walk to focus on one or two specific areas.
  • Don’t intrude too much on the learning – try to be an invisible observer. When the situation allows it, engage in conversation with teachers and children to find out more about what is going on.
  • Adopt a curious mindset and avoid jumping to hasty conclusions based on first impressions of what you see. Stay long enough in each area to get a good feel for things and don’t rush it, even though you are likely to have time pressure to finish quickly.
  • If you are working alongside someone else, make sure you spend time looking from different angles and in different places, and then comparing what you noticed – it is interesting how similar and different two opinions can be.
  • Recognise any of your biases that might have crept in. Are some of the things you notice ‘pet peeves’ or ‘the way you would have done it’, rather than things that evidence suggests are the most important to focus on?
  • Follow-up with whole-school feedback, sharing successful things you saw and any areas you need to follow up on.
  • Follow-up individually with any support or coaching conversations where help might be needed, but keep it light. If learning walks become perceived as being ‘on show’, you will stop seeing what is typical in a classroom and only get polished versions instead.

Click this link for an individual post of the Learning Walk process

 Progress Meetings

Progress Meetings: Click the image to download a pdf of this WalkThruClass Attainment & Progress (CAP) meetings are carried out with each class teacher and relevant leaders shortly after each summative assessment point. Their purpose is for everyone to reflect on the current challenges and priorities within a class or year group. A range of sources, including assessment data, is evaluated to inform adaptations to the provision, curriculum or targeted support for the next term. Leaders can often interpret data in different ways from class teachers, so it is really useful to have this joined-up discussion once a term. It helps as a check point to hold both teachers and leaders to account for their contribution towards improvement, and allows leaders to see where any additional support or intervention is needed.

  • Organise CAP meetings to take place at the end of each term after a summative assessment point.
  • Analysis should be carried out before the meeting and agreed documentation should be completed which identifies the strengths and areas of focus for the next period. Good teachers are constantly evaluating children’s progress and adapting provision throughout the term, and this just formalises their thinking.
  • The meeting should be a professional discussion that focuses on classes, groups and individual children and what can be done to address their needs, rather than crude percentages or which borderline children might get ‘over the line’.
  • The main discussion should centre on how teaching approaches and the curriculum can be adapted within the next term to address any areas of concern.
  • Try to avoid the temptation to steer the discussion onto solutions to do with teaching assistant intervention and parental engagement. These may be useful things, but the 27.5 hours of curriculum and teacher time afford the greatest opportunity to make an impact.
  • Use research to support your decision-making. If suggestions focus on specific interventions or organisation such streaming or reducing class sizes, look at what research says about these areas and read around the issues more closely.
  • Agree what the actions are going forward for the next period and make a shared record of these. Start each meeting with a review of the previous agreed actions. Did everyone do what they said they would? What impact did these actions have? What are the next steps?

Click this link for an individual post of the Progress Meeting WalkThru

Appraisal 

Appraisal: Click this picture to download a pdf of this WalkThru

An annual appraisal should be a great opportunity to have a meaningful one-to-one with members of staff to thank them for their contribution to the school in the last year and to think about how they can develop in the future. Quality reflective time with staff is a rare commodity and the 40 minutes or hour that you spend with them in appraisals is precious. If this opportunity is not grasped, these useful discussions can be lost in meetings that race through the process, ticking the boxes but missing the point. In recent years, the quality of appraisals has been limited due to a demand that they become a process where a judgement is made on performance-related pay. Since evidence now shows that the impact of performance-related pay is negligible, we should make sure that appraisals are used more effectively as developmental and motivational tools for our staff.

  • Be prepared for the meeting – make sure that you have all the documentation you need from the previous year and that the appraisee is prepared to talk through their objectives with supporting evidence. If either side is not properly prepared, stop the meeting and arrange a time to do it again; otherwise, it undermines the process.
  • Talk through the objectives set for last year and allow time for reflection and discussion on these points. Avoid simply focusing on the success criteria and whether a target has been ‘met’.
  • Celebrate successes and take time to appreciate and thank people.
  • Include a discussion on wellbeing as part of the appraisal discussion and, if the appraisee would like to, write down some commitments that you will both make to support their wellbeing for the year ahead.
  • Avoid making the meeting a drawn-out game of cat and mouse which focuses on whether to give someone a pay rise. In my view, unless someone has not met the Teachers’ Standards consistently throughout the year (in which case some kind of managed support process is warranted), they should automatically make progress on the pay scale (upper pay scale is a separate discussion).
  • Set challenges and targets that stretch and excite you both within a ‘high challenge/low threat’ discussion. Remind your appraisee that no one gets sacked for being ambitious and doing a good job.

Click this link for an individual post of the Appraisal WalkThru

Thanks to Oliver for creating these WalkThrus. You can visit his awesome website where the whole collection are available to download.

WalkThru: How to carry out a progress meeting

This WalkThru of a Progress Meeting has been adapted from Wholesome Leadership, by Oliver Caviglioli as a free downloadable resource.

The purpose of these WalkThrus is to try and ‘demystify’ the things school leaders do by making visible some of the common routines and processes they carry out. I have found, through working with Oliver, that these visual aids can help to simplify otherwise complex processes.

You can download this WalkThru here as an A3 poster or visit Oliver Caviglioli’s awesome website where there’s a collection of other school improvement processes including Learning Walks, Review Mornings and Appraisal Meetings.

Feel free to use, adapt around your own school, subject and context.

Progress Meetings

Progress Meetings Or Class Attainment & Progress (CAP) meetings are carried out with each class teacher and relevant leaders shortly after each summative assessment point, no more than 2 or 3 times a year. Their purpose is for everyone to reflect on the current challenges and priorities within a class or year group. A range of sources, including assessment data, is evaluated to inform adaptations to the provision, curriculum or targeted support for the next term. Leaders can often interpret data in different ways from class teachers, so it is really useful to have this joined-up discussion once a term. It helps as a check point to hold both teachers and leaders to account for their contribution towards improvement, and allows leaders to see where any additional support or intervention is needed.

  • Organise CAP meetings to take place at the end of each term after a summative assessment point.
  • Analysis should be carried out before the meeting and agreed documentation should be completed which identifies the strengths and areas of focus for the next period. Good teachers are constantly evaluating children’s progress and adapting provision throughout the term, and this just formalises their thinking.
  • The meeting should be a professional discussion that focuses on classes, groups and individual children and what can be done to address their needs, rather than crude percentages or which borderline children might get ‘over the line’.
  • The main discussion should centre on how teaching approaches and the curriculum can be adapted within the next term to address any areas of concern.
  • Try to avoid the temptation to steer the discussion onto solutions to do with teaching assistant intervention and parental engagement. These may be useful things, but the 27.5 hours of curriculum and teacher time afford the greatest opportunity to make an impact.
  • Use research to support your decision-making. If suggestions focus on specific interventions or organisation such streaming or reducing class sizes, look at what research says about these areas and read around the issues more closely.
  • Agree what the actions are going forward for the next period and make a shared record of these. Start each meeting with a review of the previous agreed actions. Did everyone do what they said they would? What impact did these actions have? What are the next steps?

WalkThru: How to carry out a learning walk

This WalkThru of a Learning Walk has been adapted from Wholesome Leadership, by Oliver Caviglioli as a free downloadable resource.

The purpose of these WalkThrus is to try and ‘demystify’ the things school leaders do by making visible some of the common routines and processes they carry out. I have found, through working with Oliver, that these visual aids can help to simplify otherwise complex processes.

You can download this WalkThru here as an A3 poster or visit Oliver Caviglioli’s awesome website where there’s a collection of other school improvement processes including Appraisal Meetings, Review Mornings and Progress Meetings (CAP Meetings).

Feel free to use, adapt around your own school, subject and context.

Learning Walks

Learning walks are more focused than just managing by walking around making unplanned classroom visits , but should still have a relatively informal feel. They should have one or two areas of specific focus, such as classroom behaviour routines, quality of classroom talk/language or effective direction of support staff. These could either be a follow-up from previous development work or be identified through other monitoring or data analysis.

  • Identify an area of focus and communicate a few days in advance, either by email or at a briefing, that you will carry out a learning walk to focus on one or two specific areas.
  • Don’t intrude too much on the learning – try to be an invisible observer. When the situation allows it, engage in conversation with teachers and children to find out more about what is going on.
  • Adopt a curious mindset and avoid jumping to hasty conclusions based on first impressions of what you see. Stay long enough in each area to get a good feel for things and don’t rush it, even though you are likely to have time pressure to finish quickly.
  • If you are working alongside someone else, make sure you spend time looking from different angles and in different places, and then comparing what you noticed – it is interesting how similar and different two opinions can be.
  • Recognise any of your biases that might have crept in. Are some of the things you notice ‘pet peeves’ or ‘the way you would have done it’, rather than things that evidence suggests are the most important to focus on? Or are you looking at the impact of initiative that you had a hand in planning? In which case you may well be subject to confirmation bias where you think the initiative is more successful than someone else would who is less involved.
  • Follow-up with whole-school feedback, sharing successful things you saw and any areas you need to follow up on.
  • Follow-up individually with any support or coaching conversations where help might be needed, but keep it light. If learning walks become perceived as a ‘compliance walk’ or start to encourage others to put on a show, you will stop seeing what is typical in a classroom and only get polished versions instead.

WalkThru: How to carry out a review morning

This WalkThru of a Review Morning has been adapted from Wholesome Leadership, by Oliver Caviglioli as a free downloadable resource.

The purpose of these WalkThrus is to try and ‘demystify’ the things school leaders do by making visible some of the common routines and processes they carry out. I have found, through working with Oliver, that these visual aids can help to simplify otherwise complex processes.

You can download this WalkThru here as an A3 poster or visit Oliver Caviglioli’s awesome website where there’s a collection of other school improvement processes including Learning Walks, Appraisal Meetings and Progress Meetings (CAP Meetings).

Feel free to use, adapt around your own school, subject and context.

Review Mornings

Over the last three years, we have developed termly review mornings as a way of taking a more in-depth look at the learning taking place across a year group. Rather than a series of individual observations or occasional book looks, we tend to spend most of a morning in a single year group observing, reflecting and exploring the issues. Being released from the shackles of grading lessons is empowering for both teachers and leaders, and this type of process can switch the focus from one of ‘judging’ to one of supporting improvement.

  • A couple of weeks before the review morning, send around a short survey asking staff for their opinions on what is working well in the year group at the moment, any areas that they feel might be weaker and any specific challenges or children that they would welcome support with.
  • Read the feedback and do your homework on the issues or children that have been flagged up. If concerns were raised about the behaviour of individuals, do some background checks on them. If a particular subject was flagged as a weakness, talk to the subject leader about any involvement they have had. Analyse what any available data says about this issue too.
  • Try to put people at ease. A morning can feel like a long time for staff to have leaders snooping in and around their classrooms, so it is important to try to make it feel more like a joint process of investigation rather than being judged.
  • Have access to pupil data during the morning. It is really useful to be able to check out assessment information around groups or individuals that you might notice while in the classroom.
  • Give some short general feedback to all staff involved (including support staff) at the end of the morning and then follow up with teachers individually that evening if possible. Do not give Ofsted-style grades as part of this feedback; it should focus only on helpful developmental advice and a ‘next steps’ discussion.

 

Assess Like a Consultant Doctor: Chapter 8 preview from Wholesome Leadership

As part of this series of short posts to introduce my book, Wholesome Leadership, today I’m sharing a preview of Chapter 8:  ‘Assessing Like a Consultant Doctor’.

Wholesome Leadership is now on sale for pre-order and will be published around the 22nd of May 2018. You can read some of the early reviews or find out how to order here –  www.wholesomeleadershipbook.com 

This chapter sits in the second section of the book which is focused on the ‘head’ of leadership, part of the H4 Leadership Model which captures the heart, head, hands and health of school leaders. It follows on from previous posts I’ve written including Chapter 6 – ‘Strategic School Improvement & Research’ and Chapter 7 – Healthy Accountability‘.

H4 Leadership Model: ‘The Heart, Head, Hands & Health of School Leadership…

Assessment has become a primary culprit in the ongoing challenges of teacher wellbeing, recruitment and retention. Workload in this area has spiralled out of control – particularly in areas such as marking, data inputting and evidence gathering. Alongside this, a lack of clarity around national assessment since the removal of levels and increased accountability pressure on schools to achieve in performance tables have combined to create the perfect storm.

Within the chapter, I share some of the challenges that are faced by schools in this area, suggest 10 steps to ‘sorting out summative assessment’ and talk about how we can reduce workload through revising approaches to marking and feedback. The analogy of a ‘consultant doctor’ is used to suggest how we can use assessment in a more manageable and meaningful way within schools.

Assessing like a consultant doctor

One of the perks of having a child with a disability and a complicated medical history is that you get to see the expertise of consultant doctors up close.  I have immense respect for everyone in the medical profession, but some of the specialists who have worked with Freddie have been class acts. One of the things that strikes me about these doctors is how they look beyond the obvious and avoid drawing quick conclusions. Rather than making decisions based on limited information, the most skilled and experienced doctors will examine a range of information about a patient as part of their assessment, including blood tests, scans, examinations in clinic, patient history and referrals from other medical professionals. Similarly, the most effective teachers and leaders understand the limitations of any particular test or assessment and can use their experience and expertise to interpret them wisely. And just as careful and intelligent consideration of patient information can lead to an accurate diagnosis and the prescription of helpful treatment, meaningful assessment can lead to greater understanding of gaps in learning and effective tailored teaching and intervention.

Within the chapter, I interview Daisy Christodoulou (Director of No More Marking and author of Making Good Progress and the 7 Myths of Education‘.  who kindly gives up time to offer her expertise about the challenges that remain in schools to adapt to a life without assessment levels.

Here is a summary of the chapter in one page…

Wholesome Leadership is now on sale for pre-order and will be published around the 22nd of May 2018. You can read some of the early reviews or find out how to order here –  www.wholesomeleadershipbook.com !

Wholesome Leadership Preview Chapter 7: Healthy Accountability

As part of this series of short posts to introduce my book, Wholesome Leadership, today I’m sharing a preview of Chapter 7:  Healthy Accountability.

Wholesome Leadership is now on sale for pre-order and will be published around the 22nd of May 2018. You can read some of the early reviews or find out how to order here –  www.wholesomeleadershipbook.com 

This chapter sits in the second section of the book which is focused on the ‘head’ of leadership, part of the H4 Leadership Model which captures the heart, head, hands and health of school leaders. It follows on from previous posts I’ve written including Chapter 6 – ‘Strategic School Improvement’.

H4 Leadership Model: ‘The Heart, Head, Hands & Health of School Leadership…

Within the chapter, I identify three challenges for school leaders

  1. Taking the ‘fear’ out of the ‘challenge’, creating the right conditions in our school for healthy accountability. I use Mary Myatt’s concept of ‘High Challenge; Low Threat’ as a model for thinking about how we can keep the stakes low but the challenge high within schools.
  2. Developing effective processes for school improvement that are used consistently as part of an ongoing cycle. The chapter shares some approaches to processes such as appraisal, learning walks and the use of data.
  3. Managing external accountability with confidence; staying in control of inspection. I discuss how we can stay in control of external processes such as OFSTED and also offer some suggestions as to how OFSTED might further reform in the future.

For some more thoughts on developing healthier approaches to accountability in schools, I interview Headteacher, Michael Tidd, well known for his expert views and opinion on primary curriculum and assessment. Michael offers his typical blend of pragmatic wisdom and informed opinion on the challenges.

Here is a summary of the chapter in one page, as designed by the brilliant Oliver Caviglioli.

 

Wholesome Leadership is now on sale for pre-order and will be published around the 22nd of May 2018. You can read some of the early reviews or find out how to order here –  www.wholesomeleadershipbook.com