15 March 2018
Apples and Oranges
Reforming TEF
By John Watson
Well, I suppose it is an improvement. That the results obtained by universities under the Teaching Excellence Framework (the “TEF”) should be on a course-by-course basis will introduce more granularity. Unfortunately, though, it is only granularity along one particular axis.
The TEF was introduced by the then University Minister, Jo Johnson, as an adjunct to the National Research Excellence Framework which was already in place. With students now paying for their education, it seemed only fair that they should have information about the quality of the service they were buying. That was sensible enough. What was less sensible, however, was to feed all the information collected through an algorithm and come up with a system under which universities were badged Gold, Silver and Bronze, like contestants in some Olympic track event.
Six metrics were to be taken into account:
- quality of teaching;
- quality of assessment and feedback;
- support available from staff;
- dropout rates;
- whether graduates have moved to jobs or further study within six months; and
- whether graduates are doing graduate level work.
Some of the information is collected through student surveys and some from other sources, but one only needs to look at the above headings to see what a dreadful mixture of apples and oranges they are. Suppose that teaching standards at a particular institution are low. Is that in any way compensated by better support from the staff? Some maternal figure perhaps saying “yes, dear, I know you can’t solve equations, but don’t worry, because here is a nice cup of cocoa and a voucher for an ignoramuses anonymous session”? Does good quality feedback somehow offset a high dropout rate?
Why do it? No, I don’t mean why make the underlying information available. That will probably raise standards by shining a light into dark corners where it is needed. The question is why, having got all the information, it is necessary to mulch it all up into a star system, losing the very granularity that makes it valuable? For whose benefit is this be done?
Start with the students themselves who have to decide which university to select and indeed whether tertiary education is worth the money at all. Teaching quality will clearly be important, as will the employment position of graduates after the end of the course. But an eager student (and those are the important ones) will hardly put much emphasis on the dropout rate, or indeed on the support available from staff. For them, it is only part of the underlying information which is valuable.
What then about the state which looks to the universities to turn out “product”? Here there is much more concern about dropout rates – particularly where, as is the case with the medical profession, there is a need for a set number of graduates every year. But it is the dropout figure itself which they should be looking at, not some star rating into which it has been fed.
The need to watch the underlying figures is even more important for the universities themselves. If the teaching quality is low, it is not sufficient to improve assessment or to give better support or to make the lives of students pleasanter so that none of them dropout. What is needed is to improve the teaching. Allowing compensation elsewhere only allows academics to game their overall result and to hide their weaknesses.
The answer to the question “Why?” must surely be money. When the cap on university fees rose to £9,000 in 2010, it was thought that market pressures would mean that only the top institutions would charge at this level. That market mechanism has failed and there are now annual fees of £9,250 per annum more or less across the board. How can they be contained? Well, the thinking goes, if the market won’t do it then the government must and that leaves it in need of a way of assessing the relative values of the different educations. It is surely no accident that the proposal to give separate TEF ratings to different courses within universities coincides with the suggestion that different courses should carry different fees.
This is an area where unintended consequences abound. The introduction of fees has led to a financial feeding frenzy among academics, which has diverted much the benefit and made the changes hard to reverse. It is not the moment to try to be clever again. Basing course fees on metrics is bound to lead to any number of distortions and to lose the central point that we want an educated population and not just a trained one. Of course it is a good thing that the underlying information should be available so that people can see what sort of degree they’re getting and so that the system can be improved, but that is all that it should be used for. The government should put the focus on keeping University fees generally at a level which is affordable (beginning by abolishing the ridiculous system under which the interest charged on student loans is set at a rate designed to make successful graduates subsidise those who never repay) and then give grants where there is a need for qualified people in a particular area. Yes, of course it will be expensive.
