The shaky foundations of the TEF: neither logically nor practically defensible

*Opinion piece by Dorothy Bishop

I spent Sunday reading the Green Paper “Fulfilling our Potential: Teaching Excellence, Social Mobility and Student Choice“, a consultation document that outlines radical plans to change how universities are evaluated and funded. The CDBU is preparing a response, but here’s the problem. BIS is not seeking views on whether the new structures they plan to introduce are a good idea. They are telling us that they are a good idea, a necessary idea, and an idea that they will implement. The consultation is to ask for views on details of that implementation.

The government will no doubt be braced for howls of protest from the usual suspects. Academics are notorious for resisting change, so there is an expectation that there will be opposition from many of the rank and file who work in universities, especially from those whose political allegiances are left of centre. CDBU is, however, a broad church, and disquiet with the Green Paper comes from academics covering a wide range of political views.

The idea behind the TEF is that teaching has not been taken seriously enough in our Universities, because they have been fixated on research. As a consequence, students are getting a raw deal and employers are dissatisfied that graduates are not adequately prepared for the workplace. However, the evidence for these assertions is pretty shaky. If you’re going to introduce a whole new administrative machinery, then you have to demonstrate that it will fix a problem. A number of commentators have warned that TEF is a solution to a problem that does not exist.

Let’s look at the three premises behind the justification for TEF.

Premise 1: A focus on research has led to neglect of teaching in our universities. No evidence is presented for this claim. If chasing a high REF score led to neglect of teaching, we might expect to see poorer student satisfaction ratings in institutions who did well in the REF. We don’t. This is a complex issue, not readily reduced to simple indicators, but insofar as there is a problem, we could fix it by adopting Roger Brown’s suggestion of including evaluation of teaching in the next REF, rather than setting up a whole new exercise.

Premise 2: Students are demanding ‘higher quality, transparency and value for money’. No source is given for this claim. According to HEFCE’s report on the 2015 National Student Survey, “Overall satisfaction levels among students have increased steadily”. We should, of course, never be complacent, and the overall positive picture may hide some weaker performers. But are the problems really so severe that they merit an expensive bureaucratic exercise in evaluation that takes in all HEIs in the country?  Most of the system is working to a high standard, and much of it is outstanding. This is the reason why the UK is so popular with overseas students.

Students need to make well-informed choices, but so do those introducing major administrative changes to our Universities. Let’s go for evidence-based decisions, instead of making up policy on the hoof. There are plenty of good social scientists in our Universities who could tender for conducting a survey to discover what information students used when selecting a course, and how their expectations matched up to reality.

Premise 3: The Green Paper states that “While employers report strong demand for graduate talent, they continue to raise concerns about the skills and job readiness of too many in the graduate labour pool.” Here there is at least a source: a report by BIS. However, this explicitly stated that it was based on qualitative rather than quantitative research and: “Although a large and diverse range of employers were included in this research, the interview data cannot be interpreted as being statistically representative of all graduate recruiters in England or be used to describe the numbers and proportions of organisations displaying particular characteristics or behaviours.” (p. 11).

The BIS report noted that some employers mentioned problems: skill shortages in some areas, and poor quality applicants in others. Nevertheless, it was found that “In general employers are satisfied with the graduates they recruit, with more than four out of five feeling graduates are well prepared for work and similarly the employers in this study were also mostly satisfied with the graduates they had hired” (p. 13).

When the stated rationale for change does not bear scrutiny, one has to ask whether there is a hidden agenda. In the case of TEF, it is barely concealed: the outcomes from TEF are to be used to determine whether or not Universities can raise their fees, to facilitate the entry of new providers, and to take further the programme of transforming the Higher Education sector into a free market.

I suspect the government anticipates that it will be easy to sell this new vision to Vice Chancellors, many of whom seem influenced more by money than ideology. They have gone along with other major changes, notably introduction of and escalation of student fees, and implementation of the Research Excellence Framework (REF). But will they support Jo Johnson’s Teaching Excellence Framework (TEF)? I suspect not, because it doesn’t represent a good deal for them. Students now bear the cost of their higher education, and the benefits of a good TEF will be very different from those of a good REF. A high REF score affected the amount of money the University received from central funds, as well as reputation. A good TEF will allow the University to increase their fees, but, as UCL Provost Michael Arthur pointed out, it does not look as if the additional income would compensate for the cost of the exercise: “it is fairly clear that this doesn’t represent the way forward to the sunny uplands of financial sustainability, and at this level we would be unlikely to submit to TEF for purely financial reasons!”

I could say much more about further ideological and logical objections to TEF, but these points have been better made by others, such as Sheffield VC Keith Burnett, and those giving oral evidence to BIS on this topic.

If you think universities should challenge the premises of the Green Paper, rather than merely consulting on its implementation, please lobby your Vice Chancellor on this point – and join CDBU. We need members who share our concerns about the future of British Universities.