The next day, a version of this appeared in the Guardian (without the comments on its implications for the REF. http://www.theguardian.com/higher-education-network/2015/feb/03/honours-degrees-arent-for-all-some-unis-should-only-teach-two-year-courses

Reducing overheads might indeed solve one problem, but who would make up the shortfall? Not QR because that’s just been given to the Research Councils.

“Transferring all the money to Research Councils won’t work. It would merely encourage the grossly bad behaviour that we’ve seen at Imperial, Warwick and Kings London all of whom have decreed that research must be as expensive as possible”

But that is only because there are such large overheads. If the overheads were small, then good proposals would be funded, and there would no perverse incentives for expensive grants per se. I think getting rid of REF, putting the money into research councils, and reducing the overheads might be a big improvement. What do you think? I think as problematic as the REF is the all the time spent on grants with only a very small proportion being funded (with excellent grants rejected, with no invitation to resubmit).

Jeff

Transferring all the money to Research Councils won’t work. It would merely encourage the grossly bad behaviour that we’ve seen at Imperial, Warwick and Kings London all of whom have decreed that research must be as expensive as possible.

A complete re-thinking of tertiary education is needed,

It seems to be a good thing that such a large proportion of the population now get higher education. But the university system has failed to change to cope with the huge increase in the number of students. The system of highly specialist honours degrees might have been adequate when 5% of the population did degrees, but that system seems quite inappropriate when 50% are doing them. There are barely enough teachers who are qualified to teach specialist 3rd year or postgraduate courses. And many teachers must have suffered from (in my field) trying to teach the subtleties of the exponential probability density function to a huge third year class, most of whom have already decided that they want to be bankers or estate agents.

These considerations have driven me to conclude, somewhat reluctantly, that the whole system needs to be altered. Honours degrees were intended as a prelude to research and 50% of the population are not going to do that (fortunately for the economy). Vice-chancellors have insisted on imposing on large numbers of undergraduates, specialist degrees which are not what they want or need.

I believe that all first degrees should be ordinary degrees, and these should be less specialist than now. Some institutions would specialise in teaching such degrees, others would become predominantly postgraduate institutions, which would have the time. money and expertise to do proper advanced teaching, rather than the advanced Powerpoint courses that dominate what passes for Graduate Schools in the UK.

Such a system would be more egalitarian than now too. Everyone would start out with the same broad undergraduate education, and the decision about whether to specialise, and the area in which to specialise, would not have to be made before leaving school, as now, but would be postponed until two or three years later.

If this were done, most research would be done in the postgraduate institutions, Of course there is some good research in institutions that would become essentially teaching-only, so there would have to be chances for such people to move to postgraduate universities, and for some people to move in the other direction.

This procedure would, no doubt, result in a reduction in the huge number of papers that are published (but read by nobody). That is another advantage of my proposal. It’s commonly believed that there is a large amount of research that is either trivial or wrong. In biomedical research, it’s been estimated that 85% of resources are wasted (Macleod et al., 2014Lancet 383: 101–104. doi: 10.1016/s0140-6736(13)62329-6). It’s well-know that any paper, however bad, can be published in a peer-reviewed journal. Pubmed, amazingly, indexes something like 30 jouranls devoted to quack medicine, in which papers by quacks are peer-reviewed by quacks, and which are then solemnly counted by bean-counters as though they were real research. The pressure to publish when you have nothing to say is one of the perverse incentives of the metrics culture.

The reduction in the amount of rubbish that was funded and published would allow QR related cash to be split among fewer places, with higher standards, If those standards are maintained, it could simply be allocated on the basis of the number of people in a department. Dorothy Bishop has shown that even under the present system, the amount of QR money received is strongly correlated with the size of the department (using metrics produces only a tiny increase in the correlation coefficient). In other words, after all the huge amount of time, effort and money that’s been put into assessment of research, every submitted researcher ends up getting much the same amount of money. That system wouldn’t work at the moment, because, with their customary dishonesty, VCs would submit the departmental cat for a share of the cash. But it could work under a system such as I’ve described.

Lastly, everyone should read John Ioannidis’ paper, How to Make More Published Research True http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001747

It’s true that he doesn’t mention a system such as I propose. That’s because the USA has already got such a system. It seems to work quite well there.

From the point of view of the humanities, the objection to metrics has been to citation counts: we generally cite research to show what’s wrong with it, so the most irritating work would score best. Conversely, one objection to the current system is that submitting work to be read discourages intellectual ambition. Some universities sent round instructions not to submit anything controversial for fear of antagonising the panel. (If it’s not controversial, it’s probably not worth writing.)

So what if we ruled out both citation counts and the reading of outputs? Assuming that we have to have league tables, I like the Sayer plan of assessing the environment. The submission would include current research activities, a five-year plan and data — on staff, staff development, research grants, PhDs awarded, etc. In addition, all publications in the period by all members of the department would be listed to show whether research was taking place and whether a reasonable proportion of it was thought good enough to be included in international publications. All members of the unit of assessment would be entered. Fly-ins would be made visible by a requirement to list contributions to the environment beyond delivering a paper in the last year of the assessment period.

On this basis, the illusion that the results define some ineffable quality would be seen for what it is, the process would be cheaper and less taxing, while departments divided by exclusions from the process could begin to heal their wounds. Best of all, academics could write the books and articles that they judge need to be written and at their own preferred pace. The REF would simply be a comparative snapshot of UoAs at a specific moment.

“One possibility that has been discussed is to remove the QR component of funding altogether, and give all funds to the research councils. The problem with this solution is that it would mean the research councils would have to grow in size enormously…”

The success rate of applications is currently so low, and that is a real loss of time and effort (and is perhaps the most dispiriting thing about research, especially for your researchers). If we could find a way to make success rates double or triple the size, that would be a real improvement in efficiency. It would mean an increase in size of the councils, but given that they are already dealing with so many unsuccessful applications, the increase might not be so great (just more happy outcomes). Making councils larger so that they can support more research would be compensated by the elimination of the extraordinary cost of REF.