TEF

TEF (and KEF?) – the latest developments and what it means for publishers

tef logo

Since the announcement of TEF results in June, the publishing industry has been relatively quiet on this subject (partly because of summer holidays). However, the Higher Education sector has been very busy during this period.

First of all, the TEF’s official name has changed from “Teaching Excellence Framework” to “Teaching Excellence and Student Outcome Framework”, though the acronym is to remain ‘TEF’. A “lessons learned from year 2” document was published in September, and the year 3 framework was announced earlier this month. In addition to these developments, a pilot on subject level TEF has been set up, to run during the current academic year; 30-40 institutions are involved. However, none of the results from this exercise (or names of participating institutions) will be published.

The changes identified via the “lessons learned” document are to be implemented immediately in the TEF year 3 round.  The most striking of these is certainly the decision to cut the significance of the NSS results by 50%. It could have a direct impact on the many prestigious institutions which this year were disappointed to receive Silver or Bronze, largely owing to bad NSS results. It may be possible for some of these institutions to re-apply and be rewarded with a better outcome in 2018. Another factor that might contribute to better results (and has been campaigned for by the Russell Group and others) is the introduction of benchmarking, which will help institutions where metric data did not give a true picture of life on campus, especially at institutions with high levels of part-time students, such as the Open University.

To offset the reduction in contribution of the NSS results, there will be three new main additions to the data collected for year 3: the inclusion of grade inflation metrics (the details of this to be confirmed); numbers of student contact hours (as a weighted metric); and the inclusion of Longitudinal Education Outcomes (LEO) data (more about this in a future post). All these changes signal a move away from trying to measure student satisfaction and towards the capture of more concrete outcome data.

…and then there’s KEF: at the HEFCE conference 2 weeks ago, Jo Johnson (Minister for Universities and Science) announced the introduction of a third framework, to sit along the REF and the TEF. The Knowledge Exchange Framework is being introduced to measure knowledge transfer and commercial outcomes, and will be yet another metric that universities will have to get their heads round.

What does all this mean for publishers and the book trade?

As was discussed at the ABT conference last May, there is a strong imperative for publishers and booksellers to lobby for more influence in contributing to the TEF  – especially now that the focus has moved away from NSS results: the newly introduced metrics have less direct relevance to the information sector. However, the impact of content, data and information provision on the quality of teaching cannot be denied, and it is only right to assert that the voices of those in our industry should be heard when such a key initiative is in progress. The Office for Students (OfS) – the new regulatory body for Higher Education, which will start taking charge in 2018 – is proposing from 2019 to make the TEF compulsory for all Higher Education Institutions that have more than 500 students. This is one more reason why publishers and booksellers should not underestimate the importance of the TEF for UK universities, but be willing to explore all possible opportunities to take an active role.

 

Conferences, TEF

Conference: The Incredible Machine – What next for TEF?

The TEF results were due to be released this week, coming only second to the General Election as the most anticipated day this year in the UK Higher Education sector. The day after the election, the Department of Education announced a postponement of the publication of TEF results; a new date has yet to be confirmed.

Interestingly, the other dataset eagerly awaited, the first instance of Longitudinal Education Outcomes (LEO) – statistics on graduate salaries up to 5 years after graduation – were released on Tuesday as planned.

In anticipation of the TEF result, around 140 delegates came together on Election Day to discuss the current status and the future of the TEF. The event “The Incredible Machine – What next for TEF?” had been organised by the HE Policy blog WonkHE and was attended by “leaders, managers and staff working across policy, planning, strategy, communications, marketing, public affairs, quality, registry, student experience and in students’ unions” (so went the announcement of the event), but also by HE consultants, software companies… and some publishers.

From the academic publishing sector’s point of view, it was notable how little the provision of learning resources were mentioned, and how an awareness of the importance of these on teaching outcomes seems to be lacking amongst the self-declared “TEF wonks”.

Publishers did not get a voice (or even an ear) during the conference – which was perhaps to be expected – but even university libraries seemed to play a subordinate role in the TEF discourse. Not a single librarian attended the conference, and libraries were mentioned exactly twice. In a full day of discussions about the quality of teaching, this was pretty surprising.

However, the conference itself was highly interesting. During the opening address, given by Mark Leach and Ant Bagshaw of WonkHE, the audience was asked about its attitude to the TEF, and it was obvious that the majority of those present were very sceptical about whether the TEF aims were actually being met.

Different panel sessions led the proceedings throughout the day, discussing the current situation, the metrics, and the future of the TEF. The audience was very engaged and there was plenty of time allocated for questions, comments and discussions.  Full use was made of this, and many interesting aspects were raised.

Jayne Mitchell (Deputy Vice-Chancellor, Bishop Grosseteste University), who was also a TEF assessor, Alastair Robertson (Director of Teaching & Learning Enhancement, Abertay University Dundee) and Michael Wykes (Director of Policy, Planning and Business Intelligence, University of Exeter) sat on the first panel; each gave an overview of how they had approached the TEF application process at their universities. It was fascinating to hear, as the universities they represented were very different (both by type and by geography) and therefore their attitudes opinions and the approaches they adopted towards fulfilling the TEF differed significantly. Sector wide, there has been a huge variation on how the submissions were put together, where the focus was laid and which data or qualitative information each contained. It certainly will be fascinating to  examine how varied the submissions are collectively when all have been published.

The second panel of the day focused on metrics, data and league tables. Joy Elliott-Bowman (Policy and Public Affairs Manager, Independent Higher Education), Matt Hiely-Rayner (Director of Intelligent Metrix and Head of Planning, Kingston University) and Jackie Njoroge (Director of Strategy, University of Salford) talked about the independent HE sector and the implications of data for it, about if and how the TEF data can influence the Guardian University Guide rankings (answer: it will not!) and about the benchmarking of TEF metrics. This would have been an appropriate session in which to introduce discussion of learning resources, but, as already mentioned, these played a much smaller role than I had hoped for.

After this panel session, Sue Rigby, Deputy Vice-Chancellor at Lincoln, who is also involved in the HEFCE Learning Gain initiative, spoke about Learning Gain and the use of metrics. Her focus was on the metric, and not on the “how to”, so Learning Resources were once again not mentioned. Sue came to the conclusion that “Learning Gain is not going to provide a better proxy; it is an opportunity to think hard and better about learning”.

In the last panel of the day, Mark Jones (Chief Operating Officer, Higher Education Academy), Simon Marginson (Professor of International Higher Education, UCL Institute of Education) and again Sue Rigby talked about the future of the TEF, looking at projected future developments and the future of teaching quality enhancement. It was agreed that the TEF didn’t actually measure teaching quality and that the HE sector needed more involvement in the development of the metrics.   Following the discussions at the ABT Conference, which demonstrated that the academic bookselling and publishing industry has already recognised this, it was probably the most important conclusion of the day. Maybe it could be a point of connection for the BA and PA to start their lobbying.

The discussion then moved on to performance measurement in teaching, in which individual lecturers are being measured (in this instance, the approach shows a more direct transfer from REF to TEF) and the international impact the TEF may have. Prof. Marginson said that the REF had a big impact internationally, but he doesn’t think the TEF will. (This is a moot point, given that the TEF itself is a symptom of the sea-change that is taking place in how teaching and learning are carried out, in both the UK and many other countries, rather than itself generating that change.
In this discussion, the library was mentioned a couple of times, but the quality and impact of learning resources and their provision was not in the speakers’ (nor the audience’s) minds, which was surprising and somewhat dismal to see.

This was a day with many informative discussions and lots of relevant background information for the publishing sector.  It emphasised once again the importance of lobbying by the Book Trade Industry if it doesn’t want its considerable contribution to teaching and learning be side-lined in the future developments of the TEF.

A full write-up of the conference can be found on the WonkHE website: http://wonkhe.com/blogs/live-the-incredible-machine-what-next-for-tef/