Tag: Student Engagement

Digital Publishing, TEF

While we’re on the subject…

Gold Leaf

Last week, the Office for Students released some reports and initial findings on the subject-level TEF. What are the conclusions and what does it mean for publishers?

In parallel with the third round of the current TEF, the Office for Students conducted a pilot phase for a subject-level TEF, working with 50 different universities, colleges and other HE providers. (A list of participating institutions has been published by the OfS, but the ratings awarded remain confidential). This first pilot will be followed by a second round of pilots in 2019 to refine the process. The plan is to abolish the current TEF after its forth instalment in summer 2019 and initiate the subject-level TEF in 2020 (application phase) with the first round of results being published in spring 2021.

In the pilot, two different models were being tried, and the conclusion has been made that – despite neither of the models being fully fit for purpose – a “bottom-up” approach was being favoured, though the final model is likely to be a bit of a mix of “bottom-up” and “top-down”. This means that all subjects are being assessed as part of a ‘subject group’ submission but with separate metrics for each subject, and each subject receives a TEF rating of Bronze, Silver or Gold. The subject ratings then feed into the provider-level assessment, which is still being carried out separately.
The diagram below might be helpful in illustrating this:

STEFdiagram

(Source: Office for Students)

One major factor in the lessons learned from the pilot is the need to involve students in the process – after all, the TEF is supposed to be all about students’ experience and their learning outcomes! It has been confirmed that in future rounds the students’ voice will play a more prominent role. This is where it becomes interesting for publishers of learning content, because one of the main concerns the students expressed in the feedback session was that the quality and availability of Learning Resources should be measured and carry a greater weight in the TEF scoring.
As a result a new metric for learning resources will be included in future instalments of the TEF.

Unfortunately, the Publishers’ Association doesn’t seem to have been able to get involved in this (we are aware that attempts by the PA had been made and rejected), but thankfully the students seem to be the advocates for their libraries and ultimately the publishing community – they have realised what an important part the provision of learning resources plays in measuring teaching quality.

(All reports and publications can be found of the OfS website: https://www.officeforstudents.org.uk/publications/teaching-excellence-and-student-outcomes-framework-findings-from-the-first-subject-pilot-2017-18/)

TEF

TEF (and KEF?) – the latest developments and what it means for publishers

Gold Leaf

tef logo

Since the announcement of TEF results in June, the publishing industry has been relatively quiet on this subject (partly because of summer holidays). However, the Higher Education sector has been very busy during this period.

First of all, the TEF’s official name has changed from “Teaching Excellence Framework” to “Teaching Excellence and Student Outcome Framework”, though the acronym is to remain ‘TEF’. A “lessons learned from year 2” document was published in September, and the year 3 framework was announced earlier this month. In addition to these developments, a pilot on subject level TEF has been set up, to run during the current academic year; 30-40 institutions are involved. However, none of the results from this exercise (or names of participating institutions) will be published.

The changes identified via the “lessons learned” document are to be implemented immediately in the TEF year 3 round.  The most striking of these is certainly the decision to cut the significance of the NSS results by 50%. It could have a direct impact on the many prestigious institutions which this year were disappointed to receive Silver or Bronze, largely owing to bad NSS results. It may be possible for some of these institutions to re-apply and be rewarded with a better outcome in 2018. Another factor that might contribute to better results (and has been campaigned for by the Russell Group and others) is the introduction of benchmarking, which will help institutions where metric data did not give a true picture of life on campus, especially at institutions with high levels of part-time students, such as the Open University.

To offset the reduction in contribution of the NSS results, there will be three new main additions to the data collected for year 3: the inclusion of grade inflation metrics (the details of this to be confirmed); numbers of student contact hours (as a weighted metric); and the inclusion of Longitudinal Education Outcomes (LEO) data (more about this in a future post). All these changes signal a move away from trying to measure student satisfaction and towards the capture of more concrete outcome data.

…and then there’s KEF: at the HEFCE conference 2 weeks ago, Jo Johnson (Minister for Universities and Science) announced the introduction of a third framework, to sit along the REF and the TEF. The Knowledge Exchange Framework is being introduced to measure knowledge transfer and commercial outcomes, and will be yet another metric that universities will have to get their heads round.

What does all this mean for publishers and the book trade?

As was discussed at the ABT conference last May, there is a strong imperative for publishers and booksellers to lobby for more influence in contributing to the TEF  – especially now that the focus has moved away from NSS results: the newly introduced metrics have less direct relevance to the information sector. However, the impact of content, data and information provision on the quality of teaching cannot be denied, and it is only right to assert that the voices of those in our industry should be heard when such a key initiative is in progress. The Office for Students (OfS) – the new regulatory body for Higher Education, which will start taking charge in 2018 – is proposing from 2019 to make the TEF compulsory for all Higher Education Institutions that have more than 500 students. This is one more reason why publishers and booksellers should not underestimate the importance of the TEF for UK universities, but be willing to explore all possible opportunities to take an active role.

 

TEF

A quick overview of TEF results

Gold Leaf

 

TEF results thumb

Earlier this week, ‘providers’ (i.e. participating universities and colleges) were informed by HEFCE of their Year 2 TEF results, and at midnight today all the results were made public on the HEFCE website. Alongside the final results all metrics data and written submissions have been made public, too (although universities have the right to appeal if they object to the category in which they were placed until 27th January 2017, and some are expected to do so).

So, have there been any surprises? Yes and no. It was expected that newer, more teaching-led universities like Aston and Loughborough would lead (and both of them did indeed receive a Gold status), and that traditional “red brick” universities would come out lower in the ranks. This has partially been the case.

Of the Russell Group universities, for example, the results have been a bit of a mixed bag. The majority of these 24 institutions obtained Silver status, with six of them being awarded Gold (University of Birmingham, the University of Exeter, Imperial College London, the University of Leeds, Newcastle University, and the University of Nottingham). However, three were merely given Bronze  – the University of Liverpool, the University of Southampton and – most notably – the London School of Economics. It has been revealed that most Russell Group universities ranked in the lower end of Silver, whilst the six Gold institutions only narrowly made it across the benchmark.

Which brings us to the next point: the metrics. It has widely been said that existing metrics are unsuitable for measuring teaching excellence; the publication of the results now demonstrates the important role the written submissions, which are heavily focused on qualitative data, have played. Without undertaking a detailed analysis it is difficult to say exactly why, but anecdotal evidence shows that the universities with ‘weaker’ metrics managed to obtain higher scores by submitting persuasive ‘soft’ information. For example, Durham University had a similar dataset to Nottingham, but ‘only’ received Silver status, compared to Nottingham’s Gold; in fact, Nottingham’s scores on student satisfaction were quite a bit below the set benchmark for Gold.

Clear winners of the TEF exercise are the colleges of Creative and Performing Arts. Most of them received Gold status, despite many not scoring very high on the employment and employability metrics (the latter is obviously owing to the nature of courses they teach).

What does all this mean for the publishing industry? At present, it’s difficult to gauge, but it’s worth noting that this new TEF status has been awarded for three years. Re-submissions will be accepted, but none of the Gold institutions is expected to take the risk of losing its standing. However, there have been claims that Gold might be both a blessing and a curse – some universities have already announced budget cuts for projects designed to improve student satisfaction: after all, they did receive Gold, so clearly no further improvements are needed?!?

To determine the full extent of opportunities and impacts for the publishing industry, a full analysis of the metrics and – most importantly – the written submissions will need to be carried out. Gold Leaf will do more work on this over the summer.

A quick look at just half a dozen of the submissions shows that the Library seems to feature quite prominently in the more successful ones; and while provision of learning resources tends not to play a direct role, some universities found it worthwhile to highlight their participation in EBA or PDA purchasing schemes, and / or the direct relationship with publishers they enjoy.

This could be a good starting point for a more detailed analysis of submissions.  We’ll keep you posted!

 

A list of the TEF results for the institutions that participated in the report “Resource Provision in Higher Education: Implications of the TEF and related initiatives” can be found below.

If you have any questions or wish to purchase a copy of the full report, please send us an email.
Gold Leaf also offers entirely customised seminars to individual companies featuring a 4-hour seminar on the TEF in your offices.
We would be delighted to support the industry in taking the study forward by working with individual publishers or booksellers or with syndicated groups on the future developments of the TEF, its metrics and the implications on learning resources.
We look forward to hearing from you.

 

University of Aberdeen Opted out
University of Bedfordshire Silver
BPP University Bronze
University of Cambridge Gold
Cardiff University Silver
University of Edinburgh Opted out
University of Gloucestershire Silver
University of Greenwich Silver
University of Huddersfield Gold
Imperial College London Gold
King’s College London Silver
Kingston University Bronze
University of Lincoln Gold
University of Nottingham Gold
Nottingham Trent University Gold
The Open University Opted out
Oxford Brookes University Silver
University of Reading Silver
University of Salford Bronze
Sheffield Hallam University Silver
University of St Andrews Gold
University of Surrey Gold
University of Sussex Silver
University College London Silver
University of Westminster Bronze
University of Worcester Silver
Writtle University College Bronze
University of York Silver

 

TEF

The Teaching Excellence Framework – what’s next?

Gold Leaf

The ABT Conference closed last week, and every delegate received a copy of our report. You may or may not have had a chance yet to read it, but we’d like to talk a bit more now about what’s next.

First of all, since the report went to the printing press on May Day, the release date of the TEF result was announced. Participating “providers” (i.e. HEIs that handed in a submission) will be informed – under embargo – about their results on June 12th, and all results and submissions will be made public on June 14th. Do keep checking our Blog, Facebook and Twitter accounts (@goldleaf2001) for on-time updates on this!

The Higher Education and Research Act 2017 has now passed into legislation and was published a fortnight ago; the full text can be accessed on the government website here.

The Act incorporated that a major and independent review will be carried out by 2019, which might change any of the aspects that are to be introduced in future, including linking TEF levels to tuition fees, subject-level TEF or a TEF for taught post-graduates.

The discussion about subject-level TEF and the metrics and bureaucracy to underpin this is moving forward; institutions are thinking about how to address it in future and much discussion is taking place. The universities aren’t content to await the results, they are grasping the action now to take their TEF offers into the future.

One example is the University of Manchester, which announced last week that it was to make a total of 171 staff redundant – 140 of them in teaching – and replace them with fewer and younger (and cheaper) lecturers, in order to “invest in [their] strategic priorities”, of which the TEF is known to be at the top of the list.

Conferences, TEF

Academic Book Trade Conference 2017 (part 2)

Gold Leaf

(part 2)             ABG-Conference-NEW-logo

On Thursday evening, the conference Awards dinner took place, during which this year’s ABT awards were presented jointly by Stephen Lotinga and Tim Godfray, who took the occasion to prove his talent as a singer-songwriter and presented a musical welcome.

Tim Godfray guitar
As they did last year, OUP won the “Best Academic Publisher” award, and Julie Fisher from Palgrave Macmillan won the trophy as “Rep of the year”. Blackwell’s won the Academic Chain Bookseller award, and also the Bookshop of the Year, which went to Blackwell Sheffield. Glyn Littlewood of Blackwell Sheffield also won the Individual Bookseller Award, sharing it with Hilary Piert of O’Mahony’s. Roger Horton, CEO of Taylor & Francis was presented with the Lifetime Achievement Award, which is only given very occasionally for exceptional service to the industry. Many congratulations to all the winners!

The second day of the conference was opened by Simon Walker, Head of Educational Development at the University of Greenwich. He talked about student engagement and the involvement of students in the creation of content. He said that essentially student engagement happens when there is a move away from talking to students as consumers to working with them as partners. Simon also involved the audience in an online audience engagement exercise. Mark Toole, Head of Libraries and Learning Resources of Nottingham Trent University, spoke about his university’s approach to student engagement, which is much more data-driven. He laid out how data analytics can increase student engagement at an early stage and therefore increase retention rates.  The two presentations represented opposites on the teaching / learning continuum and therefore provided excellent foils for each other.

Simon Walker

Heather Sherman, Head of Technical Library Development at Dawson Books, gave a quick overview of Resources Provision and the TEF before the concluding, and arguably most entertaining, part of the conference: the “ABT Strictly” competition: a publisher, a bookseller and a librarian presented their future plans on resource delivery to 3 student judges. The first 10-minute presentation was given by Andrew Robinson, Director of Higher Education at Cengage. Next up was Will Williams, Head of Academic Sales at Blackwell’s, followed by Martin Gill, Head of Academic Services at University of Huddersfield.

After a short Q&A session, during which the students could question the presenters, the students finally rated the presenter on a scale from 1 to 10 in a “strictly”-fashion. The winner was Martin Gill; the students were asked to justify their scores, and their views were very illuminating. It was fantastic listening to such an engaged student panel – tank you to Nguyen Hoang (University of Reading), Harriet Lowe (University of Greenwich) and Kiu Sum (University of Westminster).

student panel

After two interesting days of conversations, presentations and discussions about a topic that will stay important for Academic Publishing in future the delegates headed home (in pouring rain this time). We are looking forward to next year’s conference and the continuation of our work on the TEF in the meantime!

(c) photos: Sharon Benton
Conferences, TEF

Academic Book Trade Conference 2017 (part 1)

Gold Leaf

“Leadership and Influence in a TEF-led world”                             ABG-Conference-NEW-logo

On a sunny May 18th, more than 100 delegates came together at the Crowne Plaza Hotel in the centre of Stratford-upon-Avon for the two-day annual Academic Book Trade Conference (formerly: APS Group Conference). This year, the conference programme focused on the Teaching Excellence Framework [TEF] and student engagement.  The conference Programme Director was my colleague Linda Bennett, of Gold Leaf, who has been organising the speaker programme since 2002 and with me is co-author of the report Resource Provision in Higher Education: Implications of the TEF and related initiatives, which was sponsored by the Booksellers Association.  A copy of the report was given to each of the conference delegates.

Outgoing chairman of the ABG Group, Scott Hamilton, gave an overview of last year’s industry figures and a review of the past year from the perspective of the ABG. The first address took the form of a conversation at which conference Chairman Richard Fisher, former MD of academic publishing at CUP, engaged a dialogue with Sir David Bell, Vice Chancellor of the University of Reading.

Sir David Bell

Sir David made a number of incisive points, including that the TEF was needed because universities have been slow to respond to criticism of their approach to teaching; that although some academics may not like the TEF, it is not a life-changer in the same way that the REF has been; and that although league tables, of which the TEF will now become an important one, are important drivers for university success, they are not the only drivers.

Stephen Lotinga, CEO of the Publisher’s Association, spoke about Publishing and Politics. He noted that there was a gap in the public’s understanding of the Book Trade industry’s role and its future and that both the PA and the BA need to lobby to make the industry’s importance more visible.

Richard Stagg, Publishing Director of Pearson, told the delegates that he saw the TEF as an opportunity for the publishing industry, as long as student’s needs and expectations were kept in mind and being met. He said very clearly that the current changes in approach are not about a transition from print to digital, but about the pedagogy behind teaching and learning and about interaction. He concluded by saying that it is the academic book trade’s responsibility to show the learning gains obtained from the content it creates, and to build new relationships.

The final keynote speaker was Peter Lake, Group Business Development Director at John Smith’s, who stressed the importance of the service element in making digital products work. He illustrated this by sharing some case studies in which service has been key to the successful use of published products. He also emphasised that the BA and the PA should lobby together, particularly on the inclusion of learning response into the next revisions of the TEF metrics.

keynotes day 1

At the end of the formal programme on the first day, we presented our TEF report. It was gratifying to see how much interest it has generated; our presentation was very well received.

In the breakout sessions following our presentation, the delegates discussed the impact of the TEF on the industry and how booksellers and publishers could work together on this, and compiled possible questions to include in the NSS about learning resources. One question that was suggested by several groups (in this or similar wording) was “To what extend do the provided learning resources support your learning outcomes?”
The groups came back with a big variety of suggestions on how cooperation could make the TEF work, including the sharing of data, the joint funding of further research into the area, collaboration on standards, and – mentioned by several groups – joint lobbying. Everyone agreed that transparency, communication and cooperation had to be key to make it work for both sides.

break-out 2

(part 2 to follow)

(c) photos: Sharon Benton