Category Archives: assessment

Learning at City Conference

At this time of year, many universities run teaching and learning conferences.

I have spoken at a few of these myself. In July 2020 I presented (online) at the University of Hertfordshire Teaching and Learning Conference. In July 2021 I did a keynote at the University of Cumbria Annual Learning & Teaching Fest. The week before that I had spoken at the LJMU Active Blended Learning Conference.

What I do find though, if I am not speaking, is I usually find out about these kinds of conferences, while they are happening on Twitter.

This year we are seeing a lot more conferences happening in-person. So when I saw that City, University of London, were running their Learning at City teaching conference in-person and were welcoming external delegates, and it hadn’t happened yet, I signed up.

One of my reasons for attending was to find out more about their approach to hybrid teaching, which I had read about online.

After coffee and pastries, we had the welcome and opening keynote.

Friend or Foe? Configuring the Role of Assessment as an Opportunity to Transform the Quality of Higher Education was presented by Professor Susan Deeley, who is Professor of Learning and Teaching(Urban Studies) at the University of Glasgow.

Taking a holistic view of its functions, assessment can be utilised in versatile ways to be effective and efficient. Playing a vital role at the heart of learning and teaching, authentic and sustainable assessment is key to facilitating students’ development of skills, competencies, and graduate attributes. This is in addition to enabling students to demonstrate their academic knowledge, understanding, and critical thinking. Stepping beyond traditional assessment boundaries, a less conventional path is explored where students are actively engaged through assessment and feedback literacies within a staff-student partnership approach to learning and teaching. It is asserted that this leads to students’ deeper learning and a more democratic classroom. Configuring such a positive role for assessment transmutes it into an intrinsically motivating force that can transform the quality of higher education.

The talk covered a range of issues relating to assessment, and I did a sketch note of the talk.

One of the questions at the end of the conference was how challenging it would be to change the assessments within a module due to the validation process that was quite rigid and lacked flexibility for change at pace, or would require re-validation. This is indicative of processes that were designed for in-person courses that would change rarely, lacked flexibility and agility. Sometimes there are good reasons for that, but it does mean that sometimes though you can’t be responsive.

Having booked into parallel sessions, I did the usual thing and go to the “wrong” session.

I had intended to go to the Discover Learning Design with LEaD session, but in the end, went to a session with two papers.

Understanding student digital experiences at City, University of London and  From face-to-face to remote learning: what can we learn from student experiences of pre-recorded lectures in the pandemic?

The first presentation was about the results of the Jisc Student Experience Insights survey at City.

Whilst teaching has moved back to campus, for many students there are still online or blended elements to their learning. It is therefore important to continue to evaluate student experiences of technology for learning. This paper will present the results from the 2021/22 iteration of the survey carried out at City during Spring 2022. The session will compare the results with the national benchmark from Jisc and previous iterations of the survey to see how the Covid-19 pandemic has impacted on student digital learning experiences.

It was interesting to see how a university was actually using the survey to inform decision making.

The second paper was focused on a cohort of students.

We explore students’ experience of asynchronous learning activities, with a specific focus on pre-recorded lectures and consider their role in promoting deep learning in an online education context. 

In this subject lecturers pre-recorded their lectures to help provide a more engaging learning experience. It was also designed to be more inclusive for those attending online.

In the presentation, we heard that student feedback was in the main positive, it was seen as more flexible, more accessible, students were able to interact with the recordings when they wanted to, and could rewind, review and drop into the recordings. It was also interesting to hear about the importance of captions (or transcripts) of the recordings.

After lunch there were two more sessions, I attended The Pedagogy of Hybrid Space and Transforming your presentation slides for online learning.

These were interesting sessions and demonstrated by their delivery the challenges of delivering hybrid (as in dual mode simultaneous) sessions. I will focus on these two sessions in a later blog post.

The final session of the day was a summary of the day and a celebration of staff achievements.

Overall I had a really good day and enjoyed all the sessions I attended.

The VLE is not dead – Weeknote #167 – 13th May 2022

Image by drippycat from Pixabay

Monday morning, I was off to Queen Mary University of London for their VLE Expo. This was very much a QMUL focussed event, though they had invited a range of VLE vendors. I liked how the focus of the event was about, what do we want to do to achieve our strategic aspirations, how will the VLE help us to do that, and which platform (or platforms) will enable us to do that.

There were some excellent presentations from the academic staff on the different ways in which they were using technology including virtual reality, mixed reality and H5P. I sat on the final panel session answering questions from the floor on a range of issues. A lot of the questions were more about the use of technology for learning and teaching, than VLE specific topics. However, I did get into a few discussions about the VLE on the Twitter as a result of attending the event.

I posted another blog post in my Lost in Translation series this time with a focus on the technical aspects of recording videos or audio files.

Most institutions will (probably) have equipment which staff can use, but if there is a strategic approach to building a sustainable approach to the use of video and audio, then universities will need to reflect if they have sufficient resources to support the increased demand for cameras and microphones.

video recording
Image by StockSnap from Pixabay

Tuesday I was still in London for a briefing session, well as it happened it got cancelled, so I worked in the office.

Apple have announced that they are going to stop selling the iPod once the current stocks of iPod touch run out. So did you have an iPod and if so which one?

iPod
Photo by Cartoons Plural on Unsplash

Wednesday, I did two all-staff briefings for two directorates on the Jisc HE sector strategy. From the feedback I got they seemed to be well received.

I was reminded on the Twitter about when I took my bike to work. I made a video back then.

Mike Sharples posted an excellent Twitter thread on how AI can be used to write essays. I agree with Mike, if we are setting students assignments that can be answered by AI, are we really helping students learn?

I enjoyed the #LTHEchat on images in presentations in the evening.

These two blog posts from 2005 (and 2007) were very influential on my presentation style: Gates, Jobs, & the Zen aesthetic and Learning from Bill Gates & Steve Jobs. I also posted  a link to a presentation from an internal TEDx event about delivering presentations – A duck goes quack.

Thursday, I made my way to Harwell for a drop in session I was running at the Jisc offices there, alas an accident the closure of the M4 meant I spent nearly four hours sitting the car rather than sitting in a room talking to Jisc staff. In the end I had to abandon my visit to the office.

Friday, I had a scoping call about learning spaces in higher education. Interested in the kinds of learning spaces higher education is using, flexibility, technology and the kinds of activities spaces are being used for.

I found this WonkHE article interesting – Learning design is the key to assuring the quality of modular provision in which Nick Mount talks about building quality assurance into the design of modular programmes and micro-credentials.

Traditional providers can expect to find themselves facing the difficult job of rethinking existing assurance processes that are designed for coherent, longitudinal programmes of study, so that they can accommodate a new pick-and-mix landscape of highly portable and stackable micro-credential learning.

My top tweet this week was this one.

Using AI to write an essay

Over on the Twitter, Mike Sharples has written a thread about how students could potentially use AI to write assignments (and how academics could use AI to mark and provide feedback).

As Mike points out, existing tools such as Turnitin won’t spot these fakes.

Though I do think we should stop going down the rhetoric that all students want to cheat, I do agree with some of what Mike says in this tweet in that we do need to reflect and rethink assessment.

I also agree with Mike’s other tweet in this thread if we are setting students assignments that can be answered by AI, are we really helping students learn?  

Of course this only the beginning of how AI will impact on education.

Maybe today is the day you start wearing your mask again – Weeknote #144 – 3rd December 2021

This was a full week back at work and I was in London for most of the week. Over the summer I had enjoyed working in the London office, a change of pace, location and routine compared to the forced working from home we had endured during the pandemic. Having had a fair amount of time off work, sick with covid, it was nice to be back in the office, talking and chatting to colleagues and similarly to the summer having the change of place and routine. The office was much busier than it had been in the summer. It felt quite normal in some respects, a little quieter than it was pre-pandemic.

However it was only a couple of weeks ago that I wrote about the possibilities of in-person teaching now that 90% of university students had had at least one Covid jab. Last week though we saw a new variant of concern of the coronavirus was identified by South African scientists and labelled by the WHO as Omicron.

On Monday I wrote about the impact Omicron could potentially have on the HE sector though my main messages was that universities should prepare for a possible lockdown.

Hopefully the vaccination rollout and mask wearing will reduce the chance of lockdown, but I would still be preparing for the possibilities of another lockdown regardless.

As we reach the end of the week, there have been some stories on the spread of Omicron, across the world, spreading to Europe, as might be expected with global travel and concerns this variant would have on infection rates (being more transmissible) and the subsequent impact on health resources. There were also some positive stories about the potential of vaccination to reduce the impact of Omicron.

Having said all that I would still be preparing for the possibilities of another lockdown regardless.

As you might expect, I ensured I was wearing my mask on public transport and when entering shops, eating places and as I walked around the office.

We had an HE leadership meeting on Monday and the majority of the meeting was discussing key challenges with our new CEO.

One of the things I reflected on was the success of Learning and Teaching Reimagined (LTR) and what we should do next. In order to build on and support the sector to deliver on the recommendation and work towards the challenges, Jisc working with members produced Higher education strategy 2021-2024: powering UK higher education which outlined how Jisc would support the sector going forward.

However LTR with its focus on teaching and learning leaves the door open to other ideas. There are a range of subjects that Jisc could focus on and undertake a similar range of activities and events as we did with LTR. This, like LTR, could be a sector-wide initiative focused on providing university leaders with inspiration on what the future might hold for higher education and guidance on how to respond and thrive in those environments. We could look at the student experience, leadership, the campus… there are a range of areas in which we could focus on in.

laptop and headphones
Image by Regina Störk from Pixabay

I published a blog post about the pandemic response and what we saw though described as online learning, wasn’t online learning.

One of things I have noticed is how often much of what was done during the numerous lockdowns was described as online learning. Let’s be clear you can describe what was happening as an emergency response to a crisis, even simplistically a pivot, but what was happening across schools, colleges and universities could in no way be described as online learning.

Some of my meetings were cancelled this week, which though freeing up time, can be frustrating.

This week was the Ascilite Conference. I really enjoyed attending and keynoting the conference back in 2009. Back then the UK was in the midst of an outbreak of swine flu. I didn’t go this year, but I may think about attending next year (pandemic permitting). This year it took place online and in-person at University of New England, Armidale NSW in Australia.

Martin Bean was part of a panel session and one comment (well tweet) I saw about the session mentioned the importance of authentic assessment, which made me think.

I think there is a blog post in this.

Was reminded this week that I am rubbish at Twitter.

While eating dinner on Wednesday evening, I participated in the #LTHEChat Twitterchat, Decolonising Learning Technology  led by Professor John Traxler.

I participated and did note that so much educational technology is designed for specific sector and its cultural norms, and then adjusted for other sectors and then other cultures. It was a really interesting debate and I enjoyed the discussion.

As it was December, I started tweeting out my advent calendar posts from a few years back. I really ought to spend some time doing new ones.

At the end of the week we had a HE Team meeting.

My top tweet this week was this one.

Once more to London – Weeknote #128 – 13th August 2021

Well after a week off work, it was back to work. As with a couple of weeks ago I spent the best part of the week working in our London office. It was also a shorter week as I was on leave on the Friday. London was not very busy, but I was expecting that following my previous time in London. The office was even less busy, I was the only person in the office. It was obvious that many staff were still working from home. I don’t mind working from home. There were a few articles about the shift back to office working. Whitehall was looking to remove the London weighting for staff according to this article in the Guardian.

Whitehall officials have held high-level talks about taking away a salary boost awarded to London-based civil servants amid efforts to encourage workers back to the office.

Whilst on the BBC website an article asked: Should I be working from home or going back to the office?

People in England are no longer being asked to work from home. Instead the Prime Minister Boris Johnson is recommending a “gradual return to work”. However, in the rest of the UK, people are still being advised to keep working at home where possible.

I had quite a bit of flexibility on where I worked before the pandemic, so there is less pressure to return to the office. I have been working in the office though, I like the change in scenery and routine. After eighteen months being forced to work from home, I like the option of choice. Also during the summer holidays it makes more sense for me, when working, to be away from home.

First job was to clear that inbox full of e-mail, which to be honest didn’t take too long.

We had a team wide call on Monday, which was interesting. There will be changes in the team from September (a new manager) and we have a new CEO from mid-September.

We had an interesting meeting about the evaluation of Connect More, which people seemed to enjoy and got a lot out of.

Wednesday I had an interesting review and discussion about assessment, and what Jisc can do to support the sector to transform assessment. Despite the opportunities of digital in regard to assessment, many of the issues relating to the digital transformation of assessment, are much more about the transformation of assessment, with digital just being a catalyst. What is the purpose of assessment for example.

On Thursday I had a catchup with our (newish) HR contact. In my role I have no line management responsibilities (though plenty of matrix management responsibilities). We had a great discussion and chat about how we can make that matrix management more effective and efficient going forwards. So, we can move people from one area of Jisc to another to work on projects more easily. Something that a company like Apple do quite often.

I had a useful and interesting meeting with a university talking about our Powering HE document and the possible opportunities and challenges that universities will face over the next few years.

My top tweet this week was this one.

Unknown Unknowns and Unknown Knowns – Weeknote #117 – 28th May 2021

Monday I was focusing on one of the projects we are working on with an university looking at various scope areas and how technology and digital can make a difference. I was reminded of the NSA quote of cylinders of excellence when it comes to silo working. The concept of excellent departments, but not an excellent university came to mind, but also about the inefficiencies of silos working in isolation and not thinking about the impact of their development and change on the rest of the university.

At the end of the day we were discussing assessment. What is happening with assessment in higher education now and what changes made as a result of Covid-19 are now in place, but also the wider issues of assessment as well.

old television
Image by Free-Photos from Pixabay

Cancelled my subscription to Britbox, in the main despite there being stuff I would like to watch, I wasn’t finding the time to watch it.

Tuesday saw me back to our office in Portwall Lane for an in-person meeting with my line manager, our first meeting in-person since August last year. It was actually nice to be both in the office and in an in-person meeting.

Something that keeps coming to my attention is the future of teaching, especially the concept of dual mode or hybrid teaching. What are peoples’ experiences of “dual-mode”, “muti-mode”, hybrid teaching? What has the student feedback being like? Something I have been reflecting on this week.

Students prioritise a return to face to face teaching from September 2021, so says the findings from a report published by the UPP Foundation.

Students want universities to prioritise a return to in person teaching and are missing face-to-face interaction around their wider student experience.

This is something which isn’t too surprising and is also something that has come out of our recent research into the student experience. Though digging deeper for us, it was more the in-person interaction students were missing and less the teaching.

Wednesday afternoon myself and Isabel Lucas of HEDG and the University of Cumbria hosted a share shop, facilitated by Advance HE, on how universities can support students transitioning in HE. We looked at both new students and returning students.

In the session, How can we best support learners coming in to HE from a Covid context? We discussed the transition arrangements others have put into place for the next academic year.

In September, third year students returning to HE will not have had a normal year in higher education and it is likely that their third year will not be like it was before.

We discussed a range of issues, focusing on the known knowns and the known unknowns. More difficult to discuss the unknown knowns and the unknown unknowns!

We are aiming to share the findings from the shareshop in June.

Thursday was a light day in terms of meetings, but got even lighter, as one meeting was cancelled five minutes before it was due to start, with the other meeting, two people who had accepted were in fact on leave, so in the end the meeting lasted only five minutes.

The future of the office keeps getting discussed, with those who own offices explaining why going back to the office is so important and those who don’t explaining why it isn’t. For me a lot is about the kind of work you do, I don’t do the same thing everyday, so there isn’t a single kind of space I need all the time. Before Covid, sometimes I would be working alone, sometimes I would be in meetings, sometimes we would be collaborating and sometimes I didn’t know, so it was useful to have other people around to bounce ideas off and chat over coffee.

My top tweet this week was this one.

Powering up – Weeknote #108 – 26th March 2021

I realised that I have been walking and exercising less during the last few weeks, now the children are back in school, so this week I made a determined effort to increase the amount of walking I do.

Like last week, I have spent a lot of the week interviewing staff and students as part of a project we’re doing at Jisc. We have been talking to them about their thoughts and perspectives on digital learning. As with a lot of these kinds of interviews there are some interesting individual insights, however the real insight comes from analysing all the interviews and seeing what trends are in there. I also spent time planning a similar, but different project.

I attended a roundtable on a digital vision for Scotland and facilitated a breakout room reflecting on the vision.

If you have watched a 60 minute TV programme, you will realise few if any have a talking head for 60 minutes. Few of us have the time or the skills to create a 60 minute documentary style programme to replace the lecture, and where would you go to film it? So if you change the monologue to a conversation then you can create something which is more engaging for the viewer (the student) and hopefully a better learning experience.

In a meeting this week with staff from a university I was discussing this issue and their response was, what about comedy stand-up? That’s a monologue. That got me thinking and reflecting, so I wrote a blog post about needing a tray.

Lego Star Wars
Image by 501stCommanderMax from Pixabay

Continue reading Powering up – Weeknote #108 – 26th March 2021

Challenges for quality assurance in the wake of the pandemic

I was recently part of a panel session looking at Challenges for quality assurance in the wake of the pandemic – ensuring quality in digital learning, academic integrity in remote assessment, and emerging best practice

I gave a five minute presentation from my personal perspective.

We knew what we were doing.

We knew how to assure quality and academic integrity.

We thought everything was going to be fine….

There were early signs of the impact of Covid-19 back in December 2019 and January 2020.

We thought everything was going to be fine….

A year ago, everything changed.

We went into a national lockdown and all universities needed to start doing things remotely, at pace and at scale.

We quickly translated our practices to online versions.

Our lectures became Zoom calls.

Our seminars became Teams conversations.

Office hours became 24/7 e-mail.

However we merely translated, we didn’t transform, we didn’t think differently.

As for exams and assessment….

Well that didn’t go as planned and was challenging to move online.

Moving assessment online is a journey and one for which we had no map and no real idea of where we wanted to get to.

We thought we could do it?

We thought technology was the solution.

It wasn’t.

The thing is translating practice online doesn’t really work. You lose the nuances of what made the in-person experience so great and you don’t take the advantage of the affordances of what digital can bring.

Can we be surprised when the quality of the experience suffers

Moving exams online doesn’t work.

Moving assessment online is not only possible, but can be done in a way which maintains quality and integrity.

How can you do this when the requirements in place from university exam board, PSRBs and professional accreditation are based on an model which expects a physical in-person assessment.

You need to transform and reimagine assessment.

You can not transform assessment on your own.

However this takes time.

It takes a certain level of digital capability.

It requires an insight into what we mean by assessment and what we are trying to assess.

At Jisc we are here to help the higher education sector through the use of digital to reimagine teaching, learning and assessment and to reframe the student experience.

A shared understanding – Weeknote #105 – 5th March 2021

cogs
Image by Pavlofox from Pixabay

I started as Head of Higher Education at Jisc on the 1st March 2019. So I have done two years (and a bit), had three line managers, a changing role and, oh yes, a global pandemic.

On Monday I had an excellent conversation with Isabel Lucas from Cumbria about the HEDG meeting I was presenting at, at the end of the week.

I have been sharing internally (and externally) the draft of the Jisc HE Strategy.

Image by Free-Photos from Pixabay

Another post on language, this time from Wonkhe: Why what we mean by “online learning” matters.

As higher education institutions plan for what will happen as we move slowly towards more students being on campus, there is continuing chatter about the form that teaching and learning will take. This includes how best to deliver it and how to communicate what this might look like. In all of this discussion, there has been a proliferation of words like “remote learning”, “digital learning”, and “hybrid learning” – and these terms have largely been taken for granted in respect to their pedagogical nuance. But if the preferred solution to the problems created by the pandemic in the first semester was “blended learning”, as we tumble through a second semester it would appear that the HE sector is beginning to settle on its next term of preference – “online learning”.

We do seem to spend a lot of time discussing what we should call what we do. The article makes the point that this does matter. I disagree slightly what matters is not what it is called, but whatever it is called, we have an agreed and shared understanding of what is means for you, for me and the students. We change the term, but this doesn’t necessarily mean that we change our understanding. I recall having this discussion about the use of the term hybridthat I used in an article to mean responsive and agile, whilst someone else was using the term to describe a mixed approach. Words are important, but shared understanding actually allows us to move forward.

Image by Pexels from Pixabay

Wednesday I joined a panel at the Westminster Education Forum to deliver a session on the future use of technology in assessment.

“The future for England’s exam system – building on best practice from the 2020 series, the role of technology and ensuring qualifications equip young people with the skills to succeed post-18”

I only had five minutes, so not a huge amount of time to reflect on the challenges and possibilities. To think a year ago I would have had to travel to London by train, find the venue and then join the panel in-person. Today, I just switched on the webcam and there I was, did my presentation and then answered a few questions. I didn’t use slides, as there wasn’t always a need to use slides in these kinds of panel sessions and at an in-person event I wouldn’t have used slides.

Of course at a physical in-person event they would have provided lunch, which would then give delegates an opportunity to come and chat about what I had been talking about. That didn’t happen this time. I would say that though using Twitter as a digital back channel at physical in-person events does sometimes work, but people have to be using the Twitter. At edtech events I find a fair few people are , at other kinds of events, not so much.

campus
Image by 小亭 江 from Pixabay

I liked this post from the Independent: I’m tired of hearing that universities are closed – it simply isn’t true

Lecturers are doing all they can during the pandemic to support the myriad different ways in which students learn

It’s not as though the physical campuses are closed, they are open for those courses which require a practical element.

Then again schools are not closed, they are open for the children of key workers, as well as vulnerable children, and staff are working with them and delivering remote teaching to the children at home.

Yes the experience is variable across the country, even across a school, but to keep saying they are closed, doesn’t really tell the whole story.

Lens
Image by Rudy and Peter Skitterians from Pixabay

On Thursday I had a really good discussion with a university about digital strategy. How important it is aligned to the main university strategy, but also how it enables that strategy and other strategies as well. If you are in charge of a strategy, how does it enable others, and how do others enable yours?

At the end of the week I was involved in the HEDG meeting and did a presentation on Jisc’s Learning and Teaching Reimagined programme and where Jisc is going next. It was good meeting and the presentation seemed to hit the spot.

I had a planning meeting about a session we’re doing with Advance HE on digital leadership which looks like it will be a really good session.

Looked at the presentation I am doing next week at Digifest on the future of digital leadership, what it is and where we are potentially going.

Favourite HE story of the week was Campus capers, 1970 style from Wonkhe.

Strange things sometimes happen in universities and we’ve reported plenty of them here over the years. From hauntings and strange happenings to animal action and of course true crime events on campus. But this event which recently caught my eye is one of the oddest I’ve noticed lately. It all happened just over half a century ago at Keele University. Those were turbulent times as the world transitioned out of the end of the heady 60s era into a very different decade.

My top tweet this week was this one.

No snow – Weeknote #102 – 12th February 2021

Well no snow again… everyone else seems to have had it.

Got bogged down into internal systems and processes this week, something which I have been avoiding in my new role, well I say new role, I have been doing this job for nearly two years now. In my old role I would have to deal with contracts and suppliers, not so much as Head of HE. However with working with universities on digital strategy, digital leadership and blended learning, means I need to immerse myself back into these systems.

Did a fair amount of work on future visions this week.

I think this is an interesting concept for future online events. The concept is to record some keynote videos that can be made available for online conferences. Reminded me of how some have used TED talks in the past. However there is a difference between YouTube or Netflix and an online conference.  How do you add value to an event so that it is more than just streamed video? How do you facilitate social interactions, networking, discussion and also how do you encourage this?

Had an interesting and useful discussion on assessment this week with colleagues from the UK and Australia. The key challenges the HE sector have faced over the pandemic include: maintaining academic standards and quality as assessment is transformed, student wellbeing and engagement in regard to assessment, the skills and capabilities of staff to assess online, and how to transform at scale and at pace. There are still issues with assessing within in vocational and practical qualifications, There are challenges with PSRBs and Professional Certification.

These are the main challenges and pain points that arose from our research back in April and more recently, it is not an exclusive list and is potentially going to change as universities move again through the assessment process and learn new lessons on what they can and can not do.

  • Maintaining the academic standard and quality as required by internal and external regulations, as they translate and convert existing practice into online modes.
  • Ensuring staff have the necessary digital skills and capabilities to successfully deliver online assessment, across the assessment lifecycle. Each step of the lifecycle will require different skills to deliver.
  • Transform multiple modes of assessment to online versions at scale and at pace. Many universities have experience of designing and delivering online assessment, however they will not have done this at scale or transformed at the pace required.
  • Maintain student engagement through the next few weeks and through the assessment process, as they continue to socially isolate and study remotely.
  • Ensure student wellbeing during a time of crisis remotely and consider the impact of online assessment on wellbeing as an extra pressure and source of stress.
  • What technologies are out there that could be used to design, deliver and support online assessment? Which technologies should we be using?
  • What are other universities doing with online assessment? What best practice is out there? Who is doing it well? How do we compare?

My top tweet this week was this one.