Category Archives: assessment

assessment, weeknotes

Tulips from Amsterdam – Weeknote #241 – 13th October 2023

1 Comment

This week was all about Amsterdam, where I was attending the EDUTech conference at the Amsterdam RAI.

However, on Monday I was in our Bristol office. I had a planning catch up meeting about prospective communications and publications.

Tuesday I was off to Amsterdam. Having arrived and parked at Bristol Airport I had time for a coffee before boarding.

It was quite a quick flight, by the time we reached the Suffolk coast, we started our descent into Amsterdam. Having disembarked, I headed to passport control. There was a huge queue. Schiphol Airport had free WiFi, so I could use that in the airport whilst I had a long wait to get through passport control. Took over 90 minutes to get from the plane, through passport control, pick up my suitcase and then I could head to the railway station. It wasn’t long before I was at my hotel next to the RAI conference centre.

Following some issues with roaming in Spain in 2022 I have been a little concerned about travelling abroad and if I would have connectivity issues. However this time I had no issues.

The EDUTech Europe 2023 conference was over two days and covered a range of topics and subjects, but as you might have guessed AI was certainly up there being discussed across a range of sessions. As well as covering higher education, the conference also covers the schools sector, but the interactive programme made it very easy to filter out those sessions, so I could focus on the higher education topics.

I had been invited to participate in the Smart Campus panel session at EDUTech Europe 2023.

I had proposed that in the discussion I would chat about the following.

How do we move from a smart campus to a smarter or intelligent campus? Bringing in more data sources to get deeper insights and understanding.

What are the ethical and privacy considerations we need to be aware of before building a smart campus? Do we need to consider algorithmic bias in our planning?

A campus does not exist in isolation, how could we integrate the smart campus into the smart city (or smart community).

Across the rest of the day there were a range of sessions, I focused a lot on digital leadership. It was interesting to see the reliance on champions to deliver change. My own experience has shown that though champions can have an impact, generally the process of using enthusiasm to deliver effective change is inconsistent, and if you want to deliver holistic change across a whole organisation, then the use of enthusiastic champions generally won’t result in that change happening. Of course using champions as part of a portfolio of change levers, is certainly something that can work.

Thursday morning I attended HEAnet Group Advisory Forum via Teams. We discussed the HEAnet strategy as well as other topics related to technology and higher education in Ireland.

On Thursday I was invited to participate in the Assessment panel session at EDUTech Europe 2023, after someone had dropped out. This was an interesting session looking at the role of assessment.

Had a good discussion with a delegate from the University of Birmingham at the EDUTech Europe 2023, on various topics and issues. Have agreed to visit University of Birmingham.

Had an informative chat with a Principal Lecturer from Sheffield Hallam University on various topics and issues including smart campus and learning analytics.

Got some nice feedback from EDUTech Europe 2023:

I wanted to take a moment to express my sincere gratitude for your invaluable contribution as a speaker to EDUtech Europe 2023. Your expertise and insights added immense value to our event. The feedback we received from our attendees was overwhelmingly positive, and your knowledge and passion resonated with the audience. We look forward to the possibility of working with you again in the future.

On Friday I attended Jisc’s Research Evidence Advisory Group meeting. As we discussed research at Jisc I realised there were some key ways in which I interact with research. It started initially as three Cs, but by the end of the meeting this had grown to six.

Consumption – in my role I consume research by others, both internally and external research. Having access to that research is critical to the work I do.

Creation – I also create research, sometimes this is for projects, other times it has been for consultancy, and now and then for publication or for events.

Communication – It is important that research undertaken by Jisc is communicated internally and externally.

Community – I want to be part of the internal (to Jisc) research community, but also to be part of the external research community

Credibility – When we do research, it adds credibility to what we say and recommend. Having the evidence (gained from research) gives us the credibility we need when talking to the sector.

Co-ordination – It’s important to avoid duplication, or repetition; that we co-ordinate our activities.

Got some nice feedback from The Blended Learning in HE Conference I presented at last week.

I would like to thank you very much for speaking at The Blended Learning in HE Conference. We really appreciate the time and effort that you took in preparing to present at the event. Your address was well received by the delegates and the conference has already received some great feedback.

Did some initial organisation of Senior Education and Student Experience Group meetings, format and timings.

I spent much of the day travelling back from Amsterdam on Friday. Spent less time at passport control this time.

assessment

“Write it in your own words”

1 Comment
City of Bristol College Ashley Down Centre
City of Bristol College Ashley Down Centre by James Clay CC BY-NC 2.0

Twenty-five years ago I was a business studies lecture in a college in Bristol. I taught on various programmes including the HNC Business and Management and the Advanced GNVQ Business Studies.

A key part of those courses was assessment through assignments, this was time of continuous assessment, and no final exams.

Setting and marking assignments was always a challenge. You were assessing their understanding of the content, textbooks and lectures. I would write assignments that comprised a series of questions.

I remember colleagues would often talk to students who were struggling, to read the source material, the text books, and then write it in their own words.

In the education world there has been much discussion about ChatGPT and its impact on student assessment. Continue reading “Write it in your own words” →

assessment, conference

Learning at City Conference

1 Comment

At this time of year, many universities run teaching and learning conferences.

I have spoken at a few of these myself. In July 2020 I presented (online) at the University of Hertfordshire Teaching and Learning Conference. In July 2021 I did a keynote at the University of Cumbria Annual Learning & Teaching Fest. The week before that I had spoken at the LJMU Active Blended Learning Conference.

What I do find though, if I am not speaking, is I usually find out about these kinds of conferences, while they are happening on Twitter.

This year we are seeing a lot more conferences happening in-person. So when I saw that City, University of London, were running their Learning at City teaching conference in-person and were welcoming external delegates, and it hadn’t happened yet, I signed up.

One of my reasons for attending was to find out more about their approach to hybrid teaching, which I had read about online.

After coffee and pastries, we had the welcome and opening keynote.

Friend or Foe? Configuring the Role of Assessment as an Opportunity to Transform the Quality of Higher Education was presented by Professor Susan Deeley, who is Professor of Learning and Teaching(Urban Studies) at the University of Glasgow.

Taking a holistic view of its functions, assessment can be utilised in versatile ways to be effective and efficient. Playing a vital role at the heart of learning and teaching, authentic and sustainable assessment is key to facilitating students’ development of skills, competencies, and graduate attributes. This is in addition to enabling students to demonstrate their academic knowledge, understanding, and critical thinking. Stepping beyond traditional assessment boundaries, a less conventional path is explored where students are actively engaged through assessment and feedback literacies within a staff-student partnership approach to learning and teaching. It is asserted that this leads to students’ deeper learning and a more democratic classroom. Configuring such a positive role for assessment transmutes it into an intrinsically motivating force that can transform the quality of higher education.

The talk covered a range of issues relating to assessment, and I did a sketch note of the talk.

One of the questions at the end of the conference was how challenging it would be to change the assessments within a module due to the validation process that was quite rigid and lacked flexibility for change at pace, or would require re-validation. This is indicative of processes that were designed for in-person courses that would change rarely, lacked flexibility and agility. Sometimes there are good reasons for that, but it does mean that sometimes though you can’t be responsive.

Having booked into parallel sessions, I did the usual thing and go to the “wrong” session.

I had intended to go to the Discover Learning Design with LEaD session, but in the end, went to a session with two papers.

Understanding student digital experiences at City, University of London and  From face-to-face to remote learning: what can we learn from student experiences of pre-recorded lectures in the pandemic?

The first presentation was about the results of the Jisc Student Experience Insights survey at City.

Whilst teaching has moved back to campus, for many students there are still online or blended elements to their learning. It is therefore important to continue to evaluate student experiences of technology for learning. This paper will present the results from the 2021/22 iteration of the survey carried out at City during Spring 2022. The session will compare the results with the national benchmark from Jisc and previous iterations of the survey to see how the Covid-19 pandemic has impacted on student digital learning experiences.

It was interesting to see how a university was actually using the survey to inform decision making.

The second paper was focused on a cohort of students.

We explore students’ experience of asynchronous learning activities, with a specific focus on pre-recorded lectures and consider their role in promoting deep learning in an online education context. 

In this subject lecturers pre-recorded their lectures to help provide a more engaging learning experience. It was also designed to be more inclusive for those attending online.

In the presentation, we heard that student feedback was in the main positive, it was seen as more flexible, more accessible, students were able to interact with the recordings when they wanted to, and could rewind, review and drop into the recordings. It was also interesting to hear about the importance of captions (or transcripts) of the recordings.

After lunch there were two more sessions, I attended The Pedagogy of Hybrid Space and Transforming your presentation slides for online learning.

These were interesting sessions and demonstrated by their delivery the challenges of delivering hybrid (as in dual mode simultaneous) sessions. I will focus on these two sessions in a later blog post.

The final session of the day was a summary of the day and a celebration of staff achievements.

Overall I had a really good day and enjoyed all the sessions I attended.

apple, assessment, audio, presentation, twitter, video, vle, weeknotes

The VLE is not dead – Weeknote #167 – 13th May 2022

Leave a comment

Image by drippycat from Pixabay

Monday morning, I was off to Queen Mary University of London for their VLE Expo. This was very much a QMUL focussed event, though they had invited a range of VLE vendors. I liked how the focus of the event was about, what do we want to do to achieve our strategic aspirations, how will the VLE help us to do that, and which platform (or platforms) will enable us to do that.

There were some excellent presentations from the academic staff on the different ways in which they were using technology including virtual reality, mixed reality and H5P. I sat on the final panel session answering questions from the floor on a range of issues. A lot of the questions were more about the use of technology for learning and teaching, than VLE specific topics. However, I did get into a few discussions about the VLE on the Twitter as a result of attending the event.

I posted another blog post in my Lost in Translation series this time with a focus on the technical aspects of recording videos or audio files.

Most institutions will (probably) have equipment which staff can use, but if there is a strategic approach to building a sustainable approach to the use of video and audio, then universities will need to reflect if they have sufficient resources to support the increased demand for cameras and microphones.

video recording
Image by StockSnap from Pixabay

Tuesday I was still in London for a briefing session, well as it happened it got cancelled, so I worked in the office.

Apple have announced that they are going to stop selling the iPod once the current stocks of iPod touch run out. So did you have an iPod and if so which one?

iPod
Photo by Cartoons Plural on Unsplash

Wednesday, I did two all-staff briefings for two directorates on the Jisc HE sector strategy. From the feedback I got they seemed to be well received.

I was reminded on the Twitter about when I took my bike to work. I made a video back then.

Mike Sharples posted an excellent Twitter thread on how AI can be used to write essays. I agree with Mike, if we are setting students assignments that can be answered by AI, are we really helping students learn?

I enjoyed the #LTHEchat on images in presentations in the evening.

These two blog posts from 2005 (and 2007) were very influential on my presentation style: Gates, Jobs, & the Zen aesthetic and Learning from Bill Gates & Steve Jobs. I also posted  a link to a presentation from an internal TEDx event about delivering presentations – A duck goes quack.

Thursday, I made my way to Harwell for a drop in session I was running at the Jisc offices there, alas an accident the closure of the M4 meant I spent nearly four hours sitting the car rather than sitting in a room talking to Jisc staff. In the end I had to abandon my visit to the office.

Friday, I had a scoping call about learning spaces in higher education. Interested in the kinds of learning spaces higher education is using, flexibility, technology and the kinds of activities spaces are being used for.

I found this WonkHE article interesting – Learning design is the key to assuring the quality of modular provision in which Nick Mount talks about building quality assurance into the design of modular programmes and micro-credentials.

Traditional providers can expect to find themselves facing the difficult job of rethinking existing assurance processes that are designed for coherent, longitudinal programmes of study, so that they can accommodate a new pick-and-mix landscape of highly portable and stackable micro-credential learning.

My top tweet this week was this one.

assessment

Using AI to write an essay

Leave a comment

Over on the Twitter, Mike Sharples has written a thread about how students could potentially use AI to write assignments (and how academics could use AI to mark and provide feedback).

As Mike points out, existing tools such as Turnitin won’t spot these fakes.

Though I do think we should stop going down the rhetoric that all students want to cheat, I do agree with some of what Mike says in this tweet in that we do need to reflect and rethink assessment.

I also agree with Mike’s other tweet in this thread if we are setting students assignments that can be answered by AI, are we really helping students learn?  

Of course this only the beginning of how AI will impact on education.

assessment, jisc, twitter, weeknotes

Maybe today is the day you start wearing your mask again – Weeknote #144 – 3rd December 2021

Leave a comment

This was a full week back at work and I was in London for most of the week. Over the summer I had enjoyed working in the London office, a change of pace, location and routine compared to the forced working from home we had endured during the pandemic. Having had a fair amount of time off work, sick with covid, it was nice to be back in the office, talking and chatting to colleagues and similarly to the summer having the change of place and routine. The office was much busier than it had been in the summer. It felt quite normal in some respects, a little quieter than it was pre-pandemic.

However it was only a couple of weeks ago that I wrote about the possibilities of in-person teaching now that 90% of university students had had at least one Covid jab. Last week though we saw a new variant of concern of the coronavirus was identified by South African scientists and labelled by the WHO as Omicron.

On Monday I wrote about the impact Omicron could potentially have on the HE sector though my main messages was that universities should prepare for a possible lockdown.

Hopefully the vaccination rollout and mask wearing will reduce the chance of lockdown, but I would still be preparing for the possibilities of another lockdown regardless.

As we reach the end of the week, there have been some stories on the spread of Omicron, across the world, spreading to Europe, as might be expected with global travel and concerns this variant would have on infection rates (being more transmissible) and the subsequent impact on health resources. There were also some positive stories about the potential of vaccination to reduce the impact of Omicron.

Having said all that I would still be preparing for the possibilities of another lockdown regardless.

As you might expect, I ensured I was wearing my mask on public transport and when entering shops, eating places and as I walked around the office.

We had an HE leadership meeting on Monday and the majority of the meeting was discussing key challenges with our new CEO.

One of the things I reflected on was the success of Learning and Teaching Reimagined (LTR) and what we should do next. In order to build on and support the sector to deliver on the recommendation and work towards the challenges, Jisc working with members produced Higher education strategy 2021-2024: powering UK higher education which outlined how Jisc would support the sector going forward.

However LTR with its focus on teaching and learning leaves the door open to other ideas. There are a range of subjects that Jisc could focus on and undertake a similar range of activities and events as we did with LTR. This, like LTR, could be a sector-wide initiative focused on providing university leaders with inspiration on what the future might hold for higher education and guidance on how to respond and thrive in those environments. We could look at the student experience, leadership, the campus… there are a range of areas in which we could focus on in.

laptop and headphones
Image by Regina Störk from Pixabay

I published a blog post about the pandemic response and what we saw though described as online learning, wasn’t online learning.

One of things I have noticed is how often much of what was done during the numerous lockdowns was described as online learning. Let’s be clear you can describe what was happening as an emergency response to a crisis, even simplistically a pivot, but what was happening across schools, colleges and universities could in no way be described as online learning.

Some of my meetings were cancelled this week, which though freeing up time, can be frustrating.

This week was the Ascilite Conference. I really enjoyed attending and keynoting the conference back in 2009. Back then the UK was in the midst of an outbreak of swine flu. I didn’t go this year, but I may think about attending next year (pandemic permitting). This year it took place online and in-person at University of New England, Armidale NSW in Australia.

Martin Bean was part of a panel session and one comment (well tweet) I saw about the session mentioned the importance of authentic assessment, which made me think.

I think there is a blog post in this.

Was reminded this week that I am rubbish at Twitter.

While eating dinner on Wednesday evening, I participated in the #LTHEChat Twitterchat, Decolonising Learning Technology  led by Professor John Traxler.

I participated and did note that so much educational technology is designed for specific sector and its cultural norms, and then adjusted for other sectors and then other cultures. It was a really interesting debate and I enjoyed the discussion.

As it was December, I started tweeting out my advent calendar posts from a few years back. I really ought to spend some time doing new ones.

At the end of the week we had a HE Team meeting.

My top tweet this week was this one.

assessment, jisc, weeknotes

Once more to London – Weeknote #128 – 13th August 2021

Leave a comment

Well after a week off work, it was back to work. As with a couple of weeks ago I spent the best part of the week working in our London office. It was also a shorter week as I was on leave on the Friday. London was not very busy, but I was expecting that following my previous time in London. The office was even less busy, I was the only person in the office. It was obvious that many staff were still working from home. I don’t mind working from home. There were a few articles about the shift back to office working. Whitehall was looking to remove the London weighting for staff according to this article in the Guardian.

Whitehall officials have held high-level talks about taking away a salary boost awarded to London-based civil servants amid efforts to encourage workers back to the office.

Whilst on the BBC website an article asked: Should I be working from home or going back to the office?

People in England are no longer being asked to work from home. Instead the Prime Minister Boris Johnson is recommending a “gradual return to work”. However, in the rest of the UK, people are still being advised to keep working at home where possible.

I had quite a bit of flexibility on where I worked before the pandemic, so there is less pressure to return to the office. I have been working in the office though, I like the change in scenery and routine. After eighteen months being forced to work from home, I like the option of choice. Also during the summer holidays it makes more sense for me, when working, to be away from home.

First job was to clear that inbox full of e-mail, which to be honest didn’t take too long.

We had a team wide call on Monday, which was interesting. There will be changes in the team from September (a new manager) and we have a new CEO from mid-September.

We had an interesting meeting about the evaluation of Connect More, which people seemed to enjoy and got a lot out of.

Wednesday I had an interesting review and discussion about assessment, and what Jisc can do to support the sector to transform assessment. Despite the opportunities of digital in regard to assessment, many of the issues relating to the digital transformation of assessment, are much more about the transformation of assessment, with digital just being a catalyst. What is the purpose of assessment for example.

On Thursday I had a catchup with our (newish) HR contact. In my role I have no line management responsibilities (though plenty of matrix management responsibilities). We had a great discussion and chat about how we can make that matrix management more effective and efficient going forwards. So, we can move people from one area of Jisc to another to work on projects more easily. Something that a company like Apple do quite often.

I had a useful and interesting meeting with a university talking about our Powering HE document and the possible opportunities and challenges that universities will face over the next few years.

My top tweet this week was this one.

assessment, weeknotes

Unknown Unknowns and Unknown Knowns – Weeknote #117 – 28th May 2021

Leave a comment

Monday I was focusing on one of the projects we are working on with an university looking at various scope areas and how technology and digital can make a difference. I was reminded of the NSA quote of cylinders of excellence when it comes to silo working. The concept of excellent departments, but not an excellent university came to mind, but also about the inefficiencies of silos working in isolation and not thinking about the impact of their development and change on the rest of the university.

At the end of the day we were discussing assessment. What is happening with assessment in higher education now and what changes made as a result of Covid-19 are now in place, but also the wider issues of assessment as well.

old television
Image by Free-Photos from Pixabay

Cancelled my subscription to Britbox, in the main despite there being stuff I would like to watch, I wasn’t finding the time to watch it.

Tuesday saw me back to our office in Portwall Lane for an in-person meeting with my line manager, our first meeting in-person since August last year. It was actually nice to be both in the office and in an in-person meeting.

Something that keeps coming to my attention is the future of teaching, especially the concept of dual mode or hybrid teaching. What are peoples’ experiences of “dual-mode”, “muti-mode”, hybrid teaching? What has the student feedback being like? Something I have been reflecting on this week.

Students prioritise a return to face to face teaching from September 2021, so says the findings from a report published by the UPP Foundation.

Students want universities to prioritise a return to in person teaching and are missing face-to-face interaction around their wider student experience.

This is something which isn’t too surprising and is also something that has come out of our recent research into the student experience. Though digging deeper for us, it was more the in-person interaction students were missing and less the teaching.

Wednesday afternoon myself and Isabel Lucas of HEDG and the University of Cumbria hosted a share shop, facilitated by Advance HE, on how universities can support students transitioning in HE. We looked at both new students and returning students.

In the session, How can we best support learners coming in to HE from a Covid context? We discussed the transition arrangements others have put into place for the next academic year.

In September, third year students returning to HE will not have had a normal year in higher education and it is likely that their third year will not be like it was before.

We discussed a range of issues, focusing on the known knowns and the known unknowns. More difficult to discuss the unknown knowns and the unknown unknowns!

We are aiming to share the findings from the shareshop in June.

Thursday was a light day in terms of meetings, but got even lighter, as one meeting was cancelled five minutes before it was due to start, with the other meeting, two people who had accepted were in fact on leave, so in the end the meeting lasted only five minutes.

The future of the office keeps getting discussed, with those who own offices explaining why going back to the office is so important and those who don’t explaining why it isn’t. For me a lot is about the kind of work you do, I don’t do the same thing everyday, so there isn’t a single kind of space I need all the time. Before Covid, sometimes I would be working alone, sometimes I would be in meetings, sometimes we would be collaborating and sometimes I didn’t know, so it was useful to have other people around to bounce ideas off and chat over coffee.

My top tweet this week was this one.

assessment, jisc, twitter, weeknotes

Powering up – Weeknote #108 – 26th March 2021

1 Comment

I realised that I have been walking and exercising less during the last few weeks, now the children are back in school, so this week I made a determined effort to increase the amount of walking I do.

Like last week, I have spent a lot of the week interviewing staff and students as part of a project we’re doing at Jisc. We have been talking to them about their thoughts and perspectives on digital learning. As with a lot of these kinds of interviews there are some interesting individual insights, however the real insight comes from analysing all the interviews and seeing what trends are in there. I also spent time planning a similar, but different project.

I attended a roundtable on a digital vision for Scotland and facilitated a breakout room reflecting on the vision.

If you have watched a 60 minute TV programme, you will realise few if any have a talking head for 60 minutes. Few of us have the time or the skills to create a 60 minute documentary style programme to replace the lecture, and where would you go to film it? So if you change the monologue to a conversation then you can create something which is more engaging for the viewer (the student) and hopefully a better learning experience.

In a meeting this week with staff from a university I was discussing this issue and their response was, what about comedy stand-up? That’s a monologue. That got me thinking and reflecting, so I wrote a blog post about needing a tray.

Lego Star Wars
Image by 501stCommanderMax from Pixabay

Continue reading Powering up – Weeknote #108 – 26th March 2021 →

assessment, presentation

Challenges for quality assurance in the wake of the pandemic

1 Comment

I was recently part of a panel session looking at Challenges for quality assurance in the wake of the pandemic – ensuring quality in digital learning, academic integrity in remote assessment, and emerging best practice

I gave a five minute presentation from my personal perspective.

We knew what we were doing.

We knew how to assure quality and academic integrity.

We thought everything was going to be fine….

There were early signs of the impact of Covid-19 back in December 2019 and January 2020.

We thought everything was going to be fine….

A year ago, everything changed.

We went into a national lockdown and all universities needed to start doing things remotely, at pace and at scale.

We quickly translated our practices to online versions.

Our lectures became Zoom calls.

Our seminars became Teams conversations.

Office hours became 24/7 e-mail.

However we merely translated, we didn’t transform, we didn’t think differently.

As for exams and assessment….

Well that didn’t go as planned and was challenging to move online.

Moving assessment online is a journey and one for which we had no map and no real idea of where we wanted to get to.

We thought we could do it?

We thought technology was the solution.

It wasn’t.

The thing is translating practice online doesn’t really work. You lose the nuances of what made the in-person experience so great and you don’t take the advantage of the affordances of what digital can bring.

Can we be surprised when the quality of the experience suffers

Moving exams online doesn’t work.

Moving assessment online is not only possible, but can be done in a way which maintains quality and integrity.

How can you do this when the requirements in place from university exam board, PSRBs and professional accreditation are based on an model which expects a physical in-person assessment.

You need to transform and reimagine assessment.

You can not transform assessment on your own.

However this takes time.

It takes a certain level of digital capability.

It requires an insight into what we mean by assessment and what we are trying to assess.

At Jisc we are here to help the higher education sector through the use of digital to reimagine teaching, learning and assessment and to reframe the student experience.