Category Archives: assessment

No it’s not easy…

Time for a coffee

…but sometimes you need to think differently!

We know that change isn’t easy, if it was then all we would need to do would be buy a book on the subject and just do it.

When it comes to the embedding of digital technologies into teaching, learning and assessment I have spent over twenty years undertaking this kind of activity at a range of organisations and across different levels.

Going back to when I was a Business Studies and Economics teacher at what was then Brunel College (now City of Bristol College) I kind of fell into the use of technology to support teaching and learning. I was an ILT (or TEL) Champion before even the phrase existed. Going back a little further I was never the kind of techno geek or computer nerd many of my peers appear to be when comparing histories. I didn’t do Computer Science at school. I didn’t own a computer, I didn’t have a BBC Micro, nor the Commodore 64, ZX Spectrum or anything like that. I did have a bike though!

At University in the late 1980s we had a VAX system and it was really that kind of got me interested in technology, but it was as a tool to solve problems. I discovered I could use this thing called electronic mail to send letters to a friend at another university instead of using the post! This was quite illuminating, until I got flamed by the administrator at the other university, for not using the correct format for my e-mail… Most of the time however the use of the computers was in many ways pointless as my examinations required me to hand write essays, so why would I use a word processor, having said that I did get introduced to Word Perfect 4.2 and did think that this was better than a typewriter.

After university on a business enterprise course I was introduced to spreadsheets that I used for creating balance sheets and cashflow forecasts. For me that was probably the eye opener that got me into technology, more so than anything I had seen before, well does that make me a boring person?

By the time I was working at City of Bristol College I was using my own PC at home to create presentations, photocopying onto clear acetates as initially we didn’t have a digital projector, and we were still using OHPs. When the college did buy a projector (we had one for the whole college) it was a real effort to use it, it was the size of a small suitcase and we also had to lug the screen around as well. Due to lack of processing power, I would often bring in my own PC box, as the laptop couldn’t cope with the strain of my presentations. My PC also had a Matrox Rainbow Runner video card which I used to show full screen video. There was no internet and certainly no wireless network. My what we take for granted today, looks at his phone which can stream HD and 4K video to a projector using 4G connectivity, things do change. Things did improve and we started to see more technology in classrooms.

One outcome from all this was that as I was seen as something of an innovator in this area I was asked to support and train staff, not just from my faculty, but also other areas of the college. One clear memory of this was the impact, often I would train individuals who would then go off and do their own thing (or not). Sometimes I would train all the staff in a faculty and this is where I would often see not only the most resistance, but also the biggest impact. Where a faculty set expectations about how technology would be used, you would see the greatest impact. One faculty I taught how to use Powerpoint to (probably badly) many of the staff were quit resistant or complained they couldn’t do this technology thing, there weren’t enough PCs, not all classrooms had PCs and projectors, and so on… remember this was 1998 or 1999. The head of faculty though had made it clear that not only were all staff to do the training, and create presentation materials, but that all the presentations would be stored and shared centrally. No presentations stored on floppy disks (we didn’t have USB sticks back then) being used by individuals only.

What was a transformative moment for me was the understanding that showcasing, cascading and piloting really didn’t have the transformative impact that senior managers hoped for. Generally the main impact was that enthusiasts would become more enthusiastic and those more reluctant, would either not do anything, or just pay lip service to any initiative. What really caused institutional change was effective strategy and leadership and clarity about what was going to be done, what was expected from staff and what they needed to do and by when.

This did stick with me over the years I moved into positions where my role was to embed technology into teaching and learning. Though I often used the cascade model for staff development, but knew that this was not the ideal model for systemic holistic change across an organisation. It worked well on some individuals, but it was not transformative.

In a similar vein the use of other people’s research and running pilots was interesting and useful, but did not result in institutional change, it could inform other activities, but the idea that the best way for mainstream transformation was to run a pilot was something that I found never worked and never had the impact that others thought it would.

What I really tried to do was transform the entire institution. I would use tools such as cheeses and models, but one key aspect was culture change. Changing the culture was often about hearts and minds, but also challenging the myths and misconceptions about technology and using learning technology with learners. I would use pilots and research to inform this process.

I also knew that if something didn’t work, then to try again, but this time do it differently. Don’t keep trying to do the same thing again and again.

I know that this isn’t easy, if it was easy then we would all have done it!

One thing that came out of this was the understanding that we often make assumptions about staff capabilities and their ability to know how to embed technology and the potential of what technology can do. Just because a member of staff can has been given the training in how to use the tool or service, it doesn’t mean they know how best to use that tool or service to enhance teaching and learning, and for what function or process of the learning activity the tool would support or enhance.

I also know that isn’t easy too….

Is there a right answer?

Is there a right answer?

Complete the exercise the “right way”…

Assessing Assessment – ocTEL

This week on ocTEL we’re looking at assessment. As part of my thinking I reflected on the use of quizzes in Moodle.

Designing Moodle quizzes is much more than just been able to use the quiz tool from a technical perspective. There is a real art to crafting questions so that they not only allow the learner to test their understanding, but also require a higher level of thinking.

If we look at the following multiple choice questions, the format of which is one of many different types available on Moodle, it provides the structure and the practitioner provides the question and the answers:

Which is these is a mammal?

Shark
Dog
Spider
Crocodile

This question does not test understanding, most students would be able to guess the answer or would not find it challenging. Within Bloom’s Taxonomy this is testing knowledge only, the bottom layer of the triangle.

Bloom’s Taxonomy

In terms of feedback, you can design Moodle quizzes to provide feedback on questions. So you can explain why their answer is wrong or right and where to look for further information or support.

Onto a similar question:

What is the capital of Australia?

Sydney
Melbourne
Canberra
Melbourne

If we look at this question if you didn’t know the answer then you would need to do some research. However as with the previous question within Bloom’s Taxonomy this is testing knowledge only, the bottom layer of the triangle. It’s more challenging than the first question, but if you didn’t know it already then a quick Google search and you have the right answer.

So what about this question:

Which of these is the odd one out?

Odd One Out

The “problem” with this question is that there is no single right answer. The answer needs an explanation, and it’s the explanation that demonstrates understanding of the question, not the answer.

If we look at Bloom’s Taxonomy it is possible with this question to go all the way to the top.

However Moodle will struggle with assessing a question with no “right” answer and certainly would not be able to assess the explanation.

You could provide generic feedback on why there is no “right” answer, but that may not be useful for all learners. Feedback needs to be personalised to be really effective. Students generally don’t appreciate generic feedback.

This doesn’t mean that Moodle quizzes aren’t an useful tool for checking learning, but its limitations in assessing higher order thinking needs to be considered when designing assessment.

100 ways to use a VLE – #96 Offline Assessment

Submitting assignments online through the VLE is a great way for learners to ensure a) that the work submitted is logged, b) it’s unlikely to be lost, c) they can, once graded, refer to the grade and feedback when they want to.

The problem with submitting assignments through the VLE is when the assignment asks for the students to submit a physical thing as their assessed piece of work.

If a learner creates a sculpture for example, or makes a lasagna, welds copper tubing, these things can not be submitted electronically. Submitting a photograph is of course possible, but why? It’s an extra step that isn’t strictly necessary.

Many learners will often do a presentation for an assessment, others will do a live performance or demonstrate a technique. Now yes you could video these “performances”, but video files can be quite big and you not want to upload these to the VLE (especially if you use tape over an SD card).

However it is still useful to grade these offline activities on the VLE to inform the learners how they did and provide feedback. Of course generally that is done anyway, so why put it on the VLE?

One reason is consistency, if other (written) assignments are been uploaded to the VLE then a learner will probably want to at some point know the grades for all their assessed work regardless whether it was a written essay or a live presentation.

Another reason is that one of the key factors that aids retention is ensuring learners know in terms of their assessment what they have done, what they are doing and what they need to do. If they can find all this information in one place rather than using different systems for different assignments using differing forms of assessment then it will make life better for the learner. If feedback and grading on written work is on the VLE, it makes sense then to use the VLE for all assessment whether it be written or another activity.


Assessment

Having posted Steve’s presentation on assessment earlier, I was reminded that I had delivered some training on assessment and diverse forms of assessment a few months back.

This was the presentation I gave at the training.

The key message I wanted to get across to the participants was that just because they had assessed the way they had always done, this didn’t mean that was the only way they could assess learners. Often we assess the way that we do, we do it because we have always done it that way. There are now new tools and technologies that allow us to enhance and enrich assessment. and make it more engaging and effective for learners.

Sometimes we need to think differently, especially if the current methods of assessment are not doing what we need them to do.