Category Archives: assessment

No it’s not easy…

Time for a coffee

…but sometimes you need to think differently!

We know that change isn’t easy, if it was then all we would need to do would be buy a book on the subject and just do it.

When it comes to the embedding of digital technologies into teaching, learning and assessment I have spent over twenty years undertaking this kind of activity at a range of organisations and across different levels.

Going back to when I was a Business Studies and Economics teacher at what was then Brunel College (now City of Bristol College) I kind of fell into the use of technology to support teaching and learning. I was an ILT (or TEL) Champion before even the phrase existed. Going back a little further I was never the kind of techno geek or computer nerd many of my peers appear to be when comparing histories. I didn’t do Computer Science at school. I didn’t own a computer, I didn’t have a BBC Micro, nor the Commodore 64, ZX Spectrum or anything like that. I did have a bike though!

At University in the late 1980s we had a VAX system and it was really that kind of got me interested in technology, but it was as a tool to solve problems. I discovered I could use this thing called electronic mail to send letters to a friend at another university instead of using the post! This was quite illuminating, until I got flamed by the administrator at the other university, for not using the correct format for my e-mail… Most of the time however the use of the computers was in many ways pointless as my examinations required me to hand write essays, so why would I use a word processor, having said that I did get introduced to Word Perfect 4.2 and did think that this was better than a typewriter.

After university on a business enterprise course I was introduced to spreadsheets that I used for creating balance sheets and cashflow forecasts. For me that was probably the eye opener that got me into technology, more so than anything I had seen before, well does that make me a boring person?

By the time I was working at City of Bristol College I was using my own PC at home to create presentations, photocopying onto clear acetates as initially we didn’t have a digital projector, and we were still using OHPs. When the college did buy a projector (we had one for the whole college) it was a real effort to use it, it was the size of a small suitcase and we also had to lug the screen around as well. Due to lack of processing power, I would often bring in my own PC box, as the laptop couldn’t cope with the strain of my presentations. My PC also had a Matrox Rainbow Runner video card which I used to show full screen video. There was no internet and certainly no wireless network. My what we take for granted today, looks at his phone which can stream HD and 4K video to a projector using 4G connectivity, things do change. Things did improve and we started to see more technology in classrooms.

One outcome from all this was that as I was seen as something of an innovator in this area I was asked to support and train staff, not just from my faculty, but also other areas of the college. One clear memory of this was the impact, often I would train individuals who would then go off and do their own thing (or not). Sometimes I would train all the staff in a faculty and this is where I would often see not only the most resistance, but also the biggest impact. Where a faculty set expectations about how technology would be used, you would see the greatest impact. One faculty I taught how to use Powerpoint to (probably badly) many of the staff were quit resistant or complained they couldn’t do this technology thing, there weren’t enough PCs, not all classrooms had PCs and projectors, and so on… remember this was 1998 or 1999. The head of faculty though had made it clear that not only were all staff to do the training, and create presentation materials, but that all the presentations would be stored and shared centrally. No presentations stored on floppy disks (we didn’t have USB sticks back then) being used by individuals only.

What was a transformative moment for me was the understanding that showcasing, cascading and piloting really didn’t have the transformative impact that senior managers hoped for. Generally the main impact was that enthusiasts would become more enthusiastic and those more reluctant, would either not do anything, or just pay lip service to any initiative. What really caused institutional change was effective strategy and leadership and clarity about what was going to be done, what was expected from staff and what they needed to do and by when.

This did stick with me over the years I moved into positions where my role was to embed technology into teaching and learning. Though I often used the cascade model for staff development, but knew that this was not the ideal model for systemic holistic change across an organisation. It worked well on some individuals, but it was not transformative.

In a similar vein the use of other people’s research and running pilots was interesting and useful, but did not result in institutional change, it could inform other activities, but the idea that the best way for mainstream transformation was to run a pilot was something that I found never worked and never had the impact that others thought it would.

What I really tried to do was transform the entire institution. I would use tools such as cheeses and models, but one key aspect was culture change. Changing the culture was often about hearts and minds, but also challenging the myths and misconceptions about technology and using learning technology with learners. I would use pilots and research to inform this process.

I also knew that if something didn’t work, then to try again, but this time do it differently. Don’t keep trying to do the same thing again and again.

I know that this isn’t easy, if it was easy then we would all have done it!

One thing that came out of this was the understanding that we often make assumptions about staff capabilities and their ability to know how to embed technology and the potential of what technology can do. Just because a member of staff can has been given the training in how to use the tool or service, it doesn’t mean they know how best to use that tool or service to enhance teaching and learning, and for what function or process of the learning activity the tool would support or enhance.

I also know that isn’t easy too….

Assessing Assessment – ocTEL

This week on ocTEL we’re looking at assessment. As part of my thinking I reflected on the use of quizzes in Moodle.

Designing Moodle quizzes is much more than just been able to use the quiz tool from a technical perspective. There is a real art to crafting questions so that they not only allow the learner to test their understanding, but also require a higher level of thinking.

If we look at the following multiple choice questions, the format of which is one of many different types available on Moodle, it provides the structure and the practitioner provides the question and the answers:

Which is these is a mammal?

Shark
Dog
Spider
Crocodile

This question does not test understanding, most students would be able to guess the answer or would not find it challenging. Within Bloom’s Taxonomy this is testing knowledge only, the bottom layer of the triangle.

Bloom’s Taxonomy

In terms of feedback, you can design Moodle quizzes to provide feedback on questions. So you can explain why their answer is wrong or right and where to look for further information or support.

Onto a similar question:

What is the capital of Australia?

Sydney
Melbourne
Canberra
Melbourne

If we look at this question if you didn’t know the answer then you would need to do some research. However as with the previous question within Bloom’s Taxonomy this is testing knowledge only, the bottom layer of the triangle. It’s more challenging than the first question, but if you didn’t know it already then a quick Google search and you have the right answer.

So what about this question:

Which of these is the odd one out?

Odd One Out

The “problem” with this question is that there is no single right answer. The answer needs an explanation, and it’s the explanation that demonstrates understanding of the question, not the answer.

If we look at Bloom’s Taxonomy it is possible with this question to go all the way to the top.

However Moodle will struggle with assessing a question with no “right” answer and certainly would not be able to assess the explanation.

You could provide generic feedback on why there is no “right” answer, but that may not be useful for all learners. Feedback needs to be personalised to be really effective. Students generally don’t appreciate generic feedback.

This doesn’t mean that Moodle quizzes aren’t an useful tool for checking learning, but its limitations in assessing higher order thinking needs to be considered when designing assessment.

100 ways to use a VLE – #96 Offline Assessment

Submitting assignments online through the VLE is a great way for learners to ensure a) that the work submitted is logged, b) it’s unlikely to be lost, c) they can, once graded, refer to the grade and feedback when they want to.

The problem with submitting assignments through the VLE is when the assignment asks for the students to submit a physical thing as their assessed piece of work.

If a learner creates a sculpture for example, or makes a lasagna, welds copper tubing, these things can not be submitted electronically. Submitting a photograph is of course possible, but why? It’s an extra step that isn’t strictly necessary.

Many learners will often do a presentation for an assessment, others will do a live performance or demonstrate a technique. Now yes you could video these “performances”, but video files can be quite big and you not want to upload these to the VLE (especially if you use tape over an SD card).

However it is still useful to grade these offline activities on the VLE to inform the learners how they did and provide feedback. Of course generally that is done anyway, so why put it on the VLE?

One reason is consistency, if other (written) assignments are been uploaded to the VLE then a learner will probably want to at some point know the grades for all their assessed work regardless whether it was a written essay or a live presentation.

Another reason is that one of the key factors that aids retention is ensuring learners know in terms of their assessment what they have done, what they are doing and what they need to do. If they can find all this information in one place rather than using different systems for different assignments using differing forms of assessment then it will make life better for the learner. If feedback and grading on written work is on the VLE, it makes sense then to use the VLE for all assessment whether it be written or another activity.


Assessment

Having posted Steve’s presentation on assessment earlier, I was reminded that I had delivered some training on assessment and diverse forms of assessment a few months back.

This was the presentation I gave at the training.

The key message I wanted to get across to the participants was that just because they had assessed the way they had always done, this didn’t mean that was the only way they could assess learners. Often we assess the way that we do, we do it because we have always done it that way. There are now new tools and technologies that allow us to enhance and enrich assessment. and make it more engaging and effective for learners.

Sometimes we need to think differently, especially if the current methods of assessment are not doing what we need them to do.

e-Assessment

An interesting presentation from a keynote by that Steve Wheeler, Assessment in the Digital Age:Fair Measures?

Assessment, which in FE is heavily dictated to by the exam boards is always challenging to change to make “fit for purpose” and I do wonder if we assess because we need to assess rather than actually use it for something useful?

100 ways to use a VLE – #30 Field Trip Task

Across many curriculum areas, learners will go on field trips. Sometimes they will be to inhospitable environments such as forests, mountains, fjords or Croydon. Generally though they will be to places a little more local and urban.

Travel and Tourism students for example, may visit London to see the tourist attractions. Sports Studies could visit the site of the Olympics in 2012. Business Studies may visit a factory or a retail outlet. When I was teaching Business we visited such diverse places as an aircraft factory, a dockside undergoing urban regeneration, the Clothes Show Live, Amsterdam and Bruges, Museums and Croydon.

There were many reasons for going on field trips, a core one was to base an assignment or assessment task around the field trip. Looking at my own examples, we went to the Clothes Show Live to look at marketing and advertising. The visit to Europe was to look at the impact of cultural differences on business in the European Union. The aircraft factory was looking at how technology and changes in technology impacted on businesses processes, job roles and manufacturing.

So where does the VLE come into all this?

Well the VLE can be the place from which the task is centered and alongside the assignment brief you could place all the other information and links that learners could use for the field trip.

You do need to consider if you are expecting learners to access the VLE whilst actually on the field trip. In this case you will need to consider what your expectations are for learners in relation to the use of the VLE, and to ensure that the learner are aware of those expectations.

You will need to consider connectivity for those that will be accessing the VLE. You will need to be aware of the sort of devices that the learners will be using.

So if your learners are using the iPod touch to access the VLE, is there free wifi somewhere in the vicinity of the field trip? If they are using the iPhone or iPad, make sure you don’t have any links to Flash video or activities. If they are going to be using laptops or netbooks, where can they use them sitting down? Are there cafés or other places where they could go?

Once the logisitics of remote access to the VLE is sorted, you can then make decisions about how you are going to support the task from the VLE in the field.

Mobile connections are much slower than connections in college, so it’s vital that any resources are made to be downloaded as quickly as possible.

Simple things can be done to enable that, such as rather than using Word files (which can be quite large) ensure that you post text to the VLE (copy and paste it in) to a “label”, “discussion forum” or “web page”. That way learners can immediately access the text without needing to download a large file first. Often mobile devices are better at rendering text on webpages for smalller screens than trying to render A4 Word documents. Avoid using Powerpoint for similar reasons and as already stated unless you know that the devices are capable of using Flash, avoid Flash based content and video.

Video is possible over the mobile web, but unless you have a good connection, it is to be avoided. Audio however will download fine on even a poor 3G or GPRS connection and can be a viable alternative to video. Make sure the audio files are not too big, as some mobile phone providers have file size limits on what can be downloaded.

So what if remote access is not available, say you’re up a mountain or in Croydon?

Well the VLE could be the focus of the field trip after the event. A place to collate the thoughts, blog entries, images, video, audio from the field trip. A way of sharing resources and media easily and effectively. This will then enable any task that was started off on the field trip to be finished and finalised with all the resources that the group collected.

Field trips are a great way of making learning real for many students, the VLE can be used to support any field trip tasks and for post-trip activities too.

100 ways to use a VLE – #43 Group Task

Group assignments or tasks are a valuable part of any learning experience. They do so much more than just allow students to learn about a topic. They have to use a range of other skills too, including diplomacy, planning, monitoring, negotiation, tracking and many others.

The VLE is an ideal place to act as a portal for a group task. With a little effort it would be possible, using a VLE such as Moodle, to create a “course” for each group, duplicating the bare bones of an assignment course. This course would contain the assignment brief, notes, links, copies of handouts. It could contain tools such as discussion forums, wikis, a calendar and planning tools.

Before the advent of social networking and ubiquitous SMS students would often find it difficult to communicate and collaborate on an assignment. Today with the wealth of tools available, why would the learner use the VLE?

Well two reasons. Firstly, using the VLE does not preclude or stop the use of other online tools and services. Learners also need to recognise the time to use external tools and when to use internal tools. We do the same with internal spaces and external spaces. Usually for example seminars are held on campus, whilst informal chats in the pub are held, well in the pub… It would be expected that as well as using the VLE for the group task, learners would meet face to face and use informal communication tools such as SMS and Facebook. The VLE would be used for discussions about the assignment, sharing, collaborating and working together.

Secondly, as well as supporting learners to complete the group task, the use of the VLE could act as evidence towards the assignment itself. This is useful where the process is important part of the assignment as well as the final outcome.

As well as providing a focus for individual tasks, the VLE can also be used as a portal for group tasks or assignments.

Effective Assessment in a Digital Age Workshops

Effective Assessment in a Digital Age Workshops

From challenge to change…

Using principles of good practice, work with colleagues towards an effective model for the use of technology in assessment and feedback.

A series of free workshops based around the JISC Effective Assessment in a Digital Age publication and associated online resources will take place during January – March 2011. Workshops will be held in London (20 January 2011), Birmingham (16th February 2011), Bristol (March 2011, date to be confirmed) and Newcastle (24 March 2011). The JISC e-Learning Programme will be working in partnership with the JISC Regional Support Centres on these events.

These workshops will be exploring how the use of technology in HE and HE in FE, linked to principles of good practice in assessment and feedback, can help promote more effective learning. These workshops, which draw on the work of recent JISC-funded projects as well as related significant developments in the area of assessment, will have a practical, hands-on flavour with a focus on how to move from current challenges towards sustainable change.

The workshops will be suitable for:

  • Lecturers, tutors and course leaders who design assessment and feedback for their learners on HE-level courses
  • Intermediaries with a role in supporting practitioners with assessment, and technology-enhanced assessment (learning technologists, e-learning/ILT champions, staff developers, educational developers, academic registry)

Further information together with the registration form for the London workshop is now available from www.jisc.ac.uk/assessworkshops

100 ways to use a VLE – #9 Peer Assessment

We are often fearful of getting our peers to judge us, is that because we fear they are incompetent at assessing us, or that they are so much like us that they will do it honestly and openly and notice all our hidden flaws?

Probably for the latter reason peer assessment by learners of learner is such a valuable method of assessment. Learners giving each other formative or summative feedback to each other can be a very positive activity, even if it is (to the learners) somewhat daunting.

There are many advantages to peer assessment. It can enhance learning through learners having to not convince you that they understand a topic, but also ensure that their peers understand the topic too. It allows for a greater depth of self-reflection by learners on the learning process. Peer assessment is daunting, but as a result can increase the confidence of learners as they need to work with each other. This working together builds social interaction across a group of learners and a support mechanism.

There are some issues and fears in regard to peer assessment. It can be an administrative nightmare. It is dependent on learners all taking part and not all will want to, some may not want to give the same level of effort as others. Conflict and arguments can arise, unlikely to happen with other forms of assessment, and will the learners have the skills and the confidence to resolve these conflicts. Some staff think that it will be challenging to do and that “exam boards” may object. Even some learners may object as they see the assessment process as an end and not as a learning process in itself.

One of the key features of peer assessment is communication and though e-mail and SMS is one way, the VLE allows for the communication to be more easily aggregated, sorted and assessed.

Any piece of peer assessed work, like any other piece of assessed work, needs a detailed briefing, time to discuss, both face to face and online, and guidelines and rules need to be in place. Once more the VLE is an idea vehicle to provide the online space for the briefing, online discussions, guidelines and rules. Providing a dedicated space on the VLE for the assessment will make it easier for learners to compartmentalise the assessment from other parts of the course.

Using the VLE allows learners to undertake parts of the peer assessment at a time and place to suit them; in college, at home, in the workplace or drinking coffee!

Peer assessment offers a different process of assessment that engages learners, builds relationships and improves learning. By undertaking that peer assessment through the VLE ensures that it is easier for learners to work together, discuss, share and refer to in the future.