RSS .92| RSS 2.0| ATOM 0.3
  • Home
  • About James Clay
  • New Stuff
  • Old Stuff
  • Podcast
  • 100 Ways
  • App of the Week
  •  

    Assessing Assessment – ocTEL

    May 22nd, 2013

    This week on ocTEL we’re looking at assessment. As part of my thinking I reflected on the use of quizzes in Moodle.

    Designing Moodle quizzes is much more than just been able to use the quiz tool from a technical perspective. There is a real art to crafting questions so that they not only allow the learner to test their understanding, but also require a higher level of thinking.

    If we look at the following multiple choice questions, the format of which is one of many different types available on Moodle, it provides the structure and the practitioner provides the question and the answers:

    Which is these is a mammal?

    Shark
    Dog
    Spider
    Crocodile

    This question does not test understanding, most students would be able to guess the answer or would not find it challenging. Within Bloom’s Taxonomy this is testing knowledge only, the bottom layer of the triangle.

    Bloom’s Taxonomy

    In terms of feedback, you can design Moodle quizzes to provide feedback on questions. So you can explain why their answer is wrong or right and where to look for further information or support.

    Onto a similar question:

    What is the capital of Australia?

    Sydney
    Melbourne
    Canberra
    Melbourne

    If we look at this question if you didn’t know the answer then you would need to do some research. However as with the previous question within Bloom’s Taxonomy this is testing knowledge only, the bottom layer of the triangle. It’s more challenging than the first question, but if you didn’t know it already then a quick Google search and you have the right answer.

    So what about this question:

    Which of these is the odd one out?

    Odd One Out

    The “problem” with this question is that there is no single right answer. The answer needs an explanation, and it’s the explanation that demonstrates understanding of the question, not the answer.

    If we look at Bloom’s Taxonomy it is possible with this question to go all the way to the top.

    However Moodle will struggle with assessing a question with no “right” answer and certainly would not be able to assess the explanation.

    You could provide generic feedback on why there is no “right” answer, but that may not be useful for all learners. Feedback needs to be personalised to be really effective. Students generally don’t appreciate generic feedback.

    This doesn’t mean that Moodle quizzes aren’t an useful tool for checking learning, but its limitations in assessing higher order thinking needs to be considered when designing assessment.


    This Page has been Blocked – ocTEL

    May 2nd, 2013

    Blocked

    After posting my reflection on the Adventure Game I was reminded of Fantastic Contraption. This is a puzzle game that require logical thinking, but because of the design of the game there is no one answer, you can be very creative. It’s also very social as you can share your “contraptions” with others. I have seen some very innovative and interesting “contraptions” from others within the game. You can also return to your “contraptions” to improve them and make them better. This is a much better puzzle game and more engaging for me than I found with Lost in the City. I thought I should try and have a go at the puzzle game recommended on the ocTEL site. Alas the site was blocked, I was using the free public wifi at the local library. So despite wanting to try out the puzzle game recommended I couldn’t.

    So how about I look at Fantastic Contraption instead? Well this is a fun engaging puzzle game. It requires logical thinking and elements of trial and error. Unlike something like Lost in the City, there isn’t a single solution you need to find, there is an infinite number of solutions to each of the puzzles. You can also go back and fine tune your solutions.

    Fantastic Contraption

    This is quite engaging and there is an element of satisfaction when you find a neat or clever solution. 

There are levels that can appear to be unsolvable and these initially proving challenging can after a while be somewhat demotivating. If there is a level that you can’t solve, you probably will stop playing the game. However as the “recipe” doesn’t really change much across the different levels, you can have enough of the game and when the next game comes along you will probably stop playing, I know I haven’t played for ages.

    As for what can I learn from the game, well that’s a more challenging question. I am not entirely sure if there is too much I can learn from the game, there’s elements of physics and logic, but I am not sure how useful the skills learned will be in real life and how transferable they will be. When it comes to problem solving, I would have thought logical and fine tuning a solution would be useful, but I don’t see how this game would result in gaining those skills. If anything the game doesn’t “teach” those skills, and it would be difficult to learn those skills from the game. What is a more likely scenario is that this kind of game would appeal to someone who already has those skills.

    I think another interesting point from all of this is the fact that I couldn’t access the puzzle game from the ocTEL webiste. Site blocking is an issue with any kind of MOOC or TEL activity when the learner has to access a third party site as part of the learning and that an “administrator” has deemed to be “unsuitable”. The key question is how easy is it to have the site unblocked. In my case probably unlikely as I am using the public wifi at the local library. What is also quite interesting, but could be worrying for learners, is the comment “The request was logged”. You could imagine a learner thinking that if they went to too many blocked sites they would lose their internet access. It’s an interesting aspect of this MOOC that I suspect hasn’t really been thought about.

    Site blocking is still prevalent on public wifi, train wifi, school and college wireless too. For example CrossCountry Trains blocks YouTube and iPlayer as there is insufficient bandwidth on the train. A lot of sites are also blocked on 3G connections due to the default content control on these services. I remember a VLE I worked with in the past was blocked by Vodafone Content Control.

    When designing TEL activities (and MOOCs) as mentioned in a previous blog, consideration must be given to alternatives for those occassions when sites are blocked. Also it makes sense to check out how these activities and tools work away from university and college systems, on free wifi or 3G networks.


    Active Play – ocTEL

    May 2nd, 2013

    Lost in the City

    Have found a little time this week to look at ocTEL.

    Over recent decades, game-based learning has grown as a form of TEL. It encapsulates many principles of active learning, such as engagement in an authentic context, learning by mistake-making and reflection, experiential learning, collaborative learning and learning by problem-solving. As such, it is worth considering the techniques that games use to engage learners and what can be learned from them. Four game genres with obvious learning potential are adventure games, puzzle games, role playing games and strategy games.

    Playing “Lost in the City” reminded me of how much I hate these kinds of adventure games and how frustrating I find them. Now I did give it a fair go and spent over 15 minutes playing the game, of which the majority of time was spent trying to find a single small object on the screen. This was very un-engaging and very demotivating.

    Personally I don’t think I could learn anything from this specific game. I also think the frustrations acted as a disincentive to learning.

    This is not an “adventure” game, this is a game with puzzles that appears to be an adventure, but isn’t. These are not even “useful” puzzles as far as I am concerned, they are logic puzzles that have to be solved in a particular way or in a specific order, the “challenge” is to find that order out to solve it. The problem with this is that there is no real logic to the process and it’s much more trial and error then actual logic. Also with only one specific solution, you can’t be creative or find alternative ways of solving the problem. This does not reflect the real world.

    The other aspect is that too often you spend a long time trying to find (as in the case of “Lost in the City”) that final key, that elusive number. This is not learning, unless you’re learning to be patient! It’s merely frustrating and annoying, as far as I am concerned it’s not even a game.

    For me a true adventure game should allow for freedom and flexibility. Then it can be more of a learning experience.

    I can see how some people may “enjoy” this kind of game, but also can see how some people think such games “could” be used to teach subjects like Health & Safety, as in spot the danger. However such games would annoy many learners for the reasons I outlined above, and therefore could be a choice, but allow other ways of learning.


    Understanding the Potential

    April 22nd, 2013

    Boxes

    So what comes first, technology or pedagogy?

    For me one of the messages that comes out of the videos I have was watching last week and resonates with me, is how the most effective way of using technology is where it is used to solve a problem.

    The problem could be pedagogical, as with the teaching machine, that allows independent self-directed learning, or it could be social, as with Sugata Mitra’s hole in the wall experiment.

    What I have noticed at conferences and online in social media is that this often results in the conclusion that we should put pedagogy first and before technology.

    I do agree with that sentiment that we should start with the problem we are trying to solve and if it is a pedagogical problem then we should start with the pedagogy. We should the consider a range of technologies that could be used to solve that problem.

    There is an assumption there that a practitioner is fully conversant in the ranges of technologies available and understands their potential for solving issues.

    From working with practitioners is that they are not always aware of the different types of technology available and what their functionality and capabilities are. Without knowing what things can do and how they enhance and enrich learning, how can you make an educated choice about which is the right technology to use in which context? This is why I do think that sometimes we do need to talk about technology and how it can transform learning. This is not about solving problems with TEL, but providing TEL as a box of potential solutions to solve future problems.

    Of course you are not going to be able to know everything, or understand how everything works to make use of it, but understanding the potential means that if you encounter a problem in the future then you will have a potential solution that can be worked on.

    A secondary aspect that also needs to be considered is the transformative nature of technology. If we start talking about technology we might find there are new ways of learning, new pedagogies that could be exploited to solve problems. If we didn’t think about the potential of different technologies.

    Technology can offer new ways of learning, that if we started from a learning perspective may be missed.

    One example that comes to mind is GPS, location based learning can be transformative as it allows learning to happen in places that otherwise might be missed. If you think of those services on phones that remind you to buy something when you pass a particular shop, then put this into a learning context; imagine your phone alerting you to a building or location you are passing and how it relates to the current topic you are studying. You may not even realise that the location is connected, but GPS and location aware learning objects will help you to do just that. The problem here is about context-sensitive learning, but this just couldn’t happen in a classroom or lecture based scenario. Technology allows us to transform the potential for learning and create new ways for learners to access learning.

    I do think we need to stop thinking about what we should focus on “first”, pedagogy or technology, but actually consider the context and audience we are working with.

    If we are a teacher then more likely when working with learners, the TEL will be there to support the solving of a learning problem. The learning will come first in this instance, and the TEL will be supportive. However what if the teacher wants the learners to consider a range of technologies to support an assessment objective, in this instance then it makes much more sense to talk about the technology.

    At a conference, training session or on social media, it can make sense that

    So what comes first, technology or pedagogy? Well both do, it depends on the context.

    Image source.


    So what if I am not connected?

    April 15th, 2013

    First Great Western

    One of the challenges with any MOOC is finding the time to engage with the course, especially if you are doing it in your own time and have no dedicated staff development time slot to fit it in. I am sure that I am not that different to many others undertaking ocTEL in undertaking the course in “spare” time as I like others are generally quite busy.

    However what I have found with this MOOC, and I guess is an issue with many different MOOCs is that you need to be online! Well one of the “O”s in MOOC does stand for online.

    Well in order to be online you need an internet connection, and I am finding that when I have time to engage with the course I am either offline or have a slow 3G phone connection.

    Most of my reading and reflection is being done offline. I am writing this blog post in a word processor and will post it on my blog later when I have some internet.

    As I said though, most of the time I am engaging with this course is when I am commuting by train when I have no internet (well not a stable consistent connection).

    As a result I was unable to view the Diana Laurillard webinar recording, well that when I had a decent internet connection, I couldn’t as the machine I was on was having a problem with the Java required for Blackboard Collaborate (which I have used before on the same machine).

    Of course it is proving a little easier to engage with ocTEL through Social Media, Twitter seems to work just as well on GPRS as it does on 3G and even Google+ isn’t too bandwith hungry.

    Now we are in week one, there are a fair few YouTube videos to watch and in order to watch YouTube videos you need a relatively fast and stable internet connection. When travelling by train, this is something I don’t always have. CrossCountry for example block YouTube on their wifi on their trains, so no chance of watching it when connected on one of their rail services. First Great Western don’t even have wifi! It would have also been useful for me, as someone who listens to podcasts in the car to possible have downloadable mp3 audio versions of the “videos” to listen to. The option to download the video to an iPad or media device, or audio versions would have helped me enormously.

    It would also appear to be crucial to view these videos first before I can progress on the course. As a result I am going to need to set time aside on a computer with a decent internet connection to view the videos.

    What I have learnt from this part of the MOOC is that never assume your learners always have a decent always on internet connection. They may at times, but there may be circumstances which means that they don’t. As a result think about ensuring content is accessible in different media, or provide opportunities to convert formats into accessible formats.

    Okay time to find the time to watch those videos.