Assessment for learning

[Activity 1]

Apart from my work with the OU, I don't use summative assessment at all - it's all formative.

I teach writing skills (business and technical writing) on one or two day courses to groups of people often - but not always - from the same organisation. Because of the time constraints, the assessment process tends to be informal, happens throughout the session and influences what we're going to cover and when. These sessions are very low-tech (pen and paper, mostly) and are run face-to-face. We have exercises - which we peer-mark (discuss, really - there isn't a pass or a fail), quizzes and games. But all are used as learning tools in their own right - so the answers to a quiz will indicate why one answer is write, why another is not (or less likely to be so) and under what circumstances this isn't true. It's assessment at one level - but more diagnostic (so I can pitch at the right level - just ahead of where they are now) and formative (an integral part of the teaching).

The course I teach for the OU - TU100 - has a policy of providing a considerable amount of teaching through the feedback (and feedforward) around the assignments. It's good - but it means marking takes a very long time... I sometimes think I'm writing nearly as much as the students, and it's very frustrating when you get to the end of the year and someone says, "What feedback?" (having downloaded assignments and simply looked at the grade all year).

The course materials for TU100 incorporate a lot of formative assessment - activities followed by suggested (or actual) answers, quizzes and so on. Although they are part of a course I teach, I haven't had anything to do with implementing them.

Comparing MOOCs

[Activity 14]

After a bit of thought, I decided I might as well sign up for two courses that hold some interest for me... that way, how I feel about the subject matter is not going to adversely affect my impression of the pedagogy. I have signed up for:

• Intro to Psychology with Udacity
• DS106

Intro to Psychology

Most of the materials are delivered via online interactive videos... there is a nice clear audio track, accompanied by either a video of the teacher or slides of notes with a pen in someone's hand. Every so often, the materials paused and I was asked to answer a short (usually multi-choice) question. Although there was some interaction, it was fairly predictable... but pleasant enough.

The technology was fairly mainstream - and there was information that the videos are hosted by Amazon, so even if YouTube is blocked they should still be available.

As I was watching the 'lesson', links to related discussions are shown on the right of the window - and I could join these, or start a new discussion.

Definitely an xMOOC - traditional content (possibly from a campus-based course) moved online.

What did I think?

Well, I could probably have done the same from a book. I didn't need the online concept at all. However, I found it interesting, and also felt I could do a small bit when I felt like it and come back to exactly where I'd left off without feeling as if I'd missed something or needed to spend time hunting for something.

DS106

It actually took me a while to work out what I would be studying on this course. It wasn't immediately apparent from the page I landed on.

It sounds fascinating - and something I may well follow through, purely out of interest.

My problem with it is that I would need to be engaged and motivated from the start, because it's not that easy to find your way around. For this particular subject, and for me as an individual, it's not that important. If I miss something, I'll probably pick it up later. But that approach doesn't work for everything. I remember when I was nursing having the strong feeling that I must have missed a lecture somewhere, because something simply wasn't making sense - a bit like when you intend to skip through the adverts on a recorded TV program but actually jump 20 minutes. At first, all is fine - you're in familiar territory, with familiar characters... but you realise you've missed a chunk of the plot. If the same approach was used for a subject that I needed to fully understand and be sure I'd covered all aspects, I would be quite worried.

It's definitely more of a cMOOC than an xMOOC. The campus-based materials may have formed the basis for the course, but as even the assignments are devised by the community, the course has moved away from those beginnings.

I'm pleased that I don't seem to have to sign up for a whole host of new social media platforms to be able to get on with the course - and the list of software applications are suggestions (if you already have something, you can use it).

What did I think?

For me, it would work as a "hobby" course... something to pick up and do something with when I had a few spare minutes. Without any real concept of a "course" with start and end times, it's likely to fall down my list of priorities.

Having said that, with the limited time I have available, I can see myself dipping in to do some work on a particularly relevant topic, releasing me from the confines of a "course".

More about MOOCs... (readings)

[Activity 13] Although the activity only asked us to read one of the papers, I found I was struggling to understand the comments other people had made without reading all three - so here is my summary of them.

Making sense of MOOCs

Daniel (2012), Making sense of MOOCs: musings in a maze of myth, paradox and possibility, which provides a comprehensive review of MOOCs.

I found this a very interesting read, although it felt a little disparaging of some of the more recent MOOC offerings, especially those provided by the established, larger, universities.

It helped me unravel the differences between cMOOCs and xMOOCs (although I'm still not sure what the 'x' stands for:

• cMOOCs are the original MOOCs, which make full use of networks and connectivism. They are open in the sense of being open access, but also open-ended and (to some extent) open regarding focus and content. They have a stated topic, but those taking a cMOOC are much freer to explore some topics more than others and to create their own networks within the larger community of students of the course.
• xMOOCs are much closer to 'traditional' CBT and traditional face-to-face teaching models, having transferred existing content - supported by videos of the teacher speaking and a few interactive quizzes - to an online environment. The pedagogy is quite old.

The challenges to connectivist learning on open online courses

Kop (2011), The challenges to connectivist learning on open online networks: learning experiences during a massive open online course.

The challenges identified were many and varied - but I'm not sure they aren't similar to the challenges faced in a campus-based course or one delivered online (but not as a MOOC). In my experience, lack of confidence (both in the tools and personally), language skills and the sheer volume of information can be overwhelming in all cases. Just because a course is s MOOC doesn't mean there can't be some guidance through it.

The big challenge for many of us is that to be as fully engaged and immersed in a course as is suggested would maybe - just maybe - work for those studying full time, but is simply not possible for those studying while holding down a job, running a home and caring for a family.

The pedagogy of MOOCs

Stacey (2013), The pedagogy of MOOCs.

A relatively easy read, with several videos to break up the narrative, this reading helped set the overall scene - and I wish I'd read it first.

From the perspective of undertaking a MOOC myself - particularly a cMOOC - I found the video from Dave Cormier (Success in a MOOC), which mapped out the five steps to success - to be particularly useful.

These five steps are:

1. Orient - find out where everything is and when anything important is happening
2. Declare - somewhere to put your notes, reactions, reflections and so on. Could be a blog... or something similar.
3. Network - follow people, read what they've done, make comments.
4. Cluster - find people who are looking at things from the same perspective, a subset of the overall network.
5. Focus - remember why you're doing the course and what you want to get out of it. You need to provide your own motivation to continue.

I also found the four activities outlined for PLENK2010 to be very useful in further understanding how this type of open, flexible course works. These activitiies are:

1. Aggregate - picking and choosing what to read/watch/listen to from a selection of provided materials which cover the full range from an introduction to a topic to interviews with experts
2. Remix - Make some notes or use some other method of combining the information you encountered in step 1 (and from other activities).
3. Repurpose - take the remix and using the tools, create some new content.
4. Feed forward - share your work with other people, make it available.

My thoughts...

I like the idea of learning as participation, of learning by connecting to others, sharing experiences and so on. It sounds wonderful...

But there are downsides.

Different people bring different levels of prior experience, and that can be quite daunting to some. I'm not known for my shyness, so I coped... but I did a campus-based PGCCE a few years ago. It was a part-time course, over 2 years, and the prospectus said it was suitable for anyone working in FE, in an organisational training department or as a freelance trainer. I fitted into the last category, so thought it would be fine.

In practice, everyone bar another women and me worked in FE colleges, and had done so for several years. A lot of the "lessons" were delivered by throwing a topic out for discussion, after varying (sometimes minimal) background information... and very gradually over the course of the year I started to pick up on the terminology being used. But I got fed up of having to say, "I have no idea what you mean...", especially when someone would begin something with, "Obviously, ...". No doubt it was obvious to the speaker - and judging from the reaction, was also obvious to everyone else who worked in the sector - but I didn't have a clue. It didn't exactly encourage me to join in the discussions - I became much more of a 'lurker', even though physically in the room. It was almost necessary, as I felt that with the limited time, to take me over what to them were fundamental issues would have used too much of the valuable time. How much harder would that have been online?

I also sometimes feel - either because of pressures of time, or lack of background knowledge - that I want to say to someone, "Yes, I know I could go and have a discussion about it... but really I just want you to tell me. Is it possible or isn't it?"

Can we use MOOCs?

[Activity 12]

I'm going to look at this from three different perspectives, based on the different 'hats' I wear:

• The Open University - although 'only' an associate lecturer, I do consider myself part of the OU. I seriously considered doing this course as a MOOC when it was released. The main reason I didn't was because I knew completing (passing) it would be the final piece in the jigsaw for the MAODE, and although I would probably learn as much, I couldn't gain the credit. I think they would be particularly useful as foundation-level courses, giving people the basic study skills they need and a realistic understanding of the work level involved in studying as an undergraduate before parting with what is a significant amount of money.
• The courses I deliver as Clearly Stated... we simply don't have the resources. We're a small company (just the two of us) and no matter how you define "massive", it's simply not going to work.
• My professional body - for a while, we delivered an open course in Technical Communication, based on the City & Guilds syllabus before that course was discontinued. It would be an ideal state to provide this course for free as a MOOC, but again we are an organisation run by volunteers with limited resources.

On a personal note, I learn a lot from informal learning - and I consider MOOCs to be a hybrid between formal and informal studying. My big problem, however, is in being able to 'prove' to others that I have learned something.

In principle, I'm happy to accept people's word for the learning they've undertaken - and it's how the CPD programme I've implemented works. Our Fellows tell us what they've learnt and - ideally - how they've used it, but we don't care when, how or why they learnt it. It's now a matter of convincing academic institutions and employers that this learning is equally valuable.

Big vs little OER - advantages and disadvantages

[Activity 11]

Weller gave a very clear definition of what ‘big’ and ‘little’ OER are. He defined big OER as the large, institution-led and funded project often leading to a very polished output that may be used (viewed) by millions. In contrast, little OER is making available the smaller materials that we often create anyway to deliver our courses – perhaps with a little more polish as we are making them available outside our own inner circle, but not necessarily so.

The advantages of ‘big’ OER are primarily that of resources. Whole teams of people will work on a project, resulting in very high quality both of the presentation and of the content. Reputable experts in a field may be involved, either fronting the project or behind the scenes. The resources expended on the project also lead to several disadvantages: topic, content and longevity. The topic has to be one in which many people will be interested, or the resource being invested cannot be justified. The content may have to be modified, perhaps minimising or eliminating some of the more specialist aspects to appeal to the breadth of audience. Finally, having expended significant resources on producing a piece of big OER, it is unlikely to be updated frequently – meaning that in some cases content may become obsolete.

‘Little’ OER has almost exactly the opposite problems and advantages. Resources are usually quite limited, so the level of polish – even when the work is undertaken freely – is likely to be lower. This may be due to a combination of lack of time, and lack of expertise in the tools being used. The quality of the content in the first instance relies on the subject matter knowledge of the original author of the work. This may improve over time if the materials are repurposed, in the same way that a page on Wikipedia is modified as others read the content. There is no requirement to consider audiences beyond that required for the original use of the material – if other educators choose to use a resource, they must decide on its level of suitability or otherwise, and must make any necessary changes.

‘Big’ OER does not rely on the motivation and commitment of individuals; rather, the motivation and commitment of the organisation is the driving force. The creators of ‘little’ OER may have support from the organisation for which they work, or from the wider community in which they practice – but for the most part, the motivation has to come from them. Although theoretically ‘little’ OER is more easily updated, when time is short people are more likely to do the minimum, without thought for how others may want to use the materials later. The very fact that ‘little’ OER can be thought of as collateral damage – a by-product of normal activities – means that people may be less likely to consider the availability of a tool (a software application or a platform) or the skills required if the prime purpose is to meet their own needs.

Reference

Weller, M. (2011a) Academic Output as Collateral Damage [online], slidecast. Available at http://www.slideshare.net/mweller/academic-output-as-collateral-damage (last accessed 17 April 2014).

How useful were the OER repositories?

[Activity 8]

In common with many of my classmates, I found this an extremely frustrating exercise - probably made subjectively worse because I could write/produce many of these materials quite quickly myself (I teach TU100 - My Digital Life, and messaging/security companies have been previous employers/are current clients).

I probably didn't spend as long as I should have done in each repository - I daresay once I'd got used to the best way to search one of them, I'd find better quality materials more quickly. I did find a US government website through Merlot - and I could have done the whole course using materials from there without bothering to look elsewhere.

Some of the issues were simply those of presentation or a level of inconsistency in terminology. The bigger problems were when the resources had been written for a specific purpose and it didn't meet the needs of my audience. I had in mind an introductory course for those who would be studying online courses to make sure their technical skills were up to the task, and although I hadn't included such things as word processing or the use of imaging software, I did think the topics I'd picked would be easy to work with.

I found, though, that it was quite difficult to pick out the relevant bits and discard some of the specific. For example, I found some materials on the use of Skype in education - but they were written for teachers, not learners... and assumed you already knew how to use Skype and just wanted to find out how it could help teaching and learning.

I did find I was very tempted to modify the contents of my course to fit the resources I could find - although the topics would still need to be relevant and useful. I would be less likely to do that if I was creating them myself.

Journeys to Open Educational Practice: UKOER/SCORE Review Final Report A cumulative evaluation and synthesis of the entire HEFCE funded intervention in OER

[Activity 7]

(McGill, L., Falconer, I., Dempster, J.A., Littlejohn, A. and Beetham, H. Journeys to Open Educational Practice: UKOER/SCORE Review Final Report. JISC, 2013 https://oersynth.pbworks.com/w/page/60338879/HEFCE-OER-Review-Final-Report)

Accessibility issues

For resources to be accessible, the technology required must be readily available - and potential students must be competent to use it.

On a course I teach, we are encouraged to develop teaching materials that every tutor can use. The idea is a good one: collectively we can do more than we can as individuals. In practice, though, I find the process very challenging. Course materials designed to be used without any discussion on how they will be used and online tutorials designed to make sense when recorded and viewed later do not lend themselves well to interactive community engagement.

The open learning landscape has in this period moved definitively beyond content-based resources. Open online courses provide a context in which learning content is relatively less important than the interactions, reactions and emergent properties of the community itself.

Recognition

There is potentially a conflict between what educators may believe to be the most effective and appropriate pedagogy and what student's [sic] actually demand. This is illustrated by some of the UKOER projects reporting that students felt concerned about paying a fee and then seeing lectures being made freely available to non-payers. [Section 2.iii - Motivations of Individuals]

People pay for a course because it is far more than just the content. Students can ask questions, obtain feedback and be graded on their performance. Freely available materials are great when learning about something for interest or for something immediately relevant in my work. I either do not need to prove my competence or will demonstrate it practically. The situation is different when I need formal proof of attainment.

This has been addressed to some extent by the introduction of badges (see http://www.openbadges.org/), but it is not clear how widely these are recognised.

Continuity (and sustainability)

Money - or lack of it - is not the only factor influencing people's perceptions of OER. Time is also a consideration. There is the time it takes to create resources compared to the time it takes to re-purpose existing ones. These constraints influence what an institution is able to provide, both today and into the future. The providers of OER need to be able to sustain their provision - even if that is maintenance of existing resources rather than producing new ones. There may be other institutions and individuals (teachers or learners) relying on those resources.

In addition, embarking on a programme that relies heavily on OER may mean a reliance on increasing provision - so that the level 1 course can lead to a level 2 course and so on. Learners may begin by wanting to study a single course, but end up by wanting to obtain a qualification. Some mechanism needs to be in place to enable the learning based on OER - especially self-directed learning - to be validated as a stepping stone to further education.