(Initial) Reflecting Allowed

Maha Bali's blog has now moved to http://blog.mahabali.me

Leave a comment

Authentic & Sustainable Assessment: openly brainstorming workshop ideas

I’ll be co-facilitating a workshop with a colleague on “alternative assessment” and I have chosen to make my part of it about authentic and sustainable assessment. I plan to ask participants to brainstorm ways to modify their current assessments to make them more authentic/sustainable. (My colleague will then discuss pedagogical strategies for implementing these abstract ideas that I will discuss).

Thought I’d write this post to share my thoughts so far and see if anyone out here has good examples they’ve done in their courses that I could share. Also any ideas you have for making the workshop activities more interesting. I could then, in sharing these ideas, show by example why a sustainable, authentic piece of writing (like this blog post) can help develop ideas (and share with a wider audience) beyond doing the research all on my own and not sharing it. Does that make sense? Would this be considered crowdsourcing my workshop? (I already have books and of course google full of ideas I could use, but I have discovered I can sometimes get much more valuable stuff from people directly, like here or on twitter).

I just saw this wonderful statement by Dave Cormier where he is encouraging “blind sharing” because

It is next to impossible for you to know before you’ve shared whether it’s going to be useful to someone else.

So true. Now, moving on so I can “blind share” and encourage you to share.

So how do i define my terms?

Authentic assessment is one where learners try “real-world” applications of what they are learning. Two definitions mentioned in the Authentic Assessment Toolbox are:

A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills — Jon Mueller

“…Engaging and worthy problems or questions of importance, in which students must use knowledge to fashion performances effectively and creatively. The tasks are either replicas of or analogous to the kinds of problems faced by adult citizens and consumers or professionals in the field.” — Grant Wiggins — (Wiggins, 1993, p. 229).

One way to look at it is to consider important skills/tasks/values a “professional” in the field does, and design an assessment close to that.
An example pf a very bad assessment was my First Aid training with the Red Cross many years ago – it was a multiple choice exam. This in no way tests anyone’s ability to perform CPR under stress. A more authentic assessment would be to simulate an emergency situation and have volunteers react. The simulation, of course, would not test the volunteer’s courage and confidence to act under real conditions (quite difficult to test in an artificial environment) but at least it tests their CPR skills directly! A truly authentic alternative to this would be difficult to do in practice (e.g. Leave them on lifeguard duty and watch them from afar! ) but I have to assume that real professional life-savers (e.g. Medical people, firefighters, etc.) get more rigorous and authentic assessments than volunteers at the Red Cross.

A good continuum used in the Authentic Assessment Toolbox is this below:

Traditional ——————————————— Authentic

Selecting a Response ———————————— Performing a Task

Contrived ————————————————————— Real-life

Recall/Recognition ——————————- Construction/Application

Teacher-structured ————————————- Student-structured

Indirect Evidence ——————————————– Direct Evidence

(Note: the word “performance” can be tricky to use because it sometimes has behaviorist connotations and neoliberal ones: i.e. The emphasis is on showing a measurable skill rather than learning it deeply; however, in this context, i think the emphasis means ability to do something rather than theoretically select an appropriate response on a test).

(And in case you’re asking why I didn’t just get examples from that same website – many of the links I followed are not working).

Sustainable Assessment is an idea I got from #flsustain, Nottingham’s Sustainability, Society and You MOOC. According to Speight the author of “Learning for Sustainability” (free, open book, downloadable from here)

assessment strategies should be carefully planned to ensure that what is assessed is the development of the individual rather than their performance. … to focus upon a journey rather than a moment. A sustainable method of assessment is one that can do ‘double or triple-duty’ – it is appropriate and valid for the learning involved, takes the long view (thus making a contribution to society), and also meets the academic requirements of the university.

As part of the MOOC, I wrote the following:

Sustainability has connotations of continuity and of doing things in a holistic manner. Because my interest is in education, I am particularly interested in sustainable learning: how to design our learning environment and community and activities in ways that use sustainable methods and materials, and also promote sustainable/ongoing learning that continues seamlessly beyond any course-constrained time and space. I am just now learning how ideas of open education fit within this framework.

My personal approach to sustainable assessment has been to have all or most of my students’ work on their blogs so others outside the course can benefit. Because the content is on blogs, I am more intentional about making it useful for others beyond the course, and have invited my international networks (aka my online friends) to interact with my students via their blogs and Twitter, to everyone’s delight 🙂

Community-based learning, when done well can be both an authentic and sustainable form of assessment: learners work with real communities in their real problems, and hopefully create something that will have benefit beyond just the course, hopefully something that could endure beyond that time and space.

And just one closing quote inspired by Sean Michael Morris’ latest post on Keep Learning:

as teachers we can never be certain that our students will choose the same walls we choose for them…the space of learning is more fluid and adaptable than we might have planned on

(Disclaimer: i am quoting him slightly out of context, but it still fits brilliantly here).

This blog post, and the workshop it prepares for, is an invitation to expand our teaching beyond the walls… And I am inviting you to post suggestions in the comments! Thanks in advance!

Update: some resources from Twitter (thanks to Andrew M and Sarah S):
I was reminded of Herrington’s work (strange I did not think of it even though I cite her often in my thesis!) and pointed to this website on authentic learning, which has an authentic assessment and also points to this other good resource from UW-Stout
Another resource was HE Academy, which apparently has good projects with sustainability at their core (have not checked them out yet; hoping they are sustainable approaches to assessing learning, rather than approaches to assessing sustainability)


Digital amphibians, technological determinism, and the nuances of social phenomena

I am writing this post to reflect on technological determinism (something in #edcmooc week one we discussed as utopian and dystopian views of technology), and which i was reminded of my Shyam’s post on “digital amphibians” (love the term, did he just invent it?

Let me just first say something we all know, that is so obvious on an abstract level I doubt anyone would disagree: absolutes, black/white thinking, generalizations, one-sided views… None of these work too well to accurately describe complex social phenomena like education. We use models and metaphors to approximate what we see, but these models or frameworks or whatever rarely capture the whole, they only serve to help clarify or shed light from a certain angle. (we do so for the sake of legibility, to communicate, to clarify, but in the end we distort reality and it become unrecognizable to those living it). However good our models are, there are always multiple other angles worth considering.

Now back to Shyam’s “digital amphibians” post. I could not respond to it right away because it was full of so many ideas. I don’t think I can even summarize it here, and I have read it more than once. However, the main points in it that got me thinking (and it is quite possible I did not interpret them the way they were meant, but when does that happen anyway haha) were:

1. The digital amphibian metaphor, which I take to mean people like me who can comfortably navigate both the f2f and the online world; and by having both perspectives, be able to discover new and exciting things not very obvious or even visible to folks on only one side; but also sometimes able to be critical of both sides because they can see the other perspective;

2. Sometimes a digital amphibian can react strongly to something on either side: either with excitement and strong support; or with anger or indignation; he suggests that as “digital amphibians” mature, those reactions may become more tempered, thoughtful, etc.

3. He makes an interesting point about empathy and sensitivity, which I now realize may need to be central when a digital amphibian wants to communicate with, interact with, people on either side who are not “amphibious” themselves.

4. A very important point I think he makes and that we “amphibians” need to keep in mind is that, while we can see the pedagogical benefits (and more) of online learning/communication, its potential, we should not forget how others who care about the bottom line rather than learning, approach it. That in some ways, our passionate defense of (pedagogically sound) online learning may feed into the greedy aspirations of those who do, in fact, want to make education more efficient by reducing costs of labor. When we digital amphibians know that most quality online learning is people intensive. What makes online learning good, usually, is the people aspect of it, the interaction that occurs. Very labor-intensive, indeed, though much of it might be unpaid labor!

5. I like to think of myself as someone who avoid technological determinism. To use Sean Michael Morris’ term, a “digital agnostic” – I don’t wholeheartedly embrace technology as the solution for everything for everyone; nor do I see it as the destruction of everything for everyone. Even when technological tools help/enable me personally learn or grow or do something different, I cannot assume the same approach would work for others, or even work for me all the time…

I am still thinking about Shyam’s post, but it also made me think about a few other things…

6. In a response to a comment, I think he’s saying that he believes it is natural and acceptable for digital amphibians to not always be diplomatic. I take this to mean that it is sometimes OK to respond to extreme determinism by “partial” responses. By “partial” I mean both “incomplete” and “biased” (i love how Ellsworth 1989 uses the dual meanings of this word). Sometimes, a particular perspective gets so much hyperbole that you feel compelled to critique it, even when it has some merit. The hyperbole alone is problemtic.

some examples of my “partiality” within my dual roles

I recently co-authored an article “An Affinity for Asynchronous Learning” and you can tell by the title that it is a biased article (duh). It is not meant to discount the value of synchronicity, especially not f2f synchronicity, but it is meant to tamper the louder discourse that praises synchronicity without critiquing its limitations. I do have lots of issues with synchronous learning, but I managed to participate in 4 twitter chats last week (well, yes, there were power cuts, the kid not sleeping, the husband wanting attention, etc., but i still managed to make some of the time of each of these chats; i got lucky; it’s not always possible).

I recently also published “Why Doesn’t This Feel Empowering? The challenges of web-based intercultural dialogue” – a critique of one of the absolute best synchronous learning experience I have ever facilitated, and while I wholeheartedly believe all I wrote in the critique, I also believe in its power and potential and am trying to convince others to adopt it (yes, to adopt the experience I critique, because knowing what’s wrong with it can help us handle it better to reduce these problems rather than approach the experience with rose-colored glasses)

I mention these two first because they are peer-reviewed pieces that others have commented on and not found “too biased” to be published 🙂 But of course they are partial. They are meant to be.

But maybe one of my most partial pieces is the piece just before this one, posted on my blog a few days ago about meaningful online relationships. It was exactly as Shyam called it: an extreme reaction to something written by others who saw a threat looming large, and although I am normally aware of that threat, I did not realize that my reaction might have also been extreme. Because both articles that irritated me were quite balanced ones, to an extent, while remaining cautious about online learning, and highlighting its limitations. I don’t disagree that it has limitations, but I was responding to a particular point in both articles that irritated me, the point that online learning cannot produce affective, meaningful relationships.

I was there trying to highlight my very personal experience of forming such deep relationships, which I recognize might not work for everyone, but does work for some people.

However, I did not make clear (in my very partial post) a v important point: f2f will always (yes, a generalization worth making) have value; tactile communication (hugs, kisses) cannot be done online (yet!). However, I have had meaningless physical hugs, and deeply meaningful virtual hugs. I have been deeply touched by words I have read (even in books, where I cannot interact with the author) more than physical words/gestures by loved ones in my life.

So two final points to make, quotes that I believe in very much:

The first, by Irvin Yalom (quote not on me at the moment): when we categorize another, we lose the elements of them that are”unknowable”. Most attempts at social research will impose categories for the sake of legibility. For the sake of meaning-making. But we still need to remember there are other dimension to people and social situations beyond those we choose to use at a particular moment in time. (side not: ppl u know online recognize they do not know all of you; but people who know you f2f may not realize they do not know all of you that encompasses the online you; reminds me gain of Bonnie Stewart’s post on cyborgs which I have referred to so many times already).

The second is the notion of thinking of social research as a “crystal” (Richardson 1997 considers this a transgressive, post-modern view of social research validity), whereby the same object looks different from different angles, you look at it from one angle and you shed light on one view of it, but there are multiple other views, and social research attempts to show several of these views. And the crystal as a metaphor is slightly problematic in itself because crystallization implies a rigiidty which social phenomena, inherently dynamic, do not have. However, in thinking about it further, when we write about our research, we sort of attempt to take a snapshot and fix it in time, for that moment. And so, every time I write, when anyone writes, even if it is writing that takes many years like a doctoral dissertation, we are only capturing moment in time of our perspective/interpretation of a social phenomenon. It is not the end-all of our thoughts, nor the full picture of the social phenomenon even at that moment in time, but it is something that we can share.

I would rather share my incomplete thoughts and have them challenged and broken apart, than to keep them to myself and never take them further.


Meaningful Online Relationships

I’ve been wanting to write this post for a while. It’s about the possibilities and potential of meaningful online relationships. It’s been sort of building up and is about to explode after two articles (followed by facebook discussions, mostly on rhizo14) I read that irritated the heck out of me:

First, this one by Jason Hogan “The Campus is Dead, Long Live the Campus“. Here is the part of the article that irritated me:

Virtual communities can provide an alternative to the on-campus experience but, as yet, there is little evidence to suggest that virtual engagement with peers and with content matter experts can provide the same benefits as being immersed in the intellectual culture on campus.

After reading it, I tweeted:

To which the author responded:

(Note: in re-reading the article, it is generally quite a balanced one, otherwise, recognizing the benefits of both f2f and online learning)

BUT the author’s response raised by hackles even more, so I responded with a stream of tweets (to which he did not respond). Basically, I was saying that:

  1. Deep relationships can happen online (especially maybe for people who are more verbal/textual and possibly cyborgs – I refer here to this old post of Bonnie Stewart’s which resonated with me very much about how non-hyper-connected people had not idea what kind of lives the hyper-connected of us are living)
  2. You need to have relationships like that to understand them. My mentor is someone I only ever met f2f about 3 times total. We talked on the phone about 6 times total over a 7 year period (same for a very close friend of mine – both of them above 60, btw). I did not go into detail on twitter but will do so now. Saying online relationships can never be as valuable as f2f relationships is like being someone who has had many failed romantic relationships and assuming no one in the world can be happily married. Or it’s like being happily married in a certain way and assuming the only people can be happy is to be married and having the same kind of relationship you are. It is an absolute generalization that makes no sense. It is also understandable that someone who has not experienced a deep online relationship might think it impossible; an illusion. But you need to go through a really rough time in your f2f life with only your online friends as a lifeline to know what deep online relationships can do for you (Danielle Paradis comes to mind – this really touching post). I have several of these relationships with people I have never met f2f, and also with people I’ve only met f2f a handful of times. Don’t ask me how in a 6-week period (or even earlier) many of these happened on rhizo14. The collaborative autoethnography we’re doing might give some ideas. Or not 🙂
  3. People use text-only media to communicate in their most intimate relationships. Love letters, anyone? Text-messaging (or even more intimate uses of text messaging)? And phones, of course. So much important stuff happens with text-only media. And that’s even completely ignoring the possibilities of audiovisual online communication. Yes, you can’t get physical hugs online, but you can get virtual hugs that make you feel so warm inside they are better than physical. I remember many many many  an online communication: email, tweet, facebook message that made me feel great when my f2f day was going down the drain. I’m not talking a bad hair day here. I’m talking disastrous catastrophic days where my only solace was an online friend or two. Sometimes because I could talk it over with them, sometimes just by being themselves and lifting me up unintentionally. Regardless, I have been through hell and back several times in my personal life and my online friends have been my saviors.
  4. On a more logistical note related to learning – no one kind of education can be generalized about as being better or worse than any other kind of education. There are a multitude of ways of teaching/learning online and f2f. The same teacher can do exactly the same thing two semesters in a row (heck, the same semester with two different groups of students) and get completely different results (don’t tell me you’ve never seen that?). There are a zillion factors that can help make any learning experience good or bad, successful or unsuccessful, or anything in between. Sometimes you hit a “sweet spot” with many people and you get something like rhizo14 (Dave Cormier said something about fertile ground, which I took to mean the right conditions for something like that to blossom; I’m still trying to figure it out). But not everyone enjoyed rhizo14 as much or to the same extent; strong as I feel the community is, I know some people were in rhizo14 but are not part of the “core” of it, even though rhizomes have no center, there are still people who are more strongly connected than everyone else (I was recently deeply disappointed that I had missed the tweeting and blogging of someone I had known online in 2003  and with whom I was hoping to re-connect – who was on rhizo14 but not on the facebook group – probably because near the end I stopped following the twitter as closely)

There was a facebook discussion on rhizo14 (based on our collective irritation to a claim that “online edu like love cannot be moved online” – I don’t want to get into specifics so as not to overquote people outside the group). But the gist is that we were trying to figure out what it was that made us feel online relationships can be meaningful and intimate, sometimes even more valuable that f2f ones, or at least valuable in their own right, not as a poor second to f2f. Not all hyper-connected people are like me (I’m hyper in real life, too, talkative and hypersocial) – but there must be a group of factors/characteristics that, when found together in a person, increase the likelihood they will be able to build online relationships.

I think Simon alluded to two things that I misinterpreted but revolve around two ideas: literacy & affinity. The first, literacy, is obvious. If you’re not able to navigate social media, and then not able to do so with sound judgment (that’s the literacy part), you won’t be able to even get any relationships, deep or not. The second is affinity for digital communication. But how does that last one come about? I did my master’s online and enjoyed it tremendously. Obviously, completing it successfully meant I managed to learn online. But many others dropped out. Which means there was probably already something that helped me complete it even before I “learned to learn online”. I had not had any online learning experiences beforehand, though I’d had to work occasionally in multinational teams that met on phone conference occasionally. But that was no comparison.

Well, so I’m still not exactly sure what it is, really, that enables some people to have deep and meaningful and intimate online relationships, more than others. I don’t posit that everyone would be comfortable with this, or would trust a complete online stranger (Ary Aranguiz made up the term “frainger” on her blog earlier).

OK. This is a post of incomplete thoughts. But I wanted to get them onto my blog…

Meanwhile, I leave you with a padlet my students and  I created today in class. I gave them flowers and asked them to reflect on “Education is like flowers”… Enjoy!

Education is like flowers padlet


OERs, MOOCs, and dieting

This is just a quick post re-capping thoughts from today’s twitter discussions and especially the #nwoerchat

First, I want to point out that I remembered Rebecca Hogue’s great work on a framework for describing MOOCs and was compelled to share it with folks on #nwoer to help tamper the conversation on MOOCs beyond the typical xMOOC with all its flaws (was inspired after reading Peter’s blog post). It seemed to resonate well, and I am glad, because Rebecca did a great job.

One of the questions in today’s #nwoerchat (Storify here, great job Peter on getting it up so fast! And both Sue and Peter for facilitating) was what does the ideal MOOC look like. And well, first, most of us agreed there was no one-size-fits-all learners, contexts, etc. We also had a few rhizo14ers (Sarah, Simon, Carol, Len, who else?) and we of course waxed lyrical about rhizo14 (in this vague way that I am sure annoyed other people and made them feel excluded – I am so sorry, I remember feeling that way about other MOOCs ppl talked about and said it was an amazing experience and difficult to describe).

Speaking of no one-size-fits all, I was reminded again of the analogy of education is like dieting: fads come and go, but the reality is that each individual has different needs and different things work well for different contexts. And speaking of food analogies, Simon took my suggestion and created a padlet where we can all post our food-edu-oer analogies! Here it is if you’d like to add to it

One of the very important other conversations I have had today on fb and twitter relates to the openness/closedness of MOOCs. It is lovely to have MOOCs where you can get in any time (but I said, never leave, as in Hotel California). Several ppl on the twitter chat agreed that the best MOOCs are the ones where they never end, or at least the connections between people continue long after the official MOOC is over. This all reminded me of an earlier post about what makes a good professional development experience and it would be great if people can add to the list I initially wrote (via the comments – and then maybe I’ll do a new post integrating/attributing the comments).

Another really useful thing that came out of the twitter chat (though I did not see responses other than my own actually) was the question of how/whether MOOCs feed into our teaching. For me personally, MOOCs have fed into my teaching in many ways:

1. I can get resources to use (whether articles in xMOOCs, or peers’ blog posts in cMOOCs)

2. My students’ blogs this semester are getting loads of comments from my MOOC friends – and this fits perfectly with the course outcomes that include an element of intercultural interaction

3. Some of my students joined a Twitter chat by a MOOC friend that was done for his students originally

Flower: open and closed. Wikimedia commons

Flower: open and closed. Wikimedia commons

Among many other ways twitter has helped my teaching, e.g. when I posted my “riddle” for teaching this semester, Mark posted a “Goha” story that I used in my very first class to help me explain the dilemma to my students in a humorous way.

I’ll stop here before this all becomes too personal to make sense to anyone but me (and maybe rhizo14 coz they know me too well)

Oh… ok, one LAST personal thing: I received a (physical) gift from Clarissa (from Brazil) today – loved it Clarissa. Cannot believe I now have a rhizo14 magnet on my fridge… evidence (not that I need it) that our online relationships are much more than just that :o) And this reminds me of an annoying article I read today about online relationships not approximating f2f. Guess what? They can be richer and deeper and more engaged. Some people are just not lucky enough to be able to build them.

I’m off! One last last thing – I found a nice image for “open” and “closed” which also reminds me of Peter’s post “MOOCs: a rose by any other name”. I inserted it somewhere in the middle of this blog post. 🙂