Posts tagged assessment
WordPress Help! Private Comments/Posts & Aggregating Tags
Dec 17th
For the WordPress Mu gurus – or at least those more familiar with WP Mu than I (which does not require being a guru) – I have several questions. First, is there an available plugin that allows a post author to mark the post as private so that only they and a specific user or group of users can access/review the post? Second, is there an available plugin to allow a commenter to mark their comment as private so that only they and the post author (and system adminsitrators) can access/read the comment – even if the post isn’t private? Third, is there an available plugin that does site-wide aggregation of tags by user or tag?
That’s probably clear as mud, so I’ll explain what I’d like to do related to student portfolios and institutional assessment. I have italicized the parts of this scenario which are the source of my questions; the elements not italicized aren’t in question – I believe/know they can be done within WPMu. More >
(No Ratings Yet)
Loading ...What’s a test when you have a pocket encylopedia?
Nov 10th
Encyclopedia is “A complete copy of the Wikipedia encyclopedia on your iPhone/iPod. This app is the fastest and simplest way to browse Wikipedia on your device, and crucially, doesn’t require any internet connectivity . . . When you first launch Wikipedia, you will be required to download a copy of the database which will occupy about 2GB of space on your device.” I noticed this via LifeHacker and tracked through a Google Code page and the home page for the application. As an educator, think about this for a second. The entirety of Wikipedia on a mobile device no larger than a cell phone; that’s over 1 billion words in your pocket, literally. That’s over 25 times as many words as the next largest English-language encyclopedia, Encyclopedia Britannica. What does that mean to me? My daughter lives in a world in which she thinks “Daddy knows everything.” I’m not talking about the normal, narrow-world-view naivete of a child to whom it seems Mom or Dad knows everything; in reality, she’s never asked a question for which I didn’t or couldn’t provide an answer of some sort; even the less definitive answers to philosophical questions are still answers. Most recently, it was, “Why do we carve pumpkins at Halloween?” I have explained to my daughter that I don’t know the answer to every question; I just have a real good idea of how to find the answer to most any question. As an educator, that perspective and ability is a wonderful thing. As an educator, that perspective and ability creates a significant problem. It’s an incredible prospect to be able to find the answer to most any objective question given an internet connection and better than average internet search skills. But, what implications does that have for education, in it’s current, American incarnation? It may have been nearly 20 years since I last sat in an American History class, and many of the facts I learned then have long since escaped the clutches of my memory. However, if you were to put me in front of one of those same history tests along with a mobile device, could I not do fairly well on it? especially if I had a mobile device? In fact, why should I not have a mobile device available to me, as some Australian schools are now allowing? In terms of authentic tasks (see previous post), is remembering specific dates, facts, or information something I’ll ever have to do again? If I can use my iPod to help me answer almost every question on a test, what’s the value of the test?
(No Ratings Yet) Loading ...Stop Quizzing Learners in Second Life!!!!
Apr 15th
Finally getting back to this post I started about two weeks ago . . .
This morning/afternoon I attended a workshop that demonstrated a variety of Teaching Tools for Second Life. I absolutely appreciated the workshop and those involved in making it happen: the host institution, the faculty moderator and the facilitator. It’s these kinds of activities that make Second Life of keen interest to educators; there’s simply not another technology that brings everyone into a common workspace like Second Life does.
With that said, as an instructional designer and technologist, I have several significant pedagogical issues with the tools that were demonstrated. Each of the tools I saw demonstrated have, in my opinion, little to no useful place in Second Life. Worse yet, I believe they may actually be counterproductive to the development of quality learning experiences in Second Life. I think this is at the core of the “quality use of Second Life” question. My opinion may be perceived as being too harsh, but I’ll explain.
I don’t want to refer to the tools specifically because I do not want to denigrate the individual tools or the individual who created them; the developer is a fantastic educator for whom I hav ea great deal of professional respect. What I do want to do is to discourage the transfer of traditional assessments and learning tools into a virtual environment which has much greater capabilities and potential.
By and large, our classrooms are much more limited than the real world; it’s not easy or in many cases possible at all for learners to perform a skill in the classroom in the same manner they would perform the same skill in a real-life situation outside of the classroom. For example, in a business entrepreneurship class, it’s not possible to have learners engage the process of actually starting a business; that requires too many actions and resources that lie beyond the capabilities of the classroom. So, we cover the concepts and processes in as much detail and in as innovative ways as possible afforded by textbooks, new media, interactive technologies, classroom activities and the online environment. We then use an assessment instrument of some sort that hopefully provides a valid and reliable estimate of how learners would actually perform if they were to in fact start a business. So, out of necessity, we resort to tools that estimate how a learner would perform in the real world – in an authentic environment once they’ve left the classroom.
The virtual environment, in contrast, creates a unique opportunity to abandon the estimates and the not-so-reasonable-facsimiles of performing real world skills in a real world, authentic environment. In the case of the business entrepreneurship class, it IS possible for learners to actually start a business within a virtual environment, Second Life in particular. They can conduct market research within an actual, living market; they can fabricate actual prototypes of a product; they can bootstrap the business or seek funding for the startup; they can have actual customers purchasing products. Given enough time, they can engage every phase of the business startup process. As an instructional designer, the transfer of classroom tools and assessments that estimate how learners will perform in the real world to the virtual environment is, at best, incongruous, and at worst, counterproductive. They do not belong.
But wait . . . many will say, “We have to have some sort of assessment?!” Right, but I argue that the manner in which we assess learners needs to change; we need to take advantage of the affordances of the new environment. BUT for now, I’ll concede that argument, for argument’s sake. Even if we MUST conduct traditional, exam-type assessment, I ask, “Why do we have to do that within the virtual environment when we already have internet-based tools that do the same thing and perhaps do it much better than any currently-available tool in the virtual space?” Any learner that can access a virtual environment can also access any web-based learning management system that is guaranteed to have an integrated assessment tool. The two environments aren’t and shouldn’t be mutually exclusive; even if we want to teach a class entirely through a virtual environment, that doesn’t mean that ALL activities have to occur within that space. In fact, it’s MUCH more efficient to use both environments; we should leverage the particular strengths of each application.
If that’s not enough, one of the other participants commented, “We need to keep this as simple as possible; many of our students are not strong computer users.” We know that’s true; even strong computer users engage a learning curve within Second Life. Even if there’s disagreement about my arguments above, should we not still leverage the assessment capabilities of LMS’ rather than developing less reliable and more clunky tools to do the same thing in Second Life? for the sake of our learners?
(No Ratings Yet) Loading ...Learning Scenario: Entrepreneurship
Feb 6th
I took a few minutes to track down Ann Enigma (SL) and discuss an Internet Entrepreneurship course for which she’s providing Second Life support; they’re planning a Second Life component for the course during this Spring term. The notion is to teach “fundamental business concepts by having students start business[es] in SL,” and they’re inviting any interested educators or entrepreneurs to participate by offering guest lectures and/or professional critiques of learner work. Ann and I had discussed assessment strategy in the past, so this conversation was a follow-up to see where the project is headed.
Learners are going to engage a custom, in-house registration and orientation portal; the orientation will be facilitated face-to-face in a campus computer lab, and learners will have had a 2-3 week overview of fundamental entrepreneurial concepts. For the Second Life component, they’re going to be issued “starter kits” and then choose a business location on “Rhode Island (more space) or Casablanca (less space, more traffic).” Learners will get a “store space, and examples of vendors and products . . . Each group of 3-4 students – mixed majors – will have to devise a product or service, create a business plan for it, and actually start that business.” With the land, prefabricated resources and small L$ grant, learners can do whatever they like within the scope of the assignment and the PG restriction ;-). Ultimately, “they are expected to write a business plan, attempt to implement the plan, and reflect on the result.”
Several thoughts from my instructional design perspective.
Scaffolding/Support. The approach to orientation Ann described is a good example of the type of pre-learning support students need before engaging SL for instructional purposes. Most faculty engaging learners via SL are providing orientations and support, but this course offers learners a SL orientation and solid introduction to the course concepts as well. Also, the prefabs, workspace and L$ offered to learners are invaluable as well; that support will get learners past much of the SL distractions not essential to the course.
Simulation. Ann described, “The hope is that analyzing the SL market will provide the same type of experience as analyzing an RL industry.” Second Life should definitely provide a more authentic opportunity for learners to engage the market experience, particularly when learners are going to be able to follow up that research experience with the process of actually starting the business in Second Life.
Assessment. Currently, the plan for deliverables learners will produce include the business plan and a reflective presentation at the end. The Second Life component will definitely add a layer of authenticity to the exercise and certainly opens the potential for more than a few learners to truly amaze and astonish those teaching the course. However, Ann indicated they’re still debating how best to grade the businesses given that the limited time and new experience with Second Life may make it difficult for the businesses to flourish or develop fully.
While using SL as the market in which to develop a product and business plan (a) adds authenticity to the exercise, (b) is certainly pertinent to the content of a course titled “Internet Entrepreneurship,” and (c) creates unique insights and revelations for learners as they reflect, I think SL may create an additional opportunity in this instance.
In a typical, more traditional learning environment, learners often don’t have the opportunity to engage potential investors or to self-finance/bootstrap in an effort to actually start a business, so learners read texts, listen to lectures, and perhaps watch videos and other media describing the different sections of a business plan before write a business plan for an imaginary business. Thus, developing AND presenting a business plan via SL and/or working to finance a start-up business presents an opportunity to accomplish goals and tasks not easily performed in a non-SL environment; to me, SL makes the “punchline of the course” – actually starting a business based upon a business plan and market research – obtainable.
Imagine . . . As a learner, I don’t just develop a product or service and a business plan for it; the work to research the market, estimate financial gains and expenses, and determine resource needs is only the beginning – just as it is in the physical world. I have to prepare a precise, persuasive oral presentation to potential investors or loan providers (or experienced entrepreneurs or other business educators lending a helping hand); I might have to complete a loan application or endure an interview with a small business development agent for a bank. No matter the situation, I have a limited opportunity to get what I need to get my dream off the ground. My life as an entrepreneur can be nerve wracking, and Second Life makes learning and coping with that anxiety possible along with everything else to be learned in an “Internet Entrepreneurship” course.
Looking forward to learning how this project turns out for Ann Enigma and colleagues.
(No Ratings Yet) Loading ...Learning & Assessment in Second Life – A Scenario
Dec 10th
In a July post, I wrote about Assessing Learner Performance in Second Life. Jumping there and coming back may be useful. In short however, I suggested that Second Life (or any multi-user virtual environment) makes it more critical that we evaluate the “artifacts and performance produced” by learner experience in Second Life rather than relying on traditional forms of assessment which are designed to approximate actual performance. I did not, however, begin to explain a manner by which that may be accomplished, so the question is, “How do I begin to develop more authentic assessment of learner performance in Second Life?”
The first question to ask is, “What particular task can learners perform or what products can they produce within Second Life to demonstrate they have acquired particular skills or knowledge to be learned within the course?” An example . . . I teach an “Introduction to Computers” course in a community college; IF I were to use Second Life in that environment, I could ask learners – as a class or as a small group – to collaboratively build or explore a large, room sized , walk-in model of a PC computer. That assignment would focus on learner understanding of basic hardware.
A couple of caveats to consider when initially identifying a project, and these relate back to definitions I’ve offered previously of quality instructional uses of Second Life. The “build or explore . . . the walk-in model” is a critical choice. First, if they simply explore a model which I’ve constructed, I need to be sure to include interactive elements that make engaging the content within Second Life uniquely valuable. Otherwise, learners may be able to get the same level of understanding from less intricate technologies – web or print, text or graphics – and that’s not consistent with the suggestion I’ve made that learners should engage content “in a manner not possible through a physical or standard web-based learning environment.” Second, if I require learners to build their own model, I may be introducing skill and knowledge requirements beyond the scope of the course or objective. Learners could very easily spend more time fighting with the Second Life interface than they do on-point with the content for the course; this does not “maintain proper focus on the desired learning outcomes.”
The second question to ask is, “Based upon learner performance of a task or production of content, how do I know they’ve acquired the necessary skills or knowledge required by the course?” One choice is to ask learners to complete a written assessment, perhaps an objective exam of some sort. However, if I have the opportunity to measure actual, “real world” performance, should I not do that rather than approximating learner capabilities using less direct methods?
Back to my walk-in model of a PC . . . I can actually observe learners interacting with the model – either directly by synchronous presence or indirectly via tools I’ve developed to record their interactions with the devices/parts. Setting aside the task of creating the necessary tools in SL, the pedagogical task for me becomes identifying the different levels of performance learners may exhibit. Exactly what sort of interaction with the model constitutes “expert” level knowledge of PC parts and functions? What’s the difference between that and “average/sufficient” level of knowledge? And, finally, what sort of behavior with the device demonstrates that the learner has insufficient or poor understanding of PC hardware?
In short, I develop a list of criteria or objectives which my learners must meet, and then for each, I work to identify three different levels of performance to which point values may be assigned for grading performances. The end result is a performance rubric or matrix.
I think this type of assessment in Second Life is imperative. Transitioning different types of objective assessment tools to Second Life – quiz tools etc – is not a valuable exercise. It’s imposing a philosophically inconsistent method of assessment on the Second Life learning environment in a manner which turns the concepts of reliability and validity backwards. The ability for learners to engage authentic learning through virtual “performance” in Second Life should exclude traditional forms of assessment. Using those forms of assessment in Second Life is a sign of laziness or lack of awareness regarding more appropriate assessment tools and methods.
(No Ratings Yet) Loading ...Assessing Learner Performance in Second Life
Jul 12th
Describing skills necessary to design effective learning experiences in yesterday’s entry, I omitted one that may perhaps prove to be one of the more significant challenges for faculty teaching in Second Life: assessment.
Following yesterday’s post, I happened to meet up with an individual (will add more specific information, with her permission ;-) to discuss a few ideas. The conversation turned to a project slated at her institution for the Fall in which business entrepreneurship students will engage an experiential, authentic learning activity in Second Life: start a small business in Second Life while addressing principles of entrepreneurship. Of course, the institution wants to evaluate the effectiveness and net gain of using Second Life in comparison to more traditional modes of content delivery; to accomplish that, they’re planning to have a group using Second Life and a group not using Second Life each complete an end of course exam. Throughout the course, they’re also planning for the Second Life group to produce written work focused on entrepreneurial principles.
My question and point was and is this. In classroom environments, we use objective exams and written work to estimate how well a learner understands content because the classroom environment limits our opportunities to evaluate actual performance utilizing the content that’s been taught. Traditional meatspace or web-based classrooms for teaching entrepreneurship have a variety of constraints that prohibit learners from actually starting a business. In lieu of that actual performance, learners can read about starting a business; they can watch videos; they can listen to faculty or guest speakers talk about it; they can discuss it in small groups; or, they may be able to play serious games simulating it. But, they typically do not have a realistic opportunity actually to start a business using principles of entrepreneurship. So, we can only approximate how well they might actually “do it in the real world,” and we approximate by assessments that hopefully do a good job of measuring the ability to perform in the real world.
So, if we have the opportunity to measure actual, “real world” performance, should we not do that rather than approximating? Certainly. Will observing a learner’s actual efforts to start a business not always be a more accurate way of determining if they’ve learned the concepts of entrepreneurship? Absolutely. So, if we can design learning activities in Second Life that afford learners the opportunity to actually start a business, shouldn’t we evaluate the artifacts and performance produced by that experience? Most likely.
I’m not suggesting we have to entirely abandon approximations of performance like objective exams or written work; in many instances, local, state or national policies require those more objective, more easily standardized assessments for accountability purposes. However, there’s something terribly incongruous if learners authentically performing learned skills in Second Life are only evaluated and assessed based upon approximations of their ability to perform. Faculty teaching in Second Life should assess actual performance.
Thus, the capability to design “alternate forms of assessment” (a term used in the educational psychology literature to describe performance rubrics, portfolios, etc) will prove, I believe, to be a critical skill for successful and quality use of Second Life in educational environments.
The issue is that that type of assessment is not easy for or familiar to most faculty teaching the content. If my description of details related to entrepreneurship seem scant, there’s a reason for that; I have NO experience with or background in that field. I chose to be an instructional designer and educational technologist, not an entrepreneur (although I could easily tell you which would have been the bigger financial payoff ;-). Likewise, the individual I met has NO experience or background in educational assessment. My graduate work is in instructional design and technology; hers is in computer science and business. So, to repeat a question, slightly modified, from yesterday’s post, “How many institutions offering Second Life learning will expect faculty trained in specific disciplines (Computer Science or Business) to master a skill set which instructional designers have spent a doctoral graduate career learning?”
(No Ratings Yet) Loading ...
Hot Topics