Posts tagged instructionaldesign
Revising my grading rubric for discussion forum participation
Feb 29th
After three iterations of the course I’m teaching, I’m revisiting and potentially revising the grading rubric I’m using to assess learner participation in discussion forums. Back in August, I described the types of discussions in which my students in COSC 1401 Introduction to Computers are asked to participate and posted the grading rubric for assessing their participation. I have been using that rubric the last three terms (I’m teaching primarily 8 week terms; two last fall and one so far this spring). But, it’s not quite a perfect fit to how the discussions have progressed and how I want to grade them. So, I’m revising. I’m interested in your thoughts on this rubric. More >
(No Ratings Yet)
Loading ...Learning Outcomes ARE NOT Learning Objectives
Aug 8th
The simple fact that Learning Outcomes are NOT the same as Learning Objectives is a key principle to “Developing Effective Learning Outcomes & Objectives.” As noted in that presentation outline,
The differences lie in the level of specificity each provides and the relationship of each to assessment methods and instructional activities. Failure to understand and accommodate the differences can restrict academic freedom of faculty and complicate institutional efforts to manage curriculum and assessment.
Using the course I teach – COSC 1401 Introduction to Computers – I want to briefly illustrate the difference and the relationship between a learning outcome and a learning objective. More >
(No Ratings Yet) Loading ...Problems with Bloom’s Taxonomy?
Jul 31st
Last week or so, I came across an older article (December, 2002) from International Society for Performance Improvement that challenged the utility of Bloom’s Taxonomy on several levels. In short, Dr. Brenda Sugrue argues that Bloom’s Taxonomy is not valid, reliable or practical. Dr. Sugrue offers two alternatives which both suggest an emphasis on the application and use of knowledge. I believe there’s a great deal of truth in Dr. Sugrue’s argument, and I have a few additional thoughts. More >
(No Ratings Yet) Loading ...Continuous Improvement of Assessment Methods
(If you’re landing on the individual post page directly, this is an abstract for a conference/professional development presentation. See the speaking page for more details.)
Assessment of learner outcomes and performance has become increasingly important in a political and accreditation environment focused on evidence, data and accountability. While institutions work to establish procedures to ensure learner assessment results are documented, reported and use to improve instruction, the demand for improving learners’ workplace readiness will drive the need for continuous improvement and innovation regarding how we assess learners. Assessment is the beginning and the end of designing and facilitating effective learning experiences. With that in mind, “How do we improve assessment methods to enhance the learning and teaching process?” More >
(No Ratings Yet) Loading ...Stop Quizzing Learners in Second Life!!!!
Apr 15th
Finally getting back to this post I started about two weeks ago . . .
This morning/afternoon I attended a workshop that demonstrated a variety of Teaching Tools for Second Life. I absolutely appreciated the workshop and those involved in making it happen: the host institution, the faculty moderator and the facilitator. It’s these kinds of activities that make Second Life of keen interest to educators; there’s simply not another technology that brings everyone into a common workspace like Second Life does.
With that said, as an instructional designer and technologist, I have several significant pedagogical issues with the tools that were demonstrated. Each of the tools I saw demonstrated have, in my opinion, little to no useful place in Second Life. Worse yet, I believe they may actually be counterproductive to the development of quality learning experiences in Second Life. I think this is at the core of the “quality use of Second Life” question. My opinion may be perceived as being too harsh, but I’ll explain.
I don’t want to refer to the tools specifically because I do not want to denigrate the individual tools or the individual who created them; the developer is a fantastic educator for whom I hav ea great deal of professional respect. What I do want to do is to discourage the transfer of traditional assessments and learning tools into a virtual environment which has much greater capabilities and potential.
By and large, our classrooms are much more limited than the real world; it’s not easy or in many cases possible at all for learners to perform a skill in the classroom in the same manner they would perform the same skill in a real-life situation outside of the classroom. For example, in a business entrepreneurship class, it’s not possible to have learners engage the process of actually starting a business; that requires too many actions and resources that lie beyond the capabilities of the classroom. So, we cover the concepts and processes in as much detail and in as innovative ways as possible afforded by textbooks, new media, interactive technologies, classroom activities and the online environment. We then use an assessment instrument of some sort that hopefully provides a valid and reliable estimate of how learners would actually perform if they were to in fact start a business. So, out of necessity, we resort to tools that estimate how a learner would perform in the real world – in an authentic environment once they’ve left the classroom.
The virtual environment, in contrast, creates a unique opportunity to abandon the estimates and the not-so-reasonable-facsimiles of performing real world skills in a real world, authentic environment. In the case of the business entrepreneurship class, it IS possible for learners to actually start a business within a virtual environment, Second Life in particular. They can conduct market research within an actual, living market; they can fabricate actual prototypes of a product; they can bootstrap the business or seek funding for the startup; they can have actual customers purchasing products. Given enough time, they can engage every phase of the business startup process. As an instructional designer, the transfer of classroom tools and assessments that estimate how learners will perform in the real world to the virtual environment is, at best, incongruous, and at worst, counterproductive. They do not belong.
But wait . . . many will say, “We have to have some sort of assessment?!” Right, but I argue that the manner in which we assess learners needs to change; we need to take advantage of the affordances of the new environment. BUT for now, I’ll concede that argument, for argument’s sake. Even if we MUST conduct traditional, exam-type assessment, I ask, “Why do we have to do that within the virtual environment when we already have internet-based tools that do the same thing and perhaps do it much better than any currently-available tool in the virtual space?” Any learner that can access a virtual environment can also access any web-based learning management system that is guaranteed to have an integrated assessment tool. The two environments aren’t and shouldn’t be mutually exclusive; even if we want to teach a class entirely through a virtual environment, that doesn’t mean that ALL activities have to occur within that space. In fact, it’s MUCH more efficient to use both environments; we should leverage the particular strengths of each application.
If that’s not enough, one of the other participants commented, “We need to keep this as simple as possible; many of our students are not strong computer users.” We know that’s true; even strong computer users engage a learning curve within Second Life. Even if there’s disagreement about my arguments above, should we not still leverage the assessment capabilities of LMS’ rather than developing less reliable and more clunky tools to do the same thing in Second Life? for the sake of our learners?
(No Ratings Yet) Loading ...Educause Southwest: Does Your Campus Need a Second Life?
Feb 21st
I attended the EDUCAUSE Southwest Regional Conference in Houston, TX; my first session Thursday morning was Does Your Campus Need a Second Life? presented by Ana Gonzalez (SL: Mayela Saenz), Terry Peak (SL: Benotto Bailey) and Phil Youngblood (SL: Vic Michlalak) from the University of the Incarnate Word (San Antonio). My live blog notes – with comments/ideas from remote friends – are available online, and my thoughts and reflections are below. This group covered a lot of ground in their 45-50 minutes; do look for the presentation materials to be posted at UIW and the EDUCAUSE session site noted above.
This group noted two situations which are very fortunate when trying to explore Second Life for instructional uses. First, the CIO at UIW asked the instructional technology team to explore Second Life for relevant educational applications; a CIO requesting proactive exploration of any instructional technology seems like it would almost always be a good thing. Second, they began the project with what can fairly be described as a complete team: an instructional designer, a media specialist and a faculty member. As a result, they’ve grounded the work they’ve done on sound instructional theory and have had a successful and very accomplished pilot project that leverages unique capabilities of Second Life. Specifically, Terry spoke specifically about how they’ve used Chickering & Gamson’s 7 Principles as a framework by which to engage Second Life as a hybrid activity within a face-to-face programming/computerscience course, and Phil described a learner centered classroom that includes collaboration with two other classrooms – one in Mexico and one in France.
A couple of ideas I’ve explored recently did enter into the discussion – in shades, at least. First, given the 7 Principles, they did consider Time on Task; they set specific time requirements for completing tasks. Of course, this begs the question I’ve been asking, “To what extent does Second Life distract attention from content materials?” They may be accomplishing the time requirements, but does that account for the distractions encountered?
Directly related to that question, Terry commented that there is a learning curve for Second Life which can be somewhat steep; they had a very short time to overcome it. The UIW course is an upper level computer science/programming course, so the technology experience many of those students likely bring to the course probably mitigated the difficulty of the learning curve. While that raises the “efficiency” question (per Dodge’s PADE formula), Phil indicated that they have had 100% attendance with learners frequently staying after class to work on projects. You’d suspect that Second Life and quality designed instruction could provide an extra source of motivation, but that level of attendance and engagement by learners is extraordinary.
A couple of notable comments and question/answers from the session:
- You have to have (instructional design/media) support to teach in SL. You can’t do it by yourself.
- UIW spent a semester working on and planning the course.
- UIW specifies minimum computer hardware requirements for learners in the course.
- Learners could opt out of the SL activities in favor of alternative assignments (which also doubled as SL-is-down-assignments) but none took that option.
If you have any questions about the session, don’t hesitate to ask. I’ll answer as best I can.
(No Ratings Yet) Loading ...Learning & Assessment in Second Life – A Scenario
Dec 10th
In a July post, I wrote about Assessing Learner Performance in Second Life. Jumping there and coming back may be useful. In short however, I suggested that Second Life (or any multi-user virtual environment) makes it more critical that we evaluate the “artifacts and performance produced” by learner experience in Second Life rather than relying on traditional forms of assessment which are designed to approximate actual performance. I did not, however, begin to explain a manner by which that may be accomplished, so the question is, “How do I begin to develop more authentic assessment of learner performance in Second Life?”
The first question to ask is, “What particular task can learners perform or what products can they produce within Second Life to demonstrate they have acquired particular skills or knowledge to be learned within the course?” An example . . . I teach an “Introduction to Computers” course in a community college; IF I were to use Second Life in that environment, I could ask learners – as a class or as a small group – to collaboratively build or explore a large, room sized , walk-in model of a PC computer. That assignment would focus on learner understanding of basic hardware.
A couple of caveats to consider when initially identifying a project, and these relate back to definitions I’ve offered previously of quality instructional uses of Second Life. The “build or explore . . . the walk-in model” is a critical choice. First, if they simply explore a model which I’ve constructed, I need to be sure to include interactive elements that make engaging the content within Second Life uniquely valuable. Otherwise, learners may be able to get the same level of understanding from less intricate technologies – web or print, text or graphics – and that’s not consistent with the suggestion I’ve made that learners should engage content “in a manner not possible through a physical or standard web-based learning environment.” Second, if I require learners to build their own model, I may be introducing skill and knowledge requirements beyond the scope of the course or objective. Learners could very easily spend more time fighting with the Second Life interface than they do on-point with the content for the course; this does not “maintain proper focus on the desired learning outcomes.”
The second question to ask is, “Based upon learner performance of a task or production of content, how do I know they’ve acquired the necessary skills or knowledge required by the course?” One choice is to ask learners to complete a written assessment, perhaps an objective exam of some sort. However, if I have the opportunity to measure actual, “real world” performance, should I not do that rather than approximating learner capabilities using less direct methods?
Back to my walk-in model of a PC . . . I can actually observe learners interacting with the model – either directly by synchronous presence or indirectly via tools I’ve developed to record their interactions with the devices/parts. Setting aside the task of creating the necessary tools in SL, the pedagogical task for me becomes identifying the different levels of performance learners may exhibit. Exactly what sort of interaction with the model constitutes “expert” level knowledge of PC parts and functions? What’s the difference between that and “average/sufficient” level of knowledge? And, finally, what sort of behavior with the device demonstrates that the learner has insufficient or poor understanding of PC hardware?
In short, I develop a list of criteria or objectives which my learners must meet, and then for each, I work to identify three different levels of performance to which point values may be assigned for grading performances. The end result is a performance rubric or matrix.
I think this type of assessment in Second Life is imperative. Transitioning different types of objective assessment tools to Second Life – quiz tools etc – is not a valuable exercise. It’s imposing a philosophically inconsistent method of assessment on the Second Life learning environment in a manner which turns the concepts of reliability and validity backwards. The ability for learners to engage authentic learning through virtual “performance” in Second Life should exclude traditional forms of assessment. Using those forms of assessment in Second Life is a sign of laziness or lack of awareness regarding more appropriate assessment tools and methods.
(No Ratings Yet) Loading ...Second Life Instructional Designers Group
Dec 4th
I’ve encountered more than a few instructional designers working in Second Life, but until recently, I hadn’t encountered any specific network of instructional designers within Second Life. The natural thought was to create a group, but when I searched first, I found an “Instructional Designers” group already created by Stargazer Blazer, an instructional designer at Miami of Ohio.
The group currently has ten members, and I hope that grows over the coming months. Certainly, there’s a great deal of fantastic work being done and learning experiences being created within Second Life by very talented educators. Of course, I do believe there’s a unique contribution to be made by instructional designers. Thus, I’d like to see and hope to facilitate the SL Instructional Designers group growing, having events, generally contributing to the development of quality instructional resources within Second Life, and perhaps offering consultations to faculty working in Second Life.
If you’re interested in joining the group, simply IM me – Topher Zwiers – in world with your RL institution and a summary of your instructional design background. Currently, all group members have access to create proposals, invite new members and post group notices.
(No Ratings Yet) Loading ...Lisa Dawley on Instructional Design for Virtual Worlds
Dec 1st
I had the opportunity to watch a recorded archive of Dr. Lisa Dawley’s online presentation from Tuesday, November 27: Persistent Social Learning: An Emergent ID Model for Virtual World Design. Dr. Dawley (SL: Mali Young) is Chair & Associate Professor of the Department of Educational Technology at Boise State University and the designer of EDTECH Island in Second Life. The presentation focuses on what she believes is an emerging instructional design (ID) model for the development of virtual learning environments.
Initially, Dr. Dawley describes potential shortcomings of traditional ID models regarding the development of virtual learning environments and resources; specifically, the literature suggests the traditional models (e.g. Dick & Carey’s model and the more generic ADDIE model) are too process oriented, too static and too linear – leading to games or environments that don’t meet user expectations. I certainly understand the D&C model fitting into that category; I’ve always understood it to have more cognitivist, information processing philosophical underpinnings. The ADDIE model, as I understand it however, does not inherently have a cognitivist orientation; the instructional designer’s philosophy of teaching and learning influences specific strategies within the design phase, so the ADDIE model may be used to guide the development of more constructivist learning resources.
The challenge, according to Dr. Dawley, is to create a virtual environment that is engaging and draws learners in, and we can’t necessarily do that using traditional ID models. I agree entirely with that issue. I do believe there are two issues we, as an instructional design community, must address to meet that challenge. First, we must design and develop engaging virtual environments. Second, perhaps more importantly, we must design and develop engaging learning experiences within those virtual environments. I think the situation is somewhat analogous to an issue I’ve encountered recently regarding the use of iTunes: many colleges and universities are signing on with iTunes, but that’s only the delivery mechanism – the key to effective podcasting is the auditory content.
The presentation focuses on the first challenge: lessons learned while developing an effective and engaging virtual learning environment – specifically, Dawley’s work with EDTECH Island. For me, the first three phases of her research empirically corroborates the observational data published by Jennings & Collins (2007). For example, Jennings & Collins’ research compiled data regarding the educational institutions’ use of signage, pathways, maps and teleports on land parcels to facilitate visitor engagement and use of the space. Through the first phases of research, Dr. Dawley has collected empirical data validating the impact of those build design strategies; more importantly, her research also reveals the relative importance of building tutorials for learners, information sharing tools, meeting places, social/living areas, and group notices for increasing traffic and engaging visitors. As she summarizes, the evidence indicates navigational pathways and group notices have a definitive impact on engagement with social events and living spaces being less influential.
There were three other notable aspects of the presentation, in my opinion.
First, the data collection tools were new to me. I am familiar with visitor counters, but Maya Realities Second Life Traffic Analysis is quite impressive, to say the least. The metrics very easily evaluate the ROI of an institution’s SL space. I could try to explain it in detail, but it’s much easier to take a look at the presentation starting at the 25:30 mark (roughly halfway on the progress bar) and browse the Maya Realities website.
Second, one suggestion for increasing traffic to certain areas was to “put the milk in the back” – the grocery store notion of placing the highest volume product at the back of the store. That requires visitors to walk past everything else on the way to the high volume product. From a marketing perspective, that principle may work. From an instructional perspective, I believe it will be counterproductive. In any educational environment, we’re not trying to sell product; we’re trying to get learners to engage specific content in certain ways. Placing that content “behind” other interesting content which is ultimately a distractor is inconsistent with instructional and web design principles.
That leads to the third idea. An attendee suggested that we’re beginning to encounter and engage usability and design issues for virtual environments that are analogous to the earliest issues regarding the usability of web-based learning environments. Certainly, observing and participating in the convergence of architectural, game, instructional, and web design principles will influence the effectiveness of virtual learning environments.
(No Ratings Yet) Loading ...Can SL Transform Teaching & Learning?
Jul 24th
As an instructional technologist, I have many concerns with how we, as educators, may use Second Life. My primary concern is that we’ll simply transfer or attempt to transfer the way we’ve always taught in meatspace and in web-based spaces to Second Life. As I described in a previous post, I’ve seen too many instances where web-based instruction has failed to do anything more than transfer instructor-lead activities to an LMS in a manner that further reduces learner engagement. Rather than taking advantage of the capabilities and affordances of the web-based environment, many online courses have simply become read-the-assignments-and-take-the-quizzes type learning “experiences.” I’ve expressed the same concern regarding the design of learning experiences in Second Life.
However, after having listened to Intellagirl and Kenny Hubble’s keynotes at the SL Best Practices in Education Conference at the end of May and engaging several others in conversations at the NMC Summer Conference in Indianapolis in early June, I’m beginning to wonder if Second Life may have characteristics that give it the capability to transform how we teach and learn.
A post on the SLED Listserv in May (I apologize for not having a more specific reference) does as good a job as I could of summarizing the initial thought . . .
Personally I am struggling with what makes SL different from other online learning environments. One thing that really hit me during this session was that the learner is in complete control of their learning environment. In an online learning environment the student is confined by the program, the layout of the course, the links that the professor provides. You have to be somewhat self-directed but for the most part the teacher still takes you through the learning process.
In a virtual world, the learner truly is in control of their entire environment. Each student can and does interact with that environment on a personal level. I see the environment through “my eyes”. I am truly in control of everything. From what I view, to how I view it. I decide where to go and when to go there. There is a freedom about virtual spaces that can not be replicated in Moodle or Blackboard.
Given that explanation, does Second Life create an environment that is inherently learner centered? Is it not an intrinisically constructivist environment? Does it present an environment in which it’s natural for learners to take control of and responsibility for their own learning? Is the task at hand as much as not getting in a learner’s way and placing arbitrary constraints on the learning environment than it is coaxing learners to assume control of the learning space?
That may be too hopeful and naive. I shouldn’t underestimate our ability to impose traditional teaching methods and activities on even the most inherently constructivist space like the user-generated world of Second Life. However, will the juxtaposition of instructor-lead activities with the user-centered Second Life world be so stark a contrast that learners begin to assert themselves? Will learners have different expectations for learning in Second Life than they have for learning on the web or learning in a classroom? Will learners be more likely to ask, “Hey, why can’t we do X to learn this instead of simply reading a text and taking a quiz?” Will learners be more apt to drive the transformation of their learning spaces when that learning space resides in an environment in which they otherwise have a great deal of control?
Yeah yeah yeah. I know. But I can hope can’t I?
(No Ratings Yet) Loading ...
Hot Topics