Melissa's Learning Reflections on Assessment

This has been a great course overall. I have found a new sense of what assessment means not only online, but this will also apply in my face to face courses. How exciting it is to me to have these tools that will serve as a guide for my future courses. I look forward to creating additional courses with this in mind. I felt that the final project pulled all of the information and skills that I have learned during the course and brought them full circle. I am proud of what I was able to create and even better this will come into use in the future as I now have four tools that I can implement.

RT 310:  Professional Development for Radiation Therapists

          This upper level college course is taught as an introductory course in the fall semester when students begin the radiation therapy program. The course is designed to create awareness in the student of the importance of professional development. It is largely focused on the professional development plan for the radiation therapy program at UW-La Crosse. The students are able to learn about what types of behaviors should be modeled by a radiation therapist and a radiation therapy student. The course material also pertains to the ethics, practice standards, professional organizations, legislative movements, and patient empathy. This course is currently taught in a face to face session with some online learning activities. The course could easily be taught online and if this is done, would offer great opportunities to incorporate various educational and technological tools.

Assessment tool outcomes and final course outcome:

1. Students will summarize their course work and demonstrate an understanding of professional development by assembling a portfolio of artifacts from various activities throughout the semester using Google sites, according the rubric.
2. Students will discuss the professional development plan of the radiation therapy program at UW-La Crosse by utilizing the discussion board on D2L, graded according to the discussion rubric.
3. Upon viewing a power point about the professional organizations within the field of radiation therapy, students will demonstrate knowledge of these various professional organizations by completing a quiz on D2L.
4. Students will demonstrate patient empathy by creating a scenario based video using Camtasia Studio, according to rubric.

Final outcome: Students will demonstrate knowledge of professional development through various course activities.

Tool #1: Google Sites- http://sites.google.com

Learning outcome:

Students will summarize their course work and demonstrate an understanding of all aspects professional development by assembling a portfolio using Google sites, according the rubric for the final project of the course.

Rationale for use of this tool:

While education in the radiation therapy program thus far has largely been according to Mueller (2009), traditional assessment; students in this particular course would benefit greatly from authentic assessment as well. Instead of students recognizing correct answers and selecting one correct response that has been created by the teacher, they become the center of the assessment. In the case of authentic assessment, students perform a task to construct their mastery of certain concepts and objectives.

Advantage:

Students can construct electronic portfolios that are meaningful and a display of their best demonstrations of learning. It allows them to choose artifacts that pertain to their understanding of the various aspects of the course. Besides choosing artifacts, I will have the students reflect upon each artifact to indicate what they learned and why it was significant to them. This will allow for creativity amongst the students; as the artifacts and reflections will be unique and personal.

Disadvantage:

I feel that the strength of this assignment will come from the fact that it is student centered and a representation of their reflection and application of learning. However, if a student does not choose to engage with the material enough to create a reflection, then the real objective may not be reached. Also there is the chance that the Google sites database can disfunction and lose the student’s materials.

Portfolio Assessment rubric:

This assessment meets the analysis and synthesis levels of Bloom’s Taxonomy.

Tool #2:  D2L Discussion

Learning Outcome:

Students will discuss the professional development plan of the radiation therapy program at UW-La Crosse by utilizing the discussion board on D2L, graded according to the discussion rubric.

Rationale for use of this tool:

McCloughlin and Luca study (as cited by Bradshaw and Hinton, 2004), describe the discussion board as a necessary component of an online course to help students grow in knowledge and critical thinking. Additionally it allows for an opportunity for students to engage with one another and process the course material from a constructivist learning perspective. While it is important that the students are provided with the professional development plan for reading and application, there will be a greater chance of learning if they have to apply the information to a discussion board setting. When students demonstrate their understanding to their peers, it allows for additional ideas and perspectives, and more critical thinking about the material. For students to develop a strong understanding of the components of the professional development plan, they would benefit by discussing the material on a discussion board on D2L. When the instructor carefully crafts a discussion forum, they truly become a “guide on the side”, enabling learning to occur.

Advantages:

Students will move beyond simply reading the course materials to actively engaging with it on the discussion board. Students will also benefit by hearing perspectives from classmates regarding the material. It is desired that greater learning will occur in this constructivist setting. This discussion board will also allow students to engage socially and respectively in an online environment by exercising netiquette.

Disadvantages:

It is possible for students to not engage the way that they are required by the discussion board assignment and therefore would likely not learn as much. They could also regurgitate information back on the discussion board with little application or synthesis of its meaning.

Discussion Rubric:

This form of assessment will likely cause the students to be thinking on a comprehension and possibly the application level of Bloom’s Taxonomy.

Tool #3:  D2L Quiz

Learning outcome:

Upon viewing a power point on D2L about the professional organizations within the field of radiation therapy, students will demonstrate knowledge of these various professional organizations by completing a quiz on D2L.

Rationale for use:

By applying the information given in the power point in a quiz format to understand the radiation therapy professional organizations, students will have to be at a minimal comprehension level of Bloom’s Taxonomy.

Advantages:

This will allow the instructor to know that the students have a basic comprehension level of the various professional organizations within radiation therapy and their purposes. It will be a simple way to assess this comprehension as the quizzes will be automatically graded in D2L.

Disadvantages:

Students might guess correctly and this would indicate to the instructor that they have a good comprehension level of the professional organizations when they really do not. The quiz is also at the basic comprehension level of Bloom’s Taxonomy and can be questionable if real learning has occurred. I wonder if the student will still have a comprehension level of these concepts even as little as a day or two later.

Tool #4:  Camtasia Studio video- http://www.techsmith.com/camtasia.asp

Learning outcome:

Students will demonstrate patient empathy by creating a scenario based video using Camtasia Studio, according to rubric.

Rationale for use:

It is difficult to assess the ability of a person to demonstrate empathy by asking them questions on an exam. I feel that a better way to assess an understanding of patient empathy is to have the students actually demonstrate this skill. By utilizing Camstasia, I will be able to see the student in this scenario since we do not have face to face meetings. The video also gives the student some experience working with technology which is such an important part of the field of radiation therapy.

Advantages:

This will assess a high order level of thinking according to Bloom’s Taxonomy. I feel that this will be at the application level of thinking. The student will need to comprehend this new knowledge and then apply it to create the video. It will also allow me as the instructor to look at their body language, which is a very important part of patient empathy.

Disadvantages:

Working with technology can at times be problematic which I could see being a potential disadvantage here.

Camtasia patient empathy video rubric:

In conclusion, when authentic assessment methods are employed, a rubric is necessary for proper evaluation (Mueller, 2009).  In this assignment, the tools that were used in the higher order levels of thinking according to Bloom’s Taxonomy, required a rubric for evaluation. The D2L quiz will simply be an assessment on the basis of correct or incorrect answers. It is also important when using these tools in an online environment that the instructor plans the outcomes and how they will be assessed. The tools should be used to meet a distinct learning outcome or outcomes, and should have a planned assessment method, typically a rubric. Developing learning outcomes prior to commencement of a course, guides the assessment for the instructor and keeps needless or not meaningful assessments from occurring. Planning for assessment is crucial to obtain and meet the student learning outcomes for a course.

Sources:

Bradshaw, J., Hinshaw, L. (2004). Benefits of An Online Discussion List in a          Traditional Distance Education Course. TOJDE, 5, (3).

Mueller, J. (2009). Authentic Assessment Toolbox. Retrieved November 5, 2009,   from http://jonathan.mueller.faculty.noctrl.edu/toolbox/.

What a crazy week this was with sick kids and extra duties at work. Working Survey Monkey was a good experience and I like the idea of doing a pre-course survey. I typically do some basic assessments as part of getting to know my face to face students and I realized this week, that it is just as important in the online classroom. In fact, it might almost be more important because of the fact that we don’t have the face to face interaction. I really like to learn about students’ past experiences, learning styles, and expectations. I also do a check in evaluation at mid-term to view what they believe to be the strengths and weaknesses of the course. This gives me good feedback regarding the course and shows them that I am concerned about making the course the best it can be. It would also be easy to incorporate a pre- course survey into a face to face course, by sending this out by email before we even meet for the first time. It might bring additional focus to the course right off the bat as well.

It was a hectic week with sick kids and being out of town. I finally got the chance at the end of the week to sit down and think about my final project. I learned a lot not only from this course, but also from this assignment. It is sort of like a light bulb went off and I realized that so much of my course planning can be misguided because my assessments aren’t assessing the outcomes of the course. I enjoyed planning a course from the ground up by creating the learning outcomes and then figuring out how to use various online tools to assess that these objectives are met. I have enjoyed reading the work of Mueller in past courses, and felt that much of this assignment was creating ways to use authentic assessment. I would be interested to know what other ways besides rubrics could be used to assess authentically, authentic assessment activities. Not only was this assignment valuable when planning for an online course, but it also was helpful in creating my regular face to face lesson plans. I feel like I have guidance and purpose now when I am teaching.

This was a really good discussion in our course this week. I love the inspiration and thoughts that so many of my classmates offer. I am also very passionate about the are of putting courses online as it is a very popular topic on campus as well. There is much to be learned about teaching online which is why that I am glad that there are courses like these for teachers to receive proper training before they just jump in and do it. It is more than just putting your face to face materials online and teaching the same way. It requires a me to have different thought process, but a good one.
I feel that online teaching allows for higher order learning on the Bloom’s taxonomy because of the ability to engage each student in the online environment. Since I teach in both ways, I know that my students are engaged with the materials in the online course because I see by the way that they post comments to the course. It allows for them to read and process the meaning to them and then post to the course. I don’t see this same response in the face to face classroom, except for the 4-5 students that always ask questions or comment on the topic we are studying. In the online classroom, all students are required to participate so that brings much more thought to the course— truly a contructivist approach.

I really enjoyed working on the team midterm project for the various assessment tools for online teaching. My group worked very well together and accomplished a very strong project. I am glad that we did this project as now I have access to several great tools by looking at the other group projects.

Online Assessment Tools

A Review

by

Aldena Cherva, Phyllis Harris, Melissa Weege, Joyce Pederson

Assessment and Evaluation in the Online Classroom

University of Wisconsin – Stout

            How does one measure student progress in online education? With an exponential explosion of credit online courses, this question has no easy answer as methods are continuing to be developed to meet this unique learning option. The unique nature of constructivistic learning through collaboration inherent to e-learning, requires reconsideration of more traditional assessment methods while developing and refining new ones.  In choosing assessment tools for use in course planning, it is important to carefully examine how they work, identify their benefits and limitations, and consider how they could be used in the digital classroom. This paper reports the analyses of four assessment tools whose uses vary within the online classroom.  A range of assessment strategies included Exam builder – a tool designed for a range of testing applications, ePals Schoolblog – a secure school-based communication and learning tool, Wikis – an editable web page tool with a range of applications, and Teacherease Instructional Management System – a K-12 curriculum management system.

Exam Builder – contributed by Phyllis Harris

Introduction

            As a premier provider of online testing software, Exam Builder has clients throughout the US, Canada, Europe, Asia, Australia and South America.  Some of the test applications are for Telecommunications, Biotechnology, Aerospace, Technology and Insurance industries, as well as Government.

Advantages and Challenges

            Exam Builder is a tool used by educators, Training directors and Human Resource personnel to create and administer exams to diverse groups of students, employees, managers, and other staff.  Exam Builder can be used to manage exams, create historical reports on training, and assist in organizing the way to deliver training to all levels.  I found this software has a very detailed explanation of how it operates and is easy to operate.  Creating exams is quite easy and there is a choice of what is called “Click-and-Learn and Certificate” exams.  I found the Student Manager, Exam Scheduler and Image Manager easy to operate.  Exam Builder has a very impressive Report Manager, allowing the operator to export to PDF or Microsoft Excel.  The Report Manager also provides Analysis Reports, Exam History and Student History.

            Exam Builder is not Plug and Play software.  It takes a good technician with network service knowledge and skills, to set up the software securely.    Security is paramount whenever there is   transmission of online exams; heightened measures of security are necessary to insure protection of the test and answers.

Learning Objectives

            Exam Builder is an excellent assessment tool for highly technical application.    Having the ability to add images is a great advantage for the author of the exam.  Personalization of images to a particular technology, such as Biotechnology or the Aerospace industry, increases the use for multiple applications.

http://www.exambuilder.com/

http://www.exambuilder.com/userguide/eb_userguide.pdf

ePals SchoolBlog – A Reflection Assessment Tool – contributed by Joyce Pederson

            http://schoolblog.epals.com

Introduction

Reflective journaling through the use of Web 2.0 weblogs provides an online assessment tool that can provide formative and summative assessment information. As mentioned in Assessment and Online Teaching (p. 9), the use of reflection is a way “students are being encouraged to manage their own learning.”  Unlimited applications of blogging exist in assessing the levels of students’ developing content knowledge, their abilities to construct new understandings through collaboration, and their metacognitive skills. One unique weblog that is frequently used within elementary through university classrooms is the ePals SchoolBlog©, a free weblog available through ePals©, a provider of online school-safe educational services since 1996. SchoolBlog is a communication and collaboration program that provides blogging between students, teachers, administrators, and parents in a safeguarded and restricted use format that ePals calls a “walled garden.” Schoolblog’s primary function is to provide students with the opportunities to communicate using message boards and forums with a focus on classroom activities. Users responding using both open-ended and specific prompts, can demonstrate content understanding using constructivistic peer and teacher interactions, which can also be addressed by the teacher.  

Advantages and Challenges

 Information from the online ePals SchoolBlog Teachers Guide (2007) indicates the following user-friendly features: clear commands, manual, text formatting, curricular theme units, customizable template; customizable homepage, an assessment tool, threaded discussions and topic forums, moderation of content, and provision for RSS feeds and attachments.  Its focus on security through controlled access and internet within its school-based blog gives students a safe opportunity to develop reflective skills through journal writing and discussions of other information that stimulates critical thought.  Teachers can assess writing and collaboration skills through monitoring. It is a free site. ePals won the 2009 ComputED’s 14^th Annual Multi-Level Award in the Internet Communication Website category.

Its general inaccessibility to the broader community limits broadening students’ experiences to community and cultures beyond their school without special arrangements. It is a restricted blog with the blog open on a school-wide or classroom-wide basis with an option to add more users who can be added by the administrator. More teacher input may be needed to guide students during collaborations to expand range of thinking.  Obtaining rights to use this blog are time-consuming with an email reply needed to set up an account.

Online Learning Example

  SchoolBlogs could be used within a reading classroom as a reflection tool in which teachers post questions to which students respond or students could post questions for peers to respond to after a class reading of a book. It could be used formatively at given stop points in the book. Sharing with other classes within the user network could also occur.

Learning Objectives

 The student will demonstrate awareness of the development of a character by describing two examples of a personal characteristic such as loyalty in the story. Other students will discuss one of these examples in a response in which they agree or disagree.

 At the beginning of a class session, in a one-minute response, the student will respond to a teacher prompt related to the key point of the assignment as well as indicating an area not understood. (The teacher can then aggregate these responses and address concern areas.

Wiki – A Peer-to-Peer Collaboration Tool – contributed by Melissa Weege

Introduction

            A Wiki space is an editable Web page in which students can create pages or several pages of material or work on a group project in one place — on one site. This eliminates having to email different versions of a document back and forth, as it is worked on. Documents, graphs, and hyperlinks can be loaded on to the site to be viewed by its members. Additionally, the entire Wiki document is able to be viewed by team members in one place and can see the most current version, as well as who made the most recent edits. Wiki sites are free to use and easy to access from any Internet connection.

            ****For a good Wiki demonstration see the following brief YouTube video:                                                           http://www.youtube.com/watch?v=-dnL00TdmLY

            According to Wikipedia, the term Wiki comes from the Hawaiian language and means “fast” as it allows for quick and easy creation of Web pages. Additionally Wiki sites are easily editable by anyone allowed access to it. The Wiki is a great tool for helping students collaborate on projects to create one unified project. Instructors also have the ability to comment on student work as it is being worked on to clarify student ideas or points. Students or group members can also make comments and pages and have discussions furthering the collaborative effort on the Wiki site. I have used Wikis both in face to face courses for out of class work, in online classrooms for group work, and for collaboration with committee work. Some potential Wiki sites include:

·         Wetpaint Wiki- http://www.wetpaint.com/ This is a free site, however does have a large amount of ads unless you specifically indicate that you are an educator or buy a license to use Wetpaint. It is easy to use and very straightforward. 

·         PB Works- http://pbworks.com/ This is also a free site and is free of ads as well. It is very user friendly and additionally works well for building E portfolios.

Weaknesses and Challenges

            While a Wiki site is easy to use, it can be difficult to access it in some settings (schools, hospitals, clinics, etc.) because of firewalls. I have desired to use Wiki sites to communicate with my students at hospital internship sites, but the Wiki access has been blocked by the respective hospitals’ Internet security. So the Wiki site access is dependent upon the level of Internet security in the location of use.

            Another challenge of using Wikis is that anyone can make changes to the site, especially if the site is open and not secured. Often times I leave the site open so my students do not have trouble accessing. There are times when the site’s information may not be correct then because anyone is allowed to access it. This is true in the case of Wikipedia. Here is an encyclopedia that can be edited by anyone and while authorities keep an eye on their respective information, incorrect information may still be presented. 

            For the Wiki to be a good collaborative tool amongst peers, all peers need to participate. As is typical in some group work, there may be a participant that does not contribute on the site making it hard to work together and produce a final project or paper. However, this can happen with any tool or face to face, so it is not just the Wiki that may have this type of problem.

Online Learning Example

            Wikis are largely used for collaborative work, however it can also be used to blog, create a portfolio, and various other projects. An example of its use in an online classroom would be for a group project or paper. The students could easily establish the Wiki site and invite its group members and instructor to it. As part of my graduate work, I was assigned to work on the main themes of a certain chapter of our textbook with two other students. We needed to address three to four main points, summarize them, and present them to our class. One of my group members suggested that we do our work on a Wiki site. It worked out great and was a wonderful tool for collaboration. It was easy to see the edits made by the other group members. The Wiki site also allowed for comments to be made and discussion about various points of the paper.

            Currently my students are posting their lecture objectives for a unit in a pathophysiology course on a Wiki site. While this is a face-to-face course, this type of collaboration would also work well in an online course. For this Wiki site, when everyone has contributed, they will be able to print each page for a study guide for the unit exam. Their Wiki site is:  http://unit2patho.pbworks.com/

Online Learning Objective

Students will collaborate in groups to write a final paper.

Course Tracking Tool – Teacherease – contributed by Aldena Cherva

www.teacherease.com

Introduction

This particular product appears to be a functionally rich student and curriculum management system.  It provides tracking for student progress, lesson planning, matching of lesson plans to school standards, and curriculum mapping across the environment being supported.

Of particular note are the capabilities of the product to communicate with students and parents – hence – it is more suited to K-12 education.  I can see value in up to date status and progress of each student.  I can also see value is marrying lesson plans to school goals and standards.  Elaborate lesson plans are possible down to and including day-to-day plans.  Each of these plans can be tied to goals or standards and overlapping lessons or voids in lesson plans may be identified.  This capability, of course, assumes that all pertinent classes and lesson plans are entered into the software.

There is a very sophisticated grade book included with this software that will track any conceivable possible status of an assignment.  That is of course if the teacher enters to a great level of detail. 

Advantages and Challenges

            For the school without such a system this one appears to be easily integrated and used.  To have ready ability to communicate with parents and students is almost a given today.  This product will also produce paper status reports for those parents that do not have Internet capability.  Involvement of parents is critical to the success of a student; this tool provides one involvement.

            Although not a K-12 teacher I can see great value is having data bases of school goals and standards that I could match with my specific lesson plans.  Captured lesson plans also allow for substitutes to readily come into a class if I am unable to due to illness or other reason. 

            A sophisticated grade book would lend itself to tracking trends by student.  I imagine that such analysis as frequency of late assignments, incomplete assignments or habitually low grades would be of use.  I’m guessing that a teacher would use a paper grade book in the same way.  I know that I use my grade book in Blackboard to perform such analysis.

            I can see great advantage for the school administrator.  If each teacher were to use this tool consistently, administration of such items as adherence to standards, goal satisfaction and even performance of a teacher would be greatly enhanced.  The additional capability of curriculum mapping across various grades, schools and even districts would help to ensure consistent teaching and learning was occurring.  It also would point up gaps and overlap in teaching and learning.

            This software appears to be labor intensive.  I know that the vendor claims that grading is simplified but indeed every grade has to be entered.  In fact, to utilize some of the more extensive analysis such as an assignment submitted but not graded requires even more data entry.  I can see a teacher finding it easier to just use a paper grade book.  Strict guidelines would have to be implemented to take advantage of all the deliverables of this system.

            I could find no ability of this software to interface with course management software.  I am sure that such an interface could be developed but I don’t find the hooks already in place.

            There would be significant data entry required at the onset of such a system to enter each and every goal and standard the school might have.  Perhaps the most difficult challenge but maybe the most beneficial would the, establishment of such goals and standards.

Online Learning Objective

            This particular tool does not lend itself to specific learning objectives but rather gives the teacher the wherewithal to monitor the progress of his/her students in their fulfillment of learning objectives.  The correlation of graded assignments to learning objectives and goals and standards would aid the teacher in assessing effectiveness of such graded assignments.

Conclusion

            The emergence of online course delivery is a recent phenomenon that will expand learning opportunities to learners in a variety of settings. Measurement of student’s levels of achievement will be an integral part of overall course evaluation. Does the course make a difference in a student’s understanding of a subject area?  How can the standardized method of testing be applied in the online classroom? Are there ways in which evaluation can be done using authentic samples of their work as a means of measuring not only content knowledge but ongoing constructivistic critical thinking as well? How can we align the curriculum with assessment to more easily manage our classrooms, especially as more students participate in this option? The tools reviewed in this paper addressed these issues and have provided an introduction to them and their uses. Clearly, new applications of online assessment will continue to be available. Becoming familiar with the ones discussed in this review will be a good first step.        

I really enjoyed the concept map activity. I find that when I am getting ready to start a unit I am pretty organized and have what I know I am going to do laid out in a syllabus. However I do not always plan the appropriate assessement or learning outcomes. Or my assessment does not assess the learning outcomes that I have established. This has led me to want to be this intentional in course planning, both online and in the classroom by establishing good student learning outcomes and good ways to measure this. This should not be all that difficult since I just created an outcome assessment plan for the radiation therapy program that I direct. While that is large scale, the same concepts can be applied to units and courses. I also found the disucssions to be helpful and I am looking forward to working on the jigsaw assignment.

This had been an interesting module, mostly because I have not blogged before. However I do really like journaling or reflection to help me sort out what I have learned. It seems like I learn that much more when I reflect upon all that I have seen, heard, and experienced. I find it to be a good way to process my thoughts as well. I think if I could find a good way to assess students’ blogging, it would be something that I consider using in the future. However, it is hard to add one more account to check or read each day.
Because I really like the reflection aspect of assessment, I re-created my exam that I am to give to my students this week. The first unit of this class (the first year in the radiation therapy proram), was about professional development. Rather than try to ask exam questions related to more personal learning, I changed my exam to 4 mini-essays having them reflect on how the information. I also have allowed them to choose from 7 different essay prompts so that they can choose what they would like to reflect upon. The only downside is that there is going to be 81 essays to grade because I have 21 students. However, I do feel like this is a much better assessment method and it will show them what they have learned and hopefully be more likely and able to apply this to their lives.

Melisssa is a college professor and director of the radiation therapy program at UW-La Crosse in La Crosse, WI. She has been an educator for 2 years now and prior to that was a radiation therapist and medical dosimetrist (treatment planning) at a cancer center in Colorado Springs, CO (about two hours from Alex’s hometown of Loveland). She and her husband Matt have been married for almost 10 years and have 2 kids, Wyatt age 5 (just started Kindergarten) and Ayda, age 2 1/2 (which she emphatically mentions the 1/2 to anyone when they ask her age!). She has worked on online course development and taught online in the summer and is so excited to be able to learn about more assessment methods through this course.

I am excited to try using a blog. It is one of the Web 2.0 tools that I have not used just yet. It will be interesting to see if I learn some ways to use blogs as assessment tools, particularily in the research course that I teach.

Our class listing of blogs is: 

http://uwstoutblogs.pbworks.com/