LMS And Quality Matters Evaluation Introduction
Welcome to all members of the Executive Board,
The purpose of this webpage was to evaluate two learning management systems, Brightspace by D2L and Canvas by Instructure, on a number of different features for their utility, ease of use and depth of functionality against a Quality Matters Rubric. Quality Matters is described as a collegial review process to both provide recommendations to the course designer on strengths and areas of improvement as well as to evaluate the course in terms of course standards.
The challenge that led to this evaluation was complaints to your administration that your current learning management system does not meet the needs of its learners or faculty. The learning management system (LMS) has significant down time and the current online classes are poorly designed and disorganized. The drop out rate for both online and blended courses is much higher than face-to-face courses. Many reasons are given including: student confusion on where to start or who to contact for help, lack of engagement by both learners and faculty, poor feedback and communication, delayed and inadequate assessment tools, as well as technical difficulties.
In addition, the business and legal offices of the organization insist that the new LMS must be evaluated on such "behind the scenes" features as customer satisfaction levels, training and technical support, costs and the ability to provide analytic data. They stress legal and ethical obligations to provide a comparable learning experience for students with learning differences and disabilities. Your organization allowed a six week evaluation period as well as providing peer reviewers to assess progress.
The purpose of this webpage was to evaluate two learning management systems, Brightspace by D2L and Canvas by Instructure, on a number of different features for their utility, ease of use and depth of functionality against a Quality Matters Rubric. Quality Matters is described as a collegial review process to both provide recommendations to the course designer on strengths and areas of improvement as well as to evaluate the course in terms of course standards.
The challenge that led to this evaluation was complaints to your administration that your current learning management system does not meet the needs of its learners or faculty. The learning management system (LMS) has significant down time and the current online classes are poorly designed and disorganized. The drop out rate for both online and blended courses is much higher than face-to-face courses. Many reasons are given including: student confusion on where to start or who to contact for help, lack of engagement by both learners and faculty, poor feedback and communication, delayed and inadequate assessment tools, as well as technical difficulties.
In addition, the business and legal offices of the organization insist that the new LMS must be evaluated on such "behind the scenes" features as customer satisfaction levels, training and technical support, costs and the ability to provide analytic data. They stress legal and ethical obligations to provide a comparable learning experience for students with learning differences and disabilities. Your organization allowed a six week evaluation period as well as providing peer reviewers to assess progress.
LMS Final Recommendation
Week 1-LMS Set-Up and Content Upload
|
The course set-up in Brightspace by D2L (Brightspace) and Canvas by Instructure (Canvas) were facilitated by watching instructional videos on Atomic Learning. Creating modules was much more intuitive in Brightspace, but adding 3rd party content was easier in Canvas with their Common app. The HTML editor was easier to use and more intuitive in Canvas. Since the organization will be switching to a new LMS, ease of use for new users is the key feature for the week. Canvas seems more intuitive and to have a quicker start to launch time than Brightspace.
|
|
Week 2 - Faculty- Student Interaction
|
|
A robust method of communication between learners and instructors is necessary to keep students engaged and to provide feedback. Canvas came out the leader in nearly every area explored this week. The grade book was simple to organize and very functional. It was easy to divide a large class into groups and to provide discussion topics. Even in the trial version, the web conferencing tool was multi-featured and easy to use.
|
Week 3- LMS Assessment Capabilities
|
Both Canvas and D2L were explored for their ability to allow the creation of robust assessments, allow quick turnover for grading, provide use of rubrics for grading and methods to check for originality. Nearly all areas of the grade book were more easily set-up in Canvas than in Brightspace. Since Canvas was consistently superior in this area, a bias was considered. Canvas was evaluated after Brightspace and familiarity with grade book features may have made Canvas feel more intuitive.
|
|
Week 4- LMS accessibility Capabilities
|
|
There is a legal and ethical obligation to make sure that content is provided in an LMS in a way that is accessible for students with special needs. Both Brightspace and Canvas were evaluated for these features and online reviews by experts were also considered. Most features, except for using HTML templates with cascading style sheets were equally straight-forward and robust in both systems. Creating the HTML templates with cascading styles sheets was much easier in Canvas due to the option to import a template or use the HTML editor. Expert evaluations felt that both companies took the standards seriously and were responsive to consumer requests.
|
Week 5- Home Page Features
|
A customized homepage quickly orients students to an online or hybrid class and helps prevent dropping out due to frustration. Brightspace provided a wide range of widgets that made it easy for a new instructor to customize the look, feel and content of the home page. Customizing the navigation bar was intuitive in both LMSs, but required fewer clicks in Canvas.
|
|
Week 6- Behind the Scenes
|
|
In the final week such key features as customer satisfaction, training, technical support, costs and the ability to evaluate analytic data were explored. Both Canvas and Brightspace have very high customer satisfaction levels, with Canvas's being slightly higher. Both companies seem to provide a number of levels of training options and technical support, but Brightspace's seems to be a bit more robust. It was difficult to get an accurate assessment of cost because an individual quote will be provided based on a number of factors. Finally, reviews were mixed on the quantity and quality of the analytic data provided by both Brightspace and Canvas.
|
|
The final recommendation, after considering all factors, is Canvas. Its ease of use for new users was the deciding factor. Brightspace by D2L has a better appearance and more features, but as far as functionality and likelihood for widespread adoption by faculty Canvas is the better choice. This recommendation was based on hand-on trial, online reports and reviews and interview of administrators using the LMS at other universities.
|
|
Quality Matters Recommendation
Quality Matters is a peer review process designed to review online courses and evaluate them against forty-three standards. These standards were designed to facilitate student and faculty ease of use, utility and improve student retention rates in online courses. Quality Matters is a standardized program that provides in-depth instruction on how reviewers should evaluate courses. Directions are clearly given on how to check the online course for alignment to the standards and how to score it to determine if it meets expectations. Since the current course designs have largely been described as ineffective, having instructional designers trained in Quality Matters and tasking them with aiding faculty in developing their online courses seems to be a good use of resources. Instructional designers could develop templates and tools that could be imported into courses to facilitate rapid course design. Over a period of time, all courses, both online and blended, should be required to be evaluated against Quality Matters standards. In order to facilitate this requirement, it would be worthwhile for the organization to subscribe to Quality Matters.
During the summer before implementation, professional development sessions should be offered to allow faculty to be instructed in the use of templates designed by our instructional designers. For the first year, faculty should continue to constantly review their course and add sections aligned with the Quality Matters standards. The next summer all faculty should be required to complete the APPQMR training. They would be expected to then review their courses to make sure they meet the standards. Over the next year, all courses will be evaluated by the instructional designers to make sure they meet the standards. Faculty who do not will have a required professional development session to remediate the shortcomings in their courses. A survey of the faculty and students can be used next to answer the question of whether it is necessary for all courses to undergo an official Quality Matters review in order to be certified.
During the summer before implementation, professional development sessions should be offered to allow faculty to be instructed in the use of templates designed by our instructional designers. For the first year, faculty should continue to constantly review their course and add sections aligned with the Quality Matters standards. The next summer all faculty should be required to complete the APPQMR training. They would be expected to then review their courses to make sure they meet the standards. Over the next year, all courses will be evaluated by the instructional designers to make sure they meet the standards. Faculty who do not will have a required professional development session to remediate the shortcomings in their courses. A survey of the faculty and students can be used next to answer the question of whether it is necessary for all courses to undergo an official Quality Matters review in order to be certified.
Evaluator Background
|
Leann Poston is a student in the Masters in Educational Technology Program at Wright State University. She will be evaluating two Learning Management Systems over the next 6 weeks and then submitting an executive summary with a final recommendation on both which learning management system should be selected for the university as well as whether Quality Matters should be adopted campus wide. The university is trying to encourage development and utilization of online and hybrid college courses, but on student evaluations it is clear that these courses on the current learning management system are not successful. In addition, student evaluations indicate that there is an inconsistency in the online course content, some are excellent and some are miserable experiences. She has a B.S. degree from Xavier University and an M.D. from Wright State University as well as a history of twelve years experience teaching Anatomy and Physiology, Molecular and Advanced Placement Biology, Medical Genetics and Intro. to Clinical Medicine. She has completed the Quality Matters APPQMR workshop and is currently working in the office of Student Affairs and Admissions at Wright State University.
|
|
