Research on the Effectiveness of Online Learning, Slides of History

Download Research on the Effectiveness of Online Learning and more Slides History in PDF only on Docsity!

Research on the Effectiveness of

Online Learning

A Compilation of Research on Online Learning

September 2011

Table of Contents

 - I. Learning Outcomes - II. Growth of Online Learning 

• III. Cost of Online Learning
  • IV. Impact on Instructional Design and Delivery......................

 Most of the variations in the way in which different studies implemented online learning did not affect student learning outcomes significantly.

 The effectiveness of online learning approaches appears quite broad across different content and learner types.

A further review of experimental and quasi-experimental studies that contrasted different types of online learning practices found the following:

 When a study contrasts blended and purely online conditions, student learning is usually comparable across the two.

 Elements such as video or online quizzes do not appear to influence the amount that students learn in online classes.

 Online learning can be enhanced by giving learners control of their interactions with media and prompting learner reflection.

 When groups of students are learning together online, support mechanisms such as guiding questions generally influence the way students interact, but not the amount they learn.

Shachar M., & Neumann, Y., (2010). Twenty Years of Research on the Academic Performance Differences Between Traditional and Distance Learning: Summative Meta- Analysis and Trend Examination, MERLOT Journal of Online Learning and Teaching , Vol. 6, No. 2. http://jolt.merlot.org/vol6no2/shachar_0610.pdf

―This meta-analysis research estimated and compared the differences between the academic performance of students enrolled in distance education courses, relative to those enrolled in traditional settings, as demonstrated by their final course grades/scores, within the last twenty year (1990-2009) period, further broken down to four distinct sub-periods.‖

A large k-125 of experimental and quasi-experimental studies met the established inclusion criteria for the meta-analysis (including data from over 20,000 participating students), and provided effect sizes, clearly demonstrating that in 70 percent of the cases, students taking courses by distance education outperformed their student counterparts in the traditionally instructed courses. By dividing the two-decade time span into four sub-studies, it was determined that the probability of DE [Distance Education] outperforming F2F [Face-to-Face] increased from 1991–2009 and authors predict that it will continue to increase in strength.

Criteria for including studies in the review included the time period covered, the quality of the study, the inclusion of a control or comparison group, and sufficient quantitative data for the two groups to be analyzed. Studies in English, German, French, Spanish and Italian languages were included.

Schachar, M., & Neumann, Y., (2003). Differences Between Traditional and Distance Education Academic Performances: A meta-analytic approach, The International Review of Research in Open and Distance Learning , Vol. 4, No.2. http://www.irrodl.org/index.php/irrodl/article/viewArticle/153/

This meta-analysis research estimated and compared the differences between the academic performances of students in distance education courses relative to those enrolled in traditional settings, as demonstrated by their final course grades/scores within the 1990–2002 period.

―Eighty-six experimental and quasi-experimental studies met the established inclusion criteria for the meta-analysis (including data from over 15,000 participating students), and provided effect sizes, clearly demonstrating that: 1) in two thirds of the cases, students taking courses by distance education outperformed their student counterparts enrolled in traditionally instructed courses; 2) the overall effect size d+ was calculated as 0.37 standard deviation units; and 3) this effect size of 0.37 indicates the mean percentile standing of the DE groups is at the 65th percentile of the traditional group (mean defined as the 50th^ percentile).‖

―Based on the 86 studies and using learning outcome data from over 15,000 participating students, the results of the meta-analysis show a strong positive trend indicating that DE is an effective form of instruction. This analysis demonstrates that students engaged in DE academically outperform their F2F counterparts. We have been focusing all along on the question: ‗Is DE suitable for all students?‘ The results of this study may raise the inverse question: ‗Is F2F suitable for all students?‘ and may begin a paradigm shift in the way postsecondary education is pedagogically conceptualized.‖

Neuhauser, C. (2002). Learning Style and Effectiveness of Online and Face-to-Face Instruction, The American Journal of Distance Education , 16(2). http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?nfpb=true&&ERICExtSearch_SearchVal ue_0=EJ656148&ERICExtSearch_SearchType_0=no&accno=EJ

In this study the investigator compared two sections of the same course—one section was online and asynchronous; the other was face-to-face—by examining gender, age, learning preferences and styles, media familiarity, effectiveness of tasks, course effectiveness, tests grades, and final grades. The two sections were taught by the same instructor and used the same instructional materials.

―The results revealed no significant differences in test scores, assignments, participation grades, and final grades, although the online group‘s averages were slightly higher. Ninety-six percent of the online students found the course to be either as effective or more effective to their learning than their typical face-to-face course. There were no significant differences between learning preferences and styles and grades in either group. The study showed that equivalent learning activities can be equally effective for online and face-to-face learners.

Swan, K. (2003). Learning Effectiveness Online: What the Research Tells Us, In J. Bourne & J. C. Moore (Eds) Elements of Quality Online Education, Practice and Direction , Sloan Center for Online Education, 13-45.

“Introduction: The goal, the raison d‘etre, the stuff of education is learning. Thus learning effectiveness must be the first measure by which online education is judged. If we can‘t learn as well online as we can in traditional classrooms, then online education itself is suspect, and other clearly critical issues, such as access, student and faculty satisfaction, and (dare we say it) cost effectiveness are largely irrelevant. Indeed, when online learning was first conceived and implemented, a majority of educators believed that it could never be as good as face-to-face learning. Many still do. In fact, however, we now have good and ample evidence that students generally learn as much online as they do in traditional classroom environments.‖

“No Significant Differences:” For example, Johnson, Aragon, Shaik and Plama-Rivas [2] compared the performance of students enrolled in an online graduate course with that of students taking the same course taught in a traditional classroom. Using a blind review process to judge the quality of major course projects, they found no significant differences between the two courses. The researchers further found that the distributions of course grades in the two courses were statistically equivalent.

Maki, Maki, Patterson and Whittaker [3], in a two-year quasi-experimental study of undergraduate students, found more learning as measured by content questions and better performance on examinations among students in the online sections of an introductory psychology course.

Fallah and Ubell [4] compared midterm exam scores between online and traditional students at Stevens Institute of Technology and found little or no difference in student outcomes.

Freeman and Capper [5] found no differences in learning outcomes between business students participating in role simulations either face-to-face or asynchronously over distance.

Similarly, Ben Arbaugh [6] compared the course grades of classroom-based and Internet-based MBA students and found no significant differences between them.

In a study of community health nursing students, Blackley and Curran-Smith [7] not only found that distant students were able to meet their course objectives as well as resident students, but that the distant students performed equivalently in the field.

Similarly, Nesler and Lettus [8] report higher ratings on clinical competence among nurses graduating from an online program than nurses who were traditionally prepared.

Likewise, in a review of distance education studies involving students in the military, Barry and Runyan [17] found no significant learning differences between resident and distant groups in any of the research they reviewed.

Most recently, Hiltz, Ahang and Turoff [18] reviewed nineteen empirical studies comparing the learning effectiveness of asynchronous online courses with that of equivalent face-to-face courses. Using objective measures of content learning as well as survey responses by faculty and students, the studies provide overwhelming evidence that ALN tends to be ―as effective or more effective than traditional course delivery.‖

2. Johnson S. D., Aragon, S.R. Shaik, N. & Palma-Rivas, N. (2000). Comparative analysis of learner satisfaction and learning outcomes in online and face-to-face learning environments. Journal of Interactive Learning Research , 11(1) 29-49.
3. Maki, R.H., Maki, W.S., Patterson, M., & Whittaker, P.D. (2000). Evaluation of a web-based introductory psychology course, Behavior Research Methods, Instruments, & Computers , 32, 230-239.
4. Fallah, M. H., & Ubell, R. (2000). Blind scores in a graduate test. Conventional compared with web-based outcomes. ALN Magazine , 4 (2).
5. Freeman, M.A. & Capper. J. M. (1999). Exploiting the web for education: An anonymous asynchronous role simulation. Australian Journal of Educational Technology , 15 (1), 95-116.
6. Arbaugh, J.B. (2000). Virtual Classroom Versus Physical Classroom: an exploratory study of class discussion patterns and student learning in an asynchronous Internet-based MBA course. Journal of Management Education , 24 (2), 213-233.
7. Blackley, J.A., & Curran-Smith, J. (1998). Teaching community health nursing by distance methods: development, process, and evaluation. Journal of Continuing Education for Nurses, 29 (4), 148-153.
8. Nesler, M.S., & Lettus, M. K. (1995). A follow-up study of external degree graduates from Florida. Paper presented at the 103rd^ Annual Convention of the American Psychological Association, New York, August.
9. Barry, M., & Runyan, G. (1995). A review of distance-learning studies in the U.S. military. The American Journal of Distance Education, 9 (3), 37-47.
10. Hiltz, R., Zhang, Y., Turoff, M. (2002). Studies of effectiveness of learning networks. Elements of Quality Online Education: Volume 3 in the SloanC Series. SCOLE.

Russell, T.L. (2001). The No Significant Difference Phenomenon : A Comparative Research Annotated Bibliography on Technology for Distance Education (IDECC, fifth edition). http://www.nosignificantdifference.org/search.asp

Thomas L. Russell‘s book is a fully-indexed, comprehensive research bibliography of research reports, summaries, and papers that document no significant difference (NSD) in student outcomes based on the mode of education delivery (face-to-face or at a distance). The book also

Growth of Online Learning

Research on the growth of online learning, as tracked by the annual Sloan Surveys and others, indicate that it is fast-paced and substantial. The explanations for the growth, based on survey research, are traced to the economic downturn, the increasing adult student population, and the growing belief among academic leaders that it is as good, or better, than face-to-face learning. The College of 2020 study, Chronicle Research Services , suggests that the growth of online learning is also in response to the new college student who is older, more technologically savvy, and in need of an accessible, low cost educational option. Research now includes more and more studies of the continuing growth of online learning within and across countries, making it a global phenomenon.

Allen, E., & Seaman, J. (2010). Class Difference$: Online Education in the United States. Retrieved from http://sloanconsortium.org/publications/survey/class_differences.

The 2010 Sloan Survey of Online Learning represents the eighth annual report on the state of online learning in U.S. higher education. The survey is designed, administered, and analyzed by the Babson Survey Research Group with support from Alfred. P. Sloan Foundation and data collection conducted in partnership with the College Board. The report is based on responses from more than 2,500 colleges and universities nationwide.

Online enrollments continue to grow at rates faster than overall higher education. Enrollments in online courses increased by 21 percent between 2009 and 2010, compared with an increase of two percent for campus enrollments. More than 5.6 million students were enrolled in at least one online course in fall 2009, an increase of nearly one million students over the previous year.

―This represents the largest ever year-to-year increase in the number of students studying online,‖ said study co-author Elaine Allen, co-director of the Babson Survey Research Group and Professor of Statistics & Entrepreneurship at Babson College.

Report highlights include:

 63 percent of institutions surveyed said that online learning was an essential part of their future strategy;

 Almost 30 percent of all enrollments now are in online courses;

 Nearly three-quarters of institutions report that the economic downturn has increased demand for online courses and programs;

 More than three-quarters of academic leaders at public institutions report that online is as good as or better than face-to-face instruction; and

 Reported year-to-year enrollment changes for fully online programs by discipline show most growing, but with a sizable portion seeing steady enrollments.

Bates, T. (2011). 2@11 Outlook for Online Learning and Distance Education. Contact North. Retrieved from http://search.contactnorth.ca/en/data/files/download/Jan2011/2011%20Outlook.pdf.

Internationally acclaimed author, educator, and research expert Dr. Tony Bates highlights major developments in the ongoing rapid growth of online and distance education during 2010, identifies a number of systemic barriers limiting the progress of distance learners, and describes key opportunities for 2011.

Major developments in 2010 included the expansion of enrollments in online courses by 21 percent compared with a 2 percent expansion of campus-based enrollments. Additionally, it is noted that while much of the expansion is occurring in the aggressive, for-profit sector of education, enrollment for online courses continues to exceed that of campus-based courses in state universities as well. Many online students come from segments of the population such as lifelong learners, new immigrants, and marginalized communities which are currently underserved by the state system.

Systemic barriers to online and distance education include faculty resistance and lack of sufficient training, limited institutional and instructional goals in this area, failure to adequately project and fund the cost of online learning, and lack of a comprehensive approach to accommodate student mobility and non-traditional learners.

Dr. Bates concludes his report by describing a number of timely opportunities for growth and development in online/distance learning. These include course redesign, applications to accommodate greater student mobility, expansion of open educational offerings, greater inclusion of multimedia materials in online courses, implementation of learning analytics to improve instruction, and growth in shared services as a means of cost-saving.

Green, K.C., & Wagner, E. (2011). Online Education: Where Is It Going? What Should Boards Know? Trusteeship Magazine , Association of Governing Boards of Universities and Colleges. Retrieved from http://agb.org/trusteeship/2011/1/online-education-where-it- going-what-should-boards-know.

New data from the fall 2010 Managing Online Education Survey, sponsored by two organizations, The Campus Computing Project and the WICHE Consortium for Educational Technology (WCET), point to robust growth in online programs at many institutions across the country. Fully half of the survey participants report that online enrollments at their institutions grew by more than 15 percent over the past three years, and two-fifths expect online enrollments to jump by more than another 15 percent over the next three years.

Trustees and campus officials who remember earlier excitement about the hopes for e-learning and the internet may well be skeptical, however ―higher education has learned from the mistakes it made in the past,‖ and ―today‘s campus conversations seem to reflect a new sense of purpose and pragmatism about the challenges as well as the opportunities of online education.‖ Several factors appear to be driving the increase in online enrollments according to the survey research.

 The advancement in learner-enabling technologies,

 Economic issues, specifically the need for new skills in a changing economy,

 Greater experience with online learning, and

 The opportunity to attract new students from underserved markets thereby providing new sources of revenue.

Drawing from data compiled from the recent two surveys, the authors identify ten key trends in the ongoing development of online education at the college and university level, including: rising online enrollments, experience in making online profitable, changing organizational structures to accommodate online learning, and focus on improved quality metrics to evaluate online learning. They follow with a list of issues for trustees to consider as they evaluate online learning, including: What will it cost? How will we assess quality? How do we support faculty and students? They conclude, ―The continuing conversation about quality involves more than simply comparing the performance of students in online and on-campus courses. Ultimately, it must focus on what students learn, not where they learn, and what types of learning environments, technologies, and resources foster student learning.‖

Ambient Insight Research. (2009). “The U.S. Market for Self-paced eLearning Products and Services: 2010-2015 Forecast and Analysis” in 2@11 Outlook for Online Learning and Distance Education, www.contactnorth.ca.

In this report, growth of online enrollment for 2009 is compared to predictions for 2014. In 2009, there were a total of 27.04 million students in higher education programs:

 1.25 million students took all of their classes online (4.6 percent)

 10.65 million students took some of their classes online (39 percent)

 15.14 million students took all of their courses in traditional classrooms (54 percent)

In 2014, there will be 27.34 million students in higher education programs in total (an increase of 2 percent over the five years):

 3.55 million students will take all of their classes online (12.8 percent)

 18.65 million students will take some of their classes online (68.2 percent)

 5.14 million students will take all of their courses in a physical classroom (19 percent)

The Ambient report is predicting more than 80 percent of all higher education students will be taking at least some of their courses online by 2014 compared with 44 percent in 2009.

Nagel, D. (2010). The Future of E-Learning is More Growth. Campus Technology****. Retrieved from http://campustechnology.com/articles/2010/03/03/the-future-of-e-learning- is-more-growth.aspx?sc_lang=en.

In this article Nagel asserts that, by virtually every measure, electronic learning is experiencing unprecedented growth and will continue to do so for the foreseeable future. ―The Worldwide Market for Self-paced eLearning Products and Services: 2009-2014 Forecast and Analysis‖ released by research firm Ambient Insight in March, 2010 bolstered previous research in this area, showing that electronic learning, by dollar volume, reached $27.1 billion in 2009 and predicted this figure will nearly double that by 2014, with academic institutions leading the way. Sam Adkins, chief research officer at Ambient Insight, explained that in North America—the largest region for electronic learning—the growth is being driven by academic institutions, both preK–12 and higher education (43 percent of all North American purchases). The dollar figures in the research included expenditures for packaged content, custom content development

 The adult market will be the fastest-growing one in higher education for the foreseeable future.

 The location of a college [and its recruiting area] will be the most significant factor in determining its flow of enrollees in the next decade.

 To continue to grow, public institutions will need to enroll and be able to serve less- wealthy and less-prepared students.

Over a hundred admissions officers responded to the survey and two-thirds indicated that, while 80 percent of their current students were full-time, less than half thought that would be true by

2020. Likewise, while two-thirds said 80 percent of their current students were 18–25, only half believed that would be true in 2020.

In response to these changing demographics, the authors maintain ―nothing is as likely to change the face of higher education over the next decade as the switch to more online learning.‖ It is clear that students increasingly want and need this option.

Responding to this need, the authors report, there is growth in online enrollment across almost all disciplines. There is a strong correlation between the size of a higher education institution and the average number of its online students, and the number of colleges offering online courses continues to grow.

Community colleges and for-profit institutions will likely continue to be the avenue of educational choice for reasons of cost and convenience unless traditional colleges and universities adjust their paradigms to maintain their position in an increasingly competitive market. ―Good teaching will always be at the core of a good university, but for most colleges, higher education will become a more retail-based industry than it ever has been. The students of the future will demand it. Many colleges have a long way to go before they can fulfill that demand.‖

Christensen C., Horn M., &, Caldera, L., (2011). Disrupting College : How Disruptive Innovation Can Deliver Quality and Affordability to Postsecondary Education. Retrieved from www.innosightinstitute.org.

Christensen and colleagues provide an analysis of what the speed of transition to online learning will be in higher education. They maintain that the widespread adoption of online learning will follow an S-curve, a predictable pattern of how a new disruptive innovation begins substituting for the old or existing model. They provide a chart that predicts the pace of substitution of fully online-delivered learning versus traditional instruction in American colleges and universities. ―It

shows that about 10 percent of students took at least one online course in 2002. That fraction grew to 25 percent in 2008; was 29 percent in 2009; and will be 50 percent in 2014… In other words, the online learning in college train has left the station.‖

In an additional case study of the North Carolina Community College System, the growth of online learning ―has been so rapid that online courses accounted for 37 percent of all courses taken in 2009, up from 2.4 percent 10 years earlier.‖ Using the pace of substitution rate associated with the S-curve, it ―indicates that 50 percent of all courses will be delivered online in the second semester of the 2010 school year, and 90 percent of all courses will be delivered online by 2010.‖

Bold, M., Chenoweth, L., Garimella, N. (2010). “Brics and Clicks,” Journal of Asynchronous Learning Networks , Vol 12 (1). http://www.thefreelibrary.com/Click+and+bricks-- transforming+education.-a

This paper describes the current role of distance learning in countries described as growing economies, and focuses on global developments in distance education using the BRIC nations (Brazil, Russia, India, and China) as examples. According to the authors, ―Distance learning is the new player in a global concern: the right to education.‖ Distance learning is the route to increasing access to education and has resulted in mega universities, open universities, branch campuses by foreign institutions operating in a host country or in partnership with a local university and cross-border education.

Two billion people worldwide can be identified as the potential market for distance learning for higher education globally; ―Online delivery may exceed US $69 billion by the year 2015.‖ Investment in infrastructure is key to providing access to distance learning and is greatly impacted by regulations and government within a country and across countries. Can nations with robust online networks serve others in online education delivery? They can, and the number of exporters of higher education is growing though the U.S. is still the leading exporter of education. Australian universities and the U.K. Open University are also major providers, and Chinese and Indian open universities are increasing in size.

The World Trade Organization has been in negotiations to name education as a marketable service, subject to import and export regulations, and proposals for higher education, including distance learning, are working through General Agreement on Trade and Services (GATS) proceedings. Online learning within the agreement is referred to as cross-border and includes all distance education and virtual universities - this is a growing market with great potential.

Cost of Online Learning

Research studies on the cost of online learning approach the subject in two ways: 1) comparisons to the cost of traditional classroom learning, and 2) projections of potential cost benefits if it is conducted in an optimum way. There is a growing body of research indicating that online learning can be delivered simultaneously to large numbers of students without increasing personnel costs while still achieving desired learning outcomes. Christensen and others argue, however, that a new business model for delivering instruction will be required if online learning is to become a sustainable source of new revenue. As online learning grows and becomes more systemic, studies aimed at determining its economic benefits are perhaps the most promising new area of research.

Meyer, K. PhD. (2008). If Higher Education is a Right, and Distance Education is the Answer, then Who Will Pay? Sloan Consortium , Journal of Asynchronous Learning Networks , Vol. 12 (1), February. http://www.distanceandaccesstoeducation.org/contents/JALN_v12n1_Meyer.pdf

This policy research addresses two questions: ―(1) can state governments in the United States afford to fund this [distance education] initiative, and (2) can public higher education institutions in the U.S. fund this effort through capitalizing on cost-efficiencies of online learning.‖

To address the question of whether states can pay to build online networks, Meyer reviews state appropriations to public higher education, projected state revenue, projected high school and young adult growth, and growing percentage of family income needed to pay tuition. The conclusion: ―Absent a major change in the economy, state tax structures and willingness to fund higher education, and the public‘s willingness to tax itself, the answer may be ‗no.‘‖

To address the second question, whether public institution can pay through efficiencies resulting from online learning, Meyer focuses on cost savings in online learning. Research on studies conducted through the National Center for Academic Transformation (NCAT), led by Carol Twigg provide evidence that courses can be redesigned to increase enrollment, lower cost, and improve learning. In her landmark study on course redesign, ―with funding from Pew Charitable Trusts, 30 institutions received grants to redesign courses accompanied by a solid plan to lower costs and document improved student learning. Those 30 institutions reduced costs an average of 37%, with some projects reducing costs by 15% and others by 77%, and generated a savings of $3.1 million per year in operating costs.‖

―If higher education institutions decide to grasp the potential of online learning and put the work into making it cost-efficient for themselves and beneficial for students, the promise of a steady revenue stream may help motivate and fuel the process.‖

Twigg, C. A. (2001). Innovations in Online Learning : Moving Beyond No Significant Difference. Retrieved from The Pew Learning and Technology Program. http://www.educause.edu/Resources/InnovationsinOnlineLearningMov/

Higher education is already making use of online learning, but, according to Twigg, the current paradigm of how to use technology in course design and delivery is limiting and prevents them ―from realizing the potential in a new application of technology.‖ Specifically, the author attempts to settle the question of whether design and delivery of online learning costs more or less than traditional classroom learning.

―Cost is directly related to that of access…it is very difficult for most existing institutions to expand access, whether on campus or online, without facing significant budget increases.‖ Higher education institutions need to concentrate on ―creating an efficient course development process and supporting that process with tools that increase efficiency. The [online] model is one in which large, up-front investments are made in single courses, using the best expertise possible in the development team, with the expectation that very large numbers of students will ultimately enroll.‖

The well-designed online model allows a ―small core of full-time faculty to set academic standards, oversee curriculum, establish academic policies including degree requirements, and so on. Part-time, adjunct faculty carry out the bulk of instruction.‖ These new courses can substantially ―reduce course-delivery costs by using technology to serve large numbers of students.‖

In Twigg‘s (2003) ―Improving Learning and Reducing Costs: New Models for Online Learning, Educause Review, 38(5), 28-38,‖ the new course designs developed during the eight million dollar study by faculty in 30 universities are described. The new designs served more students and resulted in a range of 37-75 percent savings. Ninety percent of the designs also reported improved learner outcomes. www.educause.edu/ir/library/pdf/erm0352.pdf

The National Center for Academic Transformation (NCAT), directed by Twigg, is a rich source of articles and monographs on redesigning courses to lower cost, improve learning and achieve scale. www.thencat.org