Capítulo 13 - A comparative study of statistics courses: Fully Online, Blended and Face-to-Face Learning

Y. Steimberg, J. Ram, R. Nachmias y R. Israeli
Tel Aviv University, Israel

1.  BACKGROUND

The growing demand for fully online courses raises questions regarding the design of such courses and the methods of online instruction and assessment [1]. The design of a fully online course, with no face-to-face meetings, poses challenges in regard to conversion of content to digital media, planning of learning activities and assessment methods, and consolidation of ways for interaction among participants. The redesign of a face-to-face course into an online course can be described over a continuum of pedagogical and technological changes [2]. At the pedagogical level, the continuum is based on a migration from a traditional teaching and learning perception, in which the teacher is at the center as the main source of knowledge, to a perception in which the learner is at the center, taking advantage of a wide range of sources of information and actively, experimenting, interacting and collaborating with other students and with the course’s instructor. The technological continuum refers to the use of web based applications in supporting learning processes [3, 4].

In developing an online learning environment, a differentiation can be made among four teaching and learning dimensions, which encompass diverse pedagogical functions: content dimension, management, communication dimension and assessment [4]. Pedagogical change in each of these dimensions can be supported, to differing degrees, by the integration of various technological applications.

The need for redesigning statistics courses is derived from the claim that students cannot apply what they learn in the introductory courses to more advanced ones, as well as from the desire to incorporate technological applications into the teaching and learning processes [5, 6, 7]. Today’s widespread perception is that there should be a transition from teaching that emphasizes computations and problem solving on the basis of formulas, to teaching for better understanding of statistical concepts. Research has shown that active learning in statistics courses positively affected achievement [8, 9], motivation to learn, self confidence and attitude toward the field [10]. Together with the need for pedagogical change there is a need to effectively use technological applications to improve instruction of statistics [11, 12]. In the content dimension, technological applications such as applets and simulations can be used to present abstract ideas and analyze events, statistical calculators to shift the focus from computations to analysis, and dynamic data bases for the application of new concepts. In the communication dimension, discussion groups afford an opportunity for reflection, self-assessment and for practicing statistical language. The assessment dimension takes advantage of interactive thought-provoking questions for ongoing self-evaluation. The use of all these applications aims at creating a supportive learning environment that will structure the knowledge of the independent online learner from the stage of learning the basic terms in statistics through the stage of application and independent problem solving. However, research has shown that the mere prevalence of technological applications like discussion groups, applets or data bases in an online learning environment does not ensure that they will be used, and that their usefulness in learning is related to how they have been integrated [13, 14, 15].

Based on the literature and on our interest in assessing the effectiveness of a redesigned fully online statistics course, the following questions were raised: (a) Do fully online students differ in their achievements and attitude compared to blended learning students and face-to-face students? (b) Are the integrated technological applications in the redesigned fully online course useful to the online students?

2.  METHOD

At the end of the first semester, students studding in the course «Introduction to Statistics for Economists A» at Tel Aviv University, were offered the opportunity to study part B of the course, fully online. Volunteers were promised a five point bonus on their final grade (out of 100), provided that they would fill out a series of questionnaires distributed during the course. A total of 151 out of 270 students volunteered to participate in the online course. The volunteers were randomly assigned to three groups: An experiment group –a fully online group (fully online lectures and exercises)–; Two control groups –a blended learning group (face-to-face lectures in the classroom setting and online exercises)–; and a face-to-face group (lectures and exercises in the classroom setting). Comparison of final grades in the previous course «Introduction to Statistics for Economists A» between students who volunteered and those who did not volunteer showed no significant differences.

The fully online course «Introduction to Statistics for Economists B» was developed by a team of teachers from the Statistics department in cooperation and under the professional guidance of online learning experts. The main purpose was to create a dynamic learning environment which enabled self-assessment and interaction among students and between students and tutors. The course was constructed under a Learning Management System (LMS). Throughout course development, emphasis was put on integrating various technological applications to support the four course dimensions: content (use of interactive applets, interactive thought-provoking questions, self-practice with hidden solutions), management (use of weekly announcements, individual task schedule, personal tracking by web logs), communication (use of discussion groups, virtual counseling session, e-mail, students’ weekly feedback, student’s bimonthly reflective questionnaires) and assessment (use of interactive thought provoking questions, self-practice with hidden solutions, mandatory computer-based exercise, discussion groups).

To evaluate the effectiveness of the online courses, achievements and attitudes were examined by means of: (a) An identical final exam; (b) Online attitude questionnaires which included statements about the course in general; the content and learning activities; the tutoring; level of confidence in students’ understanding of the material, their opinion about the difficulty level of the course in comparison to other courses, and the estimated average time they invested in the course each week.

In order to evaluate the usefulness of the technological applications, both quantitative information on usage and qualitative information of students’ perceptions were examined. Data was collected by means of: (a) Web log files: Number of class views of course material in each study unit; Number of class views and number of class messages in each weekly discussion group and in each virtual counseling hour; Number of class views of self-practice exercise and mandatory computer-based exercise; (b) Open-ended reflective questionnaire, evaluating students’ perceptions regarding the usefulness of course material, discussion groups, virtual counseling hours, applets and exercises.

3.  FINDINGS

3.1.  EFFECTIVENESS OF THE ONLINE COURSE-ACHIEVEMENTS AND ATTITUDES

To evaluate the differences in achievements between the experiment group and the control groups, an analysis of variance was conducted to compare the grades of the final exam. No significant differences in final grades were found between the groups.

To evaluate attitude differences among students in the three groups, an analysis of variance comparing the average value of each attitude item in the online attitude questionnaire was conducted. No significant differences were found among the three groups in the following items: course content, exercises and tutoring, course in general and confidence level in understanding course material. Students from all groups were reasonably «satisfied» to «very satisfied» with the course, the lecture and the exercises. All students felt they had gained only  small to reasonable comprehension of the subject matter. Regarding the statement about the degree of difficulty, a significant difference was found between the two groups: students in the blended leaning course believed that the degree of difficulty was similar to other courses, whereas students in the face-to-face group believed that the course was difficult to very difficult compared with other courses (p < 0.01).

The students were also asked to indicate the number of weekly hours they had invested in learning. A significant difference was found between the fully online group and the face-to-face group. Students in the fully online group believed that they had invested fewer weekly hours studying course materials compared with the face-to-face group: 5.2 hours compared to 7.6 hours respectively (p < 0.01).

3.2. THE USEFULNESS OF THE TECHNOLOGICAL APPLICATIONS TO THE FULLY ONLINE STUDENTS

The second question in the study referred to usefulness of the technological applications incorporated into three of the pedagogic dimensions of the online course: content, communication and assessment. To obtain a picture of their usefulness, data was collected from web log files and open-ended reflective questionnaires. Results show:

1. Content Dimension (Use of course material and applets). Throughout the course, there was a decrease in the number of views of course material, with the most significant decrease in mid semester. Additional data about the rate of unit viewers shows that 80% of the students viewed each one of the study units throughout the course. However, as of mid course, they viewed each study unit fewer times. Moreover, a correlation was found between the number of views of course material and the grade in the course’s exam (r = 0.361; p < 0.001).
   The open-ended reflective questionnaires that evaluated students’ perceptions towards the quality of course material in the different units reveals that most students believe the study units are clear and contributed to their learning. Notwithstanding, they made several recommendations for improvement, like adding video clips of the course lectures, examples, and interactive questions.
   It appears from students’ reflective questionnaires that students’ perceptions of the applets’ usefulness are inconclusive. Most students opted not to use the applets during the learning process because they knew they would not be used in the final exam.
2. Communication dimension (Use of discussion groups). The degree of use of the discussion group decreased throughout the semester. The sharp decline in the use of the discussion group occurred at mid semester. A correlation was found between the number of messages written by a student in a discussion group and the grade in the final exam (r = 0.361; p < 0.001). From the responses to the reflective questionnaires, it emerges that the discussion groups were dropped for two reasons. The first relates to the usefulness of the information it provided, and the second has to do with the timing of the tutor’s response. Nevertheless, the discussion group which was established prior to the examination showed a great deal of traffic. The virtual counseling hour had almost no value at all. From students’ responses it is apparent that this application was not considered effective, either because the time of the synchronous meeting was inconvenient or because the tutors were readily available during the week, as several students commented.
3. Assessment dimension (Use of self-practice and weekly mandatory computer-based exercise). The degree of views on self-practice items also decreased sharply as of Unit 4. Nevertheless, the number of views of the mandatory computer-based weekly exercises was more or less consistent throughout the course.

4. DISCUSSION

The first purpose of this study was to evaluate the effectiveness of a redesigned fully online course by a controlled experiment comparing it to blended learning and face-to-face versions of the same course. The findings show that there was no significant difference among the groups regarding their achievement on the final exam. These findings are consistent with findings of some other similar studies [16, 17, 18, 19].

Both online and face-to-face students reported moderate to high satisfaction with the course content, the teaching and the homework practice. Nevertheless, there was significant variance between the groups regarding two aspects. The first had to do with the amount of time invested in the course: online students reported fewer invested hours than face-to-face students. It should be noted that other researchers [20] report opposite findings. Because students view spare time as an important factor when choosing their course load, it is vital to devise more objective tools in order to compare the amount of time invested in online and in face-to-face courses. The second aspect that showed significant differences between the groups was the degree of difficulty of the course. The students in the face-to-face course reported more difficulties compared with students in the blended learning course. This difference might be related to the fact that the blended learning course expands learning opportunities by providing both face-to-face and online learning environments.

An additional purpose of the study was to evaluate the usefulness of web based technological applications integrated into the redesigned fully online course. The findings show that students mostly used applications that were mandatory for learning (reading course materials and completing weekly mandatory tasks), but did not take advantage of other optional applications aimed at reflection and deepening the understanding of the subject matter. In the case of the applets, this seemed to be mainly because students knew that they would not be used on the final exam. Past studies have revealed a similar tendency in regards to the use of various help tools in computerized learning environments [21, 13]

Participation in the discussion group also decreased throughout the semester, although the students were encouraged in each unit to reflect and ask questions. The data show this is due to two main reasons. First, the questions raised by students were focused and did not elicit discussion. This fact is important for understanding the role of the tutor as the respondent in the discussion. It is recommended that the tutor respond to students’ questions in a manner that provokes thought and broadens understanding of the subject matter, so that the responses are beneficial to all students, not only to those who asked the specific questions. The second reason for poor participation in the discussion group relates to the timing of the tutors’ response, which attests to uncoordinated expectations between tutors and students. Students expected immediate responses and felt disappointment when they had to wait several hours. The tutors, on the other hand, attended the discussion group every day, but attempted to delay their responses in order to give the students room to engage in a discussion. Our data corroborates the findings of previous studies that a one-time disappointment with the discussion group led to dropping it entirely [22]. Another aspect that we believed would be valuable for the online student was the virtual counseling hour. The data shows, however, that it was not at all beneficial, either because of its timing, or because it did not provide significant added value beyond the ongoing debates in the discussion group.

Using an identical final exam to tap achievement in all the three groups facilitates a controlled comparison between study groups. However, at the same time, it limits the possibility to make substantive pedagogical changes in online courses. The online course had in fact two goals that were difficult to bridge. One goal aimed to significantly change the manner in which teaching and learning of statistics was conducted and the second goal was to assess the effectiveness of this change with tools that were used to assess traditional, face-to-face lectures. The need to demonstrate effectiveness based on a uniform exam significantly limited the opportunities for pedagogical change because the student’s main goal is to pass the test. Future studies must therefore find new ways to assess online learning, ways that reflect the new pedagogy, the specific nature of the learning and the goals of an online learning environment.

REFERENCES

1. HILTZ, S. R. and GOLDMAN, R. (Eds.) (2005): Learning Together Online: Research on Asynchronous Learning Networks. Mahwah, N.J.: Lawrence Erlbaum Associates.
2. LAW, N., CHOW, A. and YUEN, A.H.K.(2005): Methodological Approaches to Comparing Pedagogical Innovations Using Technology. Journal Education and Information Technologies, 10 (1).
3. DEDE, C. (1996): Evolution of Learning Devices: Smart Objects, Information Infrastructures, and Shared Synthetic Environments. The Future of Networking Technologies for Learning-U.S. Department of Education white paper.
4. DABBAGH, N. and KITSANTAS, A. (2005): Using Web-based Pedagogical Tools as Scaffolds for Self-regulated learning. Instructional Science, 33 (5-6).
5. GARFIELD, J. (2002): The Challenge of Developing Statistical Reasoning. Journal of Statistics Education, 10 (3). http://www.amstat.org/publications/jse/v10n3/garfield.html
6. RUMSEY, D.J. (2002): Statistical Literacy as a Goal for Introductory Statistics Courses. Journal of Statistics Education, 10 (3), http://www.amstat.org/publications/jse/v10n3/rumsey2.html
7. JOHNSON, H.D. and DASGUPTA, N. (2005): Traditional versus Non-traditional Teaching: Perspectives of Students in Introductory Statistics Classes. Journal of Statistics Education, 13 (2). http://www.amstat.org/publications/jse/v13n2/johnson.html
8. DELMAS, R.C. (2002): Statistical Literacy, Reasoning, and Learning: A Commentary. Journal of Statistics Education, 10 (3). http://www.amstat.org/publications/jse/v10n3/delmas_intro.html
9. HINDE, R. J. and KOVAC, J. (2001): Student Active Learning Methods in Physical Chemistry. Journal of Chemical Education, 78 (1): 93-99.
10. CHANCE, B. L. (1997): Experiences with Authentic Assessment Techniques in an Introductory Statistics Course. Journal of Statistics Education, 5 (3). http://www.amstat.org/publications/jse/v5n3/chance.html
11. MILLS, J. D. (2002): Using computer simulation methods to teach statistics: a review of the literature. Journal of Statistics Education, 10 (1). http://www.amstat.org/publications/jse/v10n1/mills.html
12. WOOD, M. (2005): The Role of Simulation Approaches in Statistics. Journal of   Statistics Education, 13 (3). http://www.amstat.org/publications/jse/v13n3/wood.html
13. DE  LIEVRE, B., DEPOVER, C. and DILLENBOURG, P.  (2006): The Relationship Between Tutoring Mode and Learners’ use of Help Tools, Distance Education. Instructional Science, 34 (2): 97-126.
14. NACHMIAS, R. and RAM, J. (2009): Research Insights from a Decade of Campus-wide Implementation of Web-supported Academic Instruction at Tel Aviv University. The International Review of Research in Open and Distance Learning, (IRRODL), 10 (2).
15. SHEMLA, A. and NACHMIAS, R. (2007): Current State of Web-Supported Courses at Tel Aviv University. International Journal on E-Learning. 6 (2), 235-264.
16. STEPHENSON, W. R. (2001): Statistics at a Distance. Journal of Statistics Education, 9(3). http://www.amstat.org/publications/jse/v9n3/stephenson.html
17. TWIGG, C. (2003): Improving Learning and Reducing Costs: Lessons Learned from Round III of the Pew Grant Program in Course Redesign. http://www.thencat.org/PCR/R3Lessons.html
18. UTTS, J., SOMMER, B., ACREDOLO, C., MAHER, M. W. and MATTHEWS, H. R. (2003): A Study Comparing Traditional and Hybrid Internet-Based Instruction in Introductory Statistics Classes. Journal of Statistics Education, 11 (3). http://www.amstat.org/publications/jse/v11n3/utts.html
19. WARD, B. (2004): The Best of Both Worlds: A Hybrid Statistics Course. Journal of Statistics Education, 12 (3). http://www.amstat.org/publications/jse/v12n3/ward.html
20. YOUNG, A. and NORGARD, C. (2006): Assessing the quality of online courses from the students’ perspective. The Internet and Higher Education, 9 (2): 107-115.
21. STEIMBERG, Y., RAM, J., NACHMIAS, R. and ESHEL, A. (2006): An Online Discussion for Supporting Students in Preparation for a Test, Journal of Asynchronous Learning JALN, 10 (4).
22. KARSENTI, T. (2006): Effects of different tutoring modalities in online learning environments. In Reeves, T. and Yamashita, S. (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Chesapeake, VA: AACE, 1.816-1.821.