Assignment #2: Critique

Eric Fink

CECS 5610

February 14, 2001

Reference: Dewhurst, D. G., Williams, A. D. (1998). "An investigation of the potential for a computer-based tutorial program covering the cardiovascular system to replace traditional lectures," Computers and Education, 31, 301-317.

Principle: A comparative study evaluating the effectiveness of two interactive computer-based tutorial programs as an alternative to lecture for first year undergraduate students at a UK university.

Type of Design: Two Pretest-Posttest Control group design studies were done.

R O X O

R O O

The first study had a total of 33 students and the second study had a total of 39 students. The students from each study were divided into 2 groups randomly. The first group had topic 1 covered by a computer-based tutorial program and topic 2 covered by lecture. The second group had topic 1 covered by lecture and topic 2 covered by a computer-based tutorial program. Each group spent the same amount of time on each topic simultaneously. Each group was given pre and post tests for knowledge and attitude. All pretests showed no significant differences between the groups for the first study, but a difference in attitude between the groups in the second study.

Factors Jeopardizing Internal Validity: In the first study, too many students were eliminated out of the already small number of students. They were eliminated by either not attending class, not completing the computer program, or not completing the test. Also many questions were eliminated due to the material not being presented or certain questions biasing the computer-based tutorial program. Non-equivalent groups in the second study effected the results.

Factors Jeopardizing External Validity: A factor jeopardizing the external validity is that the participants in these studies are all college students. Students with learning disabilities, for example, may benefit more or less from computer-based tutorial programs. The quality of the lecturer could effect the external validity since some lecturers may be better or worse than others. The level of knowledge being tested was factual information and basic understanding. A higher level on Bloom's taxonomy may produce different results.

Adequacy of Statistical Procedures Used: Data was collected through pre and post multiple choice tests and a quantitative analysis of the results was done to determine any significant differences. Construct validity of the scales was assessed using principal component factor analysis and the reliabilities quoted are Cronbach's alpha coefficient. Two-tailed P-values were also used to examine differences in the 2 groups. The second study took what was wrong in the first study and changed time factors so fewer students were eliminated. Larger case studies would be helpful.

Briefly Summarize Logic: The researchers determined that a computer-based tutorial program is as effective as lecturing disseminating factual information and that students generally prefer lectures over computer-based tutorial programs as a mode of study. They also recognized that there are advantages and disadvantages to computer-based tutorial programs.

Design Improvements: This study would be much better with larger groups.

Extension of Study: This study should be expanded to diverse groups instead of just college students. Also testing on a higher level in Blooms taxonomy would make this study more externally valid.