Assignment #2- Critique of an Experimental Study

Bill Hill

CECS 5610

16 February 2001

Reference: Gokhale, A. A. (1996). Effectiveness of computer simulation for enhancing higher order thinking. Journal of Industrial Teacher Education, 33(4), 36-46.

Principle: The purpose of the study was to incorporate the characteristics of guided discovery in the use of computer simulation activities to explore the impact on the problem-solving ability of students.

Type of Design: The study used a nonequivalent control group design.


The independent variable in this study was the method of instruction, a variable with two categories: computer-simulation and lab. The dependent variable was the posttest score. The sample for this study included 32 students enrolled in two sections of an electronics course. Sixteen students were enrolled in each section. A course in basic electronics was a prerequisite for enrollment. One section was randomly assigned to the control group while the other section was assigned as the experimental group. A pretest was administered to all subjects prior to the treatment to assess student's prior knowledge and to test the initial equivalence between groups. A posttest was administered to measure treatment effects. The pre- and posttests were not identical but parallel forms of the same test. They were designed to measure student understanding of small-signal amplifiers and belonged to the cognitive domain. The author developed the instruments used in the study.

Factors Jeopardizing Internal Validity: Biases resulting in differential selection of respondent for the comparison groups may jeopardize the internal validity of the study. The instructors may have treated the group that used the computer simulation software differently.

Factors Jeopardizing External Validity: The small number of subjects and the short time period of the study (six-weeks).

Adequacy of Statistical Procedures Used: The level of significance (alpha) for the nonequivalent control group was set at 0.05. The meaningful difference or effect size was estimated at 0.8. Since the significance criterion, effect size, and sample size were known, standard tables were used to determine the statistical power of the test. It was found to be 0.75. The Type II error rate was 0.25. A t-test was conducted on the pretest scores and it was concluded that the treatment groups were similar. Using Bartlett's test, the data were also tested for the assumption of homogeneity of variance. It was found that the variances were not significantly different (F=0.31, p>0.05).

Briefly summarize Logic (Inductive and/or Deductive): The researchers concluded that the effective integration of computer simulation into traditional lecture-lab activities enhances the performance of students. Guided computer simulation activities can be used as an educational alternative to help motivate students into self-discovery and develop their reasoning skills.

Design Improvement: The design could be improved by having a larger test group and a longer period of observation time. The use of computer simulations in different subject areas could strengthen the findings.

Extension of the Study: The study could be expanded to evaluate the effects of computer simulation on enhancing the problem-solving ability of students with different achievement levels, using different academic subject material.