Melinda Smith-Woods

CECS 5610

February 9,2001

Assignment #2-Critique of Research Article

Reference: Seng Chee Tan. (2000). The Effects of Incorporating Concept Mapping into Computer Assisted Instruction. Journal of Educational Computing Research, 23 (2), 113-131.

Principle: This study investigates the effect of incorporating concept mapping into computer-assisted instruction.

Design: Ninety-one tenth grade students in the SAP school in Singapore were divided into three randomly assigned groups.

Group

Assignment

Experimental Variable

Observation

(1) Partial Map Group

R

X

O

(2) Complete Map Group

R

X

O

(3) Menu Selection Group

R

O

The differences were that

  1. Group 1 was given partial concept maps, which they completed after each CAI topic.
  2. Group 2 complete concept maps were provided and notes were taken during each CAI topic.
  3. Group 3 took notes during each CAI topic.

Students were observed and at the end of the four-day period, the Chemistry Achievement Test, post-test, was given. The test consisted of ten multiple-choice questions and five short-answer questions. Of these questions 60% were classified as high-level, and 40 % as low-level on the Bloom's Taxonomy of Educational Objectives. Students were also tested on their ability to create a concept map.

Factors jeopardizing internal validity: Much care was taken to eliminate any threats during the study. The known fact was that there was only one grader to rate the concept maps, which made it impossible to establish inter-rater reliability.

Factors jeopardizing external validity: The only external variable that I found was that these ninety-one students were enrolled in a school that only accepted the to 10% of students in the country. Seeing as how these students have higher IQ's than "normal" students, the selection process may have caused a skew in the results.

Adequacy of statistical procedures used: Student's mid-year chemistry results were user to assist in determining both the ANCOVA and the MANCOVA. The ANCOVA was used to test differences in mean scores among the groups and the MANCOVA was used to test the performance n the high and low level questions among the groups. In addition, Bonferroni's method of multiple pair-wise comparisons was also used.

Logic summary: The way the study was designed make is hard to say if the concept mapping truly improved achievement. The gaps between the three groups was so minimal, you cannot tell if the use of concept maps affected the students learning.

Design improvement: Finding participants whose educational backgrounds are more diverse could serve to truly represent a general population.

Extension of the study: The author suggests testing some of the computer-based concept mapping tools.