home | profile | degrees | publications | expertise | past courses | 3D OLE | Computer Games Inventory | Games and Simulations
Jones, J. G., Morales, C., and Knezek, G. (2004). 3D Graphical Multi-User Online Learning Environments for Internet-based Distributed Learning: First Year Results. Proceedings of the National Educational Computing Conference, New Orleans, LA.3D Graphical Multi-User Online Learning Environments for Internet-based Distributed Learning: First Year Results
James G JonesDepartment of Technology and Cognition, College of Education
University of North Texas, Denton, Texas, USA
gjones@unt.edu
Cesareo Morales
Institute for Integration of Technology into Teaching and Learning
University of North Texas
cmorales@coe.unt.edu
Gerald Knezek
Department of Technology and Cognition, College of Education
University of North Texas, Denton, Texas, USA
gknezek@tenet.edu
Abstract: The paper presents first year (baseline) research into using a 3D on-line learning environment for teaching Computers in Education, a course at the University of North Texas for pre-service teachers (students who are training to become teachers). The study looked at nine sections of the course during the fall semester of 2003. Three of the sections used the 3D online learning system while the remaining six sections took the course in the normal face-to-face manner. Several interesting patterns emerged during exploratory analysis of this data.
Introduction
A new generation of 3D online MOOs (multi-user object orientated online constructivist environments) is becoming available for Internet-based distributed learning. These new interfaces have strong ties to their text-based cousins of the early 1990s but now provide highly collaborative, immersive environments that promote interactions among students and with the instructor. Some on-line games boast subscriber populations that rival those of many actual North American cities (J. Whiting, 2002). As a result of this popularity, the number of systems capable of supporting on-line learning environments is increasing. As computer performance on low-cost personal computers increases, these types of systems (MOOs) allow teachers to provide students with unique on-line collaborative learning opportunities in the areas of language, science, computer graphics, and other fields (Jones, 2003). With the current emphasis on offering courses via web-based delivery systems, the importance of real-time face-to-face communications between course instructor and student and among the students is more apparent than ever. An on-line 3D virtual environment supporting text, audio, and overheads allows for immersive environments to be created so that the students and instructors can interact as if they were at the University.Research indicates that the environment being presented for course delivery may impact the attitudes and satisfaction of students taking on-line courses (Walker 2003; Y. Sogabe & M. R. Finley, 2003). Dissatisfaction with courses can have several aversive consequences, such as: students dropping out of a course, students not taking distance delivered courses in the future, low evaluation ratings for the instructor, or low evaluations for the program of which the course is a part (J.R. Hill & A. Raven, 2000; J.R. Hill, 2001). While web-based delivery has increased the number of students able to attend universities without having to commute, it lacks many key features present during in-person traditional course delivery. An on-line 3D MOO allows for immersive environments to be created in which the students and instructors interact as if they were at the University, while keeping Internet connectivity to a minimum for the student. It is important to evaluate and understand how immersive environments with better feedback mechanisms impact students and instructors.
The 3D online learning environment being used in the Department of Technology and Cognition at the University of North Texas provides the ability to create a context of communications among students and between the instructor using audio chat, text chat, and overhead/whiteboard presentations in as low bandwidth as a dialup Internet connection. The Created Realities Group (CRG) has developed an online 3D virtual collaboration environment that allows users on Windows, Mac, or Linux systems to interface on as low as a dialup Internet connection. Students at remote sites assume control of an Avatar (figurine) in a shared 'created environment' on their screens such as a school building or other space. Popular features of the system include integrated real-time voice (VOIP), distributed power-point presentations, and virtual-space segmented conversation areas so that learners can move the representations of themselves to areas for small group or private discussions. Figure 1 shows a screen capture of a class meeting being held online by the University of Hawaii using the CRG system.
Figure 1 - Screenshot from an online class being done by the University of Hawaii.
This paper presents our initial research looking at students using a 3D online learning environment during the delivery of a pre-service educational course funded by an internal grant at the University of North Texas.
The Research
During the fall semester of 2003 (September-December 2003), three sections of the University of North Texas CECS 4100 course (Computers in the Classroom) out of nine were chosen to use the 3D online learning environment for the delivery of the course. Six sections took the course in the traditional face-to-face method. Instructors and students participated in pre- and post- questionnaires and follow-up interviews for the classes in the current mode of offering as well as those in the 3D on-line learning environment mode. The CECS 4100 course focuses on computer topics that would be covered in introductory and secondary school courses with an emphasis on curriculum integration. This course has been delivered in a traditional classroom format on the University of North Texas (UNT) campus and at University of North Texas at Dallas campus for the past several years.How often the classes meet was very different between the treatment and control groups. The face-to-face courses (control) meet each week on campus for a total of at least 3 hours each week. The 3D online courses (treatment) were taught as a blended course. Blended courses combine face-to-face meetings with online communications. For this semester, the treatment groups met in person a total of 7 times throughout the semester and used the 3D online system for an additional 6 online meetings. The control group therefore had much more instructor contract time than the treatment group.
It should be noted that students participating in the treatment group experienced a number of technical problems related to both delivery and lab issues. The CRG software had been used for three semesters prior to the start of this course in other UNT courses and we felt comfortable that technical problems would not play a major factor. The 3D online software was installed and demonstrated during the first two in-person meetings of the UNT Dallas campus course. The users then took CDs home with the software and installed it on their home computers. 40% of the treatment group sent e-mails to the technical support person asking questions regarding the install. These problems were solved within the first two e-mail exchanges. Only two of the students were unable to get the software operational for the class. These two students found other systems to use for class meetings. The training sessions and introductory uses looked very good. Then the course encountered about 60 days of problems. This began with a problem related to the server supporting the 3D environment. Students could not stay connected to the server for extended periods of time because a combination of issues that took three weeks to resolve. The technical problems continued when it was discovered that the computer lab at the Dallas campus had upgraded the OS. This change kept the students using the lab from attending the online course until other arrangements could be made. The instructor during this period provided the materials as attachments in e-mails sent to the participants. These problems affected the treatment group's initial use of the 3D online learning environment. The traditional format sections had no remarkable problems in the delivery of the course.
Participants
107 undergraduate students at the University of North Texas taking the CECS 4100 course participated in the research. 14 students participated as the treatment group in the three sections being taught at the UNT Dallas campus. Due to a data collection issue, less than half of the potential students enrolled in the three sections of the treatment courses were available for analysis. As will be discussed later, this lack of data affected the overall research. 93 students participated in the control group in six sections being taught on UNT campus. The demographics for the sample showed a median of 21 years of age, although there were some differences in the composition of the group by gender analysis. The 14 participants in the treatment group were all females who ranged in age from 19 to 40 with a median age of 23, whereas the 93 participants in the control group ranged from 18 to 46 and were divided into 8 males with a median age of 25.5, and 85 females with an average age of 21.
There are differences between students taking classes on the main campus in Denton and classes offered at the UNT Dallas campus. Students in Dallas tend to be slightly older and typically hold down full time jobs or have other similar commitments. This can be seen when the students are asked how far away they are till graduation. The majority of students (83.9%) hold a bachelor's degree, and there is not a definite date for graduation as a cohort. Most of the students taking courses at the UNT Dallas campus were at least 2 years away from graduating. While students taking the course on the main campus reported that they would be graduating within the next year to year and a half. Most of the students (70.1%) plan to teach at the K-3 level, thus, it is not a surprise that most of them have Interdisciplinary (elementary school), as their area of specialty.
Another interesting difference between the two groups was that of home computers. Home computers owned by the participants in the treatment group were slightly older than those owned by the control group, based on the number of support questions answered by the technical support group. 94.4% (101 students) had computer at home, and 84.1% (90 students) had Internet at home. One student in the treatment group did not have either computer or Internet at home, whereas in the control group 5 students were in that condition, and an additional 9 students had computer, but no access to Internet at home. In the future, the demographics questions will be enhanced to better track this issue.
In terms of the number of hours spent using the computer at home and at school, although most of the students reported spending about the same average time in both settings (2-3 hours), the distribution of the time is different. The majority of students spent 2-3 hours, or more at home, whereas they spent 2-3 hours or less at school. Furthermore, there were differential tendencies between the two groups. 51.6% of the students in the control group reported using the computer in the classroom 2-3 hours a week, whereas 50% of the students in the treatment group reported zero use of the computer in the classroom.
Method and Data Collection
The primary instrument used was a collection of measures gathered in the publication ÒInstruments for Assessing Attitudes Toward Information TechnologyÓ, made available by the Institute for Integration of Technology into Teaching and Learning (Knezek, G., Christensen, R., and Miyashita, K. 2000, 2004). The battery of instruments measures attitudes, dispositions, and technology proficiency among teachers. The instruments have been developed and validated over the past ten years by researchers associated with the Institute for the Integration of Technology into Teaching and Learning. All have built upon the work of previous scholars in many states and nations with support from numerous agencies, including the Meadows Foundation, the Japan Society for the Promotion of Science, the Fulbright Foundation, and the Texas Center for Educational Technology.
Regarding reliability, TAC has shown reliability coefficients from .84 to .97 across the nine subscales (Knezek & Christensen, 2002:26). The validation of GP3 yielded an Alpha of .93 (Knezek, Christensen, Morales, & Overall, 2003). CBAM, Stages of Adoption, and ACOT are single-item measures, thus, no reliability coefficient is obtainable. Nevertheless, test-retest reliability for CBAM has been estimated to be between .72-.73, and for Stages between .80- .91 (Knezek & Christensen, 2001:35).
Table 1 shows final pre- and post- data collections that were available for analysis. Due to a problem with students not taking either the pre- and/or the post- tests the ability to perform the data analysis was limited. There was 35% attrition between pre- and post- tests with the outcome being that only 55.8% of the possible sample size was available to examine. The lack of matching tests creates unbalanced groups, which result in limited data analysis alternatives. Upon discussion with the instructors involved with the research, it was determined that the problem with missing pre- or post- testing can be corrected with better tracking of the students taking the tests. This problem will be addressed in the next phase of the research.
| Groups | Pre-Tests Taken | Post-Tests Taken | Matches Pre- Post- Tests |
| UNT 6 sections (control) |
160 | 112 | 93 |
| UNT Dallas 3 sections (treatment) |
32 | 17 | 14 |
| 192 | 129 | 107 |
Table 1 - Summary Information on Pre- Post- Tests available for data analysis.
Results
In order to account for possible differences between the treatment and control groups, the first step was to test the equivalence of the groups at the starting point on the measures administered. Pretest data were analyzed, and the results are shown on Table 2.None of the measures yielded statistically significant differences between the control and the treatment groups, which suggests equivalence of the two groups on adoption, attitudes, proficiency, and preparation regarding technology. Having equivalent conditions in the pretest, an anova for the posttest was performed. The results are shown on Table 3.
The results yielded three statistically significant differences: TAC - Accommodation, TAC - Concern, and TPSA - Teaching with Technology.
Table 2 is presented in PDF version of paper.
Table 2. ANOVA for different technology integration and technology beliefs measures administered as a pretest to 14 treatment and 93 control CECS 4100 UNT students.
Table 3 is presented in PDF version of paper.
Table 3. ANOVA for different technology integration and technology beliefs measures administered as a posttest to 14 treatment and 93 control CECS 4100 UNT students
Figure 2 shows analysis for the subscale Computer Interest. What is of interest is that the 3D online learning environment (treatment) tracked the face-to-face course delivery (control). The computer interest subscale of the control group and the treatment group decrease at the same rate between pre- and post- tests. For the computer interest sub-scale, this decrease between pre- and post- tests is a normal occurrence. The difference in starting levels between control (4.23) and treatment (4.12) is most likely a result of the different student group compositions that take classes at the main campus as compared to those who attend courses at the Dallas campus. Students who attend courses at the Dallas campus tend to be older, less experienced computer users who also maintain a job or household full time.
Figure 2 - Pre/Post comparisons on the computer interest subscale.
Figure 3 - Pre/Post comparisons of computer comfort (anxiety) subscale.
Figure 4 - Pre/Post comparisons of computer accommodation (avoidance) subscale.
Figure 5 - Pre / Post comparisons of computer concern subscale
Figure 6 - Pre / Post comparisons of teaching with technology subscale
Discussion
One unexpected, universal student behavior was encountered during our baseline study that we conjecture to be unique to the 3D online learning environment. Each time the system was first demonstrated remotely to a class, the students dispersed throughout the environment to explore (learn by play) and it was very difficult to collect them back again to complete the initial training in an online mode. Because of this, we found that completing the walk-through on-line was more difficult than during a structured classroom session. Since the initial trials, we have implemented a means to globally text-chat with everyone in an area ('make announcements') in order to be able to coordinate these types of introductory sessions. We hope this will make online training sessions more easily accomplished in the future.We also conjecture that this 'spontaneous exploration' feature of the 3D online learning environment can be harnessed to better implement underutilized forms of pedagogical practice and perhaps to create new forms of teaching/learning some day. The broad area of Discovery Learning (Bruner, 1956) and popular outside of school learning techniques such as orienteering (map-based navigation) and scavenger hunts are a few of the prospective methods that immediately come to mind. It would appear that such approaches could be implemented in a 3D online learning environment in such a way that Vygotsky's Social Context for Learning (Vygotsky, 1988) could be made available through virtual game-playing partners or groups of virtual peers. Virtual peers selected at the appropriate novice-to-expert or child-to-adult level, like chess partners at or near a player's own level of skill, might provide safe, low-cost, unique avenues for expanding the sphere of readiness referred to by Vygotsky as a learner's Zone of Proximal Development. These and other theory-based pedagogical frameworks are targeted for future research.
The selection of participants in the study was not optimal. We believe that the student populations attending each campus might be a significant factor on top of the 3D online learning environment that affected the outcomes seen in the treatment group. It is our goal in the future to ensure that we have control groups based on campus location so that we can watch this issue more closely. In the future we will need to ensure that we have at least one control group at each campus for the approach being studied so as to rule out factors related to the type of student that attends either campus.
The prolonged technical difficulties encountered at the start of the semester for the treatment group had to have an impact on student's post-tests. The technical difficulties might be enough by themselves to account for the decrease in outcomes on Accommodation (Fig 4), Computer Concern (Fig 5), and Teacher with Technology (Fig 6). Only additional research will allow us to answer this question.
Lessons Learned
The primary lessons learned during these pilot tests were in areas one might expect from new technology and doing instruction over the Internet. These lessons are relevant to any Internet-based technology:
- Be certain to have several test sessions scheduled before actual course use so that students and instructors can feel comfortable with the technology and work the bugs out of the system.
- For undergraduate classes, a reward system of some type (extra points) is needed to encourage the students to participate in the initial test sessions that were not directly related to course delivery. (We did not see a problem with the graduate students grasping the concept and participating early and often in the test sessions.)
- Planning ahead and having an alternate plan and a place to post information about what was happening before and after a session is very important. This lowered student anxiety if something unexpected happened.
- Being able to communicate to the students when there were problems such as the Internet connection on campus going down the night of two classes, an instructor missing a session because of a blackout in the Northeastern United States, or students having Internet problems could be dealt with in an efficient manner that did not distract from the course delivery.
Conclusion
While the presented research needs additional studies to provide more depth to issues discovered in this first round, we feel that these initial results show positive trends in several key issues for the use of 3D online learning environments for the delivery of college courses. The experience of this exploratory research is driving our future initiatives. The next phase of our research will be to structure the participating sections such that we have a clearer picture of changes within the treatment groups. We are very much interested in seeing where 3D online learning environments might fit between total asynchronous course delivery (web-pages) and face-to-face instruction. The course to be examined during the fall semester of 2004 will allow us to compare web-based, 3D online learning environment, and face-to-face delivery all located on the main campus. This study design should allow us to have a much clearer insight into what we saw during this initial research. Additional universities will also be using the 3D online learning environment in the coming months and data collection from those courses will be compared to this data and future data as it is collected in order to gain more insight.References
Bruner, J. S. (1961). The act of discovery. Harvard Educational Review, 31, 21-32.
Hill, J.R. & A. Raven. (2000). Online learning communities: If you build them, will they stay? Retrieved August 15, 2003, from http://it.coe.uga.edu/itforum/paper46/paper46.htm
Hill, J. R. (2001). Building community in web-based learning environments: Strategies and techniques. Retrieved September 20, 2002, from http://ausweb.scu.edu.au/aw01/papers/refereed/hill/paper.html.
IITTL. (2004). Instruments for assessing attitudes toward information technology (2nd Edition). Retrieved February 9, 2004 from http://www.iittl.unt.edu/.
Jones, J. G. (2003). Internet-based 3d graphical moo software that supports distributed learning for both sides of the digital divide. Paper presented at the World Conference on Educational Multimedia, Hypermedia & Telecommunications (ED-MEDIA), Honolulu, Hawaii USA.
Knezek, G., & Christensen, R. (2001). Equity and diversity in K-12 applications of information technology. KIDS project findings for 2000-2001. Denton, TX: University of North Texas. Institute for the Integration of Technology into Teaching and Learning.
Knezek, G., & Christensen, R. (2002). Technology, pedagogy, professional development and reading achievement. KIDS project findings for 2001-2002. Denton, TX: University of North Texas. Institute for the Integration of Technology into Teaching and Learning.
Knezek, G., Christensen, R., and Miyashita, K. (2000). Instruments for assessing attitudes toward information technology. Retrieved September 20, 2002, from http://www.iittl.unt.edu/IITTL/publications/studies2b/
Knezek, G., Christensen, R., Morales, C., & Overall, T. (2003). An instrument for self-appraisal of general preparation in technology for prospective teachers. Society for Information Technology and TeacherEducation International Conference,Vol. 2003 (1), 734-737.
Sogabe, Y. & Finley, M.R. (2002). Design of attractive virtual spaces for e-learning. Retrieved September 10, 2003, from http://charybdis.mit.csu.edu.au/~mantolov/CD/ICITA2002/papers/205-3.pdf
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
Walker, S. (2003). Distance education learning environments in higher education: Associations between the psychosocial environment and student attitude. Retrieved September 16, 2003, from http://education.ollusa.edu/mtt/presentations/SERA_2003/SERA_2003.pdf
Whiting, J. (2002). Online game economies get real. Retrieved December 19, 2003, from http://www.wired.com/news/games/0,2101,55982,00.html
