Language Learning & Technology
Vol. 7, No. 3, September 2003, pp. 119-140
ONLINE LEARNING: PATTERNS OF ENGAGEMENT AND INTERACTION AMONG IN-SERVICE TEACHERS
Paginated PDF version
Faridah Pawan, Trena M. Paulus, Senom Yalcin, and Ching-Fen Chang
Language teacher education programs attempt to foster collaboration amongst pre-service and in-service teachers. The approach is in place in an online teacher education program in a Midwestern university where the current study was undertaken. Collaborative interactions are an essential element of any pedagogy which assumes that good learning is collaborative and that understanding comes through modeling, participation in, and reaction to the behaviors and thoughts of others.
This study was conducted with the following objectives: (a) to analyze the patterns and types of collaborative interactions taking place in three online classes; and (b) to use these findings as a guide in the design of instructional interventions. Our goal is to understand the practice of collaborative teaching and learning so that assistance can be provided to support instructor efforts to include collaborative interactions in their courses. We used Garrison, Anderson, and Archer's (2001) "practical inquiry" model as a framework for the study. Without instructors' explicit guidance and "teaching presence," students were found to engage primarily in "serial monologues." Based on the findings, we propose three intervention strategies that may help instructors increase collaborative interactions in online discussions.
INTRODUCTION AND PURPOSE
For the past 9 years, the language teacher education department of a large midwestern American university has offered distance education courses, primarily at the graduate level. The main clientele of this online program are in-service teachers. Some participants in online courses in the program have expressed interest in being future online educators themselves. Thus, online courses, such as those discussed in this paper, not only have the responsibility of providing quality teacher education, but also of modeling effective approaches to online instruction.
With the rise of Internet technologies, distance courses have migrated to the Web where it is possible to incorporate online discussion as a primary component of the course. One of the issues most troubling to online instructors has been the prevalence of "serial monologues" (Henri, 1991) in asynchronous discussion forums. Serial monologues are discussions in which participants share past teaching experiences and freely express their opinions with minimal effort made to connect to the contributions of others. In this study, we subscribe to Chickering's and Ehrmann's (1996) position that good learning is collaborative and social rather than isolated and competitive. We also accept Bandura's (1971) "social learning theory" which states that understanding comes through modeling, participation in, and reaction to the behaviors and thoughts of others.
While Internet technologies can enable greater synchronous and asynchronous collaboration among distance learners, there is still a lack of clarity of what online collaboration is or should be and a lack of knowledge on how to structure and engage in it. An additional challenge to effective collaboration in online courses is that the intended outcomes of collaboration have not been clearly articulated by research and/or experienced in practice. Collaborative interactions, although much touted as a means to effective, deep, and reflective learning online (Hara, Bonk, & Angeli, 2000; Hathorn & Ingram, 2002; Henri, 1992; Henri & Rigault, 1996) leave many instructors and students insecure at best and, at worst, reluctant to engage fully. Thus, we undertook this study with two objectives in mind. First, we wanted to analyze the patterns and types of interactions taking place in online classes. Second, we wanted to use these findings to guide us in the design of instructional interventions that could increase collaboration in these courses.
Language teacher education programs are beginning to foster a pedagogy of collaboration in pre-service and in-service teacher education programs (Kaufman & Brooks, 1996). This research study focuses on the types of collaborative learning through peer-to-peer interactions that take place in online instruction in three in-service language teacher education classes. Although meaningful peer-to-peer interaction is of importance to online instruction, relatively few studies have focused on the peer-to-peer structure of discourse in online forums in a way that produces valid, replicable data on the types and quality of interactions taking place.
Henri (1992) established a widely-used coding scheme to determine whether online interactions are surface-level or deep, and whether the interactions were social, interactive, cognitive, or metacognitive in nature. Hara, Bonk, and Angeli (2000) used this coding scheme to analyze asynchronous discussions used to supplement a residential graduate level course. They found that 70% of student postings reflected deep cognitive processing. Newman, Johnson, Webb, and Cochrane (1997), using a coding scheme based on Henri (1992) and Garrison (1992), used an experimental design to compare evidence of critical thinking found in undergraduate face to face groups with asynchronous groups. They found similar levels of critical thinking in both groups, but the computer conferencing group exhibited greater depth. These results seem promising for the value of online discussions, though neither of these studies was conducted in an entirely online graduate level course.
Conflicting findings are reported by other researchers who studied online instruction. Gunawardena, Lowe, and Anderson (1997) created an interaction analysis framework to examine the "social construction of knowledge in collaborative learning environments facilitated by computer conferencing" (p. 397) from themes that emerged from the analysis of transcripts of a large group asynchronous listserv debate. They discovered that the large group asynchronous listserv debate consisted almost exclusively of sharing and comparing of information. Kanuka and Anderson (1998) applied this model to study asynchronous professional development interactions and also found that information was shared rather than knowledge constructed.
Garrison, Anderson, and Archer (2001) built on this previous work to create the "practical inquiry" model (see Figure 1) that serves as the theoretical framework for this study. Their model was created specifically to assess outcomes of collaboration in a higher education online course environment.
Figure 1. Practical inquiry model (Garrison, Anderson, & Archer, 2001)
The practical inquiry model reflects four phases of critical thinking and cognitive presence: (a) the initiation phase with a triggering event that begins the dialogue about a particular issue; (b) the exploration phase in which learners move between private reflection and social exploration, exchanging information about the issue at hand; (c) the integration phase in which participants begin to "construct meanings" or solutions to the issue from the ideas explored in the previous phase; and (d) the resolution phase in which the proposed solution is "vicariously tested" (Garrison et al., 2001, p. 11).
The Garrison et al. (2001) model was selected as the theoretical framework for this study because it was developed specifically for the context we wished to examine: a computer conference held as part of a semester-long graduate level course, facilitated by both the instructor and student-moderators. In addition, the practical inquiry model accurately reflects the educational goal of our online courses: developing critical thinking skills. Thus, the practical inquiry model provided us with a tool to investigate and understand collaboration and cognitive presence. Cognitive presence is defined by Garrison et al. (2001), citing Dewey (1959) and Lipman (1991), as occurring within the framework of a critical community of inquiry in which participants "(re)construct experience and knowledge through the critical analysis of subject matter, questioning, and challenging assumptions" (p. 7).
When applied to whole-class asynchronous discussions Garrison et al. (2001) found, as did Gunawardena et al. (1997), mostly triggering events and exploration, but little in terms of integration or resolution. They analyzed only 24 messages in one course, however, leaving more room for research in this area.
This study examined online discussions from three online graduate-level language teacher education courses offered by the school of education at a large, midwestern U.S. university. Data in the study were analyzed qualitatively. Analyses were undertaken by means of descriptive statistics, content analysis (Woods, Priest, & Roberts, 2002) and the emerging design approach (Lincoln & Guba, 1986). The categories of analysis or the master codes (Woods et al., 2001) for the content analysis were derived from the Practical Inquiry Model by Garrison et al. (2001).
The following research questions are addressed in this study:
1) Are the interaction patterns evident in the online discussions characterized by one-way serial monologues or by two-way peer-to-peer interaction?
2) Does the online discourse result in the outcomes expected from collaborative learning, that is, all phases of the practical inquiry cycle?
3) What instructional factors may affect the level and type of collaborative discourse among students in online discussion forums?
4) Based on those instructional factors, what pedagogical strategies can be implemented to enhance collaborative learning outcomes in online discussions?
Table 1 describes the three courses used in this study. Literature-Based Instruction, taught by a doctoral student, had 13 students enrolled. Teaching Critical Reading Skills, taught by an adjunct faculty member, had 11 students. Twelve students were enrolled in Technology in Language Teaching, taught by an assistant professor. The faculty members had previous experience teaching online courses, while this was the first online course taught by the doctoral student.
Table 1. Description of Three Online Courses
Week of semester
Topic of discussion
discussion of story "Celebrating the Hero"; instructor initiated discussion with a short posting
discussion of professional articles; instructor initiated with a short posting
Critical Reading Skills
course management environment
students posted papers on "argumentation"; were asked to give each other feedback
students posted papers on "empathy"; were asked to give each other feedback
Technology in Language Teaching
discussion of "authentic audience"; the discussion starter (a student) posted three questions
discussion of textbook chapters; the starter (a student) posted four questions
Two courses used a threaded asynchronous discussion tool and one used a comprehensive course management tool. The course management tool included asynchronous threaded discussions as well as chat tools and an internal e-mail system. The format of the discussions in Critical Reading Skills and Literature-Based Instruction used a free and open discussion format with no specific structure. The format of Technology in Language Teaching utilized a student "starter/wrapper" technique, where students were responsible for initiating the discussions and synthesizing it at the end of the week. Discussion topics are described in Table 1.
Two weeks of discourse from each class were analyzed, comprising 160 messages.1 The discussion weeks were randomly selected, and they ranged from weeks 5 to 16 of the semester. To assure the confidentiality of the participants in the data, before the research began, all of the participant names were replaced with pseudonyms.
To answer the first research question on the prevalence of serial monologues, descriptive statistics of participation levels were calculated. These included the number of postings contributed each week by participants. Interaction maps (see, e.g., Howell-Richardson & Mellar, 1996) were also created to indicate the flow of the discussions and the direction of the postings (see Figures 2 to 7). Interaction maps provide a visual representation of the frequency of individual participation, discussion thread development, and whether discussions are one way (replying to a previous message) or two way (replying to the reply.) For these, the unit of analysis was the complete posted message.
To answer the second research question (the evidence of outcomes expected from collaborative learning), messages were coded according to the practical inquiry model. Garrison et al. (2001) used the complete posted messages as the unit of analysis. They followed a "code up" or "code down" procedure when a message reflected more than one phase of the model. However, a message-level unit did not fit this research study, because some messages contained several themes addressing different issues or questions raised during the discussions. For example, in the Technology in Language Teaching course, the discussion prompt included four questions. Some participants attempted to address all these questions within one message.
Therefore, we decided that the speech segment was the most useful unit of analysis for the coding according to the practical inquiry model . According to Henri and Rigault (1996), a speech segment is defined as "the smallest unit of delivery linked to a single theme, directed at the same addressee (all, individual, subgroup), identified by a single type (illocutionary act), having a single function (focus)" (p. 62). In this example from the Technology in Language Teaching Course, the following message by Meigi was segmented into two speech segments, each with a separate theme and focus:
I agree with you that students' levels decide whether students are concerned about their audience. My young EFL beginners don't know much about English (or don't know at all), so I wonder how I can lead them to consider their audience. Therefore, I believe the more advanced students' levels are, the more concerned they will be.
A quick but stupid question ... Why do you think students would feel more pressure as the audience becomes more authentic? For me, it's almost opposite. When I know who my audience will be, I feel more secure since I know how formal the discourse will be.
Two coders simulteanously analyzed the data according to the practical inquiry model. Coders first distinguished between on-task and off-task units. On-task units are defined as those related to the discussion at hand, such as answering a question raised by the discussion starter. Off-task units may involve small talk or asking questions about the technology being used for the discussion. For example, this post by Rebecca from the Literature-Based Instruction course was coded as off-task:Sorry guys, I couldn't find the book anywhere! I finally ordered it through ILL from the local library, but they haven't received it yet. So I haven't read the book. Hopefully, after Thanksgiving, I can participate in a discussion...are we supposed to relate this discussion to Unit 13-14, or simply discuss the book in general?
The on-task units were then coded according to the practical inquiry model. The four phases of the model are as follows (see also Table 2):
Phase 1. Triggering Events -- The Posing of Issues, Dilemmas, or Problems
Units coded as Phase 1 are often questions or take the conversation in another direction. In this example, Douglas from the Technology in Language Teaching course posts this starter question:Healey presents a table on page p.393 entitled "Settings for Autonomous Learning" that shows the relationship between content and structure in learning settings. Share your thoughts about the chart and tell whether it accurately reflects the setting in your classroom. Where does the setting in your classroom fall on this chart? What factors decided where you plotted your classroom? What about this class Technology in Language Teaching?
Phase 2. Exploration -- Engagement in Brainstorming, Questioning, and Exchange of Information
Phase 2 furthers the discussion from the triggering event to the sharing and comparing of information. This post by Berth in the Literature Based Instruction course is an example of Phase 2. Here Berth shares information from a reading and an example from her own teaching experience.I have enjoyed the Unit 6 readings by Burk, Literature in an Interdisciplinary Unit, and Probst, Reader Response Theory and the English Curriculum. In addition, I've found other articles which helped me a great deal with this unit. One of the articles, Text sets in the adult developmental reading classroom: Expanding literacy through diverse readings, is extremely relevant to my work with college freshmen, and will influence changes in my curriculum. Having said that, I am very concerned about the time necessary to develop thematic units and/or reader responses as described by Burk and Probst. I want to be creative, and I want to reach my students, but I simply need to know how to find the time to make these changes.
Phase 3. Integration -- Construction of Meaning From the Ideas Generated in the Exploratory Phase
In Phase 3 the discussion moves from sharing new information to making connections between ideas shared and creating a synthesis of new undertandings. When Phase 3 occurred it was toward the end of the discussion. An example can be seen from Matthew's post in the Technology in Language Teaching course:This week's discussion was mainly centered around autonomy in the context of not only this class but also the other classes that we alll have been a part of. Most of us put our Technology in Language Teaching class into the Programed Learning cell but as Sally points out, the chart is "too narrow". Take a closer look and you can see that the only difference between cell A and B is who controls the sequence of the class. If you think about this class and the varying levels of autonomy that we have for each assignment, I think it is obvious that the chart is "too narrow" and the level of autonomy is dependent on more factors than those listed in the table on page 393.
Phase 4. Resolution -- Finding, Testing, and Implementing a Solution to Problems Presented in the Triggering Phase
In Phase 4, the synthesis or new understanding reached in Phase 3 is vicariously tested or applied to the "real world." No examples of Phase 4 were found in the data.
Both coders analyzed all of the data to establish interrater reliability (Bauer, 2000). The two coders participated in a training session with the researchers and then independently coded a portion of the data. They came together to compare their results and to discuss any disagreements. Inter-rater agreement was 89%. Some modification to the original model was necessary because of differences between the speech segment unit of analysis and the whole message units of analysis used by Garrison et al. (2001). Due to the speech segment unit of analysis two subcategories "divergence within a single message" and "convergence within a single message" were no longer relevant and thus were dropped. We also added the parenthetical explanation of "tentative solutions" to distinguish the two "convergence" indicators in the model (see Table 2, Phases 3, 3.1, & 3.2). Coders then reanalyzed all data based on the refined coding scheme. Table 2 reflects the coding scheme used for the study. Final inter-rater agreement was 94%.
Table 2. Practical Inquiry Model (adapted from Garrison et al., 2001)
Trigger events (evocative)
1.1 Recognizing the problem
1.1.1 Presenting background information that culminates in a question
1.2 Sense of puzzlement
1.2.1 Asking questions
1.2.2 Messages that take discussion in new direction
2.1 Divergence -- within the online Community
2.1.1 Unsubstantiated contradiction of previous ideas
2.2 Information exchange
2.2.1 Personal narratives/descriptions/facts (not used as evidence to support a conclusion)
2.3 Suggestions for consideration
2.3.1 Author explicitly characterizes message as exploration; e.g., "Does that seem about right? Or "am I way off the mark?"
2.4.1 Adds to established points but does not systematically defend/justify/develop addition
2.5 Leaps to conclusion
2.5.1 Offers unsupported opinions
3.1.1 Reference to previous message followed by substantiated agreement; e.g., "I agree because…"
3.1.2 Building on, adding to others' ideas
3.2 Convergence (tentative solutions)
3.2.1 Justified, developed, defensible, yet tentative hypotheses
3.3 Connecting ideas, synthesis
3.3.1 Integrating information from various sources -- textbook, articles, personal experience
3.4 Creating solutions
3.4.1 Explicit characterization of message as a solution by participant
4.1 Vicarious application to real word
4.2 Testing solutions
4.3 Defending solutions
To answer the third research question, discussion transcripts were read for emergent themes. This question pertains to instructional factors affecting collaborative discourse and the implementation of pedagogical strategies that could enhance collaborative learning. The researchers used a qualitative "emerging design" approach (Lincoln & Guba, 1986). This method of inquiry and data analysis enabled the researchers to uncover themes that emerged from the data, without predetermining what the themes should be. Specifically in our research, in using this context-bound approach, we identified recurring themes that seem to affect and shape interactive behavior in online discussions. For example, in the classes we analyzed, instructor participation was uneven and in one case, an instructor did not post throughout the entirety of a discussion week. Based on this observation, we identified "teaching presence" (Garrison et al., 2001) in terms of instructor overt facilitation, as an instructional theme that needed to be addressed.
Table 3 provides an overview of participation levels in each class. In Literature-Based Instruction, 3 out of the 13 students enrolled in the class contributed to the discussion during week 13/14 and 9 contributed during weeks 15/16. The instructor posted three times total during these discussions. The average number of posts per students was 3.7 in the fifteenth week and 1.6 the sixteenth week. Students ranged in their number of postings from one to four times during these discussion weeks.
In Critical Reading Skills, all 11 students participated both weeks. While the instructor posted seven times in the eighth week, she did not post in the sixth week. The average number of posts per student was 3.9 (week 6) and 4.5 (week 8), based on a total 43 and 57 posts during those weeks. Posts per student ranged from one to 11.
In the Technology in Language Teaching discussions, all 12 students participated during week 5, but only nine participated in week 9. The instructor contributed three messages during the two weeks analyzed. Thirty-two messages total were posted during the two weeks, with the average number of posts per students 1.4 in week 5 and 1.3 in week 9. Students ranged in their number of contributions from one to four posts.
Table 3. Participation by Course
# of student participants out of total # enrolled
Total # of posts
# of instructor posts
Average # of posts per student
Range of posts per student
3 of 13
9 of 13
Critical Reading Skills
11 of 11
11 of 11
Technology in Lang. Teaching
12 of 12
9 of 12
Phases of the Practical Inquiry Model
Table 4. Phases of Practical Inquiry Model by Course and Week
Critical Reading Skills
Technology in Language Teaching
We will now examine the findings of the three courses in more detail, followed by suggestions about instructional factors that may have contributed to the outcomes.
Most, but not all students, participated at some level in the discussions although there was uneven participation by students with some contributing more posts than others, and overall low instructor participation. Similar to the findings of Garrison et al. (2001) and Gunawardena et al. (1997), we found that the discussions centered on Phase 2 (Exploration; 66%) , with little Phase 3 (Integration; 11%) and no Phase 4 (Resolution) at all. Out of the 229 units, over half (152) consisted of Phase 2 (Exploration), which includes personal narratives, descriptions, and facts not used as evidence to defend, justify, or develop the participants' own positions or that of others. In other words, the participants were primarily sharing information and brainstorming their own ideas in relation to the issues, problems, and questions posed by instructors or the assigned student discussion facilitator.
Participants did not, however, build upon ideas suggested by others. There were only 25 out of 229 units that were categorized as integrative. Thus only eleven percent of the units revealed attempts on the part of our participants to pull together information shared by their peers to construct and negotiate new meanings from the discussions. For the most part, the participants engaged in reflections about their own teaching experiences in relation to the initial prompts.
It is notable that no units were coded as the subcategory of divergence within Phase 3 (Integration). This reveals that students engage primarily in sharing their thoughts and exchanging information rather than challenging and questioning each others points of view. Our findings confirm those of Curtis and Lawson (2001): While students may at times refer to the comments of others (one way interaction), they do not engage in argument/counter-argument types of discussions or in the "challenge and explain" cycles of engagement that may lead to a resolution and new understanding. Some researchers may also argue that the patterns of interactions in our findings fall under the ideals of constructivism which demand the active engagement of learners in critical discussion and inquiry (see Bonk & Cunningham, 1998). Additional follow-up interactions between students (two-way interactions) are needed for greater critical thinking and extension of learning.
To find out how we might encourage greater collaborative discourse, we must take a closer look at what happened in each of the online courses. The next three sections provide a detailed look at what we found in each of the three classes in the study, namely, the Literature-Based Instruction, Critical Reading Skills and Technology in Language Teaching classes.From this we may derive possible instructional factors and pedagogical strategies that contribute to more collaborative interactions, to be investigated in a future research study.
Findings in Literature-Based Instruction
Of the three courses, the Literature-based Instruction course had the fewest number of units and had the most off-topics units. A majority of the units fell into Phase 1 (Triggering) and Phase 2 (Exploration). In fact, this course had the highest percentage of Phase 1 (Triggering) messages of the three courses. For example, in this example Rebecca triggers additional discussion on the topic of teacher responsibility:I would like to hear what other people think about this subject. Does anyone else feel frustrated at how much responsibility teachers have now? It seems to me that there's a lot of talk about teacher liability -- a negative word if I ever heard one -- but not much talk about parental liability.
The high number of triggering units show that the discussion goes in many directions based on the interests of the students.
In week 13-14 (see Figure 2), the discussion was about a story "Celebrating the Hero" that the students had read. The instructor posted the discussion prompt five weeks before the discussion actually took place. The on prompt discussion began by a student posting an initial reflection on the book (3). Soon, however, the discussion quickly fell into two threads. One thread consisted of students discussing the content of the book (9, 10, 11); the other concerned questions/problems of locating a copy of the book (4, 5, 6, 7, 8). This thread accounts for the high number of off-task posts in this discusssion. The instructor did not post again until the end of discussion, when she responded to a student who had still not found the book.
Figure 2. Literature-Based Instruction, week 13/14
NOTE: (square/rectangle box) is the initial prompt; (one-way arrow) indicates direction of the posting; "(on)" is an on-task posting; "(off)" is an off-task posting; and the numbers indicate the chronological order of the posts
Figure 3. Literature-based Instruction, week 15/16
Unlike the discussion unit in weeks 13-14 where there were many off-topic posts, the discussion in weeks 15-16 stayed on the scheduled topic of the reading assignment. Figure 3 illustrates how the discussions were initiated and discussed by Rebecca (10), Lucia (7), Ann (4), and Berth's (3) posts. In the example below we see how a discussion develops, often iteratively, from Phase 2 back to Phase 1, where Ken and Rebecca reply to other postings and follow by sharing a new viewpoint, and then into Phase 3 when Berth begins to make connections among various ideas:
Rebecca [Phase 2]: Like so many others, I too enjoyed Probst's article. I began rethinking my own teaching to see if I am spending too much time analyzing stories instead of making them alive, and I think I am. However, we shouldn't drop analytical skills or literary terms altogether. I think a lot of teachers make the mistake of NOT teaching literary terms and basic facts about writing. ... [back to Phase 1] Kids should be learning to love to read at an early age. We teachers try to take on the responsibility of "teaching" kids to love to read. I feel frustrated that we are taking so much responsibility on ourselves. The cold hard fact is that ultimately, with 99% of the students we get, they love to read because their parents love to read! Wouldn't it be great if all parents could realize the importance of sitting down with their child and a book for thirty minutes a day? If that were the case, it would be very easy to teach kids about literature.
Ken [Phase 2]: I totally agree with what Rebecca is saying about parental liability... [back to Phase 1] But unfortunately today a lot of parents can't be held accountable to teach their kids much of anything. A lot of the responsibility does end up falling on our shoulders due to this...Berth [Phase 3]: I agree with Rebecca and Ken regarding the dismal failure of many parents to take an active role in developing reading skills in children. In fact, we've seen learning toys developed to fill the void created by parents who are unwilling or unable to interact with their children. One of my pet peeves at the time was the Teddy Ruxpin (spelling ?) bear that would read stories to children … At any rate, as Rebecca and Ken have implied, we can't remedy the damage done to young lives by parents. What I think we can do is to help these children, or in my case, young adult students take personal responsibility for their own learning. I think that students must develop an intrinsic desire for learning...
During both discussions the instructor posted the prompt but did not play an active role. During the two unit discussions, the instructor posted only three times: One was an off-topic question, and the others were initiating prompts of an instructive manner, such as "Please add your thoughts/comments re: the professional articles here". The instructor in this course appears to have adopted a traditional role where the teacher is an authoritative and rather distant participant in the course. Overall, her role can be described as a triggering role because all of her posts were coded as Phase 1 (Trigger). While the instructor did not seem to play a significant role in guiding the discussion, some discussions did move from Phase 2 to Phase 3, showing that it is possible for meaningful discussion to take place without the presence of the instructor.
Findings in Critical Reading Skills
The Critical Reading Skills course had the most discussion units, likely because the task asked everyone to first post their assignments and then to make brief comments on each other's posts. While there was more discussion in this course, it remained at Phase 2. Often students provided general comments, such as "I liked your ideas for use in the classroom" or "You have described an interesting book." In this example Jenny's post from Week 6 is coded as Phase 2:You gave some excellent examples. I've also seen intimidation tried by men (in the Army) and I've seen women stand up to them. It's also interesting to watch some women in the Army who try to act like the stereotypical male, cursing, and talking loud to win the argument.
Jenny does acknowledge and affirm the previous posting, and then follows up with her own experience. However, since no new conclusions or syntheses are made related to the topic being discussed, it is not coded as Phase 3. Henry's post from Week 8 is also coded as Phase 2:I can relate to your students lack of interest in a character education program. I can remember such programs when I went to school and we just laughed at them. It was looked upon as just another scheme in which adults were telling us to be good. The scenarios were often unrealistic and we didnt relate to the content.
Henry also first acknowledges the previous post and follows with his own experience, but stops short of making a meaningful connection with the previous post. These posts are one-way because they do respond to a previous post but do not quite extend the previous idea in an integrative way. Rather, they share additional experiences and information that is related to the topic but not creating new meaning.
The Critical Reading Skills instructor participated in the week 8 discussions, but not at all during week 6, reflecting uneven involvement with the discussion. This indicates randomness of instructor involvement, at least in these two weeks of discussion. In the week in which the instructor participated actively, there was little difference between her feedback and the students' feedback to each other. The findings show that, similar to the majority of her students' postings in those two weeks (83% and 87%, respectively), those of the instructor are at Phase 2 Exploratory.
The interaction maps (see Figures 4 and 5) for this course indicate that the students and the instructor have one-way interaction, mostly in a commenting mode rather than one that is inquiry-oriented. For example, in the comments below, the instructor acknowledges the students' contributions but does not challenge the students' positions or statements.
Literature is often a great tool to help teach students empathy. If the student can step into a character's shoes and see the situation from his view-point....
Thank you for providing us with URLs and activities we can use to help teach empathy.
The structure of the course does promote discussion in terms of quantity, with each student posting their initial assignment and other students responding. However, the types of responses by both students and the instructor did not seem to promote the desired quality of cognitive presence, in which students and instructors "construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry" (Garrison et al., 2001, p. 11). Rather, the responses were primarily acknowledgements and affirmations that any contribution at all was made to the discussion.
Figure 4. Critical Reading Skills, week 6
NOTE: (square/rectangle box) is the initial prompt; (one-way arrow) indicates direction of the posting; "(on)" is an on-task posting; "(off)" is an off-task posting; and the numbers and letters indicate the chronological order of posts in response to the initial prompt
Figure 5. Critical Reading Skills, week 8
NOTE: (square/rectangle box and number) is the initial prompt; (one-way arrow) indicates direction of the posting; "(on)" is an on-task posting; "(off)" is an off-task posting; and the numbers and letters indicate the chronological order of posts in response to the initial prompt
Findings in Technology in Language Teaching
The Technology in Language Teaching course had the greatest number of Phase 3 Integration units. As explained previously, Phase 3 goes beyond information sharing to making connections and synthesizing previous ideas. In week 5, this post from Colin was coded as Phase 3:I would like to follow up on Nancy's comments regarding audience and register and relate this to Johnston's observation that written and spoken forms are converging in e-mail communication (p. 62). Nancy points out that in Japanese, "social considerations of the relative status of your audience (is) key." This is certainly true in Japan where one's age, social position, etc., influences how you address others. However, many of my students are under the impression that "Westerners" are less concerned with status and that they prefer to communicate in a direct manner. (I don't really think this is true--at least not to the extent that my students do.) What I sometimes see as a result of this thinking is overly direct or casual interaction when more indirect or formal language would have been more appropriate. I think of times when my intermediate-level students have asked me "I need you to write a letter of recommendation for me" or "Can you check this letter for me." instead of "Would you mind..." etc. (Although I wasn't really offended, someone unfamiliar with language learners could well have been.)
In the example above, Colin not only responds to Nancy's previous message but integrates it with the class reading and tests it against own own experiences as well. Matthew's post in week 9 was also coded as Phase 3:As for the table on Settings for Autonomous Learning I am in the same boat as Nancy and Rita. None of the classes that I teach or have taken part in fit into one of the cells. I think this is mainly do to the fact that as a teacher/learner all of my experience has been centered around Japaese concepts of learning. As Healy points-out on page 392, "self-direction and autonomous learning are Western concepts that fit smoothly into U.S. culture in particular." I highly agree with her on this point. I think that a higher level of autonomy is more appropriate for students in ESL, rather than EFL, becasue they have already immersed themselves in an independent and student centerd culture.
Here Matthew connects with both Nancy and Rita, his own experience, and the class readings.
The Technology in Language Teaching course also exhibited the most structured discussions of the three courses. A discussion starter and wrapper role was assigned to a student each week. These students had the responsibility of asking questions that challenge, connect, and extend information in the postings. Examples of postings by a starter (Douglas) and a wrapper (Meiqi) who undertaking their respective responsibilities are as follows:
Starter: I'm sorry to put so many questions up this week, but having three chapters does give us a lot to discuss. I'll give you some control in deciding which of these areas the discussion focuses on. Healey presents a table on page p. 393 entitled "Settings for Autonomous Learning" that shows the relationship between content and structure in learning settings. Share your thoughts about the chart and tell whether it accurately reflects the setting in your classroom. Where does the setting in your classroom fall on this chart? What factors decided where you plotted your classroom? What about this Technology in Language Teaching class?
Wrapper: As for the differences between online discourse and face-to-face communication, all of us agree that online communications lack of cues such as gesture, facial expression, tone, body language etc. However, there won't be problems of accents, pronunciation. Moreover, online discourse provides a relatively stress free environment for language learners. Of course, people don't need to worry about geographic locations via online communication -- just like us. :) Many people think compared to face-to-face discourse online communications are more casual/informal (or less stressful -- at least not for me.) Pressure, anxiety, losing security and the decrease in the level of control are possible problems aroused by authentic audience.
The discussions in the Technology in Language Teaching course mainly followed the questions initiated by the starter (see interaction maps below, Figures 6 and 7). The use of the starter and wrapper roles gave more structure to the discussion, with some interaction going beyond initial responses to the starter. However, since there were many questions posted at the same time by the starter, students attempted to address all the questions and the result was at times fairly long messages.
Figure 6. Technology in Language Teaching, week 5
Figure 7. Technology in Language Teaching, week 9
NOTE: The rectangular box () refers to the initial prompt; the one way arrow () indicates the direction of posting; and numbers indicate the chronological order of posts
Discussions were more focused in this course, perhaps due to the anchoring starter questions. The use of student discussion starters may create a learner-centered feel to the discussion, casting the instructor in less of an authoritative light. The wrapper role tried to push discussions toward integration, during which students would construct new meanings and/or solutions to ideas shared in the Exploration (Phase 2) phase. The use of the wrapper role may account for the higher number of units coded as Phase 3: Integration in this course. However, since only one student is responsible for wrapping discussions each week, and there is no expectation for anyone else to actively participate in the wrapping discussion, often the wrappers merely summarized rather than integrated ideas from the discussions.
Much as with the other two courses, the number of instructor postings was not very high in this class (2 out of 19 in week 5 and 1 out of 13 in week 9). However, the nature of the instructor involvement in the class differed from the other courses. Two instructor units were coded as Phase 2 Exploration while the other was in Phase 3 Integration -- the only instructor post to be coded at this phase in any of the three courses. This Phase 3 Integration post clarified and elaborated a point made by a specific student, giving text references and making an argument:
Hi everyone,Just responding the thread on learner autonomy. (I playing devil's advocate to my own opinions here). The following are bits and pieces I pulled from a reading by Mark Peterson at http://www.jaist.ac.jp/~mark/articlenetworking.html
In the discussion Peterson outlined some views held by critics who feel that instead of promoting autonomy, the computer-mediated communication, the internet and computer software can result to the reverse, namely:
1) Information overload can lead to technostress. Learners can be overwhelmed by the large quantities of text produced in the networked learning environment, leading to: "feelings of dislocation and discomfort" (Moran,1991:540).
2) The conventions that govern conventional communication are radically redrawn in the electronic medium. The lack of paralinguistic cues (Cooper and Selfe,1990:853), can result in "contextual deprivation" (Feenberg, 1993): a lack of information on how to respond appropriately may inhibit the learner interaction that is central to network activity.3) Critics concede that computer conferencing generates more text, however they remain sceptical that on-line communication is truly interactive (Moran,1991:55): "A......cause of our unease with the situation of the on-line conversation is that the medium encourages the production of text but discourages its reception". Moran (1991) has compared on-line discourse to a form of monologue.Is this happening to our Technology in Language Teaching class or the classes we teach? Douglas, if I am going beyond the focus of your questions, just pull me back, ok.
This contribution by the instructor responded to the overall discussion, referring to the text with the purpose of challenging the students. The instructor plays the role of clarifier, challenger, and elaborator, perhaps modeling for students through the use of outside references to support her claims and by guiding the discussion, in short, participating in a manner that the other students can then emulate.
There are several limitations to this study. The first set of limitations relates to the practical inquiry model that we were using. In the model, there was no way to code off-task posts. In the original Garrison et al. (2001) study, this meant that 33% of the posted messages in the study were not coded. In our study, only 11% of units were off-task. Nevertheless, that meant that we were unable to analyze those off-task messages. A more comprehensive coding scheme may provide a better picture of participant interactions.
Another example of a missing coding category relates to units that seem to qualify for movement from Phase 2 Exploration to Phase 3 Integration of the model (see Table 2). First, the units must refer to a previously posted message; and second, they should systematically and substantially develop an argument by use of evidence. The model, however, does not provide a category for posts that systematically and substantially develop an argument without referring to a previous message. This is certainly possible to do, and the absence of a category to describe it led to difficulty in coding. Another missing category is that of substantiated contradiction of previous ideas in addition to unsubstantiated contradiction of previous ideas (category 2.1).
Another difficulty in using the practical inquiry model was distinguishing between very similar indicators in the subcategories, which resulted in a lack of interrater reliability in the subcategories of each Phase. For example, in Phase 2 Exploration the information exchange (category 2.2) and brainstorming (category 2.4) indicators are difficult to distinguish. When an indicator in the practical inquiry model was not fully explained, precise coding of units was compromised. For example, the indicator of convergence (category 3.1.1) is defined as "reference to previous messages followed by substantiated agreement, e.g. 'I agree because…'" However, what follows "because" can easily be personal opinions, relegating the unit back to Phase 2 Exploration.
The second set of limitations stems from our analysis of discussion transcripts over a short period of time. We included only 2 weeks of discussions from each class. The use of transcripts also limited the amount of information that we had available to understand collaborative interactions in discussions. Future research could include information gathered from additional sources such as interviews with students and instructors to provide insight into additional factors responsible for collaborative interactions, or the lack, of them in discussions. For example, through our own informal conversations with our online students, we know that most of them are professionals working full time while they take online courses. As a consequence, they may be too busy to read and reflect on the voluminous textual information in the discussion forum. Additionally, informal discussions on emails and chats with our students have revealed that at times engagement in an asynchronous online environment can be solipsistic in nature in that there are no visceral pressures or invitations to engage in the discussion of others. The online discussion medium can be an easy invitation for participants to take the path of the traditional "lecture" or "soliloquy" mode in their engagement, especially when pressed for time. Interviews and surveys might also show that the in-service teachers who make up a majority of the population in our classes may enjoy sharing the wealth of experience they have in teaching, and the asynchronous medium can be an open invitation for them to do so without interruption.
Finally, research on online reading patterns might demonstrate that the organizational linearity in asynchronous discussions is an uninviting environment for dialogical or dialectic engagements. Adding on to the sequence of thoughts could be physically easier to do than engaging in a back and forth discourse and pulling apart of specific parts of texts posted in the asynchronous discussion forum that we use. Thus, conversations, interviews, personal preference surveys, reading pattern research and other research approaches might have given us a clearer understading of discussions in the online forums.
Instructional Factors and Pedagogical Interventions
One-way interaction (serial monologues) and Phase 2 Exploration posts were most common in these online discussions. Students were mostly engaging in the presentation of positions rather than in inquiry (Morgan, 2000). On the whole, there were few dialogues or critical and reflective exchanges, that is, few elements Morgan identifies as components of arguments that can bring about a "dynamic process of inquiry."
Our findings are in line with those of McKenzie and Murphy (2000). Using Henri's (1992 ) framework to evaluate the effectiveness of online discussion, they analyzed 157 messages shared by participants in a Graduate Certificate in Higher Education course offered to academic staff in Monash University in 1999. They found that although 74% of the message units could be identified as interactive contributions whereby participants were directly engaging with messages from their counterparts, "two thirds of these were commentaries rather than responses to questions" (McKenzie & Murphy, Interactivity section, para. 1). In other words, students were sharing their individual thoughts about their peers' contributions rather than taking up a line of inquiry presented by their peers.
Garrison et al. (2001) says, "often students will be more comfortable remaining in a continuous exploration mode; therefore teaching presence is essential in moving the process to more-advanced stages of critical thinking and cognitive development" (p. 10). They also state that the integration phase in the process "requires active teaching presence to diagnose misconceptions, to provide probing questions, comments, and additional information in an effort to ensure continuing development, and to model the critical thinking process" (p. 10). As seen above, more of this is needed in the course discussions. Based on our findings, we suggest three instructional interventions as follows.
1. Structure Classroom Discussions
The first recommended intervention is for instructors to clearly outline participation requirements in the course syllabus. The participation requirements could include deadlines for initial posts, required responses to others by certain dates, modeling, and description of the length contributions to the discussion.
The first two suggestions emerge from the observation that many of the initial prompts for discussions were not posted in a timely way. In fact, quite a few (including several postings from an instructor) were posted all at one time in some of the discussions and this fact may have contributed to the “serial monologue” effect. Students may not have felt that they were engaged in interactive discussions but rather in a routine that required one way responses to pre-existing prompts.
We also recommend instructor modeling of the length of postings. The prevailing presence of long messages (300 words or more) may demonstrate that students do not feel that they are in a discussion mode but rather in a presentation mode. Longer messages did not necessarily result in higher quality and frequently did not incorporate important issues raised by others or in the readings. In addition, in contrast to shorter messages, there were fewer replies to extremely long messages. Thus, issues of quantity and quality can be addressed by the instructor by including discussion expectations and guidelines in the course syllabus.
2. Demonstrate Overt Instructor Facilitation and Leadership Role
Uneven instructor participation in the discussions was clearly evident. The total number of units posted by the instructors was only 12 of the 229 units. In agreement with what was reported by Garrison et al. (2001), our findings demonstrate the importance of "teaching presence" in that overt facilitation by instructors plays a critical role in guiding students toward higher levels of learning (Anderson, Rourke, Garrison & Archer, 2001, p. 3). However, even in the Technology in Language Teaching course where the instructor attempted to model critical thinking and questioning, there was minimal effect on students who on the whole, continued to brainstorm and explore rather than integrate. The instructor's role seems similar in many ways to a physical classroom where a teacher pulls back from his/her leadership role. In such a situation, students' contributions were "more like a series of declarations" (Connolly & Smith, 2000, p. 19) that are independent of each other and did not converge to push the discussion toward integration and synthesis.
Our proposed intervention is for the instructors to frame their questions within the contexts of integration and resolution, that is in Phase 3 and Phase 4 of the practical inquiry model. If the objective of the class is collaborative learning whereby students engage in the integration and synthesis of ideas, the instructors should ask questions addressing those issues in their facilitation of discussions. In addition, instructors should be expected to participate regularly throughout the discussion and play a more visible role in guiding students toward the achievement of those learning objectives. They must make their "grand design" (Anderson et al., 2001) for the course apparent and clear throughout the discussions and in this way assert significant "teaching presence" (in contrast to teacher presence).
Anderson et al. (2001) also suggest assigning specific discussion roles to establish teaching presence in a more student-centered manner, such as those utilized in the Technology in Language Teaching class. These roles can focus the discussion and give students authority and responsibility but students require training and modeling by instructors before they can assume the roles in an effective manner.
3. Require Students to Self-Code Responses
We recommend that instructors ask students to self-code the discussion roles and types of postings they include in their discussions. This meta-cognitive strategy is based on the theory that self awareness of the purpose and outcomes of collaborative interaction has educational value (Duffy, Dueber, & Hawley, 1998). The strategy would encourage students to keep track of and to think about how their responses relate to the collaborative learning objectives set by their instructors. Self-coding their own roles and responses may raise students' awareness, for example, of the four cyclical categories of the practical inquiry model. Through these strategies, students remain in charge of their interactive behavior in discussions and that may help them find purpose and a sense of investment in what they post.
The instructors' role here is to again play an explicit role in modeling for students the self-coding and role playing procedures. Including an explanation of the different types of roles and the postings each role would require could be helpful. An example of the explanation is as follows:
Explorer: The role of the "explorer" is to "explore" ideas on the discussion topic. These ideas could include, for example, brainstorming ideas, suggestions for consideration, and personal experiences you have had related to the topic. You can draw on all sorts of resources that shed light on the topic (previous readings, other courses, etc.) The ultimate goal of the explorer is to fully investigate the possibilities of the topic in discussion.
Integrator: The integrator's role is to try to "integrate" the ideas that have been presented in the discussion. The integration may engage in developing ideas that have been presented, finding common ground among diverse ideas related in the discussion, or synthesizing important elements of the discussion. The ultimate goal of the integrator is to draw conclusions from ideas presented earlier by the other participants.
Trigger: At times you may want to ask questions or make a point that takes the discussion to another direction. This is fine and good for stimulating further discussion. Please label your post as "trigger" if you are posting for this purpose.
Applicator: At the end of each discussion it is important to figure out how our ideas can be practically applied to our teaching. The applicator will take our integrated ideas and show how they could be applied to a specific (even hypothetical) teaching situation. This "application" of our conclusions can help us see clearly the relevance of our "theoretical discussions" to our teaching.
These three interventions could improve collaboration between students in a way that creates a meaningful environment for learning in text-based discussions in asynchronous discussion forums. For the students in the class who are teachers themselves, such collaboration can be a step toward fostering long term engagement between professionals that is genuine and purposeful.
Discussions do not automatically become interactive and collaborative simply by virtue of being in an anytime/anywhere asynchronous medium. We found that while some discussions did stay on task, interactions were often one-way serial monologues. Discussions tended to explore the issues in an interesting way, but without integration or resolution of the ideas raised. No evidence of challenging ideas was found. Our findings suggest that a lack of student awareness of the purposes and structure of the discussions as well as the absence of overt facilitation by instructors may have contributed to these outcomes.
At the same time, this study demonstrates the utility of the practical inquiry model, with modifications, in studying collaborative interactions in online discussion forums. It also suggests some practical interventions to enhance collaborative engagements and thus collaborative learning in online instruction. We expect that our proposed interventions will result in more engaged and interactive discussions that not only explore, but also integrate and find solutions to the complex issues of language teaching that arise in our online classes.
There are many questions that remain unanswered, leaving many areas for further investigation. For example, since online discussions are completely text-based, are we expecting more critical thinking from discussions that take place in the forums than we do from verbal classroom discussions in residential classrooms? Are classrooms, either onsite or online, primarily venues for information sharing rather than places for knowledge synthesis and integration? Do the latter phases of knowledge construction take place while writing, for example, research papers rather than during class discussions? Are online instructors similar to onsite instructors in that they see their role in the classroom (virtual or physical) as primarily of a nurture/explorer rather than a challenger or a provocateur? We will continue to pursue these questions and more as we delve further into teaching at a distance. In our department, nearly half of our graduate students take online classes and thus we have an ongoinginterest in finding the best ways to facilitate learning for our students near and far.
ABOUT THE AUTHORS
Faridah Pawan is an assistant professor in the Language Education Department at Indiana University, Bloomington. She is director of the Interdisciplinary Collaborative Program (ICP) which offers online/onsite joint professional development opportunities for ESL and content area teachers. She is also the director of the ESL/EFL Program via Distance.
Trena M. Paulus is a doctoral candidate in instructional systems technology at Indiana University. She holds an MA in applied linguistics from Ohio University and has taught EFL/ESL in Lesotho, Ohio, and Michigan. Her research areas are professional development, technology-enhanced language teaching, and online collaboration in distance education courses.
Senom Yalcin is a doctoral candidate in language education at Indiana University, Bloomington, with a minor in instructional systems technology. Her primary interests include CALL, online learning, and instructional design. She is currently working on her dissertation on collaboration in online courses and teaching the CALL course online at IUB.
Ching-Fen Chang is a doctoral candidate in language education at Indiana University, Bloomington. Her research interests are exploring issues of computer-mediated communication in ESL/EFL education. She is currently working on her doctoral dissertation on non-native English speaking students' experiences in face-to-face and online discussions.
Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. JALN, 5(2), 2-17. Retrieved April 29, 2003, from http://www.sloan-c.org/publications/jaln/v5n2/v5n2_anderson.asp
Bandura, A. (1971). Social learning theory. New York: General Learning Press.
Bauer, M. (2000). Classical content analysis: A review. In M. Bauer & G. Gaskell (Eds.), Qualitative researching with text, image and sound (pp. 131-151). London: Sage Publications.
Bonk, C. J., & Cunningham, D. J. (1998). Searching for learner-centered, constructivist, and sociocultural components of collaborative educational learning tools. In C. J. Bonk & K. S. King (Eds.), Electronic collaborators (pp. 33-34). Mahwah, NJ: Lawrence Earlbaum Associates, Inc.
Chickering, A. W., & Ehrmann, S. C. (1996, October). Implementing the seven principles: Technology as lever. AAHE Bulletin, 3-6. Available at http://www.tltgroup.org/programs/seven.html
Connolly, B., & Smith, M. W. (2000, September).
Teachers and students talk about talk: Class discussion and the way it should be. Journal of Adolescent & Adult Literacy, 46(1), 16-26.
Curtis, D., & Lawson, M. (2001). Exploring collaborative online learning. Journal of Asynchronous Learning Networks, 5(1), 21-34.
Dewey, J. (1959). My pedagogic creed. In M. S. Dworkin (Ed.), John Dewey on education: Selections, (pp. 19-32). New York Teachers College, Columbia University.
Duffy, T. M., Dueber, B., & Hawley, C. L. (1998). Critical thinking in a distributed environment: A pedagogical base for the design of conferencing systems. In C. J. Bonk & K. S. King (Eds.), Electronic collaborators: Learner-centered technologies for literacy, apprenticeship, and discourse, (pp. 51-78). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Garrison, D. R. (1992). Critical thinking and self-directed learning in adult education: An analysis of responsibility and control issues. Adult Education Quarterly, 42(3), 136-148.
Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23. Retrieved December 1, 2002, from http://www.atl.ualberta.ca/cmc/CTinTextEnvFinal.pdf
Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.
Hara, N., Bonk, C. J., & Angeli, C. (2000). Content analysis of online discussion in an applied educational psychology course. Instructional Science, 23(2), 115-152.
Hathorn, L. G., & Ingram, A. L. (2002). Online collaboration: Making it work. Educational Technology, 42(1), 33-40.
Henri, F. (1991). Distance learning and computer-mediated communication: Interactive, quasi-interactive or monologue? In C. O'Malley (Ed.), Computer supported collaborative learning (pp. 145-161). Berlin: Springer-Verlag.
Henri, F. (1992). Computer conferencing and content analysis. In A. R. Kaye (Ed.), Online education: Perspectives on a new environment (pp. 115-136). New York: Praeger.
Henri, F., & Rigault, C. (1996). Collaborative distance education and computer conferencing. In T. T. Liao (Ed.), Advanced educational technology: Research issues and future potential (pp. 45-76). Berlin: Springer-Verlag.
Howell-Richardson, C., & Mellar, H. (1996). A methodology for the analysis of patterns of participation within computer mediated communication courses. Instructional Science, 24, 47-69.
Kanuka, H., & Anderson, T. (1998). Online social interchange, discord and knowledge construction. Journal of Distance Education, 13(1), 57-74.
Lincoln, Y. S., & Guba, E. G. (1986). Naturalistic inquiry. Beverly Hills, C. A.: Sage Publications.
Lipman, M. (1991). Thinking in education. Cambridge, England: Cambridge University Press.
Kaufman, D., & Brooks, J. G. (1996, Summer). Interdisciplinary collaboration in teacher education: A constructivist approach. TESOL Quarterly, 30(2), 231-251.
McKenzie, W., & Murphey, D. (2000) "I hope this goes somewhere": Evaluation of an online discussion group. Australian Journal of Educational Technology, 16(3), 239-257. Retrieved November 26, 2002, from http://www.ascilite.org.au/ajet/ajet16/mckenzie.html
Morgan, M. (2000). Guiding online discussions: A social argument framework. Retrieved November 26, 2002, from http://as1.ipfw.edu/2000tohe/papers/morgan.htm
Newman, D. R., Johnson, C., Webb, B., & Cochrane, C. (1997). Evaluating the quality of learning in computer supported co-operative learning. Journal of the American Society for Information Science, 48(6), 484-495.
Woods, L., Priest, H., & Roberts, P. (2002). An overview of three different approaches to the interpretation of qualitative data. Part 2: practical illustrations. Nurse Researcher, 10(1), 43-51.
Home |About LLT | Subscribe | Information for Contributors | Masthead | Archives