Language Learning & Technology
Vol. 3, No. 1, July 1999, pp. 108-113
RESEARCH QUESTIONS FOR A CALL RESEARCH AGENDA:
A REPLY TO RAFAEL SALABERRY
Carol A. Chapelle
Iowa State University
I am grateful to Rafael Salaberry for his comments on my paper "CALL in the year 2000: Still in search of research paradigms?" takes up the discussion that I hoped the paper would motivate about fundamental issues in CALL research--issues such as what the relevant research questions are, why particular research methods are appropriate, and how CALL research relates to second language acquisition (SLA) research. In the interest of moving the discussion forward, I have responded to each of the points that Salaberry has raised. Points (a) through (e) below are quoted directly from his paper:
(a) The literature review on research perspectives of classroom discourse needs to be expanded [beyond what is presented in Chapelle's 1997 paper].
The development of fruitful directions for CALL research can benefit from the perspectives of other SLA discourse work than what I illustrated in my 1997 paper if that work is linked to the study of CALL. Salaberry points out that the general questions and examples of research that I suggested represent only a fraction of the issues that a CALL research agenda should take up and therefore that sociocultural approaches to L2 classroom discourse should also be consulted. In my view, this is a reasonable suggestion, but it should include the additional step of articulating the implications of sociocultural perspectives for CALL research in terms of the research questions and methods that they imply. For example, as Salaberry points out, Warschaeur (1996) takes a step in this direction by summarizing some of the tenets of sociocultural theory and research results from studies of computer-mediated-communication. In another paper, he goes further toward what I see as necessary for arguing about the relevance of particular research paradigms:
In researching the use of new technologies by L2 learners, one might want to look at questions such as these: What new literacies does multimedia computer technology demand, both inside and outside the classroom? How does the development of these new literacies intersect with issues of class, race, gender, and identity? How does the sociocultural context of particular educational institutions or communities affect the learning and practice of electronic literacies? (Warschauer, 1998, p. 759)
Having identified these as important questions, he suggests a critical approach to the study of CALL, which might, for example, entail the perspectives and methods of critical classroom discourse analysis (Kumaravadivelu, 1999). In short, research paradigms for CALL can benefit from further exploration of second language classroom discourse perspectives, but only to the extent that they help to frame CALL research questions and suggest methods for addressing these questions.
(b) Research domains other than discourse analysis may also be relevant for the analysis of pedagogical effectiveness of CALL.
Salaberry's assertion that other domains may be equally or more relevant than SLA for investigating CALL is one that is made repeatedly in the CALL literature. His example is that computational linguistics should be as relevant as SLA for guiding research on the effectiveness of a program such as Kommissar, which relies on natural language processing technologies. To evaluate this claim, it is informative to consider the research objectives of computational linguistics. Computational linguistics investigates how rules of language and language processing can be used to write computer programs that recognize and produce human language. The focus is on the nature of the formalisms that adequately work in language generation and recognition. Research methods therefore consist of proving algorithms, writing software, and testing it on relevant data. This work is clearly of interest to those developing some types of CALL software. The principle research questions and methods, however, point in a different direction from those of concern in CALL where the questions focus on such issues as language use, language acquisition, and the nature of electronic literacies.
Looking at research questions across some of the areas that have been suggested as relevant to CALL, one can see that CALL research is concerned with either a narrower or, as in the case of computational linguistics, a different set of issues (see Table 1). Other domains may have somewhat related objectives from which CALL research can borrow methodologically; however, none addresses the specific concerns about development of L2 competence. I therefore believe it is necessary to begin by identifying the research questions essential for progress in CALL. The uncritical statement that another area may be relevant for the study of CALL threatens to make the search for research paradigms a wild goose chase. What is needed to appropriately draw from other disciplines is a clear notion of exactly what they have to offer to the development, use, and evaluation of CALL. What is needed to understand the nature of CALL evaluation is a clear formulation of research questions that pertain to language teachers' choices about CALL--the types of questions suggested by Chapelle (1997) and Warschauer (1998).
Table 1. Guiding questions of other disciplines and CALL
In... The primary question is... In CALL, the corresponding question is... Educational
How can computers best be used to improve learning? How can computers best be used to promote development of L2 communicative competence? Computers and collaborative
How can computer-assisted activities be
designed to promote learning through
How can collaborative computer-assisted language learning activities be designed to promote development of L2 communicative competence? Artificial intelligence How can rules of logic be implemented in computer programs to perform functions requiring knowledge-based analysis and judgement? How can computer programs with capability for knowledge-based analysis and judgement be used to promote development of L2 communicative competence? Computational linguistics How can rules of language, and language processing be used to write computer programs to recognize and produce human language? How can computer programs for language recognition and production promote development of L2 communicative competence? Corpus
What do descriptions and analyses of language from large corpora of texts reveal about the lexical patterns and grammatical structures that people use? How can learners' use of corpora promote
development of L2 communicative competence?
(c) The theoretical analysis of the psycholinguistic process of L2 development needs to be specified in further detail.
Many of the papers describing research within an interactionist perspective toward SLA have outlined the psycholinguistic processes in more detail than the brief summary I presented in my paper (see Chapelle, 1998, for more detail). While it is possible that additional theoretical analysis of this process may generate some useful research hypotheses and empirical research questions for CALL, in my view, it would be more fruitful to investigate empirically the research questions that the current theory implies. The problem with theorizing an excessively detailed model without a coordinated research program is that the theory does not have the necessary empirical support and the theorizers may have difficulty in articulating what empirical support would consist of (McLaughlin, 1987). Existing theory appears to provide a number of useful hypotheses and constructs that need to be operationalized in CALL research.
What seems unproductive to me is perpetuation of the argument that the field knows so little about SLA (e.g., because theories need to be specified in greater detail) that research and development in CALL should proceed with no input from work in SLA. In fact, interactionist theory is sufficiently detailed to draw principles for CALL. Table 2 illustrates some of the principles that one might posit from this theory, the research questions that they imply for CALL, and possible process-oriented research approaches for investigating these questions. Results of both process- and product-oriented research can be used to evaluate these theory-based principles and their applicability to various CALL activities. My paper illustrates only a few examples, but as Salaberry points out, examples of how relevant constructs have been, or might be, operationalized appear throughout the research on instructed SLA, and I agree that examination of these studies is useful.
Table 2. Principles of interactionist SLA that imply CALL research questions and methods
Principles of Interactionist SLA Example Research Questions Example Research Methods 1) Learners should notice the linguistic characteristics of the target language input that learners receive need to be noticed 1) Do learners attend to the linguistic characteristics of the target language input? 1) Introspective methods seeking students' reports of what they attend to. 2) Learners need to have opportunities to produce target language output. 2) Do learners produce "comprehensible output?" 2) Discourse analysis investigating learners' linguistic output in CALL tasks in which language is used communicatively. 3) Learners need to notice errors in their output. 3) Is there any evidence that learners notice errors in their output? 3) Introspective methods including learners' reports of their noticing errors. 4) Learners need to correct their linguistic output. 4) Do learners correct their linguistic output? 4) Interaction analysis identifying learners' corrections of their own language. 5) Learners need to engage in target language interaction whose structure can be modified as needed for comprehension. 5) To what extent do the learners interact with the computer or with other language users, and to what extent are "normal" interactional structures modified to focus on the form and meaning of the language? 5) Analysis of learners' language and other interactions that modify the interaction to help with comprehension..
(d) The analysis of electronic and face-to-face communication does not take into account relevant differences brought about by distinct communication media.
In response to this comment, I would ask, What are the differences that are relevant for identifying CALL research questions from interactionist theory? My point in looking at similarities was
1) that a need exists to identify research questions and methods for CALL,
2) that interactionist theory makes some hypotheses about the task conditions that may be beneficial for SLA (e.g., those that focus attention on language or that allow learners to request modified input), and
3) that CALL tasks which create these conditions might be researched through the same methods used in SLA research.
In other words, it is the similarities that are of interest because they allow for identification of features expected to be beneficial in CALL, and therefore raise possibilities for research. An analysis of the differences between CALL tasks and those used in other research, teaching, and language use is of interest for other purposes, such as to assess the generalizeability of results from research using CALL to tasks that do not rely on CALL (Chapelle, 1999). It is important to note that part 3 of my point above was not to generalize findings from research on face-to-face communication to interactions in CALL. Instead, the suggestion is to identify potential foci for research based on prior work within the relevant domain. Results of such research would provide evidence pertaining to generalizability.
An example of such research was a study of the effects of requests for modified input in a CALL listening comprehension activity. Hsu (1994) conducted a focused analysis of interactions between learners and the computer to identify their requests for modified input. The normal interaction in this part of the program consisted of learners' requests for continuation of a story with accompanying pictures on one computer screen after another. The researcher documented "interactional modifications" evident by learners' requests for repetitions, written transcriptions, or written definitions for words in the input. She also recorded the specific linguistic input associated with each of the learners' requests. She then assessed outcomes through pre- and post-tests which had been constructed specifically for the research to include the lexical phrases in the input.
Table 3 displays the data gathered for one student in this study. The first column shows the target words, and the next two columns show pre- and post-test results indicating improvement in comprehension on a number of items. Improvement is indicated in the fourth column with a plus for improvement and a minus for no improvement. The final column codes whether or not the learner requested a modification for the segment of text in which that item occurred. By first coding pluses as 1 and minuses as 0, the researcher calculated a Phi correlation of .67 for the data shown in the example. In the study, data were gathered for 15 learners each of whom completed two listening passages. For the 15 students, mean phi correlations for one of the stories was .60 and for the other was .71, which means that this research found significant relationships between requests for modified input and improvement in listening comprehension. This type of research provides some evidence for the validity of the modified input hypothesis in CALL.
Table 3. Pre-test, post-test, improvement analysis, and recorded requests for modifications for nine words in the study by Hsu (1994).
1 If learners were unable to fill in the word correctly on the pretest but were able to on the post-test, they were awarded a plus for improvement; if they did the same or worse from pretest to post test, they were given a minus.
2 If learners chose a modification (repetition, text, or dictionary) when they were listening to the segment with the target word, the word was coded with a plus for use of modifications; if they did not choose a modification, the word was given a minus.
(e) The relative importance of computer mediated communication (CMC) in the proposed research agenda appears to deserve greater emphasis given the current level of interactivity achieved by various communication media.
I question two aspects of this suggestion. First, the issue of what should be emphasized in a research agenda depends on who is conducting the research and for what purpose rather on any inherent capabilities of software. Researchers investigating intelligent tutoring systems, for example, might argue that learners' use of these systems deserves more attention because of the amount of time and money they are devoting to their construction. Teachers whose students regularly engage in CMC outside of class may be more interested in research on how to structure individualized CALL activities with particular focus on language in class. In short, what deserves emphasis in a research agenda for CALL will ultimately depended on the professional, personal, and political agenda of the researcher.
Second, I question the assumption that CALL activities based on computer-mediated communication inherently achieve a higher level of interactivity than other forms of CALL. Human-human interaction of CMC can clearly take a greater variety of forms than that of computer-learner interaction, but both the level and quality of interactivity achieved in particular CALL activities are empirical questions that can only be addressed through the analysis of interactions . For example, in a study of CALL which incorporated CMC, Lamy & Goodfellow (1999) identified what they saw as important differences in the types of interactions which they observed. They contrasted social conversation (Table 4) which requires little negotiation of meaning or stretching of competence, with reflective conversation (Table 5) in which learners talk about the target language and the learning task, thereby obviously focusing their attention on the language. The researchers claim that engaging in reflective conversation is the more worthy goal for language learning because such exchanges provide an opportunity to negotiate understanding, make explicit reference to language, and to engage in a context where control is negotiated. Research like this which aims to interpret types of interactivity on the basis of empirical data begins to appropriately tease apart the complexities inherent in the construct of interactivity. What needs to be investigated is the extent to which quality interaction can be prompted by particular task demands.
Table 4. An example of social conversation (Lamy & Goodfellow, 1999)
Student 1 A quelle heure ouvira-t-il? Je pense que je voudrais une biere... Imaginative/Social Student 2 Bonjour, E. je voudrais une bière aussi mais il n'y a pas de vertu dans une bière virtuelle... Imaginative/Social Student 3 Bonjour D. Juis suis a Caen le 23 aout. Et vous? S. Request?/Social Student 4 Bonjour S. Moi aussi je serai à Caen le vignt-troisième aoüt. Peut-être on peut recontre... Invitation?/Social
Table 5. An example of reflective conversation (Lamy & Goodfellow, 1999)
Student 1 Pour ma première liste j'ai choisi les mots suivants: déchiré, couler... Je ne comprends pas les "groups" et à quoi sert cet outil. ... Est ce qu'il y a quelqu'un (une) qui peut donner moi des examples ... Explaining learning processes
Student 2 D'apres moi, les groupes servent comme un aide-memoir Giving help
Salaberry concludes with the suggestion that a valid research proposal should be attentive to important principled distinctions incorporated into theoretical frameworks. In contrast, I have argued that such general distinctions provide no guidance for research on CALL (Chapelle, 1990). While they may be useful as general organizing principles for introducing types of CALL software, from a research perspective they offer no substantive framework because they target the wrong level of analysis. Figure 1 abstracts from the complex reality of second language classrooms a set of relationships of interest in CALL research. The arrows should be read as "influences." In other words, the horizontal reads as follows: (1) Available CALL software influences the (2) CALL activities that teachers plan, which, in turn, influence (3) the work that learners engage in; this work on CALL (hopefully) positively influences (4) learners' ability to use the language beyond the CALL activity.
Available CALL Software
Teacher planned CALL activities
Learners' work on CALL
Learners' ability to use the language
Figure 1. Interrelated influences in CALL software, teacher-planned activities, learners' CALL use, and language ability
I have suggested that the most critical research questions center around (3), learner's work on CALL, because regardless of what the teacher plans and regardless of the classification of the software as tool or tutor, it is the learners' actual language use during the activity that is likely to influence future ability. The research questions Warschauer (1998, p. 759) poses focus primarily on the factors that help to define (4), the learner's ability to use the language, in a way that can inform teachers' knowledge, and philosophy, class goals, and institutional practices. He asks, "What new literacies does multimedia computer technology demand, both inside and outside the classroom?" In other words, what kind of language abilities do learners need to function in the electronic world of the 21st century? These needs should be central to the activities that teachers plan for the language classroom. He asks "How does the development of these new literacies intersect with issues of class, race, gender, and identity? In other words, how do these learner factors influence their needs (4) and their work on CALL in the classroom (3)? He asks, "How does the sociocultural context of particular educational institutions or communities affect the learning and practice of electronic literacies?" In other words, how do institutional practices in language programs affect the CALL activities teachers plan (2), and how do institutional practices outside the language program affect the abilities learners require (4)?
If abstract, binary distinctions between types of software bear some relationship to relevant research questions about CALL, what are these questions?
In closing, I invite other readers to comment on the issues Rafael Salaberry and I have raised concerning CALL research. In my view, this critical area of our profession has received far too little attention in applied linguistics and language teaching. As a professor of TESL/applied linguistics, I am frequently contacted by graduate students wishing to pursue research on CALL but not knowing where to begin. General suggestions in the literature that many disciplines should inform the study of CALL do not help to identify the relevant research questions. The critical research issues and appropriate methods need to be formulated so development, use, and research of CALL can proceed in fruitful directions in the 21st century. To underscore Salaberry's point, we need to revisit these issues regularly--not by reinventing them but by building on previous discussion and research results.
Chapelle, C. A. (1999). Investigation of "Authentic L2 Tasks." In J.Egbert & E. Hanson-Smith (Eds.), Computer-enhanced language learning (pp. 101-115). Alexandria, VA: TESOL Publications.
Chapelle, C. A. (1998). Multimedia CALL: Lessons to be learned from research on instructed SLA. Language Learning and Technology, 2 (1), 22-34. Retrieved July 22, 1999 from the World Wide Web: http://polyglot.cal.msu.edu/llt/vol2num1/article1/index.html.
Chapelle, C.A. (1997). CALL in the year 2000: Still in search of research paradigms? Language Learning and Technology, 2 (1), 22-34. Retrieved July 22, 1999 from the World Wide Web: http://polyglot.cal.msu.edu/llt/vol1num1/chapelle/default.html.
Chapelle, C. A. (1990). The discourse of computer-assisted language learning: toward a context for descriptive research. TESOL Quarterly, 24, 199-225.
Hsu, J. (1994). Computer assisted language learning (CALL): The effect of ESL students' use of interactional modifications on listening comprehension. Unpublished doctoral dissertation, Department of Curriculum and Instruction, College of Education, Iowa State University, Ames.
Kumaravadivelu, B. (1999). Critical classroom discourse analysis. TESOL Quarterly, 33(3), 453-484.
Lamay, M-N., & Goodfellow, R. (1999). "Reflective conversation" in the virtual language classroom." Language Learning & Technology, 2(2), 43-61. Retrieved July 22, 1999 from the World Wide Web: http://polyglot.cal.msu.edu/llt/vol2num2/article2/index.html.
McLaughlin, B. (1987). Theories of second-language acquisition. London: Edward Arnold.
Warschauer, M. (1997). Computer-mediated collaborative learning: Theory and practice. The Modern Language Journal, 81, 470-481.
Warschauer, M. (1998). Researching technology in TESOL: Determinist, instrumental, and critical approaches. TESOL Quarterly, 32(4), 757-761.
ABOUT THE AUTHOR
Carol Chapelle is a professor in the Department of English and Program in Linguistics at Iowa State University where she teaches courses in second language acquisition, language testing, and computers in applied linguistics. She has published extensively in the areas of computer-assisted language learning and language assessment.
About LLT | Subscribe | Information for Contributors | Masthead | Archives