Language Learning & Technology
Vol. 3, No. 2, January 2000, pp. 44-57
INSIGHTS INTO THE CONSTRUCTION OF GRAMMATICAL KNOWLEDGE PROVIDED BY USER-BEHAVIOR TRACKING TECHNOLOGIES
Northern Arizona University
Objectivist assumptions that instruction should transfer knowledge and involve learners in deductive processes no longer dominate second-language curricula (Feng, 1996; McGroarty, 1998). Constructivist premises are increasingly compelling teachers to employ exploratory and inductive tasks, stipulating that students should be "agents" who manufacture rather than receive knowledge. Approaches to grammar instruction such as consciousness-raising tasks acknowledge the central role learners play in the acquisition process, engaging students in activities where they must hypothesize their own rules that account for patterns found in the input (Ellis, 1995; Fotos, 1994). Research, however, has concentrated on the efficacy of these premises (Ellis, 1998), such that we know much about the product, yet little about the processes affecting acquisition (Ellis & Schmidt, 1997).
In this article, the author demonstrates how computer-assisted language learning (CALL) software containing user-behavior tracking technologies can provide important insights into the construction of grammatical knowledge. It showcases these technologies' potential by reporting a study which documented the data sources (e.g., digital videos, sound files) learners utilized in a CALL-based consciousness-raising task that promoted the abilities of foreign-language learners of Spanish (N = 30) to generate indirect speech. The study also assessed whether such interactions promote grammatical development.
The most popular contexts for second-language acquisition (SLA) research are the classroom and naturalistic settings. Yet, the computer offers a multisensorial experience, increasing its attractiveness as a learning and research venue (Chapelle, 1998; Tergan, 1997). This technology provides insights into not only the product of learners' efforts but also the process, offering unprecedented opportunities for documenting acquisition "during treatment and during testing" (Hulstijn, 1997, p. 132). User-behavior tracking technologies, for instance, document learners' experiences as they acquire knowledge (Ellis & Schmidt, 1997; Robinson, 1997). Traditionally, such documentation was provided by ethnographic documentation which is quite labor intensive (Davis, 1995).
These technologies are particularly welcome today. The process-oriented educational philosophy known as Constructivism, which views knowledge acquisition as a dynamic process where learners are the architects rather than the recipients of knowledge, increasingly informs pedagogical practice (McGroarty, 1998). Indeed, grammar instruction currently adopts the assumption that learners are agents in the acquisition process (VanPatten, 1993). Yet, little is known about the extent to which learners engage grammatical data sources and the extent to which such undertakings correlate with knowledge acquisition (Ellis, 1998; Ellis & Schmidt, 1997). This article details an experiment in which foreign-language (FL) learners of Spanish developed their knowledge of indirect speech in a computer-assisted language learning (CALL) environment. Thus, the author also demonstrates the potential of user-behavior tracking technologies for providing SLA researchers with insights into how learners construct grammatical knowledge.
ADVANTAGES AND LIMITATIONS OF PRODUCT- AND PROCESS-ORIENTED RESEARCH
Process-oriented research examines knowledge acquisition while the learner reacts to learning conditions, whereas product-oriented research studies knowledge acquisition after exposure to such conditions. Most SLA research focuses on products, favoring psychometric tools that measure how much learners benefit from different instructional methods (Eisner & Peshkin, 1990; Lazaraton, 1995; Nunan, 1992). This research is advantageous because it is highly replicable (Reichardt & Cook, 1979). Furthermore, product-oriented research is highly generalizable since subject pools are relatively large and most analyses employ inferential statistics predicting population characteristics (Henning, 1986).
Nevertheless, product-oriented research routinely lacks some logical validity. According to Hulstijn (1997), even when experiments determine one method to enjoy greater efficacy than another, many do not control for the effects of variables such as previous exposure to the target language, the amount of treatment input given to different groups, and research-situation effects (e.g., Hawthorne, John Henry, and Pygmalion effects). Ellis and Schmidt (1997) contend that, on the whole, SLA research focuses on gaining "an understanding of learning and development from observations of the final state" (p. 146). They submit that a more complete understanding of the knowledge-acquisition process would result from analyses of the input, the production, and the attentional focus of individual learners during an experiment's training phase.
The process-oriented approach to data collection recognizes that SLA is a dynamic process, resulting from subjective interactions between the individual learner, L2 data sources, and situational factors (Chaudron, 1988; Nunan, 1992). And, while consideration of all these factors is an enormous task, a process-oriented approach to SLA research clearly has the potential to address the validity issues that product-oriented research faces. Unfortunately, the labor-intensive nature of process-oriented research results in small subject pools, thereby limiting the generalizability of such "qualitative" research (Lazaraton, 1995).
The combination of both product- and process-oriented research can provide an adequate understanding of SLA (Nunan, 1992), while technology can provide both perspectives with amply sized subject pools (Ellis & Schmidt, 1997). Computers can easily test treatment effects, and user-behavior tracking technologies can document numerous learner behaviors during a treatment phase. Of course, computer-based research is not problem-free. CALL-based instruction has the potential to lack ecological validity: even if certain behaviors occur during (or result from) CALL-based instruction, it is reasonable to question whether such behaviors would occur in naturalistic or classroom settings (Salaberry, 1996).
Still, these technological tools are particularly welcome today given Constructivism's rising importance. As an educational philosophy, it has shifted pedagogues' concerns from products to processes. However, McGroarty (1998) maintains that, in spite of this widespread shift, research pays insufficient attention to how students learn throughout any given methodology. Interestingly, upon proposing ways to make our research interests more process-oriented, McGroarty (1998) points to CALL investigations as providing models for SLA researchers (Avila & Sadoski, 1996; Chun & Plass, 1996).
CONSTRUCTIVIST MOTIVATIONS FOR PROCESS-ORIENTED GRAMMAR INSTRUCTION AND RESEARCH
Cognitively speaking, Constructivism is a learning theory which departs from the traditional, objectivist perspective of knowledge acquisition (Feng, 1996; Spiro, Feltovich, Jacobson, & Coulson, 1991). Objectivism, in its purest form, assumes that the essential elements of instruction are communication and deduction. In this view, when educators adequately comprehend a phenomenon (e.g., the Spanish subjunctive), they can describe it and transfer that knowledge to the student (orally or in writing). To verify that knowledge has been transferred, a teacher asks learners to utilize an explanation in tasks where they apply a construct or rule (e.g., a translation exercise, a Cloze passage). Constructivism, on the other hand, presumes that learners build knowledge actively, largely through inductive processes.
Effective pedagogy thus allows exploration and provides multiple perspectives of a targeted phenomenon. Through exploration, learners formulate, reflect on, and refine hypotheses that explain how a phenomenon works (Bruner, 1990). General cognitive problem-solving mechanisms and background knowledge relevant to a phenomenon help learners to make sense of new data. Viewing a targeted phenomenon from multiple perspectives (i.e., data sources such as a written text, a network of hyperlinks or a video) increases the likelihood that the phenomenon will become salient to the learner since features lacking salience in one context might be more salient in another (Spiro et al., 1991). For instance, two different perspectives of the imperative construction might provide distinct viewpoints, one highlighting its meaning and the other its structural features. A video with a dialog between two people who amplify their suggestions with kinesic and suprasegmental information might accentuate the imperative's pragmatic intent (e.g., ¡No lo pongas ahí! "Don't put it there!"). Alternatively, imperatives in newspaper advertisements--whose heuristic conventions often result in important lexical items being isolated and expressed in enlarged fonts (e.g., ¡COMPRE HOY! "BUY TODAY!")--might increase a student's attention to the imperative's morphological features. Another way in which constructivist instruction makes a targeted phenomenon more salient is through scaffolding, whereas Socratic dialogs between the instructor and the individual learner evoke learners' schemata for a phenomenon and give them models, or problem-solving strategies, for achieving comprehension, (e.g., Does this headline have a noun or a verb? Do you need the preterite or the imperfect here?; Bruner, 1990; Duffy & Roehler, 1986; Jonassen, 1990).
Grammar instruction today clearly adopts the constructivist perspective. The learner has been viewed as an agent in the grammar acquisition process since the groundbreaking studies on interlanguage development of the 1970s and 1980s. The morpheme-acquisition studies generated the creative construction hypothesis, postulating "the existence of universal cognitive mechanisms which enable learners to discover the structure of a particular language" (Ellis, 1990, p. 47). Researchers noticed that learners possessed an innate grammar acquisition mechanism that allowed them to acquire grammatical structures when they were ready for it, rather than when the latter were taught. While some scholars interpreted such discoveries to mean that explicit instructional intervention into learners' grammatical development would not be profitable, most researchers and educators currently concentrate their efforts on identifying instructional strategies that directly foster the acquisition of grammar (Ellis, 1998; VanPatten, 1993).
Instruction offers learners multiple perspectives of a targeted phenomenon and encourages exploration. Although educators recognize the importance of comprehensible input (Krashen & Terrell, 1983), many researchers seek to identify the conditions under which a learner will intake grammatical information (Terrell, 1991; VanPatten, 1993). Instruction increases the likelihood that intake will occur when it provides input that connects with the learner subjectively. Therefore, students are exposed to a variety of written and aural data whose content is "meaningful" given the students' background knowledge (VanPatten, 1993). Instruction also facilitates intake by enhancing the salience of a grammatical phenomenon. A targeted phenomenon becomes intake if learners "notice" (i.e., attend to) it in the input (Schmidt, 1990). "Having noticed some aspect of the environment, we can analyze it and compare it to what we have noticed on other occasions" (Schmidt, 1990, p. 132).
Many educators acknowledge that scaffolding also promotes the construction of grammatical knowledge. Learners receive negative feedback when interlocutors identify errors and somehow prompt them to correct such errors. This purportedly encourages students to notice gaps in their grammatical knowledge, and to generate new hypotheses and refine old ones (Johnson, 1988; Lightbown & Spada, 1990; Swain, 1985; Swain & Lapkin, 1995).
The core principles outlined above have spawned various "instructional options" for grammar instruction (Ellis, 1998), many of which employ inductive strategies (Collentine, 1998b). For instance, Processing Instruction (VanPatten 1993, 1997) provides learners with explicit grammatical descriptions; yet, more importantly, it promotes the analysis and contemplation of structured input that is carefully designed to draw students' attention to a targeted grammatical phenomenon, so that they might infer its semantic properties. Consciousness-raising tasks seek to increase the probability that a learner will notice a targeted phenomenon in the input by promoting metalinguistic knowledge (Ellis, 1995; Fotos, 1994; Sharwood Smith, 1993). This approach asks learners to "analyze data illustrating the workings of a specific grammatical rule" and "discover rules" (Ellis, 1998, p. 48).
Nevertheless, the research gauging the efficacy of today's instructional options largely examines products. Ellis (1998) notes that investigations into the efficacy of such instruction have not isolated the effects of particular features of a given methodology, for example, the separate effects of input, production, and explicit grammatical descriptions, "making it difficult to determine which [feature] was responsible for effects observed" (p. 54). The present author thus examined the knowledge-construction processes in which learners engage in a grammar slide show (Collentine & Collentine, 1997), fostering in a CALL-based consciousness-raising task the abilities of FL learners of Spanish to generate indirect speech. Utilizing user-behavior tracking technologies, the researcher measured the extent to which various knowledge-construction processes predicted learner benefits from the slide show. However, before reporting the study, it would be fruitful to explore the extent to which one can establish constructivist learning conditions in a CALL-based environment.
CALL'S POTENTIAL FOR GENERATING CONSTRUCTIVIST CONDITIONS
In his outline of the basic tenets of what he terms "constructive hypermedia, Yang (1996) suggests that computer assisted instruction (CAI) can readily provide a multisensory discovery-based learning environment. According to Yang, "hypermedia is a tool which can provide a rich base of information and nurture the process of sense and meaning-making, where learners guide their own learning." (p. 45).
CAI can give students access to networks of ideas, connected through nodes and links, that offer various perspectives of a targeted phenomenon. A student might initially have access to the uses and connotations of a grammatical structure in a textual format, with the option of linking to aural representations of the text, or even a digital video from which exemplars were extracted. Digital video is particularly effective at providing learners with the situational and social features that lexical and grammatical items encode (Garza, 1996). Collentine (1998a) reasons that effective CALL tasks designed to encourage grammatical hypothesis-formation combine textual with aural data. He also suggests that the technique known as "colorization," that is, presenting an item in a font color different from its surrounding context (Bell, 1984), may enhance the perceptual prominence of grammatical phenomena that lack overall salience, such as verbal or case morphology. Even though the current state of artificial intelligence limits the ability of software engineers to create algorithms that emulate Socratic interaction, context-sensitive feedback and guidance are becoming increasingly more sophisticated. This is especially true of applications whose scope is narrow, such as software modules focusing on a particular grammatical phenomenon (Nagata, 1996).
One of the most distinctive features of multimedia CAI is its non-linear presentation of information. Rather than presenting information in a highly structured, objectivist fashion, the nodes and links that connect information contained in CAI tasks empower students to search out nodes that render those perspectives of the targeted phenomenon that relate to their individual knowledge bases (Tergan, 1997; Warschaurer, Turbee, & Roberts, 1996).
Nevertheless, these multimedia conditions may not be equally effective for learners at different levels of proficiency. In summarizing the current research on the efficacy of constructivist CAI, Tergan (1997) surmises that providing a myriad of stimuli and data is minimally beneficial to novices. Accordingly, this researcher designed the grammar slide show for FL learners of Spanish with slightly advanced communicative abilities. Still, the targeted phenomenon, indirect speech, was sufficiently complex to discriminate amongst the abilities of even advanced-level learners (Collentine, 1995). Examples include Antonio le preguntó a María cuándo llegaría "Antonio asked María when she would arrive" and María le dijo a su hermana que limpiara la cocina "María told her sister to clean the kitchen."
As mentioned above, this study attempted to provide insights into the construction of grammatical knowledge while showcasing the potential of user-behavior tracking technologies to document such processes. The 40 participants who volunteered for the study were third-year, university-level FL learners of Spanish. The study utilized a grammar slide show to increase their knowledge for the production of indirect speech in Spanish. The researcher then utilized the user-behavior tracking data to determine which aspects of the instruction contributed to improvement in the participants' performance.
Before examining the relationship between learner behaviors and improvements in the use of indirect speech, it was necessary to establish that the slide show was indeed beneficial. To that end, 30 participants were randomly assigned to an experimental group which utilized the slide show and 10 to a control group which received no instruction.
Only 10 of the 40 subjects were assigned to the control group to maximize the size of the tracking-technologies database. To ascertain whether the control group's small size compromised the study's internal validity, the researcher investigated whether any significant differences existed between the two groups prior to the experiment (Borg & Gall, 1989). The control group contained 4 females and 6 males; the experimental group comprised 16 females and 14 males. There was no significant difference in the ratio of males to females between the two groups [ChiSq = 0.53, df = 1, p = 0.467]. No significant differences existed between the two groups in terms of years of high school Spanish studied [control = 2.6 (SD = 1.3); experimental = 2.4 (SD = 1.1); F(1,38) = 0.13; p = 0.717] or semesters of university-level Spanish [control = 4.1 (SD = 1.3); instructional = 3.7 (SD = 1.2); F(1,38) = 0.52; p = 0.474]. To determine whether the two groups enjoyed equal linguistic abilities before the experiment, all participants completed two linguistic competency exams two weeks prior to the experiment: (a) a computer-based grammar exam, assessing each participant's ability to use various grammatical structures in meaningful tasks; (b) a computer-based listening-comprehension exam, which measured the participants' ability to comprehend main ideas and details of authentic digital-video clips. An unbalanced, multivariate analysis of variance revealed that neither group enjoyed an overall advantage in terms of its Spanish proficiency prior to the experiment [Wilk's lambda = 0.93, F(2,37) = 1.38, p = 0.260].
An assessment task measured both groups' ability to use indirect-speech before and after the instructional group used the slide show. The ten-item task, which was computer based (written by the researcher in Macromedia's Director®) prompted the participants to generate indirect speech in writing. Each item depicted a situation graphically and aurally. For instance, a student might see a mother and a daughter chatting in a kitchen. The learner could then cue (only once) a short dialog of four to six turns accompanying that drawing. Within a dialog, a student might hear the daughter asking the mother, ¿Cuándo vamos a comer? "When are we going to eat?" The application then would instruct the participant in English to report in Spanish a particular turn, such as: "Report in Spanish what the daughter asked her mother."1 A student might type in the situation's text box, La hija le preguntó a su mamá cuándo iban a comer "The daughter asked her mother when they were going to eat."
Both groups completed the first administration of the assessment task, referred to as the pretest, two weeks before the treatment. The second administration, referred to as the posttest, took place immediately after the experimental group completed the slide show.
The researcher evaluated each sentence produced by the participants on the pretest and the posttest on a 12 point scale. The following describes the four features that the author evaluated and their relative weights.
(1) Main-clause pronoun (weight: 2 points). In indirect speech in Spanish, the addressee must be referred to by an indirect-object pronoun (e.g., Le dijimos a Carlos/Le dijimos que debía salir pronto, "We told Carlos/We told him that he should leave soon").
(2) Main-clause verb (weight: 4 points). A main-clause verb must be one of communication (e.g., decir "to tell" or preguntar "to ask") and it must be in the preterite (indicative).
(3) Subordinating conjunction (weight: 2 points). Indirect speech in Spanish requires a conjunction representing the original utterance's pragmatics (e.g., a question such as cuándo "when," si "whether," or an assertion such as que "that").
(4) Subordinate-clause verb (weight: 4 points). A subordinate-clause verb in Spanish must be in the past, and its mood must reflect the original utterance's pragmatics (e.g., Juan le preguntó cuándo lo tendría "Juan asked him when he would have it," Mamá les dijo que se bañaran "Mom told them to take a bath").
The grammar slide show, written by the researcher in Macromedia's Director® and the authoring environment's Lingo® scripting language, promoted grammatical competence through consciousness-raising techniques. Learners could infer the rules underlying a grammatical structure by exploring data sources and interacting with computer-mediated scaffolding mechanisms.
a. Sample digital-video component slide b. Sample exemplar-contemplation component slide c. Sample exemplar component slide with a consciousness-raising question d. Sample comprehension-check component slide
Figure 1. Sample screen shots of the grammar slide show
Slides are a series of screens, or windows, through which the learner can move bidirectionally (i.e., backwards and forwards) through an application using navigation buttons. The first slide in the slide show contained an advance organizer that provided general information about the discourse function of indirect speech. The students then worked with two successive cycles, each containing three components. It is important to note that the application did not require linear exploration. A student could move freely from one component back to another (e.g., to review an exemplar's corresponding digital video), or even from the second cycle back to the first. The following describes these components.
Digital-video component. The digital-video component allowed the learners to play a QuickTimeť movie. Students provided written answers to questions posed about the video and received feedback about their accuracy. The learners could freely explore the movie with the pause, advance-forward, and rewind controls. For instance, a digital-video interaction might involve a professor telling a student, "Diego, voy a estar en mi oficina a las once" ("Diego, I'm going to be in my office at eleven").
Exemplar contemplation component. The exemplar-contemplation component allowed the learners to study exemplars, or instances of indirect speech relating to particular statements in the digital video. An exemplar-contemplation screen presented sentences whose audio representations the learners could cue; additionally, colorization enhanced each exemplar's main-clause pronoun and subordinate clause. An exemplar might be as follows: El profesor le dijo a Diego que iba a estar en su oficina a las once "The professor told Diego that he was going to be in his office at eleven." Colorization would set le as well as que iba a estar en su oficina a las once apart from the rest of the sentence.
On the remaining slides of these components, the application posed consciousness-raising questions about the exemplars. Here, the exemplars had audio support but not colorization. Questions involved such phenomena as the grammatical status of the main-clause pronoun, the type of conjunction separating the main and subordinate clauses, the mood of the subordinate-clause verb, and so forth. Students typed their answers in a text box, and could then check their correctness by clicking a button. A context-sensitive routine searched for key words in an answer, and then a dialog box indicated whether the response was correct or incorrect. In the event of an error, the dialog box provided the correct answer.
Comprehension-check component. The comprehension-check component provided brief video clips, each accompanied by three sentences. The participants selected the sentence that represented an instance of indirect speech relating to the clip by clicking one of three radio buttons. The application provided feedback on the correctness of their choices. The learners could continue to select choices until they provided the correct answer.
User-behavior tracking technologies recorded all events that a student generated while using the application, as well as the time (in ticks 2) between any two events. Students generated events within a slide by interacting with its "controls," or screen elements that essentially alter the learning environment, such as a button that gives feedback on an answer. The Director authoring environment allows an application to capture various events and to document run-time conditions surrounding them, such as the time between two events, or where a student is working in an application. For instance, the application recorded students' uses of navigational controls to move back and forth from one slide to another. It could also record which icons the learners clicked to cue a sound file, as well as what they typed in text boxes.
To capture an event or a run-time condition, the programmer could assign a Lingo® "script" to any control. For instance, the following tracking script might be associated with a digital video control, writing to a file the slide number, an identification of the control, and the time of the event:
record_version1(mySlideNumber, "digital video", the ticks)
Other recordable data might include testing for conditions such as whether a learner supplied an answer to a question. This script might be associated with a button where the student checked the veracity of one of his or her answers:
if the answerTextField = "" then
alert "At least provide an answer first!"
veracity = checkAnswer(the text of answerTextField)
record_version2(mySlideNumber, "answer check", veracity, the ticks)
This script does not allow students to get feedback unless they venture a response. If a response is provided, the script calls a function that determines whether the answer provided was correct, and returns a number indicating its veracity (i.e., 1 = correct; 0 = incorrect). Subsequently, the script records in a user file the event's slide number, the type of control targeted, the result, and when the event occurred.3
To understand the knowledge construction processes that correlated with any resulting instructional benefits, the researcher examined five factors:
Audio events represents the average number of times a participant cued sound files that supported exemplars in the instructional material. Since these data sources are redundant, though different manifestations of written exemplars, this variable could represent the extent to which learners sought multiple or additional representations of a data source.
Digital-video events represents the average number of times a participant interacted with the QuickTimeť movies. It is reasonable to assume that this variable reflects the extent to which the participants obtained situational information surrounding an exemplar, such as pragmatic information (Garza, 1996).
Substantive written answers represents the average number of substantive answers that a participant provided to consciousness-raising questions within the exemplar-contemplation components. If a long answer containing three or more words was provided, it could indicate that a learner spent a significant amount of time contemplating some feature of indirect speech.
Exemplar contemplation time measured the average time (in seconds) that a participant spent on the exemplar-contemplation screens containing colorized exemplars (less the time spent on listening to aural representations of those exemplars). This factor thus partially represented the amount of attentional resources that the learners dedicated to text enhanced by colorization.
Incorrect answers represents the average number of incorrect answers that a participant provided in the comprehension-check components. Since these components prompted learners to identify instances of indirect speech and informed them whether they were correct, the learners received feedback on their hypotheses about the structural properties of indirect speech; and feedback on incorrect answers.
Results and Discussion
To examine the relationship between these factors and any treatment benefits, five regression analyses employed one each of these variables as the predictor and the pretest-posttest difference scores as the response. 4
An ANCOVA utilizing the assessment-task pretest scores as the covariant and the posttest scores as the dependent variable showed that the instructional group [M(adjusted) = 70.1, SD = 4.4] could generate indirect speech significantly better than the control group [M(adjusted) = 49.8, SD = 8.1] as a result of the treatment [F(1,37) = 4.85, p = 0.034].Thus, the instructional group benefited from the slide show.
The user-behavior tracking data suggested that certain--though not all--types of knowledge-construction processes, which were tracked during the instruction, were associated with instructional gains, which were measured by the pretest-posttest difference scores. The average number of audio events that a participant generated was a significant predictor of instructional benefit [r = 0.404, F(1,28) = 5.28, p = 0.030]. Yet, even though cueing audio files facilitated the development of indirect-speech abilities, the participants significantly underutilized this technology. The average participant cued only 8.0 (SD = 4.7) of the 12 audio links that supported exemplars of indirect speech [t(one-tailed) = -4.63, df = 29, p < 0.000].
In contradistinction to audio events, the learners over-utilized the QuickTimeť technology given its apparent ineffectiveness: even though the digital video components contained two QuickTimeť movies. The participants averaged 7.0 (SD = 4.1) digital-video events, such as pauses, or rewinding [t(one-tailed) = 6.61, df = 29, p < 0.000]; yet, digital-video events did not correlate with treatment benefit [r = 0.056, F(1,28) = 0.09, p = 0.772].
Exemplar-contemplation time was not a significant predictor of treatment benefit [r = 0.232, F(1,28) = 1.54, p = 0.226]. This result may relate to the observation that the participants underutilized these slides. An exploratory analysis showed that the participants averaged 1.1 seconds (SD = 0.3) per slide throughout the treatment, and yet they averaged only 0.7 seconds (SD = 0.5) per exemplar-contemplation screen [t(one-tailed) = 3.38, df = 56, p = 0.001]. However, exploratory analyses of the exemplar-contemplation data suggest that the dependent variable studied here hides some important tendencies.
Recall that the dependent variable (i.e., the pretest-posttest difference scores) effectively combines a participant's scores on four general features: (a) main-clause pronoun; (b) main-clause verb; (c) subordinating conjunction; and (d) subordinate-clause verb. However, the uniqueness of an exemplar-contemplation screen is that it highlights through colorization the pronoun and the subordinate clause of any exemplar. This then begs the question: Did exemplar-contemplation time correlate positively with improvement in a learner's ability to generate a main-clause pronoun and/or a subordinate clause in indirect speech? The researcher re-coded the data from the pretest and posttest assessment tasks, producing pretest-posttest difference scores for each of these four general features. The results of this analysis indicated that exemplar contemplation time correlated significantly with one factor, namely, the students' ability to produce correct subordinate-clause verbs [r = 0.412, F(1,28) = 5.51, p = 0.026].
The regression analysis suggested that the average number of substantive written answers significantly predicted treatment benefit [r = 0.425, F(1,28) = 5.96, p = 0.022]. However, this result requires careful interpretation. The learners averaged 8.0 (SD = 3.08) substantive answers to the 9 total consciousness-raising questions that the slide show posed, implying that most participants provided thoughtful answers to these questions. However, given that the mean was close to its maximum value, the positive correlation resulting from the regression analysis hints that these data were negatively skewed. A test of skewness confirmed as much (sk = -1.66; z = -3.71; p < 0.000), warranting a slightly different interpretation of the results: some learners (n = 6) did not or could not provide substantive relevant answers to the consciousness-raising questions, and these students uniformly did not benefit from the instruction as a whole.
Finally, the number of incorrect answers that the average participant provided in the comprehension-check components, and so the amount of negative feedback he or she received, significantly predicted success [r = 0.447, F(1,28) = 6.76, p = 0.015]. It is noteworthy that, although the comprehension-check components posed a total of 16 questions, the learners averaged only 1.83 (SD = 1.6) incorrect answers. In other words, they received very little negative feedback, overall.
This last observation may not be inconsequential. It indicates that, although the participants benefited from negative feedback, most of the benefits that they obtained from the treatment resulted from one of the other components. Given that the data suggested that digital-video events did not correlate with treatment success, it is reasonable to conjecture that the principal affective agent in the grammar slide show could be found in the exemplar-contemplation component.
This study attempts to broaden our perspective of the processes associated with the acquisition of grammatical knowledge. The tracking mechanisms employed in this study provide a window into knowledge acquisition processes that are difficult to observe in most product-oriented research. Since any one grammar-instruction methodology employs a number of techniques (e.g., negative feedback, input, answering consciousness-raising questions), product-oriented studies are generally limited to commenting on the effect(s) of a combination of factors on learning (Ellis, 1998). Tracking technologies, however, have allowed this researcher to study process-product relationships, providing an understanding of the effects of individual factors on learner outcomes.
What insights can we glean from the analysis about the construction of grammatical knowledge? All in all, a consideration of two factors supports the postulate that access to multiple perspectives of indirect speech is beneficial (Spiro et al., 1991). Recall that the variable termed audio events measured how often the learners cued aural representations of sentences that they read; and, exemplar-contemplation time represented, in part, how much attention the learners dedicated to textual exemplars that were colorized. Thus, these aural and visual enhancements provided alternative perceptual perspectives of textual data in the instruction, and both factors were associated with improved grammatical performance.
The data also reveal that constructing knowledge in a cooperative fashion is beneficial, even if it is with an "interlocutor" possessing limited artificial intelligence such as the computer-generated feedback in this study's slide show (Nagata, 1996). The variable termed incorrect answers measured the participants' encounters with negative feedback on their hypotheses about indirect speech; and the more negative feedback learners received, the better they could generate indirect speech. Additionally, the analysis of the variable termed substantive written answers suggests that inductive, Socratic interactions can be agents in the construction of grammatical knowledge.
Interestingly, however, some participants were hesitant to explore the information the slide show offered and to engage in Socratic interactions. First, the subjects largely moved forwards through the slide show, as the average number of backwards movements (i.e., navigating from one slide to a previous one) was insignificant, at 2.6 (SD = 2.3) per student. Second, most participants answered consciousness-raising questions cooperatively, and they benefited from doing so. However, a handful of learners provided short answers of less than three words, and none of these learners appeared to benefit from the instruction. That is, their inability or unwillingness to engage in such Socratic interactions was a strong predictor of lack of instructional benefit. Both of these observations are consistent with Tergan's (1997) findings about constructivist CAI environments, surmising that objectivist influences continue to portray highly structured, linear learning as the most productive approach to instruction. Consequently, many learners today do not exploit opportunities to engage in exploratory strategies in CAI environments.
FL pedagogues may need to consider ways of educating learners about the benefits of the "discovery" and "negotiated" learning that underscore the constructivist teaching philosophy (McGroarty, 1998). Researchers would also do well to address the following: (a) Are there certain learning conditions under which students resist the exploratory, process-oriented nature of a constructivist learning environment? (b) Are there certain types of learners that, because of their learning style (or previous training), resist constructivist environments?
Educators may find that not all instructional mechanisms which provide a multiplicity of perspectives and allow exploration promote learning. To be sure, in this study, the participants overutilized the slide show's digital video, interacting with this technology frequently, even though such interaction did not correlate with overall treatment benefit. Of course, it is important to keep in mind that the digital videos used in this study did not contain examples of indirect speech. Instead, the videos contained the speech samples upon which the indirect-speech exemplars were based. Thus, it may be unreasonable to expect to find evidence of a direct cause-and-effect relationship between digital-video use and learner outcomes, given the grammatical structure targeted here. Nonetheless, the results do raise the question of what learners expect to gain from interacting so frequently with a technology that only indirectly helps them achieve a goal.
There is a final observation to make about the data sets. The analysis indicated that the learners underutilized certain instructional features available to them. Nonetheless, the limited exposure to, for instance, audio events correlated highly with instructional benefit. Future research would do well to confirm whether small doses of certain types of data are indeed beneficial to learners. To the author's knowledge, this aspect of SLA has not been studied to date.
It is important to consider the generalizabilty of these results. CALL-based instruction lacks some degree of ecological validity compared to classroom and naturalistic learning contexts (Salaberry, 1996). Indeed, to a certain extent, this study's instructional materials both overstate and understate the potential of FL pedagogy to provide constructivist learning environments. On the one hand, the incorporation of aural, textual, and video materials into a coherent lesson on a regular basis requires a vast amount of resources, and so the lesson presented here is not representative of most daily lessons. On the other hand, the slide show metaphor may not adequately foster exploratory learning. The slide show used here asks learners to move backwards and forwards, but it contains little hyper-movement (linking to new and autonomous contexts) of the type that typifies the Internet. Nonetheless, whether the learning conditions are CALL-based or otherwise, pedagogues must encourage exploratory learning sensibly. Spiro et al. (1991), for instance, surmise that constructivist learning environments are most effective when they offer multisensory and exploratory environments while incorporating some elements of structured learning.
Finally, the scope of the above analysis is broad and the sample size limited, such that the author offers these conclusions cautiously. Nevertheless, the study achieved its goal to showcase the utility of user-behavior tracking technologies for SLA research while providing insights into the construction of grammatical knowledge.
1. These instructions were in English so as not to influence the participants' responses morphologically or syntactically. [return]
2. Ticks are a standard measure of time in programming environments, recording the time of events at a precision of 1/60 of a second. [return]
3. Prior to the instruction, the researcher described to the participants the tracking technologies underlying this application. [return]
4. Optimally, the analysis would have employed a single multiple regression analysis, depicting how much each independent variable predicted instructional benefit, holding the remaining variables constant. However, Borg and Gall (1989) recommend increasing the total sample size by 15 subjects for each predictor variable in a multiple regression analysis. This sample size was not available for the present experiment, and so the author reports the extent to which each factor predicted learner success independently of the other factors. [return]
ABOUT THE AUTHOR
Joseph Collentine is an Assistant Professor of Spanish at Northern Arizona University. He has published on the acquisition of morphosyntactically complex structures by foreign language learners of Spanish and about CALL.
Avila, E., & Sadoski, M. (1996). Exploring new applications of the keyword method to acquire English vocabulary. Language Learning, 46, 379-395.
Bell, J. M. (1984). The ColorSounds story. ColorSounds Monthly, 1, 1-2.
Borg, W., & Gall, M. (1989). Educational research. New York: Longman.
Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press.
Chapelle, C. (1998). Multimedia CALL: Lessons to be learned from research on instructed SLA. Language Learning & Technology, 2 (1), 22-34. Retrieved from the World Wide Web January 11, 2000: http://llt.msu.edu/vol2num1/article1/index.html.
Chaudron, C. (1988). Second language classrooms. Cambridge: Cambridge University Press.
Chun, D., & Plass, J. (1996). Effects of multimedia annotations on vocabulary acquisition. Modern Language Journal, 80, 183-198.
Collentine, J. (1995). The development of complex syntax and mood-selection abilities by
intermediate-level learners of Spanish. Hispania, 78, 122-35.
Collentine, J. (1998a). Cognitive principles and call grammar instruction: A mind-centered, input approach. CALICO Journal, 15, 1-18.
Collentine, J. (1998b). Processing instruction and the subjunctive." Hispania, 81, 576-587.
Collentine, J., & Collentine, K. (1997). The compatibility of computer mediated communication solutions with beginning level foreign language curricula. Computer Assisted Language Learning, 10, 411-425.
Davis, K. (1995). Qualitative theory and methods in applied linguistics research. TESOL Quarterly, 29, 427-434.
Duffy, G., & Roehler, L. (1986). The subtleties of instructional mediation. Educational Leadership, 43, 23-27.
Eisner, E. W., & Peshkin, A. (Eds.). (1990). Qualitative inquiry in education: The continuing debate. New York: Teachers College Press.
Ellis, N., & Schmidt, R. (1997). Morphology and longer-distance dependencies: laboratory research illuminating the A in SLA. Studies in Second Language Acquisition, 19, 145-172.
Ellis, R. (1990). Instructed second language acquisition. Oxford: Basil Blackwell.
Ellis, R. (1995). Interpretation tasks for grammar teaching. TESOL Quarterly, 29, 87-106.
Ellis, R. (1998). Teaching and research: Options in grammar teaching. TESOL Quarterly, 29, 39-60.
Feng, Y. (1996). Some thoughts about applying constructivist theories to guide instruction. Computers in the Schools, 12, 71-84.
Fotos, S. (1994). Integrating grammar instruction and communicative language use through grammar consciousness-raising tasks. TESOL Quarterly, 28, 323-351.
Garza, T. (1996). The Message is the Medium: Using Video Materials to Facilitate Foreign Language Performance. Texas Papers in Foreign Language Education, 2, 1-18.
Henning, G. (1986). Quantitative methods in language acquisition research. TESOL Quarterly, 20, 701-708.
Hulstijn, J. (1997). Second language acquisition research in the laboratory: Possibilities and limitations. Studies in Second Language Acquisition, 19, 131-143.
Johnson, K. (1988). Mistake correction. English Language Teaching Journal, 42, 89-101.
Jonassen, D. (1990). Semantic network elicitations: Tools for structuring hypertext. In C. Green & R. McAleese (Eds.), Hypertext: State of the art (pp. 131-162). Oxford: Intellect Books Ltd.
Krashen, S., & Terrell, T. (1983). The Natural Approach: Language acquisition in the classroom. Hayward, CA: Alemany Press.
Lazaraton, A. (1995). Qualitative research in applied linguistics: A progress report. TESOL Quarterly, 29, 455-472.
Lightbown, P. , & Spada, N. (1990). Focus on form and corrective feedback in communicative language teaching: Effects on second language learning. Studies in Second Language Acquisition, 12, 429-448.
McGroarty, M. (1998). Constructive and constructivist challenges for applied linguistics. Language Learning, 48, 591-622.
Nagata, N. (1996). Computer vs. workbook instruction in second language acquisition. CALICO Journal, 14, 53-75.
Nunan, D. (1992). Research methods in language learning. Cambridge: Cambridge University Press.
Reichardt, C. S., & Cook, T. D. (1979). Beyond qualitative versus quantitative methods. In T. D. Cook & C. S. Reichardt (Eds.), Qualitative and quantitative methods in evaluation research (pp. 7-32). Beverly Hills, CA: Sage.
Robinson, P. (1997). Generalizability and automaticity of second language learning under implicit, incidental, enhanced, and instructed conditions. Studies in Second Language Acquisition, 19, 223-248.
Salaberry, M. (1996). A theoretical foundation for the development of pedagogical tasks in computer mediated communication. CALICO Journal, 14, 1, 5-36.
Schmidt, R. (1990). The role of consciousness in second language learning. Applied Linguistics, 11, 127-158.
Sharwood Smith, M. (1993). Input enhancement in instructed SLA: Theoretical bases. Studies in Second Language Acquisition, 15, 165-179.
Spiro, R., Feltovich, P., Jacobson, M., & Coulson, R. (1991). Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Education Technology, 3, 24-33.
Swain, M. (1985). Communicative competence: Some roles of comprehensible input and comprehensible output in its development. In S. Gass & C. Madden (Eds.), Input in Second Language Acquisition (pp. 235-256). Rowley, MA: Newbury House.
Swain, M., & Lapkin, S. (1995). Problems in output and cognitive processes they generate: A step towards second language learning. Applied Linguistics, 16, 371-91.
Tergan, S. (1997). Multiple views, contexts, and symbol systems in learning with hypertext/hypermedia: A critical review of research. Education Technology, 37, 5-18.
Terrell, T. (1991). The role of grammar instruction in a communicative approach. Modern Language Journal, 75, 52-63.
VanPatten, B. (1993). Grammar teaching for the acquisition rich classroom. Foreign Language Annals, 26, 435-450.
VanPatten, B. (1997). The relevance of input processing to second language theory and second language teaching. In W. Glass & A. Pérez-Leroux (Eds.), Contemporary perspectives on the acquisition of Spanish: Volume II (pp. 93-108). Sommerville, MA: Cascadilla Press.
Warschauer, M., Turbee, L., & Roberts, B. (1996). Computer learning networks and student empowerment. System, 24, 1-14.
Yang, S. C. (1996). Designing instructional applications using constructive hypermedia. Educational Technology, 36, 45-50.
Home | About LLT | Subscribe | Information for Contributors | Masthead | Archives