Language Learning & Technology
Vol. 1, No. 2, January 1998, pp. 41-50

PDF version

Lara L. Lomicka
The Pennsylvania State University


This pilot study investigated the effects of multimedia reading software on reading comprehension. Specifically, the study aimed to explore how multimedia annotations influence the level of comprehension. Twelve college students enrolled in a second semester French course were instructed to think aloud during the reading of text on the computer screen. Participants read the text under one of three conditions: full glossing, limited glossing, or no glossing. In addition, a tracker was set up in the software to record the amount and type of glosses, and length of time that each was consulted. The raw data clearly indicate an increase in the number of causal inferences generated for students who had access to full glossing. Computerized reading with full glossing may promote a deeper level of text comprehension. Pedagogical suggestions for second language (L2) teaching and research will be discussed.


It started with the question "To gloss or not to gloss." Although the concept of glossing dates back to the Middle Ages, it has been largely unexamined by researchers until late in this century. Recent literature on glossing has provoked a controversial debate--one that remains problematic even today in foreign language (FL) reading research. Traditionally, glosses provided a short definition or note in order to facilitate reading and comprehension processes for L2 learners. Nation (1983) defined glosses as short definitions; Pak (1986) refers to them as explanations of the meanings of words. Typically located in the side or bottom margins, glosses are most often supplied for "unfamiliar" words, which may help to limit continual dictionary consultation that may hinder and interrupt the L2 reading comprehension process. Despite the perpetual debate, glossing remains a common and acceptable aid for many foreign language text books (Davis, 1989). From the first question, "To gloss or not to gloss," emerges two opposing questions: (a) "Does glossing improve L2 reading comprehension?" and (b) "Does glossing hinder fluency in L2 reading?" (Lyman-Hager & Davis, 1996).

Findings thus far related to the relationship between glossing and reading comprehension have been inconsistent (Jacobs, 1994). Johnson (1982) concludes that glossing may not promote a global comprehension of the text. While readers may not be disrupted by glosses during reading, the use of glosses does not necessarily increase reading comprehension (Pak, 1986). Jacobs, Dufon, and Hong (1994) report findings from an experiment conducted with intermediate students studying Spanish: Glossing did not significantly affect recall, but students with an above average proficiency in a second language demonstrated a higher recall when using glosses. On the other end of the continuum, research conducted by Davis (1989), Jacobs (1991, 1994), and Hulstijn, Hollander, and Greidanus (1996) tends to support the use of glossing in L2 reading. Students who were able to consult glosses before reading or
during the reading process recalled more of the text than those without glossing aids (Davis,


1989; Jacobs, 1991). Jacobs (1994) reports that third-semester students of Spanish with access to glosses performed significantly better in the amount of the text recalled than students without access to glosses. Hulstijn, Hollander, and Greidanus (1996) studied the influence of marginal glosses, dictionary use, and the reoccurrence of unknown words on incidental vocabulary learning. Advanced students of French, under one of three conditions (Marginal Glosses, Dictionary Use, Control), read a short text that had been "slightly adapted" (p. 329). Researchers found that incidental vocabulary learning is higher when L2 readers have access to the meanings of words through marginal glosses or through a dictionary. Inconsistent findings in these studies may be due to differences in text type (modified, authentic), in level (beginning, intermediate, advanced), and measures (cloze tests, recall protocols).

The issue of "to gloss or not to gloss" and the questions currently being investigated on glossing can be taken a step further and approached from a different perspective with the advent of multimedia reading software. Davis (1989) points out the use of multimedia annotations and emphasizes that hypertext is "invisible and unobtrusive," allowing the user to consult as much or as little information as s/he desires (p. 42). Jacobs (1994) also identifies the potential of glossing with computers for both learners and researchers. Multimedia program annotations are not limited to textual information and can take the form of video, sound, and pictures (Chun & Plass, 1996). Computer aided glossing can provide much more than the "traditional" glosses (definitions, translations, and grammatical notes). Through hypermedia-annotated text, readers will be able to approach the text more globally, rather than linearly (Martínez-Lage, 1997). To achieve a more global understanding of the text, other multimedia annotations such as images, sounds, cultural, historical and geographical references, and guiding questions could enhance comprehension. Some recent research involving multimedia annotations includes: Une Vie De Boy: Interactive Reading in French (Lyman-Hager, Davis, Burnett, & Chennault, 1993); Effects of Multimedia Annotations on Vocabulary Acquisition (Chun & Plass, 1996); Hypermedia Technology for Teaching Reading (Martínez-Lage, 1997); and Computers and L2 Reading: Student Performance, Student Attitudes (Davis & Lyman-Hager, 1997).

Lyman-Hager et al. (1993) examined vocabulary acquisition and student glossing choices for intermediate level students studying French. Lyman-Hager et al. concluded that students who worked with the multimedia program based on an excerpt from the story by F. Oyono, Une Vie de Boy, were better able to retain vocabulary words than students who worked with non-computerized text. Two conditions were used in this study: computerized reading and non-computerized reading. Both groups had access to glosses; the computer group had access to multimedia annotations while the text group could consult printed text with the same glosses. Immediately after reading the text, the subjects were asked to perform a written recall protocol. A week later, an in-class vocabulary quiz of "critical" words in the story was distributed following a class discussion.

In their article exploring multimedia annotations and vocabulary acquisition, Chun and Plass (1996) present the results ofthree studies with students in their second year of German who used CyberBuch, a multimedia application offering annotations through pictures, text, and video. Specifically, the goals of their investigation include exploring incidental vocabulary learning, examining the effectiveness of multimedia annotations, and investigating the relationship between look-up behavior and vocabulary test performance (p. 185). Before working with the multimedia application, students were introduced to the program and watched a video which provided an overview of the story. After reading the story and using the multimedia annotations, students took a vocabulary test and wrote a recall protocol. Chun


and Plass report that the recall protocol for visual annotations (i.e., words annotated with text and pictures, text, and video) was higher than for words annotated with text alone. While both Lyman-Hager et al. (1993) and Chun and Plass (1996) investigated vocabulary learning through the use of recall protocols, some researchers, such as Myers (1990), point out that recall or post-reading measures may be more representative of a memory test used to simply "recall" knowledge (Myers, 1990).

Martínez-Lage (1997) provides examples of multimedia annotations from a project annotated through Guided Reading (Herren, 1996). She implies the usefulness of multimedia annotations in that they can "provide immediate access to textual, sound, and visual annotations" (p. 149). This immediate access is not as intrusive as the steps required in looking up words in the dictionary. Unlike traditional glossing, the use of images through multimedia annotations can be advantageous for readers (Chun & Plass, 1996; Martínez-Lage, 1997). Martínez-Lage suggests that student interaction with text facilitates understanding because students learn not only about language, but learn with language, thus promoting active reading (p. 149).

Davis and Lyman-Hager (1997) investigated student performance and attitude regarding computerized L2 reading. Forty-two intermediate level students of French read a glossed excerpt from Une Vie de Boy on the computer screen. After the reading, participants performed a recall protocol, a multiple-choice task, and underwent an exit interview. Davis and Lyman-Hager found that even though different types of glosses were available for student consultation, they tended to utilize primarily English definitions. The interview revealed an extremely positive reaction by the participants toward the software.

In an attempt to advance toward a deeper-level understanding of L2 reading, Lyman-Hager and a team from Penn State's Educational Technology Services developed the template-based shell known as GALT (Glossing Authentic Language Texts). This shell was derived from an earlier text-based computerized reading program by Davis and Lyman-Hager, Une Vie de Boy. Both Une Vie de Boy and GALT were inspired by Bernhardt's (1991) model of L2 reading which attempts to account for the complexity of the L2 reading process. This model defines reading comprehension as a result of multiple interacting factors: word recognition (understanding individual word meanings), phonemic/graphemic decoding (recognizing words through aural or visual characters), syntactic feature recognition (understanding grammatical relationships), intratextual perception (linking statements to those that precede or follow), prior knowledge (awareness of knowledge brought to text), and metacognition (awareness of reader's own cognitive processes during the reading process) (Davis & Lyman-Hager, 1997). Davis and Lyman-Hager carefully ex plain how the different interacting factors correspond with different types of multimedia annotations. Students can click on words/expressions to obtain definitions in English or in the target language (word recognition). To prevent phonemic/graphemic errors, readers use the pronunciation feature in order to hear words or passages spoken by a native speaker. Further, students could access grammatical notes and explanations to enhance textual understanding (syntactic feature recognition). In an attempt to enhance global comprehension and to assist students in linking statements and ideas, readers are able to click on a question control button (intratextual perception). Cultural knowledge is provided to readers both during their reading (through a cultural reference control button) and in the introduction to the text, to the author, and/or to the author's country of origin (prior knowledge). Finally, the tracking device allows instructors/researchers to obtain a log of readers' strategies for gathering information (metacognition). (For a more detailed description of the software, see Davis & Lyman-Hager, 1997; Davis, Lyman-Hager, & Hayden, 1992; or Lomicka, Bradley, & Lyman-Hager, 1997).


In light of the research on glossing and multimedia annotations, it can be concluded that studies have concentrated primarily on vocabulary learning, acquisition, and retention, and have only scratched the surface of research involving computer-assisted reading comprehension and multimedia annotations. Furthermore, recall protocols seem to have become the chief measure in glossing studies. Recall protocols and other post-reading tests represent an "off-line" measure of the product of comprehension (Myers, 1990). Such "comprehension tests," as noted by Bernhardt (1991), are acceptable measures of reading comprehension and "provide a purer measure of comprehension, uncomplicated by linguistic performance and tester interference" (p. 200). However, even if the reader can answer questions or recall words, s/he may not hold a coherent understanding of the text. According to some researchers (e.g., see Myers, 1990), online measures of comprehension (reading processes and integration of text during reading), which account for reading comprehension as it is occurring, may more accurately measure comprehension.

Although online comprehension has not been extensively explored in L2 research, it has been a widely studied area for researchers in first language (L1) reading for quite some time. The assessment of online comprehension is salient to L1 theories (Just & Carpenter, 1992; Trabasso & Magliano, 1996; Trabasso & Suh, 1993; Whitney & Budd, 1996; Whitney, Ritchie & Clark, 1991). The explanation-based theory of text comprehension (Trabasso & Magliano, 1996; Trabasso & Suh, 1993) proposes that successful comprehension hinges on the generation of causal inferences. A causal inference connects events in a text at a local or global level, allowing for integration of the text, and leading to both comprehension and coherence. According to the explanation-based theory, causal inferences indicate that the reader understands the textual relationships of causality and of goal plans (Trabasso & Suh, 1993). Trabasso and Magliano (1996) have identified different types of thoughts that may occur during the comprehension process. Two types of thoughts consist of associations (information is retrieved from background knowledge but not necessarily integrated into the text), and paraphrases (mere restatements of the text). Trabasso and Magliano (1996) also distinguished two types of inferences: explanations (inferences that causally link events in a text),and predictions (anticipation of future events or future consequences of a focal event based on integration of prior text information and background knowledge).

Coupled with the fundamental role of causal inferencing in comprehension, L1 reading theories address levels of mental representation of a text that occur during comprehension (Van Dijk & Kintsch, 1983). Essential to textual understanding, these levels, as identified by Van Dijk and Kintsch, include: surface level (linguistic processes), textbase level (propositional meaning of textual constituents), and situation model (integration of textbase with background and other world knowledge). The situation model is necessary for the generation of causal inferences and successful text comprehension (Zwann & Brown, 1996). The online generation of causal inferences, integrating prior text information and causal antecedents across sentences with background and/or world knowledge, leads to the production of a coherent situation model. This implies that comprehension is achieved at many different levels, not solely at the textbase or surface level. The generation of causal inferences may be indicative not only of the online construction of a situation model, but also of the achievement of a deeper comprehension of the text.

Many studies endorse online measures to reveal causal inferences generated during reading (Ericsson & Simon, 1984; Horiba, 1990, 1996; Kern, 1994; Olson, Duffy, & Mack, 1984; Rikard & Langley, 1995; Trabasso & Magliano, 1996; Trabasso & Suh, 1993; Whitney &


Budd, 1996; Whitney, Ritchie, & Clark, 1991; Zwann & Brown, 1996). Think-aloud protocols may offer a different approach to questions of comprehension. During the procedure, participants are asked, after reading each sentence, to verbalize their understanding of that sentence within the context of the story. Think-aloud protocols have been used to show online causal inferences (Trabasso & Magliano, 1996; Trabasso & Suh, 1993), to reveal the contents of working memory (Whitney, Ritchie, & Clark, 1991), to investigate the relationship between memory and inferences (Trabasso & Magliano, 1996; Trabasso, Suh, Payton, & Jain, 1995), and to show whether a coherent situation model is being constructed (Zwann & Brown, 1996). Rikard and Langley (1995) offer recommendations for using the think-aloud procedure in L1 studies. They suggest careful selection of recording equipment and students, as well as an adequate introduction to the think-aloud process. "Thinking aloud" enables readers not only to focus their attention on what they read, but also demands that readers devote more time to thinking about what they read (Kucan & Beck, 1997).

Zwann and Brown (1996) applied Trabasso and Magliano's think-aloud protocol to L2 reading to investigate causal inferences with students studying French. They compared the number of causal inferences generated during L1 reading by skilled and less skilled readers with the number of causal inferences generated in L2 reading. Although the skilled readers could create a strong textbase, neither group could construct a strong situation model. The application of L1 theories (causal inferences, situation model) and of think-aloud to L2 research represents a salient contribution to the field of L2 reading. Zwann and Brown (1996) suggest that regardless of skill, it may be difficult for L2 students to move beyond the textbase level because L2 readers focus on establishing a textbase and are strongly affected by fluency level.

Both online assessment of comprehension and the exploration of the generation of a situation model have been applied to L2 reading comprehension. However, since glossing and research involving multimedia annotations have not traditionally used online methods to measure comprehension, L1 theories could be applied to L2 glossing in a similar manner. In this pilot study, an online assessment of L2 comprehension, the think-aloud procedure, was utilized with three conditions: no glosses, traditional glosses, and extended glosses. The generation of online causal inferences was explored, as revealed by the think-aloud data in this study, to determine whether multimedia annotations aid online comprehension of a text. Assessing online comprehension not only provides insights into the reader's comprehension process, but it also more clearly reveals whether the glosses affect the generation of causal inferences and the construction of a situation model.

For this pilot study, not only did the reader verbalize his/her understanding of the text after each sentence, but he/she also vocalized reasons for choosing glosses. In addition to using the think aloud procedure, a tracker incorporated into the software was used to determine the number and type of glosses consulted, and the length of time glosses were consulted. Noblitt and Bland (1991) suggest that data from the tracker can provide researchers with a "new tool for studying student learning" and can provide insights into student learning levels: moving from the tactical to the strategic learning level (p. 130). Cubillos (1998), also discussing the tracking feature, stated that "student logs and student records of all kinds can provide teachers with unprecedented insights into their students' SLA processes" (p. 45). This measure allowed for further specification as to where comprehension or miscomprehension occurred in the text and the effect that each gloss had on comprehension.

In considering these prior studies, the following research questions surface:


1) Do multimedia annotations aid comprehension? If so, how? What is the relationship between type and number of glosses consulted and the number of inferences generated?

2) Do multimedia annotations allow the reader to build a situation model? How do type and number of glosses consulted compare with the level of comprehension achieved?

The answers to these questions could not only have strong pedagogical implications for glossing (i.e., whether they are effective for L2 students), but they also could hold strong implications for the use of technology in the L2 classroom. By using the think-aloud procedure to determine why readers consult certain glosses and whether glossing does aid their comprehension, this pilot study aims to further explore the effectiveness of multimedia annotations on L2 reading comprehension.



Twelve students enrolled in a second semester French course at the Pennsylvania State University participated in this pilot study. The informants filled out a background questionnaire immediately after giving their consent to participate. From the information provided on the questionnaire, twelve native English speakers between the ages of 18 and 30 were selected to take part in this research.


Participants read an excerpt (first and last stanzas) from the poem Femme Noire by Léopold Senghor, which had been annotated using GALT multimedia software by Travis Bradley and Lara Lomicka (1997). Before reading the text, participants were provided with an introduction to reading in a second language, to the GALT program, to Senghor, to Senegal, and to the poem itself.


The participants were randomly assigned to one of three conditions comprised of four students each. Those in condition A read the introduction and the text with no access to glosses. Participants in condition B read the introduction and the text with access to "traditional" glosses (definitions in French and translations in English). Those in condition C read the introduction and the text with access to all glosses (definition in French; images, references, questions, pronunciation, and translation in English).


Participants were tested individually in a quiet computer lab. After reading through the series of introductions, students worked through the poem at their own pace. Each participant received instructions for the think-aloud procedure. The think-aloud scheme used in this study was based on the work of Trabasso and Magliano (1996), and Zwann and Brown (1996). Participants were provided with a demonstration of the think-aloud approach and were further given the opportunity to practice the procedure with a short text. Specifically, participants were asked to "think aloud" in English while working through the text. They were asked to think aloud about whatever came to mind during their reading of the text, to vocalize which


glosses they were using and justify their choice (for those using glosses), and finally to voice their understanding of each line in the context of the poem at the end of every 10 lines of the text. Students were able to move to any of the lines of the text during the experiment. Their verbalizations were recorded on audiotapes. In addition to the think-aloud procedure, participants' interaction with the software was logged using a tracking feature. The tracker recorded all clicks on the controls, the type of gloss used, and the amount of time subjects spent consulting each gloss. The tracker feature was used in this study as a way to map the different types of multimedia annotations that students consulted.


Data from the think-aloud protocol were transcribed, divided into clauses (which served as the basic unit of analysis), and entered into a spreadsheet program. The categorization pattern proposed by Trabasso and Magliano (1996) was utilized, parsing clauses into paraphrases (mere restatements of the text), associations (information is retrieved from background knowledge but not necessarily integrated into the text), explanations (inferences that causally link events in a text), and predictions (anticipation of future events or future consequences of a focal event based on integration of prior text information and background knowledge). Two other categories were used in addition to Trabasso and Magliano's category scheme--metacomments and evaluations--which were coined by Zwann and Brown (1996). Metacomments represent not only the participants' lack of understanding of a word or sentence (Zwann & Brown, 1996), but also comments about the text structure and the glosses. Evaluations consist of ethical, moral, or affective statements based on the participants' integration of past events in the text such as: "Hum. I feel better about that one."; "That changes the whole thing!"; or "Huh! Yeah! That is interesting!" Whitney and Budd (1996) suggest that the segment classifications developed and utilized by Trabasso and Magliano (1996) and Zwann and Brown (1996) will continue to remain useful.

Participants collectively generated 734 clauses (excluding metacomments), 154 (21%) of which were explanations. In order to control for verbosity, metacomments were excluded from this calculation. This study focused primarily on explanations; predictions were excluded in calculations since only two predictions were generated out of 734 clauses. Two trained researchers individually categorized a sample of 150 clauses. Interrater reliability was .91. The rest of the clauses were divided between the tworesearchers and categorized. Disagreements were resolved through consensus. Statistical analyses of the data indicated that there were no significant differences in the percentage of explanations generated by each group. As displayed in Table 1, participants in condition A generated a total of 35 explanations out of 160 clauses (21.7%), condition B generated 35 explanations out of 227 clauses (15.4%), and condition C generated 84 inferences out of 347 clauses (24.21%). However, the raw numbers of explanations suggest that, with a larger sample population, statistical analyses might be significant.

 Inference Statements

The three protocols illustrated in Appendix I, Tables 1-3 are representative of typical data generated under each of the three conditions: condition A (no glosses), condition B (French and English definitions), and condition C (all glosses). Vocabulary differences were apparent in data collected from all three conditions. Participant 3 (cond. A) experienced frequent vocabulary problems which often resulted in incorrect translation attempts. Despite these difficulties, participant 3 did generate a fair number of explanations based on the little


information gleaned from the text and from the introduction. Similar vocabulary problems were experienced by participant 8 (cond. B), who was able to manage with less difficulty with the aid of the French and English glosses. When encountering unglossed words, most students were able to glean the meanings of unglossed cognates such as beauté. However, for the word Eternel (" I tried to look up Eternel. I guess maybe it's a city."), participant 8 (cond. B) was surprisingly unable to recognize this (unglossed) cognate. Although limited availability of glosses did serve as an impairment, particularly for participants in conditions A and B, participant 12 (cond. C) also struggled with certain words. Vocabulary difficulties occurred even with certain glossed expressions. This type of difficulty is illustrated by participant 12's encounter with the word aigle as demonstrated in Appendix I, Table 3. Although the English gloss revealed the meaning of the phrase, participant 12 did not make the direct connection between the French word aigle and its English equivalent eagle without the assistance of the image annotation.


A significant number of metacomments generated by the participants centered around glossing procedures. Certain participants did not understand how to manipulate the glosses, others had difficulty understanding the function of each gloss and how it could aid comprehension. Examples of metacomments related to annotations or manipulation include: "I hit on the image for this one."; "All right--next line!"; "Let's look it up."; or about reading strategies, "That helps me more [referring to the French definition] cause with the English, I think you're trying to stay away from English in the definitions here which is cool, 'cause we don't want to translate"; or even regarding specific control buttons, "Line 6 is very clear now with the pictures." Although this pilot study focuses primarily on inferences, the vast number of metacomments that were produced by the participants provides substantial data for another in-depth study.

Tracker Data

Data from the tracker were tallied and incorporated in Table 2. For conditions B and C, the most frequently consulted glosses included English and French definitions. Participants in condition B, limited to French and English definitions only, consulted the French definitional glosses more frequently than the English glosses. Even though participants in condition C did have access to six different types of informational glosses, there was a strong preference for the traditional definitional glosses. Reasons for this tendency will be further addressed in the discussion and conclusion sections. Contrary to participants in condition B, those in condition C did consult the English glosses more than the French glosses. The pronunciation gloss was highly consulted only by two of the participants and did not seem to directly affect comprehension. The annotation providing grammatical explanations, although consulted moderately, did not appear to directly aid students in developing a deeper level of comprehension because inferences were not generated.

The question and reference glosses are potentially beneficial to the learners because they could lead to the construction of a situation model. The question glosses, which were accessed by each person at least once, seemed to assist readers in moving beyond the textbase (see Appendix II, Table 1 for an example). However, participants may have consulted the question glosses because they wanted to ask or seek answers to particular questions, or because the English and/or French definitional glosses were not sufficient for comprehension. Regardless of the reason, after consulting the question glosses, students usually generated one or more


inferences. The reference glosses, consulted less frequently, also appeared to assist the student in moving beyond the textbase.

It appears that participants did not understand the relevance of the image gloss because it was consulted by only one participant. Participant 12, the sole consultant of this gloss, at first did so "to see what it would do." After several consultations however, participant 12 commented that the gloss did assist comprehension: "These pictures are GREAT 'cause then I associate a picture with the word and hence I'm not translating. I'm just going straight from images. So that really helps figuring out meaning." Appendix III, Table 1 provides an illustration of the effects image glosses had on his comprehension. Although these glosses seem to have assisted participant 12 in comprehending vocabulary words and generating explanations, further research is needed in this area.


Think-aloud data, along with tracker data, seem to suggest that the major obstacle for this group of L2 readers may have been vocabulary. They seemed concerned primarily with an immediate construction of a textbase. While some participants consulted the English and French glosses to determine simple word definitions, others used annotations to verify hunches. Their use of the glosses was oriented toward the goal of translation and paraphrasing in order to achieve a minimal level of comprehension. This tendency is also evident in group C, where tracker data indicate that even with availability of all glosses the readers chose the definition glosses first. These findings support the research of Lyman-Hager and Davis (1996) who reported a strong dependence chiefly on English definitions. Lyman-Hager and Davis (1996) suggested that the participants felt "the key factor in understanding the passage was accessing word meanings in their native language" (p. 62).

Furthermore, the data suggest that the generation of explanations increases with the use of the question and/or reference glosses. These two types of annotations would seem to aid the student most in the construction of a situation model. However, as previously noted, the majority of the participants concentrated uniquely on the English and French glosses. This could be the due to habit, or perhaps students were satisfied with the construction of a textbase
level. It could also be due to the fact that the participants did not understand the relevance of the other glosses in helping them with the reading process.

The raw data gathered from the think-aloud protocol and from the tracker point to several noteworthy tendencies. Multimedia annotations, as compared to no glosses or to traditional glosses, may have a positive effect on comprehension and on the construction of a situation model. However, students at this fluency level appear to use multimedia annotations primarily to construct a strong textbase and as a result do not fully explore the potential resources available. Although research is needed before generalizations can be made, the data show tendencies which merit further investigation in using multimedia reading software. As Chun and Plass (1997) have suggested, further research that investigates the effect of multimedia materials (such as audio, graphics, video, and images) on reading comprehension needs to be conducted.


Several limitations need to be pointed out in this pilot study. First, due to the small size of the


sample population, the results were not statistically significant. However, the data do suggest that replication with a larger subject population may be significant. Second, the participants were not provided with an extensive introduction to the types of informational glosses available, enabling them to understand how each type could aid comprehension. Due to testing conditions, students did not have the opportunity to explore the software and become comfortable with it prior to the experiment. Instructors would be strongly encouraged to devote an entire class period to the introduction of such programs, so that both teachers and students are comfortable with the software and know how to use it. In addition, certain vocabulary words in the glosses themselves were not understood by the participants, at times creating additional frustration for the students. Furthermore, the introduction to the poem and to the author may have provided too much information to the participants. Certain utterances generated chiefly by participants in conditions A and B were strikingly similar to phrases in the introduction to the poem. Finally, future studies should control for text effect where different types of texts, as well as text length, would be tested.

Though the limited size of this study disallows broad generalizations, several pedagogical issues have emerged which deserve further attention and research. First, this study suggests that instructors should discuss the process of L2 reading comprehension with their students. Research could be conducted that investigates what the concept of comprehension means and may entail for students. Many students erroneously interpret comprehension as a process consisting of mere translation and paraphrasing. Students seem to be generally satisfied with achieving a minimal level of comprehension via quick and basic translation. If they are able to translate the words into their L2, then they assume they have understood the text. In doing this, students often fail to achieve a global understanding of the text. Martínez-Lage and Herren (1998) assert that "technology-based annotated texts present us with new opportunities to assist our students in getting beyond the 'mechanical' aspects of the reading process and to provide them with a means of developing good reading strategies" (pp. 146-147). In order to demonstrate the complexity of the reading comprehension process, instructors could work through texts with their students, constructing both textbase and situation models. In differentiating between these two levels of representation, the process of co-construction could underscore the importance of moving beyond translation and paraphrasing.

Additionally, further research should be conducted which examines instruction of software by the FL teacher. Instruction, along with the use of activities involving group work and discussion (during, before or after reading), may assist students in achieving a deeper level of comprehension. Since the data suggest that informational glosses such as questions might aid students in generating inferences, an increased number of interspersed questions could be incorporated into the software in order to aid the development of a situation model. Questions could be answered in written form by the students or discussed during small group activities. Reading and the effectiveness of multimedia annotations need to be adequately explained by the instructor. Students who have only been exposed to traditional glosses may not understand how other types of informational glosses could be used to enhance comprehension. Moreover, this study could be replicated with a larger sample population, and with an increased number of texts. It would also be interesting to compare results across languages and levels. A similar study could also be conducted combining an offline measure, such as a recall protocol, with an online measure, such as the think-aloud procedure. Future studies such as these will shed more light on L2 reading research involving technology.

Although more research is needed, the data from this pilot study suggest that the use of multimedia annotations may aid comprehension and the development of a situation model.


Although students are often accustomed to single word or phrase decoding, experience with multimedia annotations can help to convey the necessity of moving beyond a simple text-base comprehension. The increased integration of computers into FL curricula can facilitate more research on the relationship between different multimedia annotations and comprehension. 


I am indebted to T. Danielle Blount for her help in conducting this study and for her numerous contributions to the first draft of this manuscript. I am also very grateful to Carol Brown, Travis Bradley, and Pat Shannon for reading and commenting on various drafts of this paper. Finally, I would also like to thank the anonymous referees for their valuable comments and suggestions.


Lara Lomicka is Co-Director of Instructional Technologies at the Pennsylvania State University and a Ph.D. candidate in Foreign Language Acquisition in the Department of French. She is interested in classroom research and technology, and has been involved in several multimedia projects in French.





Top | Home | About LLT| Subscribe | Information for Contributors | Masthead