Herding Cats: A Descriptive Case Study Of a Virtual Language Learning Community
The principal aim of this study was to understand and describe an online ESL/EFL learning community. This fits in with one of the characteristics of qualitative research, which, as J. Mason (1996) states, is “concerned with how the social world is interpreted, understood, experienced or produced” (p. 4). In particular, I carried out a case study which, as Stake (1995) says, was “expected to catch the complexity” (p. xi) of a group of ESL/EFL students who were participating in the online class known as Writing through the Internet (WTI)--a class offered under the auspices of the Vandar Online English School (VOES).
The term “case study” is rather imprecise and can be used to describe almost any form of research using a non-cross-sectional organization (J. Mason, 1996). The purpose for this methodology is to retain the holistic and meaningful characteristics of real-life events and to understand the specific case under study, as well as to describe in depth how things were at a particular time and place, instead of trying to explain why things are the way they are (Stake, 1995). Stake gives the time period involved in a case study as being an hour, a day, a week, a month, a year, or several years, depending on the case under study. Since what I found in WTI was not a class in the usually accepted sense of the word--a group of students with a teacher who meet for a specific length of time, with a definite beginning and a definite end--I used an arbitrary five-month period, which corresponds approximately to a standard semester in most schools. I started my interviews on January 1, and ended them on June 1. In the next chapter I (Chapter 4: Controlled Chaos) will discuss the WTI class in depth, paying particular attention to how it has developed into a language learning community instead of being a normal VOES class. This development will be discussed more in depth in the section on Context and time frame.
In Lincoln and Guba’s (1985) naturalistic paradigm, qualitative researchers carry out their research where the phenomenon naturally occurs, because a phenomenon’s reality cannot be understood separate from the context in which it occurs. I decided to use the naturalistic paradigm for this study because, any community, virtual or otherwise, is a context dependent phenomenon with a history, set of traditions, and processes of interaction that must be studied as they naturally occur.
In carrying out my case study I used what B. Mason (1996) calls a virtual ethnography (See also, Hine 1998, 2000a, 2000b; B. Mason 1999), which is “simply an ethnography that treats cyberspace as the ethnographic reality” (B. Mason, 1996, p 4). What differentiates a virtual ethnography from a regular ethnography, then, is that it occurs completely online. The researcher becomes a part of the virtual community he/she is studying, but does not physically meet the other members of the community except by accident. According to Hine (2000a), the Internet can be considered a site for interaction that is ethnographically available because of the assumption that “what goes on within the Internet is social interaction” (p. 50).
As discussed below, there are conditions related to conducting online interviews that face-to-face interviewers seldom encounter. The email discussion list “Virtual Methods” <http://www.jiscmail.ac.uk/lists/virtual-methods.html> deals with research using the Internet. In a discussion on this list, D. Slater and Y. Waern discussed some of the problems with synchronous online interviews (Slater, message on Virtual Interviews posted to the Virtual Methods discussion list, 27 Feb 2000; Waern, message on Virtual Interviews posted to the Virtual Methods discussion list, 27 Feb 2000). Among the points they raised were:
1. The interface of most chats and MOOs are not interview-friendly. First, the channel may not be serious enough to be suitable for interviews. Second, most of these spaces need extra features such as a private room where you do not get disturbed and the ability to log the interview. Third, the people being interviewed are usually involved in quite a range of other activities while you are talking to them.
2. It is difficult to gauge the dropouts from an online interview study.
3. The synchronous nature of chats makes it more difficult to find times that suit both the interviewer and the informant, particularly when the participants are from different continents.
Paccagnella (1997) points out another problem with virtual studies. The membership of any virtual community always consists of both people who actively participate and those (often the great majority) who merely read the messages without actively participating in most discussions. Further, as B. Mason (1996, 1999) points out, much of the communication that takes place in an online community actually occurs via private email. If a researcher merely observes what takes place in the public portions of the community, he/she will miss much of the give-and-take that occurs.
According to Mann and Stewart (2000), text-based media “is not an appropriate method for research which seeks to observe the ‘real’ world” (p. 84). At the same time, they point out that recent research that focuses on virtual communities is challenging this idea. Further, they point out that most virtual interviews are carried out using email. They also found that “researchers who have carried out non-standardized interviews using asynchronous CMC are divided about their success” (p. 76). On the other hand, they note that Bennett (1998) used chat rooms to carry out in-depth interviews and preferred chat to either face-to-face or email alternatives because it allowed for a balance between the researcher and the informant as well as for negotiated meaning.
B. Mason (1996, 1999) gives two strategies to help overcome these problems.
1. The electronic survey . While electronic surveys can provide information on the attitudes and thoughts of the members of the community, B. Mason (1996, 1999) also points out that they are prone to one of the features mentioned above, the split between active participants and “lurkers” (those who do not actively participate).
2. Use of the email interview . B. Mason (1996, 1999) also points out that email interviews have the danger of leading to information overload due to the number of email messages being received. He recommends that, if a researcher uses this approach, he/she should, after receiving an informant’s consent to be interviewed, send an email message with three or four short questions and wait for a response before sending the next set. This procedure would allow the interviewer to set up a semi-structured interview that can then be manipulated according to the answers received to each set of questions. Further, each set of questions should include the subject: “Interview, part x” so that the relationship is defined within the communicative act.
A further element of the virtual ethnography is directly related to the virtual space in which the research takes place. According to Thompson, Straubhaar and Bolyard (1998), ethnographies have always taken advantage of written materials from a culture. The use of written materials, however, has usually been only a part of the data examined. In the case of virtual research, the researcher has nothing but text. The virtual ethnographer cannot observe people except through their text. Even in the case of online interviews, the data to be examined is text. Although this emphasis on text can be limiting in the amount of information the researcher can observe, it also presents the opportunity for the researcher to find more information since “all speech, behavior, community rules, and community history is, in principle, likely to be available online for the researcher's inspection” (Thompson, Straubhaar and Bolyard, 1998).
This study took place in an online ESL/EFL class originally offered under the auspices of the Vandar Online English School (VOES). The interviews and data collection/generation started on January 1, 2001 and ended on June 1, 2001.
VOES is a language school that uses volunteer teachers to offer free ESL and EFL classes to students from all parts of the world. Teachers in VOES are encouraged to develop their own classes using their experience as language teachers. While the teachers are supposed to have at least a Master of Arts in TESOL or equivalent, most of them work for the experience in online education that it offers. At the time of this study, most VOES classes were asynchronous. The class I studied (Writing through the Internet - WTI) is one of the few that were offered synchronously.
Although at the time this study began, the WTI class still received students from the VOES administration, it had become basically autonomous, accepting any student who appeared in one of the class sessions and then sent an email subscribing to the e-class.
Although a complete description of the WTI class can be found in the next chapter, a brief description of the class is given here. VOES classes currently last 6 weeks. At the start of this study, most classes lasted for 3 or 4 months with the option for students to continue in the class for more than one term. WTI students, on the other hand, sign up for the class and automatically continue until such time as they notify the teachers that they wish to leave. For this reason, there are students who have been in the class from its beginning (and had been members of previous versions of the course). Another difference between WTI and other VOES courses is the lack of a specific set of lessons. Instead, each participant is expected to take part in three basic assignments: join and post to the e-class, participate in the weekly chat sessions, and create a personal web page. Given the nature of the class, it is understood (and allowed for in the instructions found on the course web site) that not all students will carry out all three assignments. In particular, the chat sessions will be difficult for many of the students, either due to equipment and connection problems or due to time zone problems. Therefore, while it is hoped for that students will participate in all three assignments, they are expected to participate in at least one of them. A complete description of the assignments can be found in the next chapter.
The class has approximately fifty members of whom four are teachers and the rest are students, or, as the coordinating teacher prefers to call them, tutors and friends. The teachers are all native speakers of English and professional ESL/EFL teachers who volunteer their time and energy to the class.
Although the students are of all ages, most of them are adults between the ages of 20 and 60 with at least an undergraduate degree or currently studying for such a degree. There are both male and female students. The students come from various occupations. There are lawyers, engineers, college and high school students, business people, publishers, factory workers, teachers, accountants, and employees of their government. Students in this class come from Europe, Asia, South America, North America, and Oceania. But all of them have some things in common. Their command of written English is, in general, advanced. While they often have problems with pronunciation, those who participate in the voice chat sessions generally have a good command of aural English (listening skills). Also, they have an interest in improving their English and at least some knowledge of computers and the Internet.
This study describes a virtual language learning community from the point of view of 5 ESL/EFL students during six months of their participation as part of this community. One of the informants consistently participates in the synchronous portion of the class; one never participates in this portion of the class; and three participate on an irregular basis. While my main data generation arose from online interviews with these five students, I also obtained data through publicly available data concerning the community, archived assignments that these students had posted to the e-class or the class web page, publicly available (archived) chat logs posted to the class web page and, of course, my own observations as a participant in the community.
There is a general consensus that interviews are one of the best ways to generate qualitative data (Burgess, 1984; Holstein & Gubrium, 1995; Marshall & Rossman, 1995; J. Mason, 1996; Meriam, 1988; Sommer, & Sommer, 1991; Spradley, 1979; Stake, 1995; Tellis, 1997a, 1997b; Yin, 1993; Yin, 1994). In fact, as J. Mason (1996) points out, many qualitative researchers use interviewing as their main method of generating data. The interviews are generally characterized by a relatively informal style, a thematic, topic-centered, biographical or narrative approach, and the assumption that data are generated via the interaction between the interviewer and the interviewee. They may involve one-to-one interaction or larger groups (pp. 38-39).
Interviews take one of three basic forms: open-ended, focused, or structured and can involve either one-to-one interaction or a larger group (Tellis, 1997a). The open-ended interview consists of questions that ask about opinions. The focused interview consist of questions designed to generate data about a specific event, and are usually based on information received from other sources. Finally, the structured interview consists of questions with a choice of answers and is particularly useful in studies where a formal survey is required.
Lincoln and Guba (1985) identify two types of interviews: structured and unstructured. In the structured interview, the questions are in the hands of the interviewer and the respondent provides the answers. A structured approach is the mode of choice when the interviewer knows what he/she needs to learn. In the unstructured interview, both the questions and the answers are in the hands of the informant. An unstructured approach is the mode of choice where the interviewer does not know what he/she needs to learn. In other words, the structured interview is used when the interviewer has sufficient background knowledge, either through previous research or through previous contact with the culture under study, while the unstructured interview is used when the interviewer has little or no background knowledge.
Seidman (1998) recommends in-depth interviewing. The goal of this type of interview is to have the participant reconstruct his/her experience with the topic through a series of open-ended questions that enable the interviewer to build upon and explore the answers to each question. The model Seidman recommends is straightforward. The interviewer has a series of three interviews with each participant. Each interview lasts 90 minutes and is scheduled 3 days to a week after the previous one, thereby allowing time for the participant to consider the questions and responses in the preceding interview. Each of the three interviews has a different purpose. The first establishes a context for the participant’s experiences; the second allows participants to reconstruct the details of the experience, and the third encourages the participant to reflect on the meaning of his/her experience.
In this study, I used a modified version of in-depth interviewing to take into account the specific requirements of virtual research. In general, each synchronous interview lasted between 60 and 90 minutes and was focused on one general area of interest. These general areas were related to 1) background of the informant, including how he/she became a member of the community, 2) The informant’s view of the class and his/her participation in it, and 3) the informant’s view of whether or not the class is a community.
In the asynchronous interview and the follow-up questions I asked by email (generally clarification questions), each message built on the informant’s response to the preceding email.
I used the following means of data generation.
1. Participation in the WTI online language learning community.
My participation in the class consisted mainly of participating in the regular Sunday chat sessions, but also involved posting to the e-class and synchronous meetings with individual students outside of class. These extra-class time meetings occurred when students asked for my help in one of their assignments or in proofreading/editing English writing that they had to turn in either in their work environments or their regular face-to-face classes.
2. Online conversations/interviews with members of the community.
Voice chat--When I first joined the community it used a voice chat room that was provided by HearMe. Unfortunately, HearMe was discontinued about the time I started my interviews, so I was only able to hold one interview using this client. While there were attempts to find another client that would work as well, by the time we found one, most of the interviews had already concluded.
Text-based chat: At the request of my informants, most of the interviews were held using ICQ. The advantages of ICQ were threefold. 1) Everybody was already familiar with it since they use it in their class. 2) ICQ automatically saves all chat sessions, so both the informant and I had copies of the interviews. 3) ICQ time stamps each entry so there is a record of how long it took for a response.
E-mail based interviews -- Follow-up questions after the third interview were submitted by e-mail, thereby allowing the participant more time to consider their answers. In one case, the entire interview was by email.
3. Publicly available documents that the informants had submitted to the e-class and/or placed on their class web pages.
These documents were either sent through email to the e-class or were announced to the e-class by the coordinating teacher. In this second case, the coordinator also requested that members of the class read and respond to the posting. These postings were also placed on the web page.
4. Publicly available chat-logs of the regular community chat sessions.
All weekly chat sessions were logged and placed on the class web page together with a short analysis of what had gone on during that session. Once posted, the coordinating teacher would send a message to the e-class inviting the members of the community to read them and make comments on what was discussed in the session.
Data collection and analysis go hand in hand to promote the emergence of substantive theory grounded in empirical data. For this reason, the researcher starts analyzing very early in the research process (Marshall and Rossman, 1995; Merriam, 1988; Rubin and Rubin, 1995; Swaffar, 1998; Stake, 1995; Warschauer, 1999).
According to Marshall & Rossman (1995), there are five modes in qualitative analytic procedures, each of which involves analysis and interpretation of data. These modes consist of organizing the data; generating categories, themes and patterns; testing the emergent hypotheses against the data; searching for alternative explanations; and writing the report. Of these steps, the generation of categories, themes and patterns is one of most basic. In this basic level of analysis, Merriam (1988) suggests using an “unordered meta-matrix” (p. 155). This is a large chart organized by categories of interest to the researcher and contains illustrations of what the category includes. From this descriptive display, one can advance to higher levels of analysis.
Marshall & Rossman see qualitative data analysis as “a search for general statements about relationships among categories of data” (p. 111). They also see generating categories of data to collect, or cells in a matrix, as an important focusing device for the study.
Further, Rubin and Rubin (1995) see coding as the “process of grouping interviewees’ responses into categories that bring together the similar ideas, concepts, or themes you have discovered” (p.238). This process proceeds in stages. First, by setting up a few main coding categories as suggested by the original reading of the interviews and the intended purposes of the study and then making a list of new codes while re-reading the interviews. Further, the researcher can code on anything he or she thinks may later help in the data analysis. In addition, coding involves “the development of conceptual categories, typologies or theories that interpret the data for the reader” (Merriam, 1988, p. 133) by looking for recurring regularities in the data and asking which units of information go with each other by comparing one unit of information with the next.
After each interview, I formatted the session log for ease of reading and to aid in recovery of data from the session. After formatting the session, I began to analyze the log for the following elements: 1) questions that needed further exploration, 2) new questions that should be asked related to the topic, and 3) themes that could be seen as arising from the data.
I used the following categories in coding my data.
1. Reasons the informant decided to study online
2. Reasons the informant joined this particular class
3. Manner in which the informant learned about both VOES and WTI
4. Attitudes related to the e-class
5. Attitudes related to the synchronous chat sessions
6. Reasons for participation/non-participation in the synchronous chat sessions
7. Attitudes related to the assignments
8. Types of assignments the informant preferred to answer
9. Experiences the informant has had in the class
10. Elements of community that are present in the class
11. Attitudes related to usefulness of online ESL/EFL courses
12. Attitudes related to the use of the internet
13. Experience in taking online courses
14. Experience in using the internet
15. Problems the informant has in the class
16. Modifications necessary for online interviews
According to Lincoln and Guba (1985), trustworthiness suggests credibility in place of internal validity, transferability in place of external validity, dependability in place of reliability, and confirmability in place of objectivity (pp 281-84 and 289-331). According to Glesne and Peshkin (1992), a researcher has established trustworthiness when the participants agree that what the researcher has written is true. I established trustworthiness by triangulation.
According to Tellis (1997a), triangulation is the means the researcher uses to confirm the validity (trustworthiness) of the methodology he/she is following. The term comes from the multiplicity of methods of generating data or, as J. Mason (1996) says, “triangulation involves using more than one method” (p. 42). Stake (1995) also indicates that the use of multiple sources of data is one means of triangulation. Further, according to Feagin, Orum, and Sjoberg, (1991), Snow and Anderson asserted that triangulation can occur with data, investigators, theories, and even methodologies. According to Lincoln and Guba (1985), triangulation consists of validating each piece of information against at least one other source, such as a second interview and/or another method, such as an observation as well as an interview.
Of the four types of triangulation identified by Denzin (1984), Data source triangulation, Investigator triangulation, Theory triangulation, and Methodological triangulation, I used Data source triangulation, which Denzin identifies as occurring when the researcher uses different contexts to obtain data and expects it to be the same in each context.
To triangulate my data, I used three contexts: first, participation-observation as a member of the class; second, synchronous and asynchronous interviews; third, analysis of publicly available documents related to the class.
It is always important to protect the anonymity of any human participants within a study of this type. To protect the identity of the participants, I changed the name, nationality and pertinent personal information of each person. To avoid unconscious identification of names with specific nationalities, I used only English names. Further, for purposes of confidentiality, I changed the name of the school and the particular course I was observing.
The next chapter describes the WTI community in detail, paying particular attention to how it developed from a class into a community and how it changed over the course of the study. It also discusses how students interact in this community both within regular class sessions and outside of class time.