Quantitative and Qualitative Studies

Research Redesign

Introduction

In 2002, I served as the head principal of a large junior high school in Seattle, Washington. My responsibilities included supervising student teachers placed in my building by various colleges and universities in the state. On a daily basis, I conducted a building-wide “walk through” that enabled me to observe the student teachers.

I noticed how some seemed to mold naturally into the role of classroom teacher while others struggled with even the most mundane tasks. This piqued my interest to examine more closely the art of teaching.

            In the fall of 2003, I entered the Ph.D. in Leadership and Change program at Antioch University. A unique feature of the program was its focus on individualized approaches to learning products (papers). Because of my interest in teaching as described above, I chose two research studies as part of the Research Redesign learning product (paper) requirement: Teacher Clarifying Behaviors: Effects on Student Achievement and Perceptions  (Smith, 1977) and Does the “Art of Teaching” Have A Future? (Flinders, 1997). This document will explore how these studies, conducted using one method, can be redesigned using the opposite method.

The first study involves the correlation between students’ achievement and perception and certain phrases that teachers use to communicate explanations of concepts. In addition, it identifies teachers’ verbal behaviors that have a negative effect on students’ task involvement and in turn on students’ achievement and perception. The lecture notes used as handouts are supposed to counterbalance the aforesaid negative effect. It is a quantitative study that will be redesigned into a qualitative study.

The second study concerns the artistic dimensions of teaching. Focused themes (communication, perception, cooperation, and appreciation) emerged gradually over the course of the research, which is a qualitative study that will be redesigned using a quantitative approach.

This paper is formatted into three main sections: (1) the quantitative redesign, (2) the qualitative redesign, and (3) the general conclusion. Before redesigning, it is important to explain the general frame within which the redesign is conducted in general. Many researchers asked a question where the boarder between the qualitative and quantitative research lies. Trochim points out that “the heart of the quantitative-qualitative debate is philosophical, not methodological” (Trochim, 2000). According to Trochim, an “approach” to inventing any design is “a general way of thinking about conducting […] research”, which “describes, either explicitly or implicitly,  the purpose of the[…]  research, the role of the researcher(s), the stages of research, and the method of data analysis. So, in the “Delimitation and Assumption” parts for both redesigns the changes on the aforesaid points in comparison to the original cases are specified.

To proceed, the same source provides the researcher with four basic design elements, which are time; program(s) or treatment(s); observation(s) or measure(s); and groups or individuals. The changes on the level of elements are also stated in the appropriate sections and in the “Delimitation” sections for both redesigns. Both main parts contain “Checks for reliability and validity” section. Trochim states that “[…] validity refers to the approximate truth of propositions, inferences, or conclusions” (2000) and reliability in research means “repeatability” or “consistency”. The measures to make the redesigns valid and reliable are pointed out as well. Each of the two main parts contains nine sections, and the final conclusion refers to the both redesigns.

To conclude the introduction, from the literature review made for the paper the general claim for any researcher to think flexibly and strategically when attempting to redesign a study is vivid. “While no definitive approach for designing designs exists, we might suggest a tentative strategy based on the notion of expansion”, states Trochim (2000). The way the original cases were expanded in both the redesigns will be explained for each part. The conclusion summarizes the debates on the qualitative-quantitative dichotomy in the modern research area and suggests the general frame for any further expansion of the researches of interest.

Research Redesign of a Quantitative Study

Quasi-experimental Study:

Teacher Clarifying Behaviors: Effects on Student Achievement and Perceptions

(Lyle R. Smith, Augusta College)

Introduction

The general frame for the present redesign is the quantitative quasi-experimental research Teacher Clarifying Behaviors: Effects on Student Achievement and Perceptions by Lyle R. Smith. In turn, this researcher gained inspiration from the report by Jack Hiller et al (1969), as well as from the work of Jacob Kounin. Hiller with the colleagues and Kounin independently concentrated on examining teachers’ communicative patterns used to explain concepts to the students. Smith based his research upon the aforesaid reports about a negative correlation between students’ task involvement and in turn achievement, and certain teachers’ behaviors.  In addition, Smith included ideas from other research about the effectiveness of teachers’ use of lecture notes as handouts.

The independent variables of teachers’ clarifying behaviors in the Smith’s case were “bluffing”, “uncertainty”, “discontinuity” and “handouts”. The first three variables were reported as negative clarifying behaviors and the latter one was observed as positive. The target population for Smith’s research was 448 social studies students from 8 public high-schools in 3 counties of Georgia.  Each student participating was assigned to one of 16 groups (n=28 each) according to the ability to comprehend social studies material presented in tape-recorded lessons.

In the quasi-experiment the uncertainty condition in teacher behavior negatively correlated to the student performance while the handouts usage had a significant positive effect on the dependent variable of student performance. The variables of discontinuity and bluffing showed no significant main effect on the student performance though had a strong intercorrelational effect.

Smith states, “The results of my research were rewarding. I found lessons in which the teachers’ communication contained certain negative behaviors identified by Hiller and his colleagues that have an impact on students’ achievement and perception.” Smith adds, “Also, I found that some teaching behaviors affect one another in interesting ways. For example, if a teacher uses many phrases reflecting uncertainty and discontinuity, the use of lecture notes as handouts appears to compensate for the negative effects of these behaviors.”

According to the task, the quantitative quasi-experimental study by Smith needs to be redesigned into the qualitative one with the main research question staying the same: if the use of lecture notes could counteract the negative effects that specific verbal behaviors of teachers have on students’ achievement and on their perceptions of the quality of the teaching they receive.

Delimitations and Assumptions of the Re-designed Study

According to the Trochim’s three-fold classification, Smith’s case was a quasi-experimental design, where multiple groups were non-randomly assigned to the treatment of handouts as a positive teacher clarifying behavior and of negative clarifying conditions, and where multiple waves of measurement were used (one pretest measure of the 16-item test on the comprehension ability; two posttest measures of the 20-item test on the comprehension ability and the 11-item evaluation).

The term quasi-experiment was introduced by Campbell and Stanley (1963). “Since then, the researchers [Campbell and Stanley (1963, 1966); Cook and Campbell (1979); Kidder and Judd (1986); Rossi and Freeman (1985)] have tended to see this area as involving primarily two interrelated topics: the theory of the validity of casual inferences and a taxonomy of the research designs that enable us to examine causal hypotheses” (Trochim, 2000). Smith also struggled to obtain “[t]he issue of causality and, hence, predictability” (Cohen et al, 2000: 211). “Smith claims the high ground for the experimental approach, arguing that it is the only method that directly concerns itself with causality; this, clearly is contestable […]” (ibid.).

This model, premised on notions of isolation and control of variables in order to establish causality, may be appropriate for a laboratory, though whether, in fact a social situation either ever could become the antiseptic, artificial world of the laboratory or should become such a world is both an empirical and a moral question respectively. Further, the ethical dilemmas of treating humans as manipulable, controllable and inanimate are considerable. (Cohen et al, 2000: 212)

From the epistemological point of view, “qualitative researchers believe that the best way to understand any phenomenon is to view it in its context” (Trochim, 2000). On the ontological level, they “[…] don’t assume that there is a single unitary reality apart from our perceptions” (Trochim, 2000).  Given these assumptions, let us state briefly the logic of this particular redesign.

The redesign is likely to combine different qualitative approaches. Firstly, on the preparatory stage of the research and throughout the whole study, the researcher freely uses ethnography approach, by which “an entire culture” of the U.S. public high-school social studies classes is studied. The researcher(s) implements a method of direct observation to gather qualitative data in the redesigned research. Secondly, on the concept firmament stage and further throughout the entire evaluation of different teacher clarifying behaviors, the phenomenology is of help as this “school of thought […] emphasizes a focus on people’s subjective experiences and interpretations of the world” (Trochim, 2000).

But what is immensely directing and framing the redesigned research is the “Grounded Theory” approach, originally developed by Glaser and Strauss (1967).

The self-defined purpose of grounded theory is to develop theory about phenomena of interest.  But this is not just abstract theorizing they’re talking about.  Instead the theory needs to be grounded or rooted in observation […]. Grounded theory is a complex iterative process.  (Trochim, 2000)

The “Grounded Theory” frame for the present redesign originates from several “specific theoretical expectations” as Trochim puts it, to be specific, from the hermeneutic empiricism (Peirce, Ch. S.; Weber, M.; Mead, G. H.; Dewey, J.; and Parsons, T.); modern intrapersonal communication theory and constructivism.

Hermeneutic theories generally hold that human behavior is organized in action signs which are understandable as indicative of what is being done. Human behavior is always a symbolic expression never simply an objective fact (Parsons, 1937; Schutz, 1965; Weber, 1974). (Anderson in Salwen & Stacks (Eds), 1996: 46)

With the linkage to the communication processes, the hermeneutic scholars strive to “test their theories in light of daily events, in commonly-placed situations, framed by the interaction of their participants” (Anderson, 1996: 45), and focus on the “accounts of everyday life” (Stacks et al, qtd. ibid.). Thus, in the research the communication between a teacher and a student is viewed from a “transactional or coorientational perspective”.

That is, two (or more) people coordinate their communication to reach a shared perspective satisfactory to all. Of paramount concern is the relationship between the two people and the perceived well-being of the “other.” (Salwen & Stacks (Eds), 1996: 234)

The valuable for the present redesigned research is also the theory of the “intrapersonal communication” – “the study of how people process messages” (Goss, in Salwen & Stacks (Eds), 1996: 335) with the attention to the “differences in brain activity” which result in “differences in behavioral patterns” (ibid.). Lastly, the new roles of the participants within the communication in the class system are taken into account.

Constructivism leads to new beliefs about excellence in teaching and learning and about the roles of both teachers and students in the process. In constructivist classrooms, students are active rather then passive; teachers are facilitators of learning rather than transmitters of knowledge (Stein et al., 1994: 26).

To put it in a nut-shell, the epistemological assumptions for the redesign would be different in comparison to the Smith’s case. As the processes under research are assumed to fit into the “communication field”, the researcher “faces the challenge of successfully developing synergy and synthesis among its most accepted and popular theoretical frames […]” (Atwater in Salwen & Stacks (Eds), 1996: 539-40). The redesign adopts, firstly, a constructivist contextualist worldview, treating the classroom environment as non-static text that has to be actively interpreted and construed to establish the mutual understanding among the participants (teacher and students). Secondly, it follows the constructive alternativism, where knowledge is viewed as a process of construction and reconstruction of different meanings. Third, it adopts pragmatic and predictive usefulness as justification criteria, since no proposition is considered to be ultimately truthful. Fourth, the redesign within the constructivist paradigm treats human beings (both teachers and students) as proactive and future-oriented.

The grounded theory methodology (Glaser & Strauss, 1967) in the redesign allows the researcher to explore and understand the perception held by high school students of teacher uncertainty, bluffing, discontinuity and the use of notes / handouts in instruction.  “The perceptions of different participants [is] brought together to find agreement or inconsistencies and some kind of action programme [is] devised and evaluated in the light of how people interpreted events” (Wragg, 1999: 21).

On the level of the purpose of the present qualitative redesign, the task stays the same like in the Smith’s case: to probe the ideas of the teachers and students of the U.S. public high-schools about the phenomenon of specific teacher clarifying behaviors and their effect on student achievement and perception of the lecture quality. In the redesign, though, the researcher seems to play a greater role than in the original case. In the qualitative research the central viewpoint “[…] recognizes that the researcher’s subjectivity deeply affects the research” (Hara, 1995), while the quantitative purist Smith, believed as it is evident from his research that “[s]cientists are responsible for putting aside their biases and beliefs and seeing the world as it ‘really’ is” (Trochim, 2000).

From the point of research stages, the redesign does not sweep aside completely the results achieved by Smith in his quasi-experiment, but expands them in a number of ways, thus, across time, observations and groups. However, the “program” so far as it may be related to the qualitative study will stay the same: it consists of presenting three negative and one positive teacher clarifying behaviors among those, which are identified by the participants of the redesigned research. It should be noted here, that when the qualitative data on the research topics would be collected and analyzed, there would likely emerge additional “program” patterns, which may in turn affect the variable of student achievement and perception. Then, the expansion across the program and results as well would occur.

Hypothesis

Following the hypotheses of Smith case, the redesign adopts one-tailed hypothesis pattern:

H₀ –  the use of lecture notes either doesn’t significantly counteract or increase the negative effects that specific verbal behaviors of teachers have on public high-schools students’ achievement and on their perceptions of the quality of the teaching they receive

H_A – the use of lecture notes either significantly counteracts or decrease the negative effects that specific verbal behaviors of teachers have on public high-schools students’ achievement and on their perceptions of the quality of the teaching they receive.

The redesign seeks to establish the causal relationships between two models. The first consists of the conceptual frame of lecture notes used as handouts, their significance for the learning process and the structure. The second model consists of several variables correlating: the students’ achievement and perception; and the teachers’ specific verbal behaviors consisting of bluffing, uncertainty and discontinuity as well as other conditions. If there is no evident cause-effect relationships between the variables, the correlation effects to be found are also valuable.

Method

Unlike the non-random assignment and non-random sampling in the Smith’s case, the redesign uses the multi-level sample strategy, where the preference is given to the random patterns. It is the US public high-school students, on the one hand, and their teachers, on the other hand, who form the theoretical population of interest to the study.  The redesign adopts the multi-level sampling strategy.

Firstly, the researcher begins with a state (Washington) sample₁ of school districts stratified by economics and educational level. The socio-economic factors may have either implicit or explicit effect to the ability of the students to participate in the communication processes (the language ability of listening, reading and speaking skills, the reading and listening habits and so on) and to the ability of teachers to use different communication techniques (the bad conditions in schools may affect the practice of using handouts). The educational level may correlate to the student achievement and perception levels. Presumably three districts in a state were randomly selected from the sampling frame.

Within selected districts, the researcher does a simple random sample₂ of schools (6 schools per each of three districts, thus, 18 schools). Within schools, the researcher chooses a stratified random sample₃ of classes (by student performance and teacher qualification levels [high/medium/low]). It is nine classes in each school, which represent cases in the study. The total number of cases is 146 then.

Within classes, several subsamples are formed at different stages of the research. On the first and the final stages, all the teachers and students of nine classes in each school are asked to complete the survey questionnaire. Besides, the subsamples of texts (lesson plans, handouts) and tapes (video and audio) are taken into account. For the in-depth interviews, the researcher selects the subsamples of students on a stratified random method by the reaction to the teacher clarifying behaviors and achievement. All the nine teachers in each school participate in the interviewing.

Instruments/Rationale

The main rationales for choosing the instruments for the redesign are conceptualization and evaluation. Trochim, e.g., gives two definitions of evaluation:

Evaluation is the systematic assessment of the worth or merit of some object. […] Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object. (2000)

In the case of redesign the “ambiguous term ‘object’” (Trochim, 2000) refers to, firstly, the communication program/policy (positive vs negative teacher clarifying behaviors), and secondly, to the person implementing the program, thus, the teacher. “The generic goal of most evaluations is to provide “useful feedback” to a variety of audiences” (Trochim, 2000) which here includes students, teachers, and, probably, methodologists and educators. “[F]eedback is perceived as “useful” if it aids in decision-making” states Trochim.

In the redesign case, the evaluation strives to get the feedback on the best communication techniques in order to improve students’ achievement and perception as well as the teachers’ achievement and perception. Another goal of the evaluation is to find out the most adequate scheme of proportional use of handouts and other positive clarifying strategies during the lectures. The strategy chosen for evaluating the model here is the ‘grounded theory’ approach of Glaser and Strauss among others belonging to the qualitative/anthropological models, which “[…] emphasize the importance of observation, the need to retain the phenomenological quality of the evaluation context, and the value of subjective human interpretation in the evaluation process’ (Trochim, 2000).

With the objective of deriving the qualitative data two measurement instruments of the questionnaire and in-depth interview are used.

Questionnaires are designed to reveal people’s attitudes through the opinions they express. A researcher asks for opinions on the assumption that information about people’s preferences can help explain and predict their behavior in decision-making situations. (Thomas, 1998: 162)

The questionnaire (I a, b) (Appendix A: 26) is used on the first stage of the redesign in order to obtain the concept map of specific teachers’ clarifying behaviors, both positive and negative. The respondents name these behaviors, and rank them, firstly, on the degree of their personal appreciation, and secondly, on the degree of the every-day practice occurrence. Besides, the respondents provide the researcher with their background information, important for establishing the variables and the possible side effects to correlations and possible results. The questions are open-ended so that to give the respondents the free space to map their concepts and to provide the researcher with the wider range of concepts.

The questionnaires are precedent of the interview survey of teachers and students at the second redesign stage in order to deepen the concept map of notions interrelating and showing /not showing the propositional cause-effect correlation. Each interview covers two sets of variables (multidimensional constructs”: firstly, the concept of the specific teachers’ behaviors intercorrelating with students’ achievement and perception of the teaching quality; and secondly, the concept of the presence/absence of handouts intercorrelating with the presence/absence of specific teachers’ behaviors negatively affecting students).

In the suggested interview plan (Appendix B: 27) the items were sequenced according to the Funnel approach [“a question sequence that begins with a very broad query, then progressively narrows the scope of questions to address specific points” (Thomas, 1998; 172)]. In the interview of presumably 15 questions, 2 questions are aiming in general at either awareness or unawareness on the matter; the rest 11 are arrowed at deriving the categories and 2 at establishing the correlation between the concepts.

The researcher intends to draw the grading of the correlations between concepts on the second stage of the survey, the interview. Qualitative interviews are particularly well suited to this study on the first stage after the questionnaire. This approach allows the researcher, to “engage in conversation which in turn will allow me to get to know the participants” (Kvale, 1996) in the context of this study.  These interviews provide for direct interaction between the researcher and the participants.  By interviewing the participants, the researcher is able to gain an understanding of the student’s views on classroom instruction as it relates to teacher communication and student achievement. In addition, the researcher is able to unfold the meaning of the students’ experiences (Kvale, 1996) and is free to move the conversation in any direction relevant to the subject area (Trochim, 2002).

At the third stage of the redesign, the questionnaire II (a, b) (Appendix C: 28) is used, the more “sharpened” one comparing to the first questionnaire. The researcher aims at not just establishing the concept map, but also checking the concepts found out and constructing the character of correlations within the “Clarifying behaviors affecting the students” model. The questionnaire II is a three-part instrument with the 5-point Likert scale.

Procedure/Redesign 

“Complex designs might involve a lengthy sequence of observations and programs or treatments across time”, stated Trochim (2000), and it is true with the qualitative researches. The redesigned research takes presumably 1 ¼ term with the vacation between the terms to analyze the data gathered during the preparatory stage. In each school works a team of researchers, presumably three people, who alternate in observing cases and join their efforts in interpreting the data.

The redesigned research consists of three main stages with the precedent preparatory stage. On the preparatory stage, researcher enters the field, visiting the lessons in the selected classes, taking field notes and giving the participants a chance to get used to the person sitting at the lectures.

One way of deciding the aspects of classroom interaction on which to focus is to undertake a pilot study of a few lessons in the field under observation and make notes, not only about events which catch the eye [in our case the test scores or facial expression of the students’ not receiving handouts but obliged to answer the questions unanswerable without them], but also about routine matters which can easily be taken for granted. Some classroom observers are influenced by a particular perspective or theoretical influence, so this will often partially determine the emphasis. (Wragg, 1999: 26)

The teachers present the researcher to the pupils as a special guest but do not point the special aim of his constant visits at first. The introduction may sound like “This is an educational researcher who is interested in some educational topic to be observed. That does not affect your marks though you all take part in the school refinement and your cooperation is appreciated”. The researcher is interested in the background information on both the teacher and pupils in each case/class and in the details of natural environment. During the vacation the field notes are analyzed to serve the ground for the next stages of the research.

The main part of research takes a term with a week cycle, the days when there are no lectures is used to analyze the data). Three researchers in each school are assigned to three classes each, alternating days when (s)he is present in each class. The frequency of visiting each case is proportional. The lectures, which are not visited by the researcher, are video- and audio-taped. The lesson plans and handouts are collected from every lecture, whenever the lectures were observed by the researcher directly or through the taping.

On the first main stage, in the very first day of the direct study, the researchers spread out the first survey instrument, the questionnaires I (a, b) among the students and teachers of the selected classes in each school. Throughout each week, the in-depth interviews are conducted with students and teachers.  In the end of the term the students and the teachers are asked to fill in the questionnaire II (a, b).

The final stage is dedicated to the analysis of the qualitative data, to the checks for reliability and validity, and to the deriving of conclusions.

Analysis of Data 

On the stage of concept mapping, the analyses of different qualitative data (derived from the field notes, questionnaires, textual and taped material) are done through different strategies. One of it is coding, a useful strategy “for both categorizing qualitative data and for describing the implications and details of […] categories” (Trochim, 2000). While developing some initial categories, the researcher(s) do(es) open coding, later, moving to selective coding with respect to a core concept. Besides, the researcher(s) record(s) “the thoughts and ideas of as they evolve throughout the study” (Trochim, 2000) with memoing.  “[E]arly in the process these memos tend to be very open while later on they tend to increasingly focus in on the core concept” (Trochim, 2000).

The classification of categories is done through the cluster analysis (Appendix D: 31). On the basis of this analysis, a distinction between types of clarifying behaviors is found. After that a comparison is made by means of analysis of variance between the scores of different types of teacher clarifying behaviors. To answer the question about the relation between teachers’ clarifying behaviors and students’ achievement and perception, with the help of analysis of variance the scores of the identified types of teacher clarifying behaviors are compared to the scales of the student achievement and perception.

Checks for Reliability and Validity

To have the consistent estimates of the same phenomenon (teacher clarifying behaviors), the inter-observer reliability is checked, so far as on the research project the team of researchers is working. “Studies of [qualitative] matters have shown that raters’ successive judgments of narrative data (essays, letters, descriptive oral responses, and the like) are often quite inconsistent” (Thomas, 1998: 185). So, in each case (school) a few observers “independently classify the same data (narrative answers to questions) into the study’s categories”.

The interrater reliability is then reported as the percent of times the coders’ judgments matched.  […]. Consequently, if raters’ assignment of data to classes is to be not only reliable but also valid, it is important that the raters be obliged to make “blind” judgments, that is, to reach decisions unmarred by information that could compromise the objectivity of those decisions. (Thomas, 1998: 185)

The integrative work of pulling all of the detail up together is done in group sessions. The conclusion validity is reached, because there is surely a relationship between the variables of teacher clarifying behaviors and student performance and perception. The construct concurrent validity is reached by incorporating the questions into the measure instruments, which help to discern those students, who are positively affected by certain clarifying behaviors, and those, who are negatively affected. The average scores are calculated for every scale (e.g., bluffing, uncertainty, discontinuity, handouts) and converted to percentile scores. To determine the internal consistency of the scales alpha values are calculated. The intercorrelations coefficients for all the scales are found to improve the convergent and discriminant validity.  The external validity is improved by first, the mostly random method of sampling, second, the replicating of study in week-cycles, third, by the concept mapping approach.  The internal validity is checked through incorporating the items into the research patterns (such as background questionnaires), which would help to establish, if the relations between the variables are of case-effect nature.

The strong hermeneutic position in qualitative research (Rosaldo, 1993) holds researchers responsible for both the intended and unintended consequences of their research. Something is true because we create the conditions by which it can be held to be true. Each of us, therefore, is responsible for the claims we make. (Anderson, 1996: 56)

The researcher bias may certainly influence the reliability and validity of research.

Possible Results

In the qualitative redesign the researcher (a research team) is studying the hypothesized compensatory effect of handouts on students’ achievement and perception, affected by the negative verbal teachers’ behaviors on a measure of formative evaluative survey such as questionnaires and in-depth interviews.

The redesigned after the Smith’s study into the qualitative study research reports correlations between students’ achievement and perception variables and the variables of certain teacher clarifying behaviors. The classification scheme represents assumingly three clusters of categories offering information about input variables, contributing to the model of teacher clarifying behaviors, while a fourth cluster focuses on students’ reading/listening abilities and habits, achievement and perception. Three sets containing a total of input (likely causal) variables may be the following (Appendix D: 31).

In their narratives, the teachers show (significant/insignificant) awareness on their clarifying behaviors. Among them they point out positive (handouts, no bluffing, certainty, continuity) and negative (no handouts, bluffing, uncertainty, discontinuity) conditions. They show (significant/insignificant) awareness of the effect the certain clarifying behaviors produce on pupils.

In their narratives, the students show (significant/insignificant) awareness on their teacher clarifying behaviors. Among them they point out positive (handouts, no bluffing, certainty, continuity) and negative (no handouts, bluffing, uncertainty, discontinuity) conditions. They show (significant/insignificant) awareness of the effect the certain clarifying behaviors of the teachers produce on pupils.  The results of the study are generalized to all the population of high-school students and teachers so far as the district and school effects do not dump the significance of correlations of the variables under research (certain teacher behaviors and student performance).

                             Research Redesign of a Qualitative Study

Case Study:

Does the “Art of Teaching” Have A Future?

(David J. Flinders, University of Oregon)

Introduction

The general research frame for the present redesigned study is the qualitative research conducted by David J. Flinders in his doctoral dissertation (1997). While he was completing his graduate coursework, Flinders served as a research assistant on a project to collect information about high school curriculum and instruction. As part of that work, Flinders visited several local schools to observe and interview a small number of teachers. Flinders discovered that talking with teachers and watching them work raised some compelling questions:

1. What do we know about the day-to-day professional lives of teachers?
2. How do teachers experience their work?
3. What are teachers’ strongest concerns?
4. From what sources do teachers derive their everyday satisfactions?

These questions became the initial focus of his study.

The information Flinders collected through interviews, observations, and written documents was analyzed and described using an approach known as educational criticism. This genre of qualitative inquiry takes its lead from the arts and humanities rather than from the social sciences. Presumably for this reason, it is not surprising that this case study gives particular attention to the artistic dimensions of teaching which one may call the implicit strand of the notion under the research.

Different dimensions of teachers’ professionalism are widely explored by modern educators, but up to the recent moment the emphasis was laid upon explicit matters. Both Flinders’ and redesigned cases follow a widely used approach. It views professionalism as “the conduct, aims, or qualities that characterize or mark a profession or a professional person” (Merriam-Webster Online Dictionary) ” where the term profession is defined as “a calling requiring specialized knowledge and often long and intensive academic preparation”. Restricted to the teacher occupation, professionalism is defined as the whole of teachers’ ideas about their job and the knowledge and skills they have (cf. Hoyle, 1980). Yet Flinders went further the common definitions and expanded them.

It is interesting to mention in this connection that before the 1990s there was a tendency to scrutiny in the classroom/outdoor area the behavioral patterns of teachers (the use of power or the proportion of different types of work conducted) and the subsequent students’ performance. The researchers measured the explicit teacher professionalism (e.g., the mechanics of lesson planning, test construction, and curriculum development). “Since then, the emphasis has shifted to an exploration of the internal world of teachers, including their thoughts, perspectives, knowledge, ideals, and values” (Ornstein, 1999, paraphrased in Chen, 2003: 24+) or to researching the implicit strand of teaching professionalism. Actually Flinders was not the only one, who paid attention to the implicit constituents of the teaching.

In answer to the age old question “is teaching an art or a science?” I lean toward art. Yet it is insufficient to say teaching is an art and be done with it. Describing teaching as an art produces as many questions as it answers. (Morgan-Fleming, 2000: 159)

The research question in Flinders’ report is formulated as the open-ended:

What exactly does it mean to be a professional? Does it mean simply possessing a body of expert knowledge and a repertoire of technical skills? Climbing a career ladder toward greater autonomy and increased occupational rewards? Or, for classroom teachers, does professionalism mean something more? (Flinders, 1989)

  It is evident that the educator concentrated on exploring not the traditional explicit characteristics of teachers’ professionalism but the less yielding for the statistical analysis implicit characteristics. With the aforesaid research question, the quantitative redesign needs to create the statistically assessed model of teacher professionalism, which since then is formulated and referred to as the implicit professionalism model.

Delimitations and Assumptions of the Re-designed Study

The purpose of the redesigned study with the research question stays the same.  However, the redesign presents a correlational analysis of the relationship between the teacher background and work situation, and teacher implicit or artistic mode of professionalism. The specific purpose is to determine the nature of implicit professionalism, the nature of the relationships, both direct and indirect, which exist between perceived distinctive features of implicit professionalism, the demands of teaching, and teacher background factors in the teaching environment.  A causal model is proposed and used to investigate these relationships.

The possibility of researcher bias may have affected the validity of Flinders’ findings. Flinders was a former English teacher and, in graduate school, he studied under an art educator. As a result, Flinders is probably more likely to see teaching as involving artistic elements than would a researcher with a different background. Flinders used interview data as well as his own perceptions of what the teachers did. In the quantitative redesign the role of the researcher is less constructivistic. The emphasis is shifted from the interpretation and observation towards the categorizing and finding correlations.

Such qualitative methods as field observation and survey in the form of the personal in-depth interviews surely took much time and effort from Flinders. The scientist was more interested in constructing the concepts of implicit professionalism then in assessing it. The redesign involves both linear and inferential statistics to expand the Flinders’ findings in this direction.

To proceed surveying the elements of design, Flinders focused on English language teachers of the U.S. public high-school establishments because his own training was in this area, and he wanted to bring to the study as much subject-matter knowledge as possible. Focusing at the high school level was also an attempt to capitalize on his previous teaching and research experience. In the redesign, the researcher is interested in more generalization concerning the theoretical population of interest, that is why the special instrument for measuring the background of the teachers is used.

It is important to specify the particular nature of teaching so that relevant means for studying and appraising the art can be found. It is important first to describe teaching, giving emphasis to its defining characteristics before developing frameworks to appraise the art form or train new artists. (Morgan-Fleming, 2000: 159)

I […] recognized that the artistic side of teaching is often neglected. The professional and academic community rarely stops to scrutinize what makes for a beautiful lesson or a well-orchestrated class discussion. I hope that the study will help educators more fully recognize the intricate skills that good teaching demands. In making them explicit, we open the door for teachers to use these skills more deliberately, and to share their expertise more often with their colleagues. In this way, we can stay in close touch with our own professional lives. (Flinders, 1998: ?)

With all the aforesaid, the redesign is organized in the following manner.

Hypothesis

H_O1 – Implicit teaching skills either doesn’t significantly affect or negatively affect the quality of teaching.

Ho₂ – Classroom teachers with more teaching experience always significantly more often demonstrate implicit teaching skills than classroom teachers with less teaching experience.

H_O2 – Classroom teachers with greater autonomy and increased occupational rewards always significantly more often demonstrate implicit teaching skills than classroom teachers with less autonomy and increased occupational rewards.

H_A0 – Implicit teaching skills consists of skills in communication, perception, cooperation, and appreciation.

H_A1 – Implicit teaching skills in communication, perception, cooperation, and appreciation significantly positively affect the quality of teaching.

H_A2 – Classroom teachers with more teaching experience doesn’t always significantly more often demonstrate implicit teaching skills than classroom teachers with less teaching experience.

H_A2 – Classroom teachers with greater autonomy and increased occupational rewards doesn’t always significantly more often demonstrate implicit teaching skills than classroom teachers with less autonomy and increased occupational rewards.

Method

While Flinders selected six teachers from a pool of volunteers after talking with the English department chairs at two schools, the researcher of the redesigned study experiences the need to choose the sample of a larger size. The aims of the researchers in both the original case and the redesign in picking up the sample coincide though. Both aim at selecting a sample that represents the typical range of high school English teachers in terms of experience, gender, educational background, and type of teaching assignment. The researcher conducting the redesigned study strives to achieve more quantitative and statistically valid research pattern, thus (s)he gives up the non-random sampling method in benefit of the random one with the regards to four major variables in assessing the teacher background.

Careful consideration is given to making the schools as representative as possible of government schools in the Washington state keeping in mind such a variable as students’ achievement as one of the measures for teaching professionalilsm. Like in the previous redesign, the multi-level sampling method is used. The chosen sampling frame on the first level is the statewide WASL achievement roster (Washington Assessment of Student Learning), from which three top schools, three average scoring schools and three lowest schools are randomly stratified. On agreement with the school administration all the teachers participate in conducting the survey questionnaire.

Instruments/Rational

For the redesign the complex instrument in the form of a survey questionnaire is used. The survey questionnaire is a good tool when the research sample size is large (Nardi, 2003). On the first stage, a background instrument collected demographic data including the following parameters: 1) years spent at the present school, 2) sex, 3) employment status (full-time or casual), and 4) the school in which one taught.  Specific information on 5) age, 6) subject area is also included.

The questionnaire comprises four different instruments designed to collect information about implicit professionalism, experience and autonomy. The parameter of occupational rewards is measured with the instrument for teacher autonomy. The teacher implicit or artistic side of professionalism instrument is developed from Flinders (1989). The teacher experience and autonomy instrument are developed afresh using a similar approach to that used with the implicit professionalism questionnaire. 

To gain insight into the teachers’ professional orientation the teachers are asked to indicate on a five-point scale how far they agreed to the presumably 24 statements (the five-point scale runs from 1 = ‘Highly disagree’ to 5 = ‘I highly agree’). ‘I highly disagree’ is taken in this investigation to be a characteristic of a teacher with a formative explicit-professionalism orientation and ‘I highly agree’ as a characteristic of one with an extended implicit professional orientation. Thus, the questionnaire part for finding scales to measure the teacher professionalism is called “What should a professional teacher be aware of?” and consists of 24 items (see the list of assumed items in the Appendix E: 32). Some of them are taken from the Flinders’ research. The others are incorporated as the “fillers”. The concern with the implicit dimension of the professionalism is measured through the subsection: “What belongs to the art of teaching?”. The items are drawn from the concept mapping conducted beforehand. The teachers are asked to indicate on a 5-point scale how far they agree that these items belong to the artistic dimension of teacher professionalism.

The selection of specific scales to measure teacher experience and the items for the scales is guided by the analysis of teachers’ work provided from the Professional Certificate (advanced certification) program. It officially declares that “[i]n order to teach in a public school classroom in Washington State, a person must become certified by completing a state-approved teacher certification program” (http://www.teachwashington.org/certification.php). There are three types of certification programs in Washington: firstly, the Bachelor’s with certification; secondly, the Post-Baccalaureate certification; thirdly, the Master’s with certification. “Upon successful completion of a state-approved teacher certification program, the student earns a Residency Certificate” which is “valid for five years”. “Within the first 5 years in the profession, teachers are required to return to college to complete a Professional Certificate (advanced certification) program”.  The official information restricted the category of the target population on the variable of experience into the 4 subcategories on the scale of certification. The interval for measuring the experience variable is chosen for 5 years.

Data on the experience of teachers is gathered through a self-report part of the questionnaire which is designed to check-list experience constituents.  To evaluate experience teachers are asked to indicate the level of “experience demand” associated with various tasks, duties and responsibilities on a five point Likert Scale. While no hint is given to guide respondents in the interpretation of the term “experience”, the items are designed to reflect the conceptualization of experience.  The categories are not necessarily mutually exclusive and cover different aspects of experiences and levels of expertise which influence the way teachers see their work.

There are multiple sources for deriving the scales for measuring teacher autonomy and occupational rewards model. Youngman (1982) provided a system for describing teachers’ work having identified 14 areas of teacher activity. These included aspects of classroom, organizational and administrative procedures.  The description of the duties based approach to teacher appraisal is provided by Scriven (1989) which also outlined the duties and areas of responsibility of the teacher.  To the framework which emerged from the aforesaid two sources also incorporated the ideas of Hoyle (1980), who discerns two forms of “professionality”, viz. “restricted professionality” and “extended professionality”. Both forms of professionalism differ both in orientation and in behaviour or activities (Hoyle, 1969, 1974, 1980; Nias, 1989). Jongmans & Beijaard (1995, 1997) have investigated, among other things, teachers’ involvement with school policy. The instrument for measuring this scale is also the self-report questionnaire filled in by the participants.

Procedure/Redesign

1) Measurement of teacher implicit professionalism: Four teacher implicit professionalism scales are hypothesized.  Flinders in his research pointed out certain topics, which the teachers from his sample associated with the teaching artistry. The focused themes (communication, perception, cooperation, and appreciation) emerged gradually over the course of the study. They emerged in part because Flinders “…found them to be salient features of the teachers’ day-to-day work.” The instrument for measuring the implicit professionalism in the redesign takes into account the Flinders’ findings, though the respondents may name some other constituents. Factor analytic and reliability procedures are used to confirm and refine these scales.  The analysis either confirms or rejects the existence of four latent professionalism variables.

2) Measurement of teacher experience: The concept of the perceived level of knowledge and skills associated with different teaching experience has been recognized as a significant element of teaching artistry and is referred to here as “experience”. It is measured by four (presumably) variables. Several of the independent variables are related to teacher experience: e.g., the teacher’s education, the years of teaching practice, class allocations and the composition of classes taught. These variables provide information on aspects of the teaching context but cannot provide information concerning how teachers feel about their work. Consequently, some variables should be designed to assess the attitudes of teachers towards the demands of specific aspects of their work.

3) The measurement of teacher autonomy: The model being pursued examines the possible mitigating influence of teacher autonomy and rewards on implicit professionalism. The scales are taken from the self-reported questionnaire and the content analysis of textual documents.

4) A sequential path model is developed to demonstrate the hypothesized relationships between the variables under consideration.  A separate form of the model is tested for each of the four teacher implicit professionalism variables.

Analysis of Data

To investigate how far a distinction into types of professional orientation occur (implicit and explicit variables) the researcher carries out a cluster analysis. On the basis of this analysis of the 24 items measuring professional orientation the researcher finds a distinction between two types of explicit/implicit orientation. After that the researcher makes a comparison by means of analysis of variance (N-way Analysis of Variance, ANOVA) between the scores of the two types of  “professional orientation” (F[24, N (number of cases adjusted for degrees of freedom)]=x; p < 0.05: p = indicates the level of statistically significant difference among the means, the value of α is typically set at .05 in the social sciences though a newer, but growing, tradition calls out to achieve a statistical power of at least .80.).

Possible summary statistics for the implicit professionalism scales is likely to be arranged in the form of a table.

Table 1:    Scale Statistics: Teacher implicit professionalism – scale mean scores

(N=X FOR ALL SCALES)

The selection of specific scales to measure teacher implicit professionalism and the items for the scales is guided by the analysis of teachers’ work provided from 3 different sources (questionnaires, videotaping, and lecture plans’ analyses). Reliability for the scales ranged from a high of x for some specific variable to a satisfactory x₁ for another scale.  Mean scores have a potential maximum of y_max indicating teachers recognition “highly agree” of implicit professionalism from the factor and a minimum of y_min indicating “highly disagree” implicit professionalism derived from the factor.  The mid-point of the scales is y_medium, indicating that moderate amounts of implicit professionalism were derived from the factor.

The researcher looks then at which scale has the highest mean scale score with a mean of x indicating the average experience of implicit professionalism from the specific source in the range between “sometimes” and “a fair bit agree”.   The researcher compares the average scores of other scales to observe the teachers’ experience of “a fair bit disagree’ implicit professionalism from these factors.

Then, the researcher looks at the inter-correlations between the scales. At the maximum the researcher calculates how much per cent of the variance in one scale could be accounted for by another.  Consequently, the scales are treated as independent or dependent measures of teacher implicit professionalism.  The relationships in the causal model are tested separately for the four different implicit professionalism outcomes (see Models 1 to 4).  The relative importance of independent and intervening variables is expected to differ reflecting the complex nature of teacher implicit professionalism.

Confirmatory factor analysis is used to refine the experience scales and internal consistency of the scales is assessed with reliability analysis. This process may provide four experience variables: 1. Education; 2. Assessment (The demands of teacher evaluation, including formal examination and assessment.); 3. Teaching (The demands associated with face-to-face teaching.); 4. Resources (The demands associated with finding, developing and producing teaching resources. e.g., learning to use new resources, seminars, courses and etc.). Possible summary statistics for the teacher experience scales is likely to be arranged in the form of a table.

Table 2:    Scale Statistics: Teacher experience – scale mean scores

(N=X FOR ALL SCALES)

The three scales developed in measuring the teacher autonomy are as follows: 1. Administration (Teachers’ school level administration – the demands associated with duties beyond the classroom. e.g., involvement with school-community activities); 2. Rewards (either pecuniary or not); 3. Cooperation with the colleagues. Scale means are computed and compared as in the preceding models. Possible summary statistics for the teacher autonomy scales is likely to be arranged in the form of a table.

Table 3:    Scale Statistics: Teacher experience and rewards – scale mean scores

(N=X FOR ALL SCALES)

A sequential path model is a saturated three stage one in which each variable is potentially influenced by all prior variables.  The pattern of causation moves from left to right, meaning that teacher implicit professionalism is potentially influenced by all variables to the left.

Figure 1:   SEQUENTIAL PATH MODEL – TEACHER WORK-RELATED STRESS

            SUMMARY MODEL

Teacher &         Teacher                      Teacher                       Teacher

School    —->   Experience   —->       Autonomy &     —-> Implicit

                                                            Occupational                Professionalism

Background                                                Reward

(6 vars)            (4 scales)                    (3 scales)                     (4 scales)

The three stage models are refined using multiple linear regression analyses.  Standardised partial regression coefficients (beta coefficients) are used to indicate the strength of causal paths within each form of the model.

             Checks for Reliability and Validity

Before suggesting the survey questionnaire “What does the teaching artistry consist of?” to the sample, the researcher invites the “testing pool” of teachers not participating in the redesign, to fill in the opinionnaire and to refine background and other scales. After all the teaching staff representation in terms of experience and other variables is relatively similar when comparing “apples with apples”. In order to obtain the sufficient return rate along with the survey, the researcher includes a brief, friendly letter of explanation as well as a stamped, self-addressed envelope. Thus, the inter-rater reliability is pursued. Besides, confirmatory factor analysis is used to refine the scales and internal consistency of the scales is assessed with reliability analysis.

School and Subject Area Effects: As school and subject areas are nominal variables, it is necessary to consider now these variables would be treated in the model before proceeding to results of the analyses. As there are nine (presumably) schools involved and several subject areas identified, these variables need to be represented in regression analyses by “dummy” variables in each case.  The initial analysis may reveal that when these dummy variables are included in analyses they may tend to mask other small, but significant, relationships in the models.  To overcome this problem each of these groups of variables is combined into a composite variable designed to control for the effects of school and subject area.

For school, this process may consist of converting the dummy variables into a composite variable related to the overall experience of implicit professionalism (an average of implicit professionalism in all four models).  An ANOVA of overall implicit professionalism by school, with a multiple classification analysis, may be used to obtain the adjusted deviation from the grand mean for each school (Williams et al, 1980).  The deviation scores are then used as weights in the construction of a composite variable.  The composite variable created in this way reduces the number of variables in the regression analyses and thus the complexity of the causal models.

As a consequence, it should be noted that the paths in the models relating to school and subject area represent only broad general effects, which may then be compared with the effects of other variables in analyses.  In the case of school, the composite path coefficient may represent the effect of working in a high or a low evaluated for teachers’ professionalism school.  In the case of subject area, the path relates to working in a high or a low teachers’ implicit professionalism subject area.  One would need to return to the multiple classification analyses to determine which schools and which subject areas are notable in their relationships with teacher implicit professionalism.

Possible Results

On the basis of analyses of 4 models (implicit professionalism arising from communication, cooperation, appreciation and perception) it may be evident that each model displays the complex of relationships which exist between the background contextual variables, the intervening variables and implicit professionalism due to each of the 4 factors.  The effects depicted in each of the 4 models either show or not consistent patterns of causation.  The school one teaches in and the number of years spent teaching at that school either have significant effects on implicit professionalism from each factor.  The experience and autonomy (+ rewards & recognition) variables have some effects (e.g. direct positive, indirect positive, direct negative, mitigating, small but significant indirect and etc.) on implicit professionalism from each factor.

The models may show the importance of either the non-obligatory supplementary refinement programs or the pecuniary rewards in determining the level of teacher implicit professionalism.  Contextual factors such as the characteristics of the school in which one works, the years spent at this school, and the nature of classes taught, even sex also may have direct effects on some factor related implicit professionalism.  It is hard to state for sure the degree and the character of correlations between the models without the data for analyses. However, behind the direct effects there is a complex network of interactions at work.

Education and the sex of teacher may also have significant causal paths to implicit professionalism in these models. The aging of the teaching service means that the particular problems of experienced classroom teachers are likely to become increasingly important.  A school’s internal organisation may also be an important consideration in terms of rewards. Cole (1989) claimed that it is in times when intrinsic pedagogical rewards are low that extrinsic rewards such as pay become predominant.  If wage rates are adequate for the working conditions, qualifications and experience of teachers they may mitigate frustrations arising in this domain.

Both the experience and the autonomy/rewards and recognition model may explain some per cent of the variance in implicit professionalism in each domain. To conclude, experience and autonomy seem to be important factors in determining implicit professionalism. However, in the process of analyzing specific data in the specific area new factors may arise. Kyriacou’s (1987) caution that each study should be placed in its own context is important in this case.

Conclusion

For more than a century, the advocates of quantitative and qualitative research paradigms have engaged in ardent dispute. From these debates, purists have emerged on both sides (Campbell & Stanley, 1963; Lincoln & Guba, 1985).

Quantitative purists (Ayer, 1959; Maxwell & Odaney, 2004; Popper, 1959; Schrag, 1992) articulate assumptions that are consistent with what is commonly called a positivist philosophy. That is, quantitative purists believe that social observations should be treated as entities in much the same way that physical scientists treat physical phenomena (Creswell, 1994). Further, they contend that the observer is separate from the entities that are subject to observation. Quantitative purists maintain that social science inquiry be objective. That is, time- and context-free generalizations (Nagel, 1986) are desirable and possible, and real causes of social scientific outcomes are determined reliably and validly. According to this school of thought, educational researchers eliminate their biases, remain emotionally detached and uninvolved with the objects of study, and test or empirically justify their stated hypotheses. These researchers have traditionally called for rhetorical neutrality, involving a formal writing style using the impersonal passive voice and technical terminology, in which establishing and describing social laws is the major focus (Tashakkori & Teddlie. 1998).

Qualitative purists (also called constructivists and interpretivists) reject what they call positivism. They argue for the superiority of constructivism, idealism, relativism, humanism, hermeneutics, and sometimes, postmodernism (Guba & Lincoln, 1989; Lincoln & Guba, 2000; Schwandt, 2000; Smith, 1983). These purists contend that:

• multiple-constructed realities abound
• time- and context-free generalizations are neither desirable nor possible
• research is value-bound
• it is impossible to fully differentiate causes and effects
• logic flows from specific to general (e.g., explanations are generated inductivity from the data)
• knower and known cannot be separated because the subjective knower is the only source of reality (Guba, 1990).

Qualitative purists prefer detailed, rich, and thick (empathic) description, written directly and somewhat informally rather than a detached and passive style of writing,

Both sets of purists view their paradigms as the ideal for research and implicitly if not explicitly advocate for the incompatibility thesis (Howe, 1988), which posits that qualitative and quantitative research paradigms, including their associated methods, cannot and should not be mixed. The quantitative versus qualitative debate is so divisive that some graduate students from educational institutions with an aspiration to gain employment in the world of academia or research are left with the impression that they have to pledge allegiance to one research school of thought or the other. Guba (a leading qualitative purist) clearly represents the purist position when he contends that “accommodation between paradigms is impossible . . . we are led to vastly diverse, disparate, and totally antithetical ends” (Guba, 1990: 81). A disturbing feature of the paradigm wars is the relentless focus on the differences between the two types of research. Indeed, the result of the two dominant research paradigms is two research cultures, “one professing the superiority of ‘deep, rich observational data’ and the other the virtues of ‘hard, generalizable’. . . data” (Sieber, 1973: 135).

“Methods are always more than technique”, assumes Anderson (1996: 58), and many researchers evidence on the necessity to avoid formalization when designing the social research. A number of studies are dedicated to the dichotomy of qualitative and quantitative research systems though “[b]oth […] are equally empirical in privileging our experiencing of the phenomenal world over its formal analysis” (Anderson, 1996: 47). It is not the question of whether the numerical data of the quantitative design is more valid and reliable than the discourse data of the qualitative one. After all “the experience of interest is in the meaning of things” (Anderson, 1996: 49). And many researchers of the modernity vote for the synthesis of qualitative and quantitative designs.

However, both strategies should answer the common criteria, a relatively exhaustive list of which is given by Trochim.  “Good research strategies reflect the theories which are being investigated [being theory-grounded]; […] reflect the settings of the investigation [situational]; […] [g]ood designs can be implemented [feasible]; […] have some flexibility built into them [redundant]; […] strike a balance between redundancy and the tendency to overdesign [efficient]” (2000).

Whether quantitative or qualitative in nature, most research studies in education have looked at particular programs, courses, and students in single institutions. More studies are needed to look across institutions so that more general conclusions can be reached. More studies are needed that relate specific parts of teachers’ preparation (subject matter, pedagogy, clinical experiences) to the effects on their teaching practice, and perhaps on student achievement. Studies that compare the relative importance of specific parts of teacher preparation would be useful to those designing and revising teacher education programs.

            The design of future studies should include more sensitive measures that describe specific features of instructional quality. Research studies should include comparisons among plausible variables. A stronger interplay between research about particular contexts and research that seeks general conclusions across teacher education programs

is needed. Teacher preparation research must be explicit about connections to the improvement of student achievement and about the contexts in which graduates of teacher preparation are working.

            Several promising areas for future research exist. For example, the subject matter preparation of teachers needs more attention, with close examination of content, quality and differences across subject areas. Researchers do not yet know enough about the effects of close, long-term connections between K-12 schools and teacher preparation programs. Research could show how policies that are designed to influence teacher education actually affect program components and what prospective teachers learn. Further, research should examine the effects of “education methods” and “education foundations” courses.

            There is enormous potential for research to lead the ongoing reform and improvement of teacher education in the United States. By building on previous work and conducting rigorous studies that ask important questions, the research community can contribute to ensuring that a well-qualified teacher is available in every classroom for every child.

Appendix A

Smith Redesign

Questionnaire Ia for students

Students’ Awareness of the Specific Teachers’ Behaviors

1) List 10 things, which the teacher does to make the subject of the lesson clearer for you.

2) List 10 things, which the teacher does to make the subject of the lesson not clear and dull for you.

Rank 20 items from the most appreciated by you personally towards the less appreciated by you.  Rank 20 items from the most common in your practice towards the less common in your practice.

II.

Please, provide the research team with some information on your background:

1) How old are you?

2) What grade are you in?

3) Do you find it a positive experience to study here?

4) What things make you happy about studying here?

5) What things make you unhappy about studying here?

6) What task(s) do you find the most difficult in your learning experience?

7) What task(s) do you find the easiest in your learning experience?

8) Do you consider yourself to be a successive student?

Questionnaire Ib for teachers

Teachers’ Awareness of the Specific Teachers’ Behaviors

I.

1) List 10 things, which the teacher does to make the subject of the lesson clearer for students.

2) List 10 things, which the teacher does to make the subject of the lesson not clear and dull for students.

Rank 20 items from the most appreciated by you personally towards the less appreciated by you.  Rank 20 items from the most common in your practice towards the less common in your practice.

1. Please, provide the research team with some information on your background:

1) How long have you been teaching? At this particular school?

2) What is your qualification?

3) Do you find it a positive experience to teach here?

4) What things make you happy about teaching here?

5) What things make you unhappy about teaching here?

6) What task(s) do you find the most difficult in your teaching experience?

7) What task(s) do you find the easiest in your teaching experience?

8) Do you consider yourself to be a successive teacher?

Appendix B

Smith Redesign

Interview Scheme

• Does your good achievement at the lesson always correlate with your good perception of the lesson?
• What is a good achievement at the lesson for you?
• What is a good perception of the lesson for you?
• In order to have good achievement at the lesson, what are the most important things you think teacher should do? What things should he/she pay attention to?
• In order to have good perception of the lesson, what are the most important things you think teacher should do? What things should he/she pay attention to?
• What part do you think the handouts play when used play in maintaining good achievement?
• What part do you think the handouts play when used play in maintaining good students’ perception?
• What kinds of teachers’ behaviors do you think are most important for good achievement?
• What kinds of teachers’ behaviors do you think are most important for good perception?
• What kinds of teachers’ behaviors do you believe are most damaging to adequate presentation of the material?
• What kinds of teachers’ behaviors while presenting the subject to the class would you call specific?
• How do you think using handouts influences teachers’ specific behavioral effects on your achievement?
• How do you think using handouts influences teachers’ specific behavioral effects on your perception?
• Do you get support from handouts when the teacher demonstrates specific behavioral patterns through the lesson?
• What kinds of support give you handouts?

Appendix C

Smith Redesign

Questionnaire II

1. Students’ Awareness of Clarifying Teachers’ Behaviors

1) Please, state, how your understanding of the subject material and your perception of the lecture quality were affected by the following teachers’ behaviors. The ranking runs on the 5-point scale: 1 – “Applies very little to me”; 2 – “Applies little to me”; 3 – “Applies sometimes”; 4 – “Applies often”; 5 – “Applies very often to me”.

2) Agree/disagree with the following statements. The 5-point scale: 1 – “Highly disagree”; 2 – “Disagree”; 3 – “Sometimes, not sure”; 4 – “Agree”; 5 – “Highly agree”.

Handouts are useful for understanding the subject at the lesson.

__Handouts help to understand the lesson.

__ Handouts help to fulfill the post-test.

__ Handouts help to control the teachers’ discourse when the teacher is uncertain on the subject-matter.

__ Handouts help to control the teachers’ presentation of the subject on the level of facts or sequence of facts.

__ Handouts help to control the sequence of the lesson.

3) Students’ Assessments of Instructors

Agree or disagree with the statement about the instructor during the time that you spend in your class. The 5-point scale: 1 – “Highly disagree”; 2 – “Disagree”; 3 – “Sometimes, not sure”; 4 – “Agree”; 5 – “Highly agree”.

The instructor:

__is truly an expert in the subject-matter.

__presents material in an easily comprehended sequence.

__speaks clearly.

__uses distracting wording.

__holds your attention.

__ provides material for the better understanding of the subject material.

__ is bluffing. (adopted from Thomas, 1998: 167)

Questionnaire II

1. Teachers’ Awareness of Clarifying Teachers’ Behaviors

1) Please, state, how your presentation of the subject material and your perception of the lecture quality were affected by the following teachers’ behaviors. The ranking runs on the 5-point scale: 1 – “Applies very little to me”; 2 – “Applies little to me”; 3 – “Applies sometimes”; 4 – “Applies often”; 5 – “Applies very often to me”.

2) Agree/disagree with the following statements. The 5-point scale: 1 – “Highly disagree”; 2 – “Disagree”; 3 – “Sometimes, not sure”; 4 – “Agree”; 5 – “Highly agree”.

Handouts are useful for understanding the subject at the lesson.

__Handouts help to understand the lesson.

__ Handouts help to fulfill the post-test.

__ Handouts help to control the teachers’ discourse when the teacher is uncertain on the subject-matter.

__ Handouts help to control the teachers’ presentation of the subject on the level of facts or sequence of facts.

__ Handouts help to control the sequence of the lesson.

3) Teachers’ Assessments of Instructors

Agree or disagree with the statement about yourself as the instructor during the time that you spend in your class. The 5-point scale: 1 – “Highly disagree”; 2 – “Disagree”; 3 – “Sometimes, not sure”; 4 – “Agree”; 5 – “Highly agree”.

The instructor:

__is truly an expert in the subject-matter.

__presents material in an easily comprehended sequence.

__speaks clearly.

__uses distracting wording.

__holds your attention.

__ provides material for the better understanding of the subject material.

__ is bluffing. (adopted from Thomas, 1998: 167)

Appendix D

Cluster I: Teacher characteristics: (1) education, (2) experience, (3) gender, (4) readership (amount and type of reading done by teacher), (5) planning (amount and type of lesson plans and handouts), (6) teacher’s aims in instruction.

Cluster II: Teaching strategies: (1) time allocation (continuity/discontinuity), (2) relation to the subject-matter (certainty/uncertainty), (3) skill orientation, (4) strategy orientation, (5) functional orientation (practical use of handouts and lecture notes), (6) assessment for accountability (providing reports for the school’s administrators), (7) assessment for instruction (providing information to improve teaching methods).

Cluster III: Proximal teaching conditions: (1) class size, (2) instructional time, (3) percentage of students speaking a language other than the one, in which the learning is conducted, (4) number of handouts per topic.

One set of four categories offered information about the target variable – that is, the output variable of pupils’ reading and listening competence, or better say their responsiveness to the teacher clarifying behaviors:

Cluster IV: Students’ reading/listening skills and habits: (1) narrative comprehension, (2) expository comprehension, (3) document comprehension, (4) frequency and types of voluntary reading.

Appendix E

Items for the scale “Implicit Professional orientation”

Five-point scale: higher value = highly agree/applies very strongly to me.

References

Anderson, J. A. (1996). Thinking Qualitatively: Hermeneutics in Science, in M. B. Salwen & D. W. Stacks (Eds), An Integrated Approach to Communication Theory and Research, 45 – 58. Mahwah, NJ: Lawrence Erlbaum Associates.

Atwater, T. (1996). Communication Theory and Research: The Quest for Increased Credibility in the Social Sciences, in M. B. Salwen & D. W. Stacks (Eds), An Integrated Approach to Communication Theory and Research, 539 – 45. Mahwah, NJ: Lawrence Erlbaum Associates.

Ayer, A. (1959). Logical positivism. New York: The Free Press.

Campbell, D. T., & Stanley, C. (1963). Experimental and quasiexperimental

designs for research. Chicago, IL: Rand McNally.

Chen, D. D. (2003). A Classification System for Metaphors about Teaching. JOPERD –The Journal of Physical Education, Recreation & Dance. 74(2), 24+.

Cohen, L., Manion, L., & Morrison, K. (2000). Research Methods in Education. London: Routledge Falmer.

Cole, M. (1989). The Politics of Stress in Teaching, in M. Cole & S. Walker (Eds), Teaching and Stress, Ch.9. Milton Keynes, Stony Stratford, UK: Open University Press.

Creswell, J. W. (1994). Research design: Qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications.

Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage Publications.

Flinders, D. J. (1989). Does the “Art of Teaching” have a future? Educational Leadership, 46(8), 16-20.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory. Chicago, Il.: Aldine.

Goss, B. (1996). Intrapersonal Communication. In Salwen, M. B. & Stacks, D. W. (Eds.). An Integrated Approach to Communication Theory and Research. (335 – 343). Mahwah, NJ: Lawrence Erlbaum Associates.

Hara, K.  (1995). Quantitative and qualitative research approaches in education. Education, 115(3), 351 -356.

Hiller, J. (1969). A methodology for estimating relative cost-benefits of alternative pretesting procedures (Technical report). Washington D.C.: U. S. Army Research Institute for the Behavioral and Social Sciences.

Howe, K. R. (1988). Against the quantitative-qualitative incompatability

thesis, or, Dogmas die hard. Educational Researcher. 17. 10-16.

Hoyle, E. (1980) Professionalization and deprofessionalization in education, in E. Hoyle & J. Megarry (Eds) World Yearbook of Education 1980,, 42-57. London: Kogan Page.

Kyriacou, C. (1987). Teacher stress and burnout: an international review. Educational Research, 29 (2), 146-152.

Kvale, S.  (1996). Interviews: An introduction to qualitative interviewing. Thousand Oaks, CA: Sage Publications.

Lincoln, Y. S., & Guba, E.  (1995). Naturalistic inquiry. Beverly Hills, CA: Sage Publications.

Maxwell & Odaney, 2004 Maxwell, S. E., & Delaney, H. D. (2004). Designing

experiments and analyzing data. Mahwah, NJ: Lawrence Erlbaum.

Morgan-Fleming, B. (2000). Characteristics of the Teaching Art and Nature of Competencies That Insure Learning. Education. 121(1), 159.

Nagel, T. (1986). The view from nowhere. New York: Oxford University Press.

Nardi, P. M. (2003). Doing survey research: A guide to quantitative methods.

Boston, MA: Pitzer College.

Popper, K. R. (1959). Theology of scientific discovery. New York: Routledge.

Salkind, N. J. (2004). Statistics for people who think they hate statistics (2nd Ed.). Thousand Oaks, CA: Sage Publications.

Schrag, P. (1992) The Resources of rationality: A response to the postmodern challenge (Studies in continental thought). Bloomington, IN: Indiana University Press.

Schwandt, T. A. (2000). Three epistemological stances for qualitative

inquiry: Interpretivism, hermeneutics, and social constructionism. In

1. K. Denzin & Y. S. Lincoln, Handbook of qualitative research

(pp. 189-213). Thousand Oaks, CA; Sage.

Scriven, M. (1989). The State of the Art in Teacher Evaluation, in J. Loken & P. McKenzie (Eds) Teacher Appraisal: Issues and Approaches. Australian Education Review (No.28). Hawthorn, Victoria: ACER.

Sieber, S. D. (1973). The integration of fieldwork and survey methods.

American Journal of Sociology, 73, 1335-1359.

Smith, J. K. (1983). Quantitative verses qualitative research: An attempt to clarify the issue. Educational Researcher, 12(3), 6 – 13.

Stein, M., Edwards, T., Norman, J., Roberts, S., Sales, J., Alec, R., & Chambers, J. (1994). A Constructivist Vision for Teaching, Learning and Staff Development. Unpublished manuscript, Wayne State University Detroit, MI.

Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining

qualitative and quantitative approaches. Applied Social Research Methods

Series (Vol. 46). Thousand Oaks, CA; Sage.

Thomas, R. M. (1998). Conducting Educational Research: A Comparative View. Westport, CT: Bergin & Garvey.

Trochim, W. M. (2000). Research Methods Knowledge Base. Retrieved May 15, 2004, http://trochim.cornell.edu/kb/statdesc.htm

Wragg, E. C. (1999). .An Introduction to Classroom Observation. London: Routledge.

Youngman, M. B. (1982). A system for describing teachers’ jobs. Educational Studies, 8 (1), 23-30.