Download Structured Observation and Other Data Sources in Social Research and more Exams Sociology in PDF only on Docsity!
Social Research Methods: Chapters 7, 8, 9,
Structured Observation is not used very often by social researchers because: -
- Certain types of behaviour are difficult to observe
- Difficult to generalize findings from the samples oftentimes selected for this type of research, i.e., small non-random samples Main advantage of Structured Observation: - behaviour is observed directly rather than inferred from a participant's report (survey response) Structured Observation: Observation Schedule - A device used in Structured Observation that specifics the categories of behaviour that are to be observed and gives instructions on how behaviour should be allocated. Structured Observation: Strategies for Observing Behaviour -
- Incidents
- Observation and recording of a wide variety of behaviours in a short or long period of time
- Time sampling Structured Observation: Strategies for Observing Behaviour: Incidents: - Recording incidents means waiting for something to happen and then recording what follows from it. Structured Observation: Strategies for Observing Behaviour: Time Sampling: - Researchers generally sample the times at which they make their observations. The times at which the observations are made are chosen in advance, either systematically or randomly. Structured Observation Issues of Reliability: Inter-observer consistency: - This entails considering how closely two or more observers of the same behaviour agree on how to code it. (Cohen's Kappa 0.75) Structured Observation Issues of Reliability: Intra-observer consistency: - The degree of consistency in the application of the observation schedule by a single observer over time. Consistency is difficult to achieve because people tend to behave differently depending on the context. Measurement Validity - The degree to which a measure of a concept actually measures what it is supposed to measure. Face Validity - A type of validity that is achieved if, on inspection, an indicator measures to see if they cluster into one or more groups.
Concurrent Validity - A type of validity that is tested by relating a measure to an existing criterion or a different indicator of the concept to see if one predicts the other; one of the main forms of measurement validity. Structured Observation Issues of Validity: Error in Implementation - Is the observation schedule being administered as directed? Do people change their behaviour when they known they are being observed? Field Experiment - A study in which a researcher directly intervenes in a natural setting to observe the consequence of that intervention. Covert Observation - Participants are not aware that they are being studied Structured Observation: Ethical Issue of Field Experiments - Deception Criticisms of Structured Observation -
- Risk of using an inappropriate observation schedule
- Does not readily allow for the observer to get at the meanings people attach to their behaviours; observer may also not pay due attention to the context in which the behaviour is taking place
- Generation of too much "small" data = difficulty in developing general themes
- BUT .. It is still likely to be more accurate and effective than getting people to report their behaviour in a survey Cohen's Kappa - A measure of the level of agreement between two peoples coding decisions that takes into consideration the possibility that agreement will occur by chance. Rule of Thumb, if the coefficient is:
.75 = very good .75< x >.6 = good .4< x >.59 = fair. Structured Observation - A relatively undressed method in social research. It entails the direct observation of behaviour, which is analyzed using categories that are devised before the observation begins. Other Sources of Data - Includes: Personal documents, Secondary analysis, and Official Statistics (unobtrusive measures). Validity - A research criterion concerned with the integrity of the conclusions generated by a particular study. Other Sources of Data: Personal Documents -
- Diaries, Letters, Autobiographies
◦ Ecological fallacy - data gathered at region or household level is used to make statements about individuals ◦ No control over data quality ◦ Absence of key variables Other Sources of Data: Official Statistics Advantages - ◦ Based on populations not samples ◦ Since data is not being gathered in a study format, reactivity is less pronounced ◦ Increased prospect of analyzing data longitudinally and cross- sectionally Other Sources of Data: Official Statistics Disadvantages - ◦ Misleading, e.g., crime rates - what about under-reported or not reported crimes Reactivity/ Reactive effect - The effect on research participants of knowing that they are being studied, which may result in atypical or inauthentic behaviour. Reliability - The degree to which a measure of a concept is stable or consistent. Other Sources of Data: Official Statistics Reliability and Validity Issues -
- Variations in definitions and policies shift/ differ over time
- Unobtrusive measures Other Sources of Data: Unobtrusive method - a. Physical traces b. Archived materials c. Simple observation Other Sources of Data: Unobtrusive measure: Physical Traces - The "physical signs left behind by a group" and include things such as graffiti and trash, or paper trails of financial transactions. Other Sources of Data: Unobtrusive measure: Archive Materials - Includes documents and any other information collected by governmental and non- governmental organizations. Other Sources of Data: Unobtrusive measure: Simple Observation - Refers to "situations in which the observer has no control over the behaviour... in question, and plays an unobserved, passive, and non-intrusive role in the research situation". Qualitative Analysis - Focuses mainly on words and images rather than numbers. Qualitative researchers tend to product inductive, constructionist, and interpretivist studies, although they do not necessarily subscribe to all three of those perspectives. The Nature of Qualitative Research: Inductive: - An approach to inquiry that begins with the collection of at a, which are then used to develop theories, hypotheses, and concepts.
The Nature of Qualitative Research: Interpretivist: - An epistemological position that requires the social scientist to grasp the subjective meanings that people attach to their actions and behaviours. The Nature of Qualitative Research: Constructionist: - An ontological position (the antithesis of objectivism) according to which social phenomena and their meanings are continually being created by social actors. The Nature of Qualitative Research: Naturalistic Approach: - A style of research designed to minimize disturbance to the natural or everyday social world. The Nature of Qualitative Research: Ethnography/Participant observation - A research method in which the researched is immersed in a social setting for an extended period of time, observing behaviour, asking questions, and analyzing what is said in conversations both between subjects ad with the fieldworker. As a term, ethnography is more inclusive than participant observation, which emphasizes the observational component. Written accounts of ethnographic research are often referred to as ethnographies. The Nature of Qualitative Research: Unstructured Interviewing - An interview in which the interviewer is free to explore any topic, although an interview guide is often used. the questioning is usually informal and the content, phrasing and sequencing of questions may vary from one interview to the next. The Nature of Qualitative Research: Focus Groups - A form of group interview in which there are several participants; there is an emphasis in the questioning on a particular topic or related topics; and interaction within the group and the joint construction of meaning is observed. The Nature of Qualitative Research: Discourse Analysis - An approach to the analysis of talk and other forms of communication that emphasizes the way language can create versions of reality. The Nature of Qualitative Research: Conversation Analysis - The fine drained analysis of talk (recorded in naturally occurring situations and then transcribed) to uncover the underlying structures in interaction that make social order possible. The Nature of Qualitative Research: Qualitative content analysis - An approach to constructing the meaning of documents and text that allows categories to emerge out of data analysis and recognizes the significance of the context in which items appear. The Nature of Qualitative Research: Participatory action research - Research in which local people affected by a particular social problem collaborate as equals with professional researchers and government officials to generate knowledge pertinent to the problem and to take action to ameliorate it. The Nature of Qualitative Research: The Main Steps -
Ensuring that researcher's interpretations are congruent with the participants' lived experiences. Difficulties: Defensive reactions from participants; participants may be reluctant to be critical; language incongruence The Nature of Qualitative Research: Transferability - Geertz's (1972) "thick descriptions" The Nature of Qualitative Research: Dependability - "auditing" approach with peers as auditors The Nature of Qualitative Research: Confirmability - ensuring that the researcher has acted in "good faith;" one of the objectives of the external auditors The Nature of Qualitative Research: Issue of Criteria - Need for reliability and validity checks to ensure "scientific rigour" of the study The Nature of Qualitative Research: Main Goals -
- Seeing through the eyes of people being studied a. Description and the emphasis on context
- Emphasis on process
- Flexibility and limited structure The Nature of Qualitative Research: Critiques -
- Too subjective
- Issues with replication
- Problems of generalization 4. Lack of transparency Qualitative data analysis software like Nvivo 11 = greater standardization in data analysis (advanced organization and classification systems) The Nature of Qualitative Research: Life History - A method, often referred to as the biographical method, that emphasizes the inner experience of individuals and its connections with larger societal events throughout the life course. it usually entails life history interviews and the use of personal documents as data. Contrasts between Qualitative and Quantitative Research -
- Numbers / Words
- Researcher point of view / research participant point of view
- Researcher distant / researcher close
- Theory testing / theory development
- Structured / Unstructured
- Generalizable Knowledge / Contextual Understanding
- Hard, Reliable Data / Rich, Deep Data
- Macro / Micro
- Behaviour / Meaning
- Artificial Settings / Natural Settings
Ethnography and Participant Observation - Require extended involved in the activities of the people under study Four Types of Ethnography -
- Overt role in open/public setting
- Overt role in closed setting 3. Covert role in open/public setting
- Covert role in closed setting Open = e.g., libraries, parks, sidewalks Closed = e.g., schools, social movements, corporations Ethnography: Key Points about 4 Types -
- Open vs. closed settings is not "hard and fast"
- Overt vs. covert distinction can vary from context to context even in the same study
- Preferred choice is that of an overt role
- Advantages of covert role: easier access; less reactivity
- Disadvantages: problem of taking notes; not being able to use other methods (i.e., interviews); anxiety; and ethical problems Covert Ethnography - The researcher does not reveal his or her true identity and/or intentions. Such research may violate the ethical principle of informed consent. Overt Ethnography - Overt observations refer to the researcher being open about their intentions in the field and ensuring all members of the social group are aware of what is happening. Ethnography: Access to closed settings - ◦ Use of social networks to gain access ◦ Recruit "sponsors" and "gatekeepers" ◦ Offer something in return, e.g., a final report ◦ Provide a clear explanation of your objectives and methods ◦ Be prepared to negotiate ◦ Be honest about the amount of people's time likely to be needed ◦ Pay attention to dress and demeanour Ethnography: Access to Open Settings - ◦ Similar issues as gaining access to closed settings ◦ The importance of "sponsors" and "gatekeepers" ◦ "hanging around"- pay attention to dress and demeanour Ethnography: Ongoing Access - ◦ Suspicion of "researcher's" presence and/or intent ◦ Concern over researcher's level of discretion ◦ Participants may "react" against the researcher Ethnography: Ways to ensure continued access -
Feminist Ethnography - Form of Participatory Action Research (PAR) Goal: giving "voice"to vulnerable or marginalized groups of women Non-exploitative - negotiated relationship; reciprocity From a female standpoint Understanding women in context Ethnography: Exiting the Field - Reasons for exiting: ◦ Personal ◦ Time constraints ◦ Research questions have been sufficiently answered -"saturation" of themes Protocol and ethical commitments, i.e., maintaining anonymity of participants Unstructured Interviewing - ◦ Can be a narrative - one question to begin ◦ Similar to a conversation ◦ Series of prompts with very few questions Semi-Structured Interviewing - ◦ Series of questions and prompts ◦ Interview guide or schedule Interviewing in Qualitative Research: Steps to Formulate Questions (GSIFRPIRF) - general research area > specific research questions > interview topics > formulate interview questions > review/revise interview questions > pilot guide (test) > identify novel issues > review interview questions > finalize guide Interviewing: Types of Questions (IFPSDISSI) - Kvale's nine types:
- Introducing
- Follow-up
- Probing
- Specifying
- Direct
- Indirect
- Structuring
- Silence
- Interpreting Interviewing: Recording and Transcribing - A. Audiotaping and/or videotaping interviews requires interviewee's consent (added) B. Transcription of interviews should take place soon after the interview ◦ Every one hour of tape requires four to five hours of transcription ◦ Importance of a good transcriptionist - well- trained and attentive Moderator/Facilitator - The person who guides the questioning of a focus group, also called a facilitator. Interviewing: Features of Focus Groups -
◦ Allows researcher to elicit a wide variety of perspectives on an issue ◦ Interviewees may challenge each other, eliciting more realistic accounts ◦ Reflect the processes through which meaning is constructed in everyday life - more naturalistic than one-on-one interviews Interviewing: Selecting Focus Groups - ◦ Generally 6-10 per group ◦ Selection on basis of socio-demographic (e.g., gender, class, age, ethnicity) or other characteristics (e.g., professional group/category) Interviewing: Group Interaction - Complementary vs. argumentative Interviewing: Limitations of Focus Groups -
- Researcher has less control over the discussion => can get "out of hand"
- "unwieldy" amount of data may be produced
- Data may be difficult to analyze
- Can be difficult to arrange/set up
- Group effects may be an issue, e.g., groupthink, silencing
- Not suitable for all personalities, situations, or topics (sensitive, private experiences) Sampling: - Process of recruiting participants into a study Census - A count of an entire population. Sampling Error - Differences between the characteristics of a random sample and the population from which it is selected. Probability Sample - A sample, selected at random, in which each unit in the population has a known probability of being selected. Non-Probability Sample - A sample selected using a non-random sampling method. Essentially, this means that some units in the population are more likely than others to be selected. Representative Sample - A sample that is similar to the population in all important respects. Sample - The segment or subset of the population selected for research. The selection may be based on either probability or non-probability sample. Sampling Frame - A listing of units in the population from which a sample is to be selected. Population -
The Convenience Sample -
- Locate a group of people that are easy to find/access once you have decided on your location/site for data collection
- e.g., undergraduate students, café patrons, nursing staff in a long term care facility
- Interview and/or observe them in their "natural" setting The Snowball Sample -
- Find a few individuals that are relevant to your topic
- e.g., street-involved youth, single mothers living in co-operative housing, lesbian parents, grandchildren living in skipped generation families
- Ask these individuals to refer you to more individuals in their social network The Quota Sample -
- (+) when you don't have a lot of time
- Determine what the population looks like in terms of specific characteristics, e.g., gender, ethnicity, immigration status, SES
- Create "quotas" based on those qualities
- Select people to fill each quota Theoretical Saturation - In grounded theory, the point where emerging concepts have been fully explored and no new insights are being generated. Ethnographers Sample: - People, Places, Contexts, Times, Events Interviewers Sample: - People, Places, Times Content analysts sample: - Media, Dates Sampling: How Many? -
- Qualitative researchers seek "saturation"
- Quantitative researchers seek statistical validity Sampling: Improving Response Rates - Be tenacious - call back/follow-up Personalize the invitation to participate Write a good cover letter Don't make the questionnaire too long Offer incentives -- no strings attached ◦ e.g., copies of reports, food, child care, transportation, honoraria Quantitative Data Analysis: Nominal Variables - There are only qualitative differences between categories ◦ Categories cannot be ordered by rank ◦ Cannot do arithmetic or mathematical operations with the categories
◦ e.g., religious affiliation, marital status, province of residence Qualitative Data Analysis: Ordinal Variables - The categories of the variable can be rank ordered ◦ e.g., self-reported health status: categories are "excellent,"very good," "good," "fair," and "poor" ◦ Distance or amount of difference between categories may not be equal ◦ Cannot do arithmetic or mathematical operations with the categories Qualitative Data Analysis: Interval/ratio: - ◦ Distance or amount of difference between categories is uniform, e.g., Number of siblings: 0 siblings, 1 sibling, 2 siblings, etc. ◦ Can do arithmetic and mathematical operations with the categories, e.g., 1 sibling + 3 siblings = 4 siblings Interval/ratio variables can be reduced to ordinal or nominal variables (although that involves a loss of information) ◦ e.g., ageàage categories (young, mid-life, old) Qualitative Data Analysis: Univariate Analysis: Frequency Tables - Often, the first step in the analysis is to create frequency tables for the variables of interest. When interval/ratio variables are shown in frequency tables, some of the categories may be combined, otherwise the table would become too large ◦ e.g., age, income Qualitative Data Analysis: Univariate Analysis: Diagrams - Diagrams can be used to illustrate frequency distributions These include bar charts, pie charts, and histograms ◦ Bar and pie charts - nominal + ordinal level variables ◦ Histograms - interval/ratio level variables (may be collapsed categories or not) Measures of Central Tendancy: Mode - Mode: The most frequently occurring score, category or value ◦ Can be used with all levels of measurement Measures of Central Tendancy: Median - Median: The middle score when all scores have been arrayed in order (typically from lowest to highest) ◦ Can be used with ordinal or interval/ratio data ◦ If there is an even number of cases, take the mean of the middle two cases Measures of Central Tendancy: Mean - Mean: sum of all scores, divided by the number of scores ◦ Can be used with interval/ratio data ◦ Vulnerable to outliers (extreme scores - low or high) Measures of Dispersion: range - Maximum (highest) score minus minimum (lowest) score ◦ Influenced by outliers
l Fewer 'clear cut rules' l Uses guidelines l Coding is the main feature l Inductive l Iterative or reiterative Qualitative Data Analysis: Analytic induction - an approach to the analysis of qualitative data in which the collection of data continues and the hypothesis is modified until no cases inconstant with it are found. example: l Research question - How do people experience grief? l Hypothetical answer - '5 stages' l Researcher would collect data l Universal explanation would be found - Maybe grief is a more positive process when grief is considered legitimate by others (legitimation of experience critique: l Lack of useful guidelines (e.g., number of cases) l Explanations generated can be too broad Qualitative Data Analysis: Grounded Theory - an approach to the analysis of qualitative data in which the goal is to use the data to generate theory; the data collection and analysis proceed in an iterative fashion. Tools: l Theoretical saturation - the point where no new data arises l Constant comparison - process of comparing new data with data already collected in the study l Coding Example: l Select sample l Conduct interviews l Immediately transcribe, code (open, axial, and selective) and memo l Practise constant comparison (e.g., explore relationships, develop and saturate categories) l Theoretically sample l Develop a 'grounded theory' Outcomes: l Concepts - 'labels given to discrete phenomena' l Categories - an abstract collection of concepts l Properties - 'attributes or aspects of a category' l Hypotheses - 'initial hunches about relationships between concepts' l Theory - A collection of related categories intended to explain a phenomena Critiques: l Is it possible to suspend all other/prior knowledge? l Transcription l Reaching saturation is problematic l Fragmentation and loss of context
Qualitative Data Analysis: Narrative analysis - an approach focused on the search for and analysis of stories that people use to understand their lives and the world around them. Four Models:
- Thematic - what is said
- Structural - the way a story is told
- Interactional - dialogue between storyteller and listener
- Performative - narrative/story as performance Critiques: l Researchers may treat stories uncritically l Researchers may superimpose meanings on narratives Grounded theory: Coding - l Fluid process l Codes are intended as indicators l First step in generating a theory l Three types of coding according to Strauss and Corbin: Open (initial) coding; Axial coding; and Selective coding (focused) l Some researchers avoid axial coding as they feel it closes off the coding process too early Quantitative Research and Interpretivism - Many quantitative researchers are interested in issues of meaning, e.g., researchers who examine attitudes using surveys e.g., How do you feel about the changes to the Sociology curriculum, i.e., the introduction of new courses? ◦ Satisfaction scale that measures attitudes Quantitative Research and Constructionism - Quantitative methods (such as quantitative approaches to content analysis) can be used to establish how people create their sense of reality e.g., one could study the content of TV shows quantitatively to illustrate a particular way in which gender socialization takes place Research Methods and Epistemological and Ontological Considerations - Researchers using a particular research strategy do not always share the same epistemological and ontological assumptions Use of certain research methods may not be accompanied by the expected epistemology and ontology The Quantitative/Qualitative Contrast: A Nuanced View - Behaviour vs. Meaning: ◦ Many perspectives in quantitative social science now consider meaning to be important ◦ Qualitative research often involves the study of behaviour Theory and Concepts Tested in Research vs. Those Emerging from the Data - Much quantitative research (e.g., survey research) is largely inductive ◦ Hypotheses and theories may not emerge until after the data have been gathered
