Research Issues: Validity, Reliability, Objectivity, Representativeness And Ethics As Important Concepts In Assessing The Value Of Different Research Methods (Copy)

VALIDITY

Meaning of Validity

• Validity = whether the research truly measures or captures what it claims to study
• Concerned with truthfulness, accuracy, and authentic representation
• High validity → deep, realistic, context-rich understanding
• Central to interpretivist evaluation of methods

Types of Validity

1. Internal Validity

• Accuracy within the research setting
• Whether findings reflect participants’ true experiences
• Threatened by:
  • Researcher influence
  • Leading questions
  • Social desirability
  • Environmental manipulation

2. External Validity

• Whether findings can be generalised beyond the research context
• Limited in:
  • Small samples
  • Unique settings
  • In-depth ethnographies

3. Ecological Validity

• Whether behaviour observed reflects natural, real-world settings
• Highest in:
  • Covert observation
  • Naturalistic ethnography
• Lowest in:
  • Laboratory studies
  • Artificial experiments

4. Interpretive Validity

• Whether researcher correctly understands participants’ meanings
• Achieved through:
  • Verstehen
  • Rapport building
  • Participant validation

Research Methods with HIGH Validity

• Unstructured interviews
• Participant observation
• Ethnography
• Life histories
• Personal documents
• Covert observation (natural behaviour)

Research Methods with LOW Validity

• Structured interviews
• Questionnaires
• Experiments
• Official statistics (may misrepresent)

Threats to Validity

• Demand characteristics
• Leading questions
• Hawthorne effect
• Researcher bias
• Misinterpretation of behaviour
• Cultural misunderstanding
• Sensitive topics causing concealment

Ways to Increase Validity

• Triangulation
• Respondent validation
• Reflexivity
• Prolonged field immersion
• Neutral, non-leading questions
• Naturalistic observation

RELIABILITY

Meaning of Reliability

• Reliability = consistency and repeatability of findings
• If repeated under the same conditions, results should remain similar
• Central to positivist evaluation of methods

High-Reliability Methods

• Structured questionnaires
• Structured interviews
• Content analysis using coding frames
• Official statistics (standardised collection)
• Experiments
• Systematic observation

These use standardised procedures → easier replication.

Low-Reliability Methods

• Unstructured interviews
• Participant observation
• Ethnography
• Covert observation
• Life histories
• Interpretive documents

Behaviour varies across situations → low replicability.

Threats to Reliability

• Researcher effects
• Emotional influence
• Inconsistency in asking questions
• Human unpredictability
• Changing field conditions
• Unique interactions

Ways to Increase Reliability

• Standardised interview schedules
• Closed questions
• Structured observation categories
• Clear operationalisation
• Training of interviewers
• Detailed coding frameworks

Written and Compiled By Sir Hunain Zia, World Record Holder With 154 Total A Grades, 7 Distinctions and 11 World Records For Educate A Change AS Level Sociology Full Scale Course

OBJECTIVITY

Meaning of Objectivity

• Objectivity = absence of researcher bias
• Findings reflect reality rather than researcher’s:
  • Politics
  • Emotions
  • Values
  • Identity
• Positivists argue objectivity essential to scientific research

Sources of Researcher Bias

• Personal values
• Political ideology
• Cultural background
• Gender or ethnicity
• Class position
• Emotional involvement
• Desire for certain findings

High-Objectivity Methods

• Surveys
• Structured interviews
• Experiments
• Official statistics
• Secondary quantitative data

These minimise interaction → less bias.

Low-Objectivity Methods

• Unstructured interviews
• Ethnography
• Participant observation
• Interpretive document analysis

Researcher highly involved → bias risk.

Ways to Increase Objectivity

• Detached researcher role
• Standardisation of procedures
• Use of quantitative data
• Statistical techniques
• Reflexive practice
• Using multiple researchers (investigator triangulation)

Critiques of Objectivity

• Complete objectivity impossible
• All research choices reflect values
• Funding sources influence agenda
• Professional ideology shapes interpretation
• Interpretivists argue:
  • Values not a threat → they enrich understanding

REPRESENTATIVENESS

Meaning of Representativeness

• Representativeness = whether the sample accurately reflects the wider population
• High representativeness → findings generalisable
• A key concern of positivist methodology

Factors Affecting Representativeness

• Size of sample
• Sampling technique
• Diversity of participants
• Response rate
• Sampling frame accuracy
• Access to hidden groups

Methods with HIGH Representativeness

• Large-scale surveys
• Census data
• Probability sampling:
  • Random sampling
  • Stratified sampling
  • Systematic sampling
• Official statistics (in some cases)

Methods with LOW Representativeness

• Snowball sampling
• Convenience sampling
• Participant observation
• Small-scale ethnography
• Unstructured interviews
• Case studies

Barriers to Representativeness

• Difficulty accessing certain groups:
  • Homeless
  • Criminals
  • Wealthy elites
  • Undocumented migrants
• Low response rates
• Gatekeeper control
• Self-selection bias
• Cultural language barriers
• Fear of disclosure on sensitive topics

Ways to Increase Representativeness

• Larger sample size
• Stratified random sampling
• Incentives for participation
• Multi-stage sampling
• Improving sampling frame accuracy
• Using official registers

Written and Compiled By Sir Hunain Zia, World Record Holder With 154 Total A Grades, 7 Distinctions and 11 World Records For Educate A Change AS Level Sociology Full Scale Course

ETHICS

Meaning of Ethics in Research

• Ethics guide how researchers treat participants
• Sociologists must avoid:
  • Harm
  • Deception (unless justified)
  • Privacy violations
  • Coercion
• Ethics central to evaluating whether a method is acceptable

Main Ethical Principles

1. Informed Consent

• Participants know:
  • Purpose
  • Process
  • Risks
  • Use of data
• Special protection for:
  • Children
  • Disabled individuals
  • Vulnerable groups

2. Avoiding Harm

• Physical harm
• Psychological harm
• Emotional distress
• Social harm
• Loss of reputation
• Retraumatisation

3. Confidentiality and Anonymity

• Names removed from transcripts
• Locations disguised
• Secure storage of digital files
• Encryption used

4. Right to Withdraw

• Participants can leave at any stage
• Cannot be coerced
• Particularly relevant during emotional interviews

5. Deception

• Allowed only when:
  • No alternative method possible
  • Public behaviour observed
  • Ethics committees approve
• Common in covert observation → always controversial

6. Protection of Vulnerable Groups

• Strict protocols for:
  • Prisoners
  • Children
  • Elderly
  • Victims of crime
  • Domestic violence survivors

7. Researcher Safety

• Danger during:
  • Gang ethnographies
  • Drug environments
  • Hostile communities
• Method may be changed to reduce risk

Ethical Strengths and Weaknesses Across Methods

Highly Ethical Methods

• Questionnaires
• Structured interviews
• Content analysis
• Official statistics
• Secondary data

Ethically Problematic Methods

• Covert observation
• Deceptive experiments
• Deep emotional interviews
• Ethnography in illegal settings

USING THE FIVE CONCEPTS TO ASSESS RESEARCH METHODS

QUESTIONNAIRES

Validity

• Low (superficial answers, limited depth)

Reliability

• High (standardised)

Objectivity

• High (little researcher involvement)

Representativeness

• High with large samples

Ethics

• Strong (minimal harm)

UNSTRUCTURED INTERVIEWS

Validity

• Very high (authentic meanings)

Reliability

• Very low (unique interactions)

Objectivity

• Low (researcher influence)

Representativeness

• Low (small samples)

Ethics

• Sensitive topics risky

Written and Compiled By Sir Hunain Zia, World Record Holder With 154 Total A Grades, 7 Distinctions and 11 World Records For Educate A Change AS Level Sociology Full Scale Course

PARTICIPANT OBSERVATION (OVERT & COVERT)

Validity

• Very high (natural behaviour)

Reliability

• Very low (context-specific)

Objectivity

• Low (immersion leads to bias)

Representativeness

• Very low (one group studied)

Ethics

• Overt → moderate
• Covert → highly problematic

EXPERIMENTS

Validity

• Low (artificial settings)

Reliability

• High (controlled)

Objectivity

• High (measurable outcomes)

Representativeness

• Low (small, artificial groups)

Ethics

• Experiments involving deception controversial

OFFICIAL STATISTICS

Validity

• May be low (institutional bias)

Reliability

• Very high (standardised collection)

Objectivity

• High (collected by institutions)

Representativeness

• High (large-scale population data)

Ethics

• Safe (secondary data)

ETHNOGRAPHY

Validity

• Extremely high

Reliability

• Extremely low

Objectivity

• Low (interpretive)

Representativeness

• Very low

Ethics

• Often complex (covert methods, illegal behaviour, emotional impact)