Understanding Reliability and Validity in Research: A Comprehensive Guide, Slides of Research Methodology

Download Understanding Reliability and Validity in Research: A Comprehensive Guide and more Slides Research Methodology in PDF only on Docsity!

Finding Ideas to Research

Generating Topics

• Translate ideas into valid and reliable ways of

measuring them

• Collect evidence

• Unique Topics

• Innovative but Difficult

Reliability and Validity -- Relationship

• The relationship between reliability and validity can be

confusing because measurements and research can be

reliable without being valid, but they cannot be valid

unless they are reliable.

• For a study to be valid it must consistently (reliability) do what it purports to do (validity).
• For a measurement to be judged reliable it should produce a consistent score.
• For the research study to be considered reliable each time it is replicated it too should produce similar results.

Definition; Reliability

• Reliability is the consistency of your

measurement, or the degree to which an instrument measures the same way each time it is used under the same condition with the same

subjects.

• It is important to remember that reliability is not measured, it is estimated.

Estimating Reliability

• Internal consistency estimates reliability by grouping

questions in a questionnaire that measure the same concept.

• One common way of computing correlation values

among the questions on your instruments is by using

Cronbach's Alpha.

• Cronbach's alpha splits all the questions on your instrument every possible way and computes correlation values for them all (SPSS will do this).
• Like a correlation coefficient, the closer the value is to one, the higher the reliability estimate of your instrument.

Definition; Validity

• Cook and Campbell (1979) define validity

as the "best available approximation to

the truth or falsity of a given inference,

proposition or conclusion."

• Basically, were we right?

Measurement Concepts

• Measurement is the process of assigning numbers to

represent the amount of a variable (a characteristic, attribute, trait present in a person, object, situation under study). Measurement results that contain little error are said to be reliable.

• Sources of measurement error include
  • the instrument (eg, improper calibration)
  • the environment (eg, noise level)
  • the researcher (eg, fatigue, mood)
  • data processing (eg, data entry error)

How to find a topic?

• Curiosity and Experience
  • Too big or Narrow
• Assignments, Theses, and Grants
  • RFPs, RFAs, Work related assignments
• Other Research Findings
  • Scholarly articles, secondary sources, replication, ‘filling in the hole’
• Serendipity (by accident)
  • A finding that you were not expecting

Searching for Research

• Internet

• Academic versus Nonacademic

• Library Databases

• Ask New Questions
• Once have articles, use those references
• Popular newspapers/magazines

Literature Reviews

• Evaluate Previous Research
  • Create a database (or collection)
• Attention to Methodology
  • Sampling
  • Questions/Hypotheses
  • Variables
  • Measurement
  • Analyses
  • Conclusions
  • Limitations

Writing a Literature Review

• Your literature review should reflect the

important thinking in the area that will impact your work, and should provide a context for the background and importance of the question. You should identify existing knowledge and the gaps in the knowledge, and indicate methodologies that have been used in other similar research questions. The literature review is often included as part of your research proposal.

Two levels of Review

• Conducting a literature review

• Your research investigation of the literature

• W riting a literature review

• The review you write for your own project

Deductive Reasoning

• Deductive reasoning works from the more general to

the more specific.

• It is informally called a "top-down" approach.
• We might begin with thinking up a theory about our

topic of interest. We then narrow that down into more

specific hypotheses. We narrow down even further

when we collect observations to address the

hypotheses. This leads us to be able to test the

hypotheses with specific data; a confirmation (or not)

of our original theories.

Inductive Reasoning

• Inductive reasoning moves from specific observations

to broader generalizations and theories.

• Sometimes called a "bottom up" approach
• In inductive reasoning, we begin with specific

observations and measures, begin to detect patterns

and regularities, formulate some tentative hypotheses

that we can explore, and finally end up developing

some general conclusions or theories.