Reviewing the Literature

P9419

Class #4

October 20, 2003

Now you have EndNote

And dataset

And readers

And ~research question

What next?

Literature search

Retrieve other articles based on data from your dataset

Check the lists of references in those articles

Search readers’ articles on related topics

Generate list of search terms based on

research question

Variables/categories in the dataset

Experiment with combinations of or subsets of search terms

Keep track of your search terms

Home hazards and falls

Home hazards and falls à 72 refs

Home hazards à 637 refs

Falls à 870 refs

Falls and community living à 288 refs

Falls and nursing homes à 360 refs

Falls and Kelsey J à 16 refs

Falls and elderly à 5279 refs

Falls and fractures à 108 refs

Home hazards and fractures à 27 refs

So many articles, so little time . . .

Exclude publications in languages you don’t read

Exclude publications before a certain date except landmark articles frequently cited

Rethink your research question

Where to begin reading

Articles based on your dataset

Recent review articles about your research question

Start plowing through list and eliminating the ones that obviously don’t belong

Master’s thesis literature review ¹ introduction/background section of journal article

Show that you really understand the issues

Familiar with the work of key contributors to the field

Strengths and weaknesses of prior work

Generic intro

X is common in many countries with a high prevalence of Y (1-15).

Prior research suggested that X causes Y (16-30).

More recent studies have suggested that Y causes X (31-45).

Only a few studies have considered the association of X and Z (46-50) or Y and Z (51-53).

We conducted a study to test the hypothesis that Z is associated with both X and Y.

Systematic review ¹ meta-analysis

Meta-analysis refers to the analysis of analyses. I use it to refer to the statistical analysis of a large collection of results from individual studies for the purpose of integrating the findings. It connotes a rigorous alternative to the casual, narrative discussions of research studies which typify our attempts to make sense of the rapidly expanding research literature.

(Gene Glass, 1976)

When studies are similar in design

Meta-analysis can help you investigate the relationship between study features and study outcomes. You code the study features according to the objectives of the review. You transform the study outcomes to a common metric so that you can compare the outcomes. Last, you use statistical methods to show the relationships between study features and outcomes.

from Rudner, Glass, Evartt, & Emery (2002). A user's guide to the meta-analysis of research studies

Problems of meta-analysis

Studies are often not similar in design, population characteristics, etc.

If they are not similar in design, then they should not be meta-analyzed.

If they are similar in design, they may have biases in common.

Pooling the results of many small biased studies gives you a biased result that is statistically significant.

Impressive but bad science.

Systematic review

First author, year

You may want to use two columns for your database so that you can later sort alphabetically by author.

Chronology is important; research builds on past results.

Don’t look just at first author.

Study design

Laboratory studies

Ecological

Case-control

Cohort

Clinical trials

Controlled

Randomized

Sample

Sample size, ratio of controls to cases, different kinds of comparison groups

Types of controls (hospital, community, RDD, screened, etc.)

Geographical location, age group, gender

Dates when data were collected (time from data collection to publication may vary)

Exposure/treatment

What is the exposure or treatment?

Dosage

Duration

Measured how?

Biologically effective dose (biomarker)

Outcome

Disease

Death due to disease

All-cause mortality

Disease recurrence

Recovery/remission

Criteria for the above

Result

Measure(s) of effect

Assessment of statistical significance

Identification of confounders/effect modifiers

Comments, strengths/weaknesses

Sample size and power

Handling of known confounders/effect modifiers

Human subjects

Other categories?

Create your own

Compare apples to apples

Play with hierarchy of categories

Come back to your research question/hypotheses

Come back to your search

That’s why they call it research . . .


	Retrieve other articles based on data from your dataset
	Check the lists of references in those articles
	Search readers’ articles on related topics
	Generate list of search terms based on
		research question
		Variables/categories in the dataset
	Experiment with combinations of or subsets of search terms
	Keep track of your search terms


	Home hazards and falls à 72 refs
	Home hazards à 637 refs
	Falls à 870 refs
	Falls and community living à 288 refs
	Falls and nursing homes à 360 refs
	Falls and Kelsey J à 16 refs
	Falls and elderly à 5279 refs
	Falls and fractures à 108 refs
	Home hazards and fractures à 27 refs


	Exclude publications in languages you don’t read
	Exclude publications before a certain date except landmark articles frequently cited
	Rethink your research question


	Articles based on your dataset
	Recent review articles about your research question
	Start plowing through list and eliminating the ones that obviously don’t belong


	Show that you really understand the issues
	Familiar with the work of key contributors to the field
	Strengths and weaknesses of prior work


	X is common in many countries with a high prevalence of Y (1-15).
	Prior research suggested that X causes Y (16-30).
	More recent studies have suggested that Y causes X (31-45).
	Only a few studies have considered the association of X and Z (46-50) or Y and Z (51-53).
	We conducted a study to test the hypothesis that Z is associated with both X and Y.


	Meta-analysis refers to the analysis of analyses. I use it to refer to the statistical analysis of a large collection of results from individual studies for the purpose of integrating the findings. It connotes a rigorous alternative to the casual, narrative discussions of research studies which typify our attempts to make sense of the rapidly expanding research literature.
	(Gene Glass, 1976)


	Meta-analysis can help you investigate the relationship between study features and study outcomes. You code the study features according to the objectives of the review. You transform the study outcomes to a common metric so that you can compare the outcomes. Last, you use statistical methods to show the relationships between study features and outcomes.
	from Rudner, Glass, Evartt, & Emery (2002). A user's guide to the meta-analysis of research studies


	Studies are often not similar in design, population characteristics, etc.
	If they are not similar in design, then they should not be meta-analyzed.
	If they are similar in design, they may have biases in common.
	Pooling the results of many small biased studies gives you a biased result that is statistically significant.
	Impressive but bad science.


	You may want to use two columns for your database so that you can later sort alphabetically by author.
	Chronology is important; research builds on past results.
	Don’t look just at first author.


	Laboratory studies
	Ecological
	Case-control
	Cohort
	Clinical trials
		Controlled
		Randomized


	Sample size, ratio of controls to cases, different kinds of comparison groups
	Types of controls (hospital, community, RDD, screened, etc.)
	Geographical location, age group, gender
	Dates when data were collected (time from data collection to publication may vary)


	What is the exposure or treatment?
	Dosage
	Duration
	Measured how?
	Biologically effective dose (biomarker)


	Disease
	Death due to disease
	All-cause mortality
	Disease recurrence
	Recovery/remission
	Criteria for the above


	Measure(s) of effect
	Assessment of statistical significance
	Identification of confounders/effect modifiers


	Sample size and power
	Handling of known confounders/effect modifiers
	Human subjects


	Create your own
	Compare apples to apples
	Play with hierarchy of categories
	Come back to your research question/hypotheses
	Come back to your search
	That’s why they call it research . . .