The Goals of Literature Reviews
As part of a scholarly work, a literature review offers an overview of the existing knowledge and scholarship against which the current contribution can be interpreted and judged. In the world of scholarship, a literature review has an implicit ethical responsibility combined with a practical aim. With it, the authors of a scholarly work (one that professes to offer an original intellectual contribution) claim to provide a fair and reasoned overview of the previous scholarship one needs to know in order to evaluate the research and arguments developed in the work. If it falls short, other scholars with authoritative knowledge will judge it either inadequate or deceptive. A first principle of literature reviews is that they represent what scholars who work in the area of our research consider important. We may add materials we personally judge important, regardless of the regard they hold with other scholars. However, we should not exclude works that other scholars largely consider important based on our disagreement with that judgment. Instead, we include them while adding the criticisms that motivate our less favorable judgments.
Writing a literature review involves several overlapping tasks. First, we must identify the relevant materials, the ingredients of our review. Second, we want to catalog (1) the theoretical arguments, (2) the empirical findings, and (3) any critical methodological insights across all the materials. In parallel, we want to catalog the flaws, shortcomings, or omissions we perceive in the ideas, findings, or methods. We also want to look carefully for explicit or implicit disagreements, and catalog them as well. Third, we decide how to organize the review around ideas, findings, and methods.
We organize the review around the ideas and the findings, highlighting the widely accepted and the disputed. We do not organize the literature review directly around the articles (and books) being reviewed, simply providing summaries of each, although some authors find writing summaries of each piece a useful intermediate step. Sometimes the reviewed materials only appear as citations in our literature review, as when we state that several studies have found some empirical pattern or argued for some interpretation, and then simply list the relevant citations. Sometimes we will discuss the ideas, procedures, or findings of a particular study in greater depth, either because it is important or because it exemplifies an approach. We do not usually use a chronological organization for the literature review as a whole, but we do sometimes use it for sections of the review when it seems appropriate to show how some line of research or theory developed over time.
Overall, our literature reviews should accomplish two tasks. (1) They assure scholars with authoritative knowledge about the area that we know what we are doing. (2) They provide other scholars with the scholarly context of our research and arguments so that they can see how our work is a contribution and assess its value (and, if desired, further investigate the prior work based on our references).
How do we produce a literature review? Let me suggest two basic strategies. First, we can search databases to identify the materials we need to review, and then devise an appropriate organizing framework to present the review. Second, we can find some published studies that include literature reviews aimed at research problems similar to our own, and use them as a source of references and organizing strategies. Generally, we should expect to use both these strategies, with each leading to the other.
Building on the Literature Reviews of Others
The easiest starting point for a literature review is to follow the lead of existing literature reviews, if they exist. Literature reviews generally exist in two forms. First, most scholarly research articles will include a literature review relevant to their research. Second, some literature reviews appear as independent articles. The Annual Review of Sociology and annual reviews for other disciplines exclusively publish literature reviews, while regular scholarly journals publish dedicated literature reviews unpredictably.
The literature reviews in research articles, when done well, will cover both the published research directly on the same or closely related research questions and will discuss principal relevant findings and theoretical perspectives of the broader subject categories that have significance for the articles’ research. These reviews will also seek to provide good coverage of recent contributions and selective coverage of “classic” earlier contributions that still influence work today. In short, they aim to do exactly what you aim to do in your literature review, which is why they are such a good starting point.
The full literature review articles include articles that present narrative reviews of the relevant literature as well as meta-analyses of the findings in many research articles. Of these two, the narrative, integrative reviews are closer to your goals. Compared to a literature review that is part of a research article, a narrative review article will usually cover more works or cover works in more depth and assess the works more on their independent merits than on their value to a specific research question. The goals of a meta-analysis differ, typically aiming at a more unified portrait of the results in the published research articles that pursue a similar problem, by aggregating them through a statistical analysis. When the statistical analysis provides a convincing summary of the research findings, we may be able to use the meta-analysis as a substitute for all the literature it assesses. In general, literature reviews in research articles on a similar topic are the most valuable to us, with full literature review articles and meta-analyses helping us interpret and summarize select sets of research publications more efficiently.
Ideally, we would like to find at least three literature reviews. We cannot be confident about the completeness, selection logic, or interpretive accuracy of any particular review. If we have several, we can look for the overlaps and the discrepancies. Of course, if we have several literature reviews, we want to check the likelihood that one has influenced another, implying they are not independent sources; the simplest question is to check if any of the articles containing a review cites one of the others where. To the degree that we have independent literature reviews, we can infer that wherever they agree in their references to research and theoretical work, those materials will be relevant to our literature review. Note that we do not want to look only for exact correspondence in the sense that they cite the same article, but also for similarities, such as them citing different articles from the same author, or referring to the same theoretical argument but citing different presentations of that argument. Of course, if we can find only one relevant literature review, we can still use it as a starting point for our explorations of the literature. The difference is that we cannot have confidence in the importance of the specific materials cited, so we have to find more evidence before we judge the significance of the works.
How do we use the literature reviews after we have found them? First, we, of course, place the research articles that contain the literature reviews on our provisional literature review list. Second, we read the literature reviews with the aim of identifying the references that appear to be appropriate for our own literature review and add them to our provisional list. (Note that we do not yet need to read the rest of the research articles that contain the literature reviews – leave that until we have our list more complete.) Third, if any references now added to our provisional list appear to be directly relevant to our research question, we retrieve them to obtain their literature reviews and then read those literature reviews to expand our provisional list again.
In short, we try to use the literature reviews found in relevant research articles to identify materials that are prospects for our own literature review, and where possible repeat that process based on the materials identified in the previous round, until we feel that we have gone as far with this method as is reasonable. When we are lucky, meaning that we find a number of research articles that concern research questions closely related to our own research question, we will quickly be able to assemble a good provisional list of relevant past work. If the literature reviews we use are at least adequate, then our provisional list will include an appropriate range of recent and classic contributions, of research and theory focused near our topic and relevant work that is more general.
What do we do with this provisional list of references? That part is easy. We read it all. Initially, we skim everything to determine its usefulness and start to organize a working list of references for the literature review, in which we categorize the references by their relevance, and by whatever criteria we think make sense for ordering our review (the organizing criteria will change as we get better understanding of the material).
We also potentially get an extra value from previous literature reviews – ideas about how to organize the materials. Recall that we should organize a literature review around the ideas and findings – which are what we really are reviewing – not around a sequential summary of the references. If we can find some good literature reviews, they will provide valuable ideas about ways to categorize the relevant publications, about the theoretical or empirical cleavages that divide the researchers in this area, and about the shortcomings. We are free to borrow and build upon the organizing principles and comparative insights in others’ literature reviews, as long as we remember to cite them properly. Borrowing ideas from other authors’ reviews without properly citing them constitutes plagiarism. Usually all we need to do is something like ‘Jones (2012) suggests that we can usefully divide the findings [theories] regarding the relation between having children and divorce into three distinct explanatory concerns’ followed by a description of the categories and their motivation.
One temptation in the use of other authors’ literatures we want always to avoid. We cannot legitimately rely on others' interpretations of the works we are reviewing rather than reading them ourselves. Anytime we do want simply to restate some other author’s summary or interpretation of a published work, we must cite the author whose ideas we are borrowing. We cannot avoid this obligation by rewriting the ideas in our own words or meshing the ideas from several authors so they do not resemble any one of them – we still have to cite the sources of the ideas.
Of course, before we use existing literature reviews, we must find them. For this we turn to database searches.
Searching Databases to Find Relevant Publications
What do we do if we do not have existing literature reviews that can guide us? We look for them. Of course, we have to consider the possibility that we cannot find any articles that aim at research questions similar to our own and are recent enough that their literature reviews can be our starting point. In both cases, we must pursue systematic search strategies to find publications that relate to our research question. If the searches uncover good literature reviews in research articles (or, to a lesser degree, stand-alone reviews), we can switch to the strategy based on using existing literature reviews. If the searches fail to identify sources for helpful literature reviews, we must rely on a full-scale literature search of our own. Fortunately, with modern databases available over the internet, this is a fairly straightforward task, although it does require a significant investment of time.
Usually, we do this by a series of successive searches that build on each other. First, we use basic keyword searches. If our research is on the possible effects of having children on divorce in the early years of marriage, for example, we might search on “(reproduction OR birth) AND divorce” as a first, very broad search pattern. We could do this using the Social Science Citation Index, in Google Scholar, or in Sociological Abstracts, for example. The SSCI has the valuable quality of allowing us to sort the findings by year or number of citations (and has the most complete coverage of journals). Google Scholar is simplest to use and does the best job including books as well as articles. Sociological Abstracts is particularly good for finding older contributions. All will provide links to NYU library to access those publications that are available online. These initial keyword searches commonly produce many “misses,” works that are not really of value to us. Then, we have to comb through many pages of references, looking at abstracts if the titles seem at all possible, picking out the ones that seem most likely. If we are lucky, we will find some good starting points quickly. If we are not, this can take a while.
Once we find some relevant articles, particularly ones that are close to our research question, we will want to move to citation searches. This is especially the case when the past research we find using keywords is old. Using the SSCI (recommended) or Google Scholar or both (most recommended), we search for the publications that have cited the research articles we have found related to our question. (In SSCI, we choose Cited Reference Search rather than Basic Search; in Google Scholar we search for our target article, then click on the link where it says “cited by N” after the entry.) These search tools will then list all the articles (all they “know” about) that have cited the target article (or book). Again, we scan through the resulting lists of citing articles, looking for titles that seem relevant, opening abstracts for further information as seems desirable, and selecting the references that seem the best bets. For publications that are not too recent, we can use the number of citations they have received as an indicator of their perceived significance (this approach has significant limitations, but it works fine as a selection device at this stage). Of course, once we find relevant publications, we can use them to begin more citation searches. (Usually, somewhere in the midst of this we are going to find some research articles with useful literature review, even if they are not directly on our research question.)
A first rule for any search engine, which includes databases, is to learn how they work. If you look around a bit, you will find a “Help” link or a link to a guide. Read these carefully. Every search engine has its own rules and many have useful settings. People who use these search engines or databases without taking the time to learn how they work commonly spend more time while getting poorer returns.
Most databases and search engines allow Boolean searches, using AND, OR, parentheses, and possibly wildcards. Each of them usually has some unique rules about Boolean searches and they often have special capacities (such as ways to find similar items or specifying how close together two terms must be). Effective use of Boolean search capabilities greatly enhances the quality of search outcomes.
To save time, discover and use each search engine’s capacity to store and organize your search results. For example, the Social Science Citation Index allow you to register, after which you sign in to your account each time, and you can mark and save any references you find that you consider useful. You can also link to Endnote online through SSCI. With Google Scholar, you can do something similar by setting up your “library” in which you can save references. Using these capacities allows you to save references online so that you can access them from any location where you can access the internet, you can use the search results as starting points for further searches as needed, and you can later download them as a group when convenient.
The greatest difference between relying sole on this method of successive searches and relying more on other’s literature reviews as your starting point is not the difficulty of finding the relevant works. Rather, it is that you must develop the organization of ideas and findings in the literature on your own. This is a challenge and an opportunity. However, it always requires far more work.
Note that even when we have some existing literature reviews that can serve as starting points to our own literature reviews, we want to spend time doing some serious searches using databases. This is usually the only way that we can find recent work that we can use, because published literature reviews will commonly fail to cover recent years. In addition, database searches allow us to check if the literature reviews we find have neglected some important work relevant to our research.