Although scholars spend much time and effort writing
critiques of published work, and graduate students spend even more,
systematic treatments of the criteria for, and strategies toward, good
critiques are rare. Here are just a few points to consider.
- A critique can never be better than its author's understanding of
the work being criticized. And a reader cannot appreciate the
intent of the critique beyond their grasp of the critic's reading of
the original work. This means that to write a good critical
review, we must
- first be sure we understand what we are criticizing and,
- second, we must present a clear summary of that understanding
as part of our critique.
- Although there are important exceptions, most critical reviews
really are elaborations of the answer to a simple question.
After reading the piece, do I, the critic, find the argument
worthwhile or not? However elaborate or simple, with whatever
style of presentation, the review is largely an effort to present a
sustained defense of that evaluation.
- A good review is always fair. We should never shy from
identifying a flaw or calling a mistake what it is. But we
should always try to use the language and tone that we hope
reviewers will use when they reveal the similar failings in our
work.
- We review manuscripts, articles, and books. We do not
review people. Brilliant scholars occasionally write dim
papers--they are not dim as a result. A wise reviewer
avoids referring to the author, and concentrates on the
strengths and weaknesses of the work being reviewed. For
example, saying that "the arguments in the last and first
sections contradict each other" is preferable to saying "in the
last section, the author contradicts what she said in the first
section". Attributing thoughts and intentions to an author
is even worse (e.g., "the author seems to believe", or
"apparently the author is trying to ..."). Sitting in judgment
of the author's intellect, effort, or morality is the worst and
we should always avoid such judgments, no matter how much we may
be thinking them. Again, we review written works, we do
not review people.
- As reviewers, we commonly want to be clear about two
interpretations of what has been accomplished in the work being
reviewed: that of the author and that of the reviewer.
Sometimes these will be the same; often not. A
critic has no obligation to agree with the author's view of the
what has been done in the work. But the scholarly critic
does have an obligation to grasp and accurately present the
author's aims and orientation as they are conveyed in the
work. Once we reach the part of our review dedicated to
assessing and criticizing the material, we may express an
opinion that, for example, the theoretical argument or
inferences stressed by the author seem a poor fit to the
evidence and suggest better substitutes. Before that,
however, we should have included a summary of the author's
expressed intentions and interpretation. (Sometimes,
rarely, the author's aims and arguments may be too muddled or
incomplete for us to supply a concise, logical summary - in that
case we can only do our best.)
- When an author sees a review, the author should not feel (1)
the reviewer has said or implied I said something that I never
said or implied or (2) the reviewer has said or implied that I
failed to consider something that is explicitly part of my
presentation. Good scholars do not mind critical reviews,
even highly critical ones, that are accurate and fair.
Good scholars despise reviews that are inaccurate or
unfair, even if they are positive.
- In general terms, most reviews include three components, although
they may be merged rather than appearing separately.
- What does the reviewed
material say? In simplest form, this is a summary.
It conveys the original author's thesis, evidence, and logic
of argument. We are telling the reader of our review
what the piece achieves according to the author.
- How does the content of
the reviewed piece relate to the relevant existing
literature? With what theories or findings by others
does the piece agree or complement or disagree? We are
telling the reader of our review where we believe this work
fits in the intellectual or scholarly conversations about the
relevant topic. Where possible, this will show how the
piece builds on other work, how it seeks to challenge other
work,
- What are the strengths and
the shortcomings of the work? Does it offer new ideas or
findings, and, if so, are they compelling? Does it
address prior work in a clear, effective, and sufficiently
complete way? Does it have serious flaws of logic,
evidence, analysis, or omission? We want to remember
that we should provide the evidence or logic needed to support
our criticisms; we do not offer unsupported opinions.
- Note that for the first two components - conveying what the
reviewed material says and how it relates to other work and
ideas - we are stressing an impartial portrait. In
the third component - the assessment - we dispense with the
neutral position, developing a judgment of the quality,
creativity, validity, and significance of the work.
- Note that in the third component - the assessment - we should
also include our ideas about how to rectify its flaws. Our
goal is not simply to poke holes in someone's work, but to offer
ideas (when we have them) how we can make the best of it.
We can also discuss how others might build on the work being
reviewed, but this is more the original author's terrain, so we
need not feel obligated to pursue this.
- It may seem daunting to review a scholarly work without
authoritative scholarly knowledge in the relevant fields.
Unavoidably, the depth and effectiveness of all three
components, but particularly the critical assessment, will
depend on the reviewer's authoritative knowledge and
confidence. Even when we lack authoritative knowledge in a
field, however, so long as we could reasonably be considered as
part of the intended audience, we can write thoughtful,
accurate, and useful reviews (we can also sometimes do the job
when we are not the intended audience but we will skip that
here). If we are part of the intended audience, then any
aspect of the work that is not presented in a way we can follow
and understand must either be superfluous to our assessment
(i.e., the author may include material intended only for
"experts" and their special concerns, with the understanding
that the main argument does not depend on understanding this
material) or reflects an error in presentation by the author
(e.g., if the author presents a theoretical or methodological
component section that is necessary to the overall argument but
left incomprehensible to the a large part of the presumed
audience). This means that we can responsibly and
effectively review the work, so long as we explicitly "bracket"
the aspects of the original material that we will neglect.
- Our critiques can be stronger and more useful in general if we
consistently distinguish between several levels of error and their
implications.
- Errors of presentation. It is commonplace for
authors to make mistakes. They leave gaps in arguments,
misconstrue the meanings of other authors, neglect an
implication of the data analysis, and the like. These
mistakes have little significance to the argument; they are
simply embarrassments. These are mistakes that we expect
the author would agree about and correct if given the chance.
All of these errors deserve to be pointed out, but we
want to recognize that most of them do not challenge the
validity or value of the argument that has been made. Most
errors of presentation are just that. They detract from
the polish or the completeness of the work, but do not diminish
its plausibility or merit.
- Significant but correctable errors. More serious
flaws appear when the author makes a mistake of logic, evidence,
or omission that weaken the the main argument or theory, or
even, in the worst cases, prevent it from working or making
sense. The question facing the critic is this: is it
possible to correct the error without changing the identity of
the theory or argument? The first challenge for the critic
is to show that the author's argument is seriously hampered by
this error: its predictions are unreliable, its claims
contradict knowable facts, it contains internal contradictions
or ambiguities that prohibit rigorous application, or the
like. This is often a difficult task (after all, published
works commonly reflect long and hard work on the arguments, rely
on considerable authoritative knowledge, and have survived
examination by other authoritative reviews). But the
harder task - which should be compulsory at least to attempt but
commonly is not - is to suggest a repair to the original
argument that would overcome the shortcoming. Showing how to
repair a significant error is a scholarly contribution beyond
the common goals of a review, and should be highlighted as such.
- Fatal flaws. A critical flaw in a theory,
argument, or claim is an error that cannot be corrected
without changing the nature or identity of the original
argument. If the original argument relies on an empirical
claim that is wrong, or if the inherent logic of the argument is
internally inconsistent or otherwise unworkable, then it is
fatally flawed. Only a fundamentally different theory,
argument, or claim has a chance of validity. Critics often
seek to unearth fatal flaws but rarely succeed. Critics
often present correctable errors as fatal flaws, sometimes
because the critic cannot see the strategy for fixing the error
any better than the original author; sometimes just because the
critic hopes for the status gain associated with finding a fatal
flaw. We always want to avoid making a serious error while
we are arguing that someone else made a serious error.
- If we systematically consider at what level each flaw lies and
adopt a fitting response, our assessment becomes more accurate
and more accessible. It is surprisingly easy to confuse
ourselves, and once we do that we are certain to confuse
others.