7024 | Assessment Initial Draft | Final Draft comments |
Abstract | Okay. | Formally lacks an
abstract, but we will call the introduction the abstract substitute. Given that, it is clear and effective. (Note that the last two sentences should be removed unless this were a grant proposal submitted to an advocacy organization. Otherwise, a research proposal should stress the value of the research to improving knowledge, not that it could become a political tool. Designing research to meet political ends is considered a violation of academic standards.) |
Research Question | The general research interest is clear, but the specific research questions not so much. Try to create a simple, single sentence statement of the central research problem. | The new research question, like that in the original draft, is clear enough in a general sense, but a bit elusive in the specifics. Possibly this is because the concept identity is at the core of the project, but what it means is never entirely clear. The intended causal conditions concern the degree to which the nation or religion are central to an individual's identity, but what this means - beyond the instrumentation that people answer something is important to their identity - remains unclear. That people can answer a question does not mean that the questions and answers have consistent and clearly interpretable meaning. |
Literature Review | This is a good
literature review, well organized, broad in coverage, clearly linked to the
proposed research agenda. One relevant question that seems overlooked by the literature review concerns the relationship between attitudes and actions. It could usefully review some work that considers how much attitudes towards immigrants (as measured by surveys) reflect peoples' real-world behavior towards immigrants. Although this is not the subject of the proposed research, most readers will infer that the value of the research depends on the assumption that attitudes at least mirror people's behavior (or, more strongly, influence or guide that behavior). Addressing the legitimacy of this assumption could help give weight to the value of this project. We should note that whether or not the dominant national values perspective is analytically sensible is a rather contentious issue among scholars. Still, a good number of scholars and publications do consider it worthwhile, so it is a legitimate departure point for the proposal. |
The new literature
review shifts attention to the issues relevant to the refocused goals of the
research, but it seems to have a hard time linking the research ideas and
findings to the project goals. As a
result, we are introduced to a good range of relevant prior work, but we are
not shown how different parts of this work relate to each other, or, more
important, how the past work relates to and guides the research design. The effort to distinguish values from attitudes seems an appropriate goal, but the effort here is not successful. What is needed is a clear and concise claim about what distinguishes values and attitudes (at least for the purpose of this project), which the remainder of this section could then elaborate and show its implications for this study. As written, this section is confusing. The "Attitudes Toward Immigrants" section considers a reasonable breadth of materials, but does not do so in a way that clearly harnesses them to achieve a goal. That some variations in religious membership or adherence seem related to variations in attitudes toward immigrants is certainly relevant to the project, but the presentation here does not indicate any overall pattern or logic deemed important. It is not apparent why the comments on cosmopolitanism are in the same paragraph, and the final paragraph covers varied relationships without suggesting what matters to this project. The "Social Identities and Attitudes Toward Immigrants" section again discusses some relevant materials, but does connect the previous work to the project goals in a clear way. |
Data & Analysis | Overall, the data
seem appropriate and practical for the proposed project. It is not apparent why the proposal suggests it will assign each nation an "appropriate number" based on the Inglehart-Welzel cultural map. A more obvious approach would be to use the two principal I-W scales and an interaction variable based on their product. If there is some reason for representing the two scales as one unified scale, the proposal should make it clear. It is not obvious what the draft proposal means by saying the research will "conduct a stratified random sample of respondents" from these data. From what else is stated, part of the idea is to select a subset of the nations including in the data. It is not self-evident why. The proposal also suggests that the research will select only citizens, but this seems inconsistent with the section that offers hypotheses about differences in attitudes between citizens and non-citizens. Aside from the possibility that the size of the dataset is awkward, no reasons are offered for not including all the respondents from those nations that are used in the study. If there is a good reason, the proposal should clarify and defend it. For national economic indicators, the research design could consider some that could offer more signs of economic stress, beyond the GDP, such as national unemployment or changes in GDP (to capture recessions). The draft proposal suggests that the national values scores of the World Values Survey do not represent a simple aggregate of individual respondents’ recorded values. The proposal should indicate how they calculate the scores in the final draft. |
The data appear generally appropriate but it is not entirely clear how well they can work for the intended project. The principal dependent variable is identified as whether respondents choose "illegal immigration" as one of the two most important issues facing the country (from a list of 12 issues - not indicated in the proposal). This has two apparent potential problems. First, although we might expect that the (frankly, ill-conceived) wording "illegal immigration" would mainly raise the attention of those opposed to immigration, we do not know this. The wording of the question is such that people who consider immigration issues politically critical, regardless of their position on those issues, have only this item to choose to indicate their concerns - so we cannot actually know if a respondent indicating this item is important is expressing a negative orientation toward "illegal" immigrants. Moreover, this item gives no idea what a respondent thinks about "legal" immigrants, or what the respondent understands as the distinction between "legal" and "illegal" immigrants. Second, we do not know how respondents who do not choose this item as one of the two most important issues feel about it unless they choose it as one of the "least" important issues (an indicator ignored in the proposal). So, if a large proportion of the sample do not select immigration as one of either the most or the least important issues, the size of the sample who have any expressed opinion may be small. Presumably, you can place this residual part of the sample in some vaguely middling position relative to the two others to form a rough three level ordered scale, but the proposal shows no recognition of this. |
Causal Interpretations | This project involves
two levels of causal processes affecting individuals, one being the
characteristics that vary by individual biographies and circumstances, the
other being the national conditions that have a presumed common effect on all
in a country. The principal underlying
research design is not clearly and directly stated. It appears to be to discover if national
conditions have effects that cause the overall distribution of attitudes
within countries (caused by individual level varying causes) to look
different across countries, presumably in the central moments
(averages). In essence, this
explanatory goal is built on two assumptions.
The first is that aggregate differences across countries are not due
to compositional differences, they do not result simply from differing
distributions of individual characteristics (such as income or religion or
experience of historical hardships like war).
The second is that the aggregated differences can be explained by some
national characteristics that essentially distinguish the experience of all
people in one nation from those in another.
These assumptions are both difficult to demonstrate, but they should
be kept clearly in mind as it is easy to lose tract of them behind
explanatory rhetoric. Past research suggests that attitudes towards immigrants vary for different kinds of immigrants (e.g., high skilled vs. low skilled, or the racial or ethnic identity associated with immigrants). This implies a potential issue with unmeasured variations in context. For example, if people in countries receiving mainly high-skilled immigrants think about high-skilled immigrants when they answer the survey question, their responses mean something different than respondents who are thinking about low-skilled immigrants when they answer the survey question. More generally, the extensive variation in immigration experiences across nations - including the magnitude of immigration as well as the characteristics of immigrants - unavoidably implies that the cultural references available to respondents describing their attitudes on immigrants will also vary widely. (This could, however be seen as an opportunity, as one could seek to incorporate data from other sources on the characteristics of immigration patterns.) Be wary of trying to explain one individual level attitude (such as feelings about immigration) with other individual attitudes (such as trust of others). When we step back to think about it, we can see that it is difficult to propose a mechanism by which one attitude can cause another attitude. Attitudes do not do things to other attitudes in our cognitive processing. When people talk about some attitudes causing others they are commonly weaving imaginative tales based on the apparent moral and logical relationships between attitudes. It could be interesting to look at what can be found using indicators of national social conditions in contrast to the national values. For example, the project could use one of the established indices of national gender inequality. There are also indices of national conditions like democracy. There is no need to go down this road, it just might be interesting. |
The discussion of
causal possibilities shows some promise, but remains scattered and a bit
muddled. The stress on attitudes
causing attitudes is open to considerable criticism, but by itself this puts
the project in the same circumstances as many published works doing the same,
so in that sense it is an acceptable strategy. Still, it would be a good idea to put more
effort into considering what explains the distributions of all the relevant
attitudes. Although there is now passing mention of possible reciprocal causation with respect to some work discussed in the literature review, this idea is not well developed with respect to the research agenda. For example, realistically, we could plausibly expect that people strongly opposed to the number of immigrants now in the U.S. or anticipated in the future, regardless of the conditions causing that attitude, could as a result become people conceiving themselves as believing that their national identity is important to them. How? People troubled by immigrants need to justify their position, and they have readily available ideological presentations that associate such things as love of country and patriotism with opposition to immigrants. These ideas offer people who do not like immigrants a kind of positive identity association as a means to explain and justify xenophobia. Rather than having to see themselves as self-serving bigots, they get to be patriots. This possibility is a problem that the research needs to face and the proposal should acknowledge. |
Research Contribution | To strengthen the statement of the research value, try to be more specific about the ways that the proposed project will provide knowledge absent from past research. This does not need to be a list of many possibilities, but at least one way that is sufficient to justify the research effort. | While the proposal does not address this directly (and it should), the substance of an answer is scattered about. |
Citations & Bibliography | The reference structure is fine. It appears some citations may not appear in the bibliography. | Well done. |
Quality of Writing & Organization | Very good. | Well written. |
Priorities for Revising / Responsiveness to Feedback | Overall, this proposal is developing well. For the next iteration, you might stress clarifying exactly what data will be used, how the national and individual level characteristics will be analyzed together, and what are the principal causal assumptions and expectations at stake. | Fully rewritten. Because of the significant changes between drafts, much of the earlier feedback was not directly applicable to the final proposal. |
Miscellaneous Notes | The draft proposal states that if the research does not find significant regression coefficients, it will mean that “in reality, the variables have a negligible effect on one another.” A small point, but remember that operationalized variables do not really exist “in reality” — they are approximations of concepts. | |
Proposal | Fairly well done. The general goals of the project seem sensible and the work that has gone into developing it stands out. The proposal still, however, has to contend with the meaning of "identity" in this context and the problem of causal directions when looking at attitudes. | |
Class Overall | Sound work through the class and a good pattern of development. | |