9512 | Assessment |
Abstract | No abstract in this draft, but it should be easy to extract from the existing research question section. |
Research Question | The general research aims seem pretty clear, although the
specific statements are sometimes awkward or ambiguous. For example, the most focused statement of
the research question suggests that the project will look at the relationship
between changing neighborhood composition and student achievement - this was
probably meant to be changes in student achievement. All this means is that it will benefit from
further editing for clarity and typographical errors. Note that "math" was left out of the sentence giving the final, formal statement of the research question. |
Literature Review | The literature review is in good shape, well organized with good coverage. It also will benefit from a bit of close editing (particularly to ensure that the causes and effects being discussed are always clear), and the very first sentence should be omitted. |
Data | The data seem okay for the project. Assuming the data are available, the
research might benefit from using the 2011-12 academic year as well. With this data added, then the same cohort
that were in the 3rd grade in one round are in the 8th grade in the next
round. Then, any compositional changes
within the cohorts would be evident (while comparisons of different cohorts
in different periods combine changes over time within cohorts and differences
across cohorts). (For example,
economic cycles could alter the proportion of a cohort who qualify for free
lunch.) This is not essential, rather
an opportunity to consider. Moreover,
using more than two points in time provides significantly more leverage for
arguing that changes in dependent variables are due to independent variables. It might be worth some effort to see if anyone has done a useful mapping between census boundaries and school district boundaries or neighborhoods, for previous research, that the project could simply adopt for attaching census data to the school neighborhoods. Otherwise this can be time consuming. It is not clear from the presentation if the data on student performance are available at the individual level (which potentially would let you assess changes in the performance of students relative to their neighborhood conditions). If individual level data are available, it is not clear why the research design would focus the research on aggregates of students performance. Regardless of the answer to the previous point, it might help to think through what ideal data would look like to solve the project goals. This might involve: individual level data on students' performance and their families' circumstances followed over time, plus aggregated data on schools and neighborhoods over time. Such data would then allow for an analysis that compares students' performances over time, looking simultaneously at the influence of parents income, neighborhood economic status, quality of students in relevant school, and other characteristics of relevant schools. This would allow quality of neighborhoods, schools, and family circumstances all to vary, including variation independent of the family (neighborhood change) and due to family "choice" (relocating to a new neighborhood). Thinking through how such "ideal" data would permit isolating the competing causes, showing how each of the shortcomings of the data actually available reduce what the research can hope to discover. |
Causal Interpretations | The basic causal idea is clear, but the mechanisms could use
further development, particularly because this is decisive for the research
design. If we look at possible
mechanisms - for example, simple compositional changes in the student body
(more or fewer students from high performance families), derivative changes
in the neighborhood (e.g., exposure to violence or books), or derivative
changes in the school atmosphere (e.g., shifts in the status implications of school
performance or associating with alienated cliques) - we have different things
to look for in the data. Also, relative to the underlying motivation for this research, consider the reality that the school neighborhoods in NYC are to some degree a zero sum game (they are not to the degree that NYC experiences changes through population exchanges with other areas, particularly surrounding ones, or through changes in the NYC economic conditions). So, to the degree we are shifting families' locations among a fixed set of neighborhoods, a rise in the average income in some implies a decline in others. This means that the economic diversity argument somewhat relies on the assumption that returns to diversity decline at some point of economic improvement. Or, to put it differently, the difference in educational outcomes between poor and moderate income neighborhoods is considerably greater than the difference between moderate and high income neighborhoods. If this is not true, then high income people have a strong interest in sustaining neighborhoods of concentrated affluence. And the diversity argument becomes one of advocating for the egalitarianism of mediocrity. Those of us favoring egalitarian policies may find this unappealing to consider, but we would be analytically incompetent not to recognize the issues. Also, what are the implications of people's capacities to exercise choice over the schools their children attend in New York City? Presumably this is less of an issue (for research) at lower grades, but this possible issue should be addressed if only to show that it is not a problem. That is to say, how much does school choice potentially attenuate the effects of residential realignments on changes in the student composition of schools? Is the research design able to distinguish the changes in neighborhood residential composition from changes in student body composition if there is potential variation in this relationship? And if this variation is possible, what does it imply about the causal expectations? |
Research Value | The effort to explain the potential value of the project would
be much stronger if it started by stressing what it can add to the existing
scholarly literature. However much the
information is relevant to policy, it is only a valuable contribution because
it discovers things not already known in the scholarship. |
Timetable | Probably the major concerns with the projected timetable involve how difficult it will be to assemble the data into a form that is ready to analyze. The data analysis as well may require more time than anticipated. |
Citations & Biblio | The reference structure seems fine, except that some of the bibliographic entries appear incomplete. |
Quality of writing | The writing is fine. A thorough editing would be beneficial. |
Priorities for Revising for Final Draft | The priorities would seem to include clarifying the causal analysis, refining the research question, and clarifying what exists in the data and how it will be used, all in relationship to each other. |
Miscellaneous Notes |