1210 | Assessment Initial Draft | Final Draft comments |
Abstract | The abstract is clear and provides a good sense of the project. | Good (unchanged). |
Research Question | The research question is initially presented as three related questions. Of those, the second and third are clear and consistent with the rest of the proposal (although the third could be better phrased), but the first is posed as a historical question that appears to be irrelevant to the research design. | Clearly stated (as revised). |
Literature Review | The paper has a sound initial literature review. To develop it further, try organizing more around the ideas rather than the authors, and in that context broaden the range of materials cited to provide a more effective portrait of the relevant scholarship. In particular, see what more research exists on the causes and consequences of student debt. | The literature review
is good. The additions helped,
although it still neglects the causes of student debt. The causes of debt are particularly
relevant for those who have a choice between more prestigious, more expensive
colleges and less prestigious, less expensive - for them the effects of their
decisions (in this project's focus on the debt to income ratio) depend on the
extent to which the more expensive school repays with higher income jobs.
Note that efforts to compare the value of college for people of varying origins suffer from several difficult limitations. First, such comparisons must cope with selection effects, as the people from disadvantaged backgrounds who attend college will differ on the average from those who do not in terms of their aspirations, parental support, and early scholastic achievement (so the outcome differences are not simply college effects). Second, if we try to measure the value of education by social mobility, then those with affluent beginnings have less room to climb. Researchers can cope with these and other issues with good data and proper techniques, but they limit the substantive validity of our comparisons even under the best circumstances. The fundamental class difference is one that does not require research: effective use of educational opportunities is a clear avenue by which those from disadvantaged origins may achieve a good position, while educational failure is a condition that can limit the success of those from advantaged origins. Or, to put it differently, what it takes in initiative and effort for the disadvantaged to achieve a high-status occupation through education is considerably greater than what is required of the advantaged. |
Data & Analysis | The data seem reasonable for the project, but the limitations of information available in the data, as mentioned in this draft, is somewhat worrying. It is not obvious that this data will allow sufficient specification of the causal process and consideration of important possible influences to achieve resolution of the research question as intended. If the data do not offer sufficient information, we will have no way of knowing if college debt is functioning as a causal influence, or if it is one more symptom (or effect) of the same causal conditions that guide the variation in post-graduate incomes. | Data and data discussion unchanged. The data are reasonable, but suffer some limitations that make their suitability for answering the key questions uncertain. Without a means to assess the motives behind college debt choices or the choices among colleges with varying costs, and having information on employment extending only four years after graduation, the possibilities for assessing the effects of college debt on employment outcomes seem problematically curtailed. The proposal would become significantly stronger by facing these issues head on. |
Causal Interpretations | This seems to be a
fairly clear causally oriented analysis.
It is seeking the effects
of college debt. This implies that we
should see a fairly well-specified causal model, one that identifies what
circumstances and influences need examination. Such a model presumably would consider the
determinants of college attendance, the determinants of borrowing for
college, and the possible causal pathways through which debt may affect
future income. Where is this
model? One possible conundrum -- Let's ask what people experience as the alternative to taking on debt. Is it going to a cheaper school, foregoing college, making alternative financial arrangements (such as holding a job), or something else? This matters, because, as the proposal begins to recognize in the brief reference to counterfactuals in the conclusion, assessing the impact of debt depends on comparing debtors to appropriate alternatives. We can try to control for family income and a few other things, but, in a way, this makes the problem more evident. Once we control for influences like income, we can think of ourselves as comparing people with college debt with others who have similar family income (and other characteristics) who did not acquire college debt. At this point we are faced with the critical problem: what causal conditions distinguish those who borrow from those who do not, and are those conditions irrelevant to subsequent earnings. Or, to put it differently, can we assume that after our controls, borrowing is a random event? Also, the proposal suggests that graduates with higher levels of debt have greater need for earnings to make loan payments, which would seem to imply that student loan indebtedness should be positively correlated with future employment earnings after controls for the reasons students acquire debt (because students with a lot of debt are more motivated to get a job, especially a well-paying job, that will allow them to pay off their debt). If the dataset has detailed information about post-graduation occupations, it could allow testing whether it is true both that students with debt are more likely to enter occupations that offer a decent starting salary, and that they yet have less potential for salary growth over time. |
The relevant parts of
the proposal are largely unchanged, with the addition of an alternative
hypothesis that debt could motivate decisions that lead to higher
earnings. What is here is reasonable. The absence of a better specified causal
model is still cause for some concern, given the difficulties for
establishing the appropriate research design. Again, this is a study where careful consideration of counterfactuals appears particularly important for determining what comparisons are meaningful. One strategy that might help is to consider a simplified model of actors, conditions, and decisions. For example, you could define a model with high school graduates having either strong or mediocre credentials; coming from families with high, moderate, or low income; and facing, as appropriate, choices between prestigious expensive college, mediocre moderate cost college, and no college. With such a simplified model, we can try to work through the possibilities, considering what choices face each type of person (or family). Doing something like this can provide insight into what kinds of comparisons are ideal for identifying the effects that the project seeks, which can then be used to evaluate the comparisons that are possible with the data. |
Research Contribution | The discussion of the proposed research's potential value, should pay more attention to the anticipated scholarly contribution. | The proposal still does not describe what it contributes beyond past work. We can make some inferences about this by comparing the research design to the reviewed literature, but a proposal should make this clear, not an inferential test on readers. |
Citations & Bibliography | The reference structure seems okay (but note the Hout reference is not in the bibliography). | Reference structure is fine. |
Quality of Writing & Organization | The writing is fine. | Well written with proper professional air. |
Priorities for Revising / Responsiveness to Feedback | Probably figuring out the data availability issues in conjunction with developing the causal model(s) is most crucial to this proposal. | The revisions considered some feedback but also ignored some that it should have taken into account, particularly relating to the data and causal interpretation. |
Miscellaneous Notes | ||
Proposal | The proposal is fairly good, but should have been better. | |
Class Overall | Good work through the class, but it could have been better with a bit more effort. | |