2974	Assessment Initial Draft	Final Draft comments
Abstract	Okay	Essentially unchanged.
Research Question	The general sense of the research goals are fairly clear, to assess whether emergency care treatment outcomes for the insured and uninsured have changed since the passage of Affordable Care Act, depending on state adoption of alternative responses.  How this would be translated into a more specific research question is unclear.	Although a paragraph has been added to acknowledge the issue with the uninsured seeking different emergency room services (feedback to the draft proposal), the research question remains ambiguous.  While the primary research design concerns emergency room outcomes, the presentation of the research question and the motivating concerns repeatedly conflate overall health outcomes, overall medical treatment, and emergency room outcomes.
Literature Review	The literature review shows promise, but seems a bit eccentric at this point.  While there are many reasons to have interest in research in other nations, it seems doubtful that much is to be gained from studies in Ghana and India that have very different insurance systems, medical systems, cultures, and economic conditions.  This is not my area, but I suspect there exists considerable research on insurance and medical treatment in the U.S.	The literature review has been expanded a bit, but remains rather skimpy and arbitrarily selective. The statement "those with no insurance coverage or Medicare and Medicaid insurances may appear to have better treatment outcomes due to the higher rate of visits to the emergency department with conditions that are treatable by primary care (Pukurdpol, Wiler, Hsia, & Ginde, 2014)" contradicts the actual finding in the source that Medicare patients have a lower rate of visits to emergency departments with conditions treatable by primary care than those with private insurance.  Reporting the findings of  previous research incorrectly is not acceptable.  The overall tone of the literature review also seems peculiarly at odds with what is probably the most relevant review to which it refers, the Sommers, Gawande, & Baicker (590-1): "One question experts are commonly asked is how the ACA — or its repeal — will affect health and mortality. The body of evidence summarized here indicates that coverage expansions significantly increase patients’ access to care and use of preventive care, primary care, chronic illness treatment, medications, and surgery. These increases appear to produce significant, multifaceted, and nuanced benefits to health."  One is free to dispute one's sources, but simply ignoring their arguments is not a winning strategy.
Data & Analysis	The HCUP data indicated in the proposal appear to be excellent data on hospital patients.  However, according to the HCUP web site, it seems that state identifying information has been withheld from the data beginning in 2012, which makes it difficult to see how this data will be used for an analysis for select states.  Is this a mistaken understanding? The draft proposal provides little information on the dataset (including its sampling strategy) or the variables planned to use in this research project.  The final draft should include this information,  explaining and justifying the choices of independent, dependent, and control variables.  In addition, since the proposal suggests a plan to use relatively advanced statistical techniques (instrumental variables and difference-in-differences), it should describe how these techniques are an improvement on traditional OLS results, and be more specific about the models that will be used. It is entirely legitimate to "borrow" a research design from a published work to apply with adaptations to different data.  In this case, it appears that all aspects of the intended research procedure are borrowed directly from the Sommers, et al. "Three-Year Impacts Of The Affordable Care Act ..." article.  To do this, however, we need to show that we understand all aspects of the research methodology and that it is applicable to our data and research problem.  Here that would mean showing an understanding of instrumental variables strategies and difference-in-differences models.  The draft proposal does not accomplish this.	The final proposal acknowledges that the restricted data are required to have the state identifiers, abandons the three state design of Sommers et. al., and removes the instrumental variables strategies and difference-in-differences models as suggested.  Unfortunately, beyond incorporating direct suggestions, the analysis does not display progress. How the research will use the data remains somewhat mysterious.  It states, "Individual states will be the unit of analysis in the study, but individual level data for each patient will be used for the analysis."  The proposal gives no indication how individual data will be used for an analysis based on states.  The HCUP data are on individual cases.  Without any indication how this individual case data will be aggregated to the state level and used to pursue the stated problem, this does not constitute a research design. The proposal seems to overlook that the data shows the diagnosis categories, which would appear to be the primary means for overcoming the differential treatment seeking of the uninsured.  The proposal also neglects the selection effects of the broadened insurance coverage via the ACA - even if the states fully using the ACA were otherwise the same as those that did not, the additional insured in the ACA using states would be comparable to the uninsured in the ACA non-using states, not the insured.  This means that it is not only the proportion insured that differs, but the kind of people who are insured and not insured.  This in addition to the realistic expectation that the states using ACA differ significantly from those that did not.
Causal Interpretations	The draft proposal does not really provide causal interpretations.  It focuses on the purely empirical questions of the quality of health outcomes under different conditions.  This is not, of itself, wrong, but it does leave the research somewhat unmoored.  The proposal also seems to lose track of the meaning of its own plan to study the outcomes for emergency room arrivals.  Much of medical care in the U.S. does not involve emergency rooms, which are the common destination of the unexpected severe medical condition of the insured and wide ranging medical needs of the uninsured.  A difference-in-differences strategy may make some sense for assessing changes in the profiles and outcomes for the insured and the uninsured who come to emergency rooms.  It seems considerably less likely to allow effective assessment of the value that insurance has for medical care, which is implied by the focus on the passage of the ACA. Passage of the ACA meant many more people were insured, which also means that use of the emergency room has changed.  In short, which people arrive at emergency rooms with insurance or arrive without insurance would both be expected to change as a result of the ACA.  This creates a potential issue with selection bias in the comparisons.  This seems unrecognized in the proposal.	The proposal still largely ignores causal analysis.  It sticks to the simple empirical question, do emergency rooms treat uninsured people differently than the insured.  Likely because it lacks a causal foundation, it appears to ignore implications of the basic causal concern that the uninsured and insured have different outcomes because the people differ (e.g., biographically, economically, support networks, health, or medical history). The proposal shows no recognition in the research design that positive effects of greater insurance coverage could result in the appearance of less positive outcomes in emergency rooms.  If the medical conditions at arrival in emergency rooms are more dire on the average (because newly insured people get less problematic issues resolved elsewhere), then the outcomes are likely to be worse.  One might think that this could be resolved by looking at different medical issues separately, but this appears doubtful.  Instead, it is possible to see circumstances that could result in either better or worse ER outcomes, even if insurance effects are overall positive.  Take, for example, heart issues.  It could be that the ER outcomes are improved because some of those who would have arrived in bad condition receive treatment for those bad conditions without using the ER due to new insurance coverage (for example, they are directly admitted to the hospital for surgery).  Or, it could be that some who would have quietly died at home end up being sent to the ER because of better care through new insurance leading to more unfortunate ER outcomes.  Which is to say that there seems to be an inherent potential selection bias that could happen in either direction that probably cannot be handled with the data. In short, there does not seem to be any obvious prediction about ER outcomes if greater insurance improves health care.  Which leaves the aims of the project obscure.  This is an example why one needs to consider causal alternatives even if the research aim is mainly descriptive.
Research Contribution	The claim that it would be valuable knowing how having or not having insurance affects the health care people receive seems unassailable.  However, it is not apparent that this project can provide such information and it is also not apparent from the draft proposal what knowledge already exists about such questions.	No changes.
Citations & Bibliography	The citations and bibliography are formally okay.	Okay.
Quality of Writing & Organization	The writing is clear and sparse.	Clearly written, but ultimately frugal to the point of insufficiency.  The proposal will improve by developing various parts more deeply and carefully.
Priorities for Revising / Responsiveness to Feedback	While it is possible to see the foundations for a research project in the draft proposal, it appears to need considerable work.  The specific research question, the literature review, the data to be used, the causal models in the background, and the research design all seem underdeveloped here.	Response to feedback is uneven at best.  The text of the final proposal amounts to two and one-half pages.  While it is good to be concise, one must also be adequate and effective.
Miscellaneous Notes
Proposal		The proposal concerns a worthy topic, but it still needs work on both the conception and design to become practical.
Class Overall		Progress has been uneven, but moments of insight suggest an unmet potential if enough effort is invested.