Correct use of hypothesis testing and choosing appropriate comparison groups when estimating the impacts of location based policies: A response to Neumark and Young

Abstract

We are grateful to Neumark and Young (2017, hereafter NY hereafter) for spotting an error in Ham, Swenson, İmrohoroğlu, and Song’s (2011, hereafter HSIS) 1990 poverty rate data. Our corrected estimates reported here of the impacts on the Poverty Rate of Enterprise Zones (ENTZs), Enterprise Communities (ENTCs), and Empowerment Zones (EMPZs) are smaller than those in HSIS. However, they are still quite sizable and statistically significant. We show here that NY obtained similar results with the NCBD data to our new estimates reported here. The NCBD data uses a different approach than that used by HSIS to deal with the changing borders of some Census tracts over time. However, we find NY’s criticisms of the HSIS results for ENTZs, ENTCs and EMPZs to be deeply flawed, and suffer from several important errors. First, their criticisms arise from their making a fundamental error in hypothesis testing. Second, they use an incorrect approach for comparing parameter estimates from different studies. Third they use a comparison group for EMPZs where about half the comparison tracts are impacted by other labor market programs, leading to downward biased estimates of the EMPZ impacts. We argue that this bias is over 50 percent of their (under)estimated treatment effect for EMPZs.