QUANTIFYING THE VALUE OF MODELS AND DATA: A COMPARISON OF THE PERFORMANCE OF REGRESSION AND NEURAL NETS WHEN DATA QUALITY VARIES

Abstract

Under circumstances where data quality may vary, knowledge about the potential performance of alternate predictive models can enable a decision maker to design an information system whose value is optimized in two ways. The decision maker can select a model which is least sensitive to predictive degradation in the range of observed data quality variation. And, once the "right" model has been selected, the decision maker can select the appropriate level of data quality in view of the costs of acquiring it. This paper examines a real-world example from the field of finance -- prepayments in mortgage-backed securities (MBS) portfolio management -- to illustrate a methodology that enables such evaluations to be made for two modeling alternative: regression analysis and neural network analysis. The methodology indicates that with "perfect data," the neural network approach outperforms regression in terms of predictive accuracy and utility in a prepayment risk management forecasting system (RMFS). Further, the performance of the neural network model is more robust under conditions of data quality degradation.