Stern School of Business >
IOMS: Information Systems Working Papers >
Please use this identifier to cite or link to this item:
|Title: ||A Bootstrap Evaluation of the Effect of Data Splitting on Financial Time Series|
|Authors: ||LeBaron, Blake|
Weigend, Andreas S.
|Keywords: ||Model evaluation|
Time series prediction
Linear bias of early stopping
Superposition of forecasts
|Issue Date: ||1997|
|Publisher: ||Stern School of Business, New York University|
|Series/Report no.: ||IS-97-013|
|Abstract: ||This article exposes problems of the commonly used technique of splitting the available data into training, validation, and test sets that are held fixed, warns about drawing too strong conclusions from such static splits, and shows potential pitfalls of ignoring variability across splits. Using a bootstrap or resampling method, we compare the uncertainty in the solution stemming from the data splitting with neural network specific uncertainties (parameter initialization, choice of number of hidden units, etc.). We present two results on data from the New York Stock Exchange. First, the variation due to different resamplings is significantly larger than the variation due to different network conditions. This result implies that it is important to not over-interpret a model (or an ensemble of models) estimated on one specific split of the data. Second, on each split, the neural network solution with early stopping is very close to a linear model; no significant nonlinearities are extracted.|
|Appears in Collections:||IOMS: Information Systems Working Papers|
Items in Faculty Digital Archive are protected by copyright, with all rights reserved, unless otherwise indicated.