|
Archive@NYU >
Stern School of Business >
CeDER Published Papers >
Please use this identifier to cite or link to this item:
http://hdl.handle.net/2451/27752
|
| Title: | Discover Interesting Patterns for Investment Decision Making with GLOWER
- A Genetic Learner Overlaid With Entropy Reduction |
| Authors: | Dhar, Vasant
Chou, Dashin
Provost, Foster |
| Keywords: | data mining
knowledge discovery
machine learning
financial prediction
genetic algorithms
rule learning
investment decision making
systematic trading |
| Issue Date: | 7-Nov-2008 |
| Series/Report no.: | CeDER-PP-2000-01 |
| Abstract: | Prediction in financial domains is notoriously difficult for a number of
reasons. First, theories tend to be weak or non-existent, which makes
problem formulation open ended by forcing us to consider a large number
of independent variables and thereby increasing the dimensionality of
the search space. Second, the weak relationships among variables tend to
be nonlinear, and may hold only in limited areas of the search space.
Third, in financial practice, where analysts conduct extensive manual
analysis of historically well performing indicators, a key is to find
the hidden interactions among variables that perform well in
combination. Unfortunately, these are exactly the patterns that the
greedy search biases incorporated by many standard rule learning
algorithms will miss. In this paper, we describe and evaluate several
variations of a new genetic learning algorithm (GLOWER) on a variety of
data sets. The design of GLOWER has been motivated by financial
prediction problems, but incorporates successful ideas from tree
induction and rule learning. We examine the performance of several
GLOWER variants on two UCI data sets as well as on a standard financial
prediction problem (S&P500 stock returns), using the results to
identify one of the better variants for further comparisons. We
introduce a new (to KDD) financial prediction problem (predicting
positive and negative earnings surprises), and experiment with GLOWER,
contrasting it with tree- and ruleinduction approaches. Our results are
encouraging, showing that GLOWER has the ability to uncover effective
patterns for difficult problems that have weak structure and significant nonlinearities. |
| URI: | http://hdl.handle.net/2451/27752 |
| Appears in Collections: | CeDER Published Papers
|
All items in Faculty Digital Archive are protected by copyright, with all rights reserved.
|
