KNOWLEDGE DISCOVERY FROM DATABASES: THE NYU PROJECT

Abstract

More and more application domains, from financial market analysis to weather prediction, from monitoring supermarket purchases to monitoring satellite images, are becomingly increasingly data-intensive. The result is massive databases that are growing at a rapid rate - it has been estimated that the world’s electronic data almost doubles every year. With this rate of data explosion, there is a pressing need for computers to play an increasing role in analyzing these huge data repositories which are impossible to penetrate manually. The challenge is to ferret out the regularities in the data that will prove to be interesting to the user. A group in the Information Systems department at the NYU Business School has been working in this area for a number of years. The focus of our project is now on the discovery of patterns from time series data. In this paper we give an overview of the kinds of databases we are "mining” and the kinds of temporal patterns and rules which we are attempting to discover. In the first phase of this research, we have developed a taxonomy of patterns as a way to organize our research agenda. We wish to share the taxonomy with the research community in the "knowledge discovery in databases" area since we have found it useful in classifying the universe of regularities or patterns into distinct types, that is, patterns which differ in terms of their structure and the amount 6f search effort required to find them. Although the primary focus of our project is on time series data, and the examples we will present are chosen from this arena, the taxonomy is general enough to apply to any type of data.