A FIELD EVALUATION OF NATURAL LANGUAGE FOR DATA RETRIEVAL
Turner, Jon A.
Stohr, Edward A.
White, Norman H.
|Publisher:||Stern School of Business, New York University|
|Abstract:||Although a large number of natural language database interfaces have been developed, there have been few empirical studies of their practical usefulness. This paper presents the design and results of a field evaluation of a natural language system - NLS - used for data retrieval . A balanced, multifactorial design comparing NLS with a reference retrieval language, SQL, is described. The data are analyzed on two levels: work task (n=87) and query (n=1081). SQL performed better than NLS on a variety of measures, but NLS required less effort to use. Subjects performed much poorer than expected based on the results of laboratory studies. This finding is attributed to the complexity of the field setting and to optimism in grading laboratory experiments. The methodology developed for studying computer languages in real work settings was successful in consistently measuring differences in treatments over a variety of conditions.|
|Appears in Collections:||IOMS: Information Systems Working Papers|
Items in FDA are protected by copyright, with all rights reserved, unless otherwise indicated.