This Article Full Text (OnlineFirst PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Google Scholar Articles by Wang, Y. D. Articles by Forgionne, G. Social Bookmarking                   What's this?

Testing a decision-theoretic approach to the evaluation of information retrieval systems

¹ Department of Applied Information Technology, George Mason University, USA
² Information Systems Department, University of Maryland Baltimore County, USA

^* To whom correspondence should be addressed.

	Abstract

With information overload a real problem, especially on the Internet, there has been much interest in developing effective and efficient information retrieval (IR) systems. The various information retrieval approaches will require accurate evaluation to justify the requisite substantial development and implementation investment. Recently, a comprehensive and integrated evaluation model has been proposed and illustrated. By analyzing the evaluation measures using the analytic hierarchy process (AHP), the model transforms IR evaluation into a multi-criteria decision making (MCDM) problem, which assesses both the IR outcome and the interactive IR process. This paper extends that research by refining the evaluation model and by testing the research question through mathematical testing and simulation. The tests confirm the need to include both process and outcome criteria in any IR evaluation and prove the superiority of the proposed decision-theoretic approach over the traditional evaluation methodologies that focus on the IR outcome alone.

Key Words: information retrieval system; evaluation; information search process; decision making; multi-criteria model; analytic hierarchy process

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?