Probabilistic Neighborhood Selection in Collaborative Filtering Systems

Abstract

This paper presents a novel probabilistic method for recommending items in the neighborhood-based collaborative filtering framework. For the probabilistic neighborhood selection phase, we use an efficient method for weighted sampling of k neighbors without replacement that also takes into consideration the similarity levels between the target user and the candidate neighbors. We conduct an empirical study showing that the proposed method alleviates the over-specialization and concentration biases in common recommender systems by generating recommendation lists that are very different from the classical collaborative filtering approach and also increasing the aggregate diversity and mobility of recommendations. We also demonstrate that the proposed method outperforms both the previously proposed user based k-nearest neighbors and k-furthest neighbors collaborative filtering approaches in terms of item prediction accuracy and utility based ranking measures across various experimental settings. This accuracy performance improvement is in accordance with ensemble learning theory.