Limits of predictability in top-N recommendation

Top-N recommendation systems aim to recommend a small group of N items to users from many products, and the accuracy of the system is a commonly used metric to evaluate its performance. Existing methods can only obtain the predictability of Top-1 recommendations. This study aims to evaluate the highest accuracy, or predictability, of Top-N recommendations. To extend the predictability to a broader range of scenarios, we first investigated the correlations among N most likely actions and described the distribution of user behavior using information theory. Subsequently, we estimated the predictability of Top-N recommendations using the Fano inequality. Experimental results demonstrate that our method not only quantifies the predictability of N targets but also yields predictability that is closer to the true values compared to current methods, reducing the evaluation error by a factor of 5 in Top-1. Combining several real-world datasets, existing recommendation methods can approach 70% predictability at Top-10. Moreover, users exhibit a more pronounced bias towards Top-N items offline compared to online shopping.