Google scholar arxiv informatics ads IJAIS publications are indexed with Google Scholar, NASA ADS, Informatics et. al.

Call for Paper


July Edition 2021

International Journal of Applied Information Systems solicits high quality original research papers for the July 2021 Edition of the journal. The last date of research paper submission is June 15, 2021.

A Literature Review of Empirical Studies of Recommendation Systems

Nikhat Akhtar, Devendera Agarwal. Published in Information Sciences

International Journal of Applied Information Systems
Year of Publication: 2015
Publisher: Foundation of Computer Science (FCS), NY, USA
Authors: Nikhat Akhtar, Devendera Agarwal
Download full text
  1. Nikhat Akhtar and Devendera Agarwal. Article: A Literature Review of Empirical Studies of Recommendation Systems. International Journal of Applied Information Systems 10(2):6-14, December 2015. BibTeX

    	author = "Nikhat Akhtar and Devendera Agarwal",
    	title = "Article: A Literature Review of Empirical Studies of Recommendation Systems",
    	journal = "International Journal of Applied Information Systems",
    	year = 2015,
    	volume = 10,
    	number = 2,
    	pages = "6-14",
    	month = "December",
    	note = "Published by Foundation of Computer Science (FCS), NY, USA"


In the last twelve years, the number of web user increases, so intensely leading to intense advancement in web services which leads to enlargement the usage data at higher rates. The purpose of a recommender System is to generate meaningful recommendations to a collection of users for items or products that might interest them. Recommender systems differ in the way they analyze these data sources to develop notions of congeniality between users and items which can be used to identify well-matched pairs. The recommender system technology intentions to help users in finding items that match their personal interests. It has a successful usage in e-commerce applications to deal with problems related to information overload proficiently. In this paper, we will extensively present a survey of six existing recommendation system. The Collaborative Filtering systems analyze historical interactions alone, while Content-Based Filtering systems are based on profile attributes, Hybrid Techniques attempt to combine both of these designs, Demographic Based Recommender systems aim to categorize the user based on personal attributes and make recommendations based on demographic classes, while Knowledge-Based Recommendation attempts to suggest objects based on inferences about a user’s needs and preferences, and Utility-Based Recommender systems make recommendations based on the computation of the utility of each item for the user. In this paper, we have recognized 60 research papers on recommender systems, which were published between 1971 and 2014. Finally, few research papers had an influence on research paper recommender systems in practice. We also recognized a lack of authority and long term research interest in the field, 78% of the authors published no more than one paper on research paper recommender systems, and there was miniature cooperation among different co-author groups.


  1. Adomavicius G, Tuzhilin A (2005). Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE Trans. Knowl. Data Eng. 17(6):734-749.
  2. J. Beel, S. Langer, and M. Genzmehr, “Sponsored vs. Organic (Research Paper) Recommendations and the Impact of Labeling,” in Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013), 2013, pp. 395–399.
  3. J. Beel, S. Langer, M. Genzmehr, and A. Nürnberger, “Persistence in Recommender Systems: Giving the Same Recommendations to the Same Users Multiple Times,” in Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013), 2013, vol. 8092, pp. 390–394.
  4. Herlocker, J. K. (August 1999). An algorithmic framework forperforming collaborative filtering. Proceedings of the 22nd ACM Conference. Berkely,CA.
  5. S. Gottwald, “Recommender Systeme fuer den Einsatz in Bibliotheken / Survey on recommender systems,” Konrad-Zuse-Zentrum für Informationstechnik Berlin, ZIB-Report 11-30, 2011.
  6. O. Küçüktunç, K. Kaya, E. Saule, and U. V. Catalyürek, “Fast Recommendation on Bibliographic Networks with Sparse-Matrix Ordering and Partitioning,” Social Network Analysis and Mining, vol. 3, no. 4, pp. 1097–1111, 2013.
  7. Linden G, Smith B, York J (2003). Recommendations: Item-to-Item Collaborative Filtering. Published by the IEEE Computer Society, IEEE Internet Comput. 7(1):76-80.
  8. Das A, Datar M, Garg A, Rajaram S (2007). Google News Personalization Scalable Online Collaborative Filtering. In Proceedings of the 16th International Conference on World Wide Web, WWW ‟07, Banff, Alberta, CANADA, ACM, pp. 271-280.
  9. Kim, H. N., Ji, A. T., Ha, I., & Jo, J. S. (2010). Collaborative filtering based on collaborative tagging for enhancing the quality of recommendation. Electronic Commerce Research and Applications, 9, 73–83.
  10. Sarwar, B., Karypis, G., Konstan, J. A., & Riedl, J. (2000b). Analysis of recommendation algorithms for e-commerce. Proceedings of the ACM E-Comm., 158–167.
  11. Y. Shi, M. Larson, and A. Hanjalic, “Collaborative Filtering Beyond the User-Item Matrix: A Survey of the State of the Art and Future Challenges,” ACM Comput. Surv., vol. 47, no. 1, pp. 3:1–3:45, 2014
  12. Lopez-Nores, M., Garca-Duque, J., Frenandez-Vilas, R. P., & Bermejo-Munoz, J. (2008). A flexible semantic inference methodology to reason about user preference in knowledge-based recommender systems. Knowledge-Based systems, 21, 305–320
  13. P. Lops, M. Gemmis, and G. Semeraro, “Content-based recommender systems: State of the art and trends,” Recommender Systems Handbook, pp. 73–105, 2011.
  14. M. Ge, C. D. Battenfeld, and D. Jannach, “Beyond accuracy: evaluating recommender systems by coverage and serendipity,” in Proceedings of the fourth ACM conference on Recommender systems, 2010, pp. 257–260.
  15. D. Jannach, M. Zanker, A. Felfernig, and G. Friedrich. Recommender Systems – An Introduction. Cambridge, 2011.
  16. X. Su and T. M. Khoshgoftaar. A survey of collaborative filtering techniques. Adv. in Artif. Intell., 2009:4:2–4:2, January 2009.
  17. L. Palopoli, D. Rosaci, and G. M. Sarné, “A Multi-tiered Recommender System Architecture for Supporting E-Commerce,” in Intelligent Distributed Computing VI, Springer, 2013, pp. 71–81.
  18. M. Pazzani and D. Billsus. Content-based recommendation systems. In P. Brusilovsky, A. Kobsa, and W. Nejdl, editors, The Adaptive Web, volume 4321 of Lecture Notes in Computer Science, pages 325–341. Springer Berlin, Heidelberg, 2007.
  19. L. Rokach, P. Mitra, S. Kataria, W. Huang, and L. Giles, “A Supervised Learning Method for Context-Aware Citation Recommendation in a Large Corpus,” in Proceedings of the Large-Scale and Distributed Systems for Information Retrieval Workshop, 2013, pp. 17–22.
  20. Michael J. Pazzani,” A Framework for Collaborative, Content-Based and Demographic Filtering”,Artificial Intelligence Review - Special issue on data mining on the Internet archive Vol. 13 Issue 5-6, Dec. 1999, Pages 393 - 408 , Kluwer Academic Publishers Norwell, MA, USA.
  21. R. Burke, Knowledge-based Recommender Systems, Encyclopedia of Library and Information Science, 69(32):180-200, 2000.
  22. Shiu-li Huang ,”Designing utility-based recommender systems for e-commerce: Evaluation of preference-elicitation methods”,   Electronic Commerce Research and Applications, Volume 10, Issue 4, July–August 2011, Pages 398-407
  23. Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’08, pages 426–434, New York, NY, USA, 2008. ACM
  24. W. Hill, L. Stead, M. Rosenstein, and G. Furnas. Recommending and evaluating choices in a virtual community of use. In Proceedings of the SIGCHI conference on Human factors in computing systems, CHI ’95, pages 194–201, New York, NY, USA, 1995. ACM Press/Addison-Wesley Publishing Co.
  25. Ricci, F., Cavada, D., Mirzadeh, N., Venturini, A.” Case-based travel recommendations. In: D.R. Fesenmaier, K.Woeber, H.Werthner (eds.) Destination Recommendation Systems: Behavioural Foundations and Applications, pp. 67–93. CABI (2006).
  26. Montaner, M., L´opez, B., de la Rosa, J.L.: A taxonomy of recommender agents on the internet. Artificial Intelligence Review 19(4), 285–330 (2003).
  27. Fisher, G.: User modeling in human-computer interaction. User Modeling and User-Adapted Interaction 11, 65–86 (2001)
  28. Berkovsky, S., Kuflik, T., Ricci, F.: Cross-representation mediation of user models. User Modeling and User-Adapted Interaction 19(1-2), 35–63 (2009)
  29. Mahmood, T., Ricci, F.: Improving recommender systems with adaptive conversational strategies. In: C. Cattuto, G. Ruffo, F. Menczer (eds.) Hypertext, pp. 73–82. ACM (2009).
  30. Burke, R.: Hybrid web recommender systems. In: The AdaptiveWeb, pp. 377–408. Springer Berlin / Heidelberg (2007).
  31. D. Goldberg, D. Nichols, B. M. Oki, and D. Terry, “Using collaborative filtering to weave an information Tapestry,” Communications of the ACM, vol. 35, no. 12, pp. 61–70, 1992.
  32. P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl, “GroupLens: an open architecture for collaborative filtering of netnews,” in Proceedings of the 1994 ACM conference on Computer supported cooperative work, 1994, pp. 175–186.
  33. J. B. Schafer, D. Frankowski, J. Herlocker, and S. Sen, “Collaborative filtering recommender systems,” Lecture Notes In Computer Science, vol. 4321, p. 291, 2007.
  34. R. Dong, L. Tokarchuk, and A. Ma, “Digging Friendship: Paper Recommendation in Social Network,” in Proceedings of Networking & Electronic Commerce Research Conference (NAEC 2009), 2009, pp. 21–28.
  35. A. Vellino, “Usage-based vs. Citation-based Methods for Recommending Scholarly Research Articles,” Arxiv, vol., 2013.
  36. J. B. Schafer, D. Frankowski, J. Herlocker, and S. Sen, “Collaborative filtering recommender systems,” Lecture Notes In Computer Science, vol. 4321, p. 291, 2007.
  37. S. Sosnovsky and D. Dicheva, “Ontological Technologies for User Modeling,” International Journal of Metadata, Semantics and Ontologies, vol. 5, no. 1, pp. 32–71, 2010.
  38. P. Lops, M. Gemmis, and G. Semeraro, “Content-based recommender systems: State of the art and trends,” Recommender Systems Handbook, pp. 73–105, 2011.
  39. Y. AlMurtadha, M. N. Sulaiman, N. Mustapha, and N. I. Udzir, “Improved web page recommender system based on web usage mining,” in Proceedings of the 3rd International Conference on Computing and Informatics (ICOCI), 2011, pp. 8–9.
  40. Rocchio, J.: Relevance Feedback Information Retrieval. In: G. Salton (ed.) The SMART retrieval system - experiments in automated document processing, pp. 313–323. Prentice- Hall, Englewood Cliffs, NJ (1971).
  41. Herlocker, L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating Collaborative Filtering Recommender Systems. ACM Transactions on Information Systems 22(1), 5–53 (2004).
  42. Rich, E.: 1979, ‘User Modeling via Stereotypes’. Cognitive Science 3, 329-354.
  43. Krulwich, B.: 1997, ‘Lifestyle Finder: Intelligent User Profiling Using Large-Scale Demographic Data’. Artificial Intelligence Magazine 18 (2), 37-45.
  44. Mahmood, T., Ricci, F.: Towards learning user-adaptive state models in a conversational recommender system. In: A. Hinneburg (ed.) LWA 2007: Lernen - Wissen - Adaption, Halle, September 2007, Workshop Proceedings, pp. 373–378. Martin-Luther-University Halle-Wittenberg (2007).
  45. Bridge, D., G¨oker, M., McGinty, L., Smyth, B.: Case-based recommender systems. The Knowledge Engineering review 20(3), 315–320 (2006)
  46. Towle, B. and Quinn, C.: 2000, ‘Knowledge Based Recommender Systems Using Explicit User Models’. In Knowledge-Based Electronic Markets, Papers from the AAAI Workshop, AAAI Technical Report WS-00-04. pp. 74- 77. Menlo Park, CA: AAAI Press.
  47. Schmitt, S. and Bergmann, R.: 1999, ‘Applying case-based reasoning technology for product selection and customization in electronic commerce environments.’ 12th Bled Electronic Commerce Conference. Bled, Slovenia, June 7-9, 1999.
  48. Burke, Raymond R. (2002), "Technology and the Customer Interface: What Consumers Want in the Physical and Virtual Store," Journal of the Academy of Marketing Science, Vol. 30, No. 4, pp. 411-432.
  49. Guttman, R. H., Moukas, A. G. and Maes, P.: 1998, ‘Agent-Mediated Electronic Commerce: A Survey’. Knowledge Engineering Review, 13 (2), 147-159.
  50. R. Burke, “Hybrid recommender systems: Survey and experiments,” User modeling and user-adapted interaction, vol. 12, no. 4, pp. 331–370, 2002.
  51. Burke, Robind D.: Hybrid Web Recommender Systems. In Brusilovsky, Peter, Alfred Kobsa and Wolfgang Nejdl (editors): The Adaptive Web, Methods and Strategies of Web Personalization, volume 4321 of Lecture Notes in Computer Science, pages 377–408. Springer, 2007.
  52. Sarwar, B.M., Konstan, J.A., Riedl, J.: Distributed recommender systems for internet commerce. In: M. Khosrow-Pour (ed.) Encyclopedia of Information Science and Technology (II), pp. 907–911. Idea Group (2005)
  53. A¨ımeur, E., Brassard, G., Fernandez, J.M., Onana, F.S.M.: Alambic : a privacy-preserving recommender system for electronic commerce. Int. J. Inf. Sec. 7(5), 307–334 (2008)
  54. McSherry, F., Mironov, I.: Differentially private recommender systems: building privacy into the net. In: KDD ’09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 627–636. ACM, New York, NY, USA (2009)
  55. Xiaoyuan Su, Taghi M. Khoshgoftaar, Xingquan Zhu, and Russell Greiner. Imputation-boosted collaborative filtering using machine learning classifiers. In SAC ’08: Proceedings of the 2008 ACM symposium on Applied computing, pages 949–950, New York, NY, USA, 2008. ACM.
  56. Prem Melville, Raymond J. Mooney, and Ramadass Nagarajan. Content boosted collaborative filtering for improved recommendations. In Proceedings of the Eighteenth National Conference on Artificial Intelligence (AAAI- 02), pages 187–192, Edmonton, Alberta, 2002.
  57. Jessenitschnig, M., Zanker, M.: A generic user modeling component for hybrid recommendation strategies. E-Commerce Technology, IEEE International Conference on 0, 337–344 (2009). DOI
  58. Robin Burke, Bamshad Mobasher, Runa Bhaumik, and Chad Williams. Segment-based injection attacks against collaborative filtering recommender systems. In ICDM ’05: Proceedings of the Fifth IEEE International Conference on Data Mining, pages 577–580, Washington, DC, USA, 2005. IEEE Computer Society.
  59. Averjanova, O., Ricci, F., Nguyen, Q.N.: Map-based interaction with a conversational mobile recommender system. In: The Second International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies, 2008. UBICOMM ’08, pp. 212–218 (2008).
  60. Nguyen, Q.N., Ricci, F.: Replaying live-user interactions in the off-line evaluation of critique based mobile recommendations. In: RecSys ’07: Proceedings of the 2007 ACM conference on Recommender systems, pp. 81–88. ACM Press, New York, NY, USA (2007).


Recommendations System, Utility Based, Collaborative Filtering, Contents Based Methods, Demographic Based, Knowledge Based, Hybrid Methods, Knowledge Sources.