TỐI ƯU HÓA HỆ THỐNG ĐỀ XUẤT: TÍCH HỢP LỌC CỘNG TÁC VỚI CÁC KĨ THUẬT BAYESIAN VÀ GAUSSIAN
Nội dung chính của bài viết
Tóm tắt
Một phương pháp thường được sử dụng trong hệ thống đề xuất là lọc cộng tác (Collaborative Filtering - CF). Tinh chỉnh các siêu tham số (hyperparameters) của các thuật toán CF vẫn là một công việc khó khăn ngay cả với những khám phá mới trong việc mô hình hóa người dùng và các sản phẩm/dịch vụ. Nghiên cứu này đề xuất một phương pháp thay thế cho công việc này thông qua tối ưu hóa Bayesian sử dụng quá trình ngẫu nhiên Gaussian trong quá trình thay đổi các siêu tham số. Phương pháp này giảm thời gian và công sức cần thiết cho việc tinh chỉnh thủ công bằng cách tự động điều chỉnh các siêu tham số cho hai thuật toán lọc cộng tác cơ bản (và đơn giản) trên ba tập dữ liệu phổ biến: Netflix Prize, Movielens 1M và Movielens 10M. Do đó, nó có thể giúp các nhà thực hành cải thiện hiệu suất của hệ thống đề xuất, đồng thời rút ngắn đáng kể thời gian và công sức dành cho việc tinh chỉnh hệ thống của họ.
Từ khóa
tối ưu hóa Bayesian, lọc cộng tác, Movielens, Netflix Prize
Chi tiết bài viết
Tài liệu tham khảo
Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734-749. https://doi.org/10.1109/TKDE.2005.99
Aljunid, M. F., & Doddaghatta Huchaiah, M. (2020). Multi‐model deep learning approach for collaborative filtering recommendation system. CAAI Transactions on Intelligence Technology, 5(4), 268-275. https://doi.org/10.1049/trit.2020.0031
Bardenet, R., Brendel, M., Kégl, B., & Sebag, M. (2013). Collaborative hyperparameter tuning. Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, II–199–II–207.
Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. The Journal of Machine Learning Research, 13, 281-305. https://doi.org/10.5555/2188385.2188395
Chen, Y., Mensah, S., Ma, F., Wang, H., & Jiang, Z. (2021). Collaborative filtering grounded on knowledge graphs. Pattern Recognition Letters, 151, 55-61. https://doi.org/https://doi.org/10.1016/j.patrec.2021.07.022
Cho, Y. H., Kim, J. K., & Kim, S. H. (2002). A personalized recommender system based on web usage mining and decision tree induction. Expert Systems with Applications, 23(3), 329-342. https://doi.org/https://doi.org/10.1016/S0957-4174(02)00052-0
Frazier, P. I. (2018). A Tutorial on Bayesian Optimization. https://arxiv.org/abs/1807.02811
Garnett, R. (2023). gaussian processes. In Bayesian Optimization (pp. 15-44). Cambridge
University Press.
Guido, R., Groccia, M. C., & Conforti, D. (2023). A hyper-parameter tuning approach for cost-sensitive support vector machine classifiers. Soft Computing, 27(18), 12863-12881. https://doi.org/10.1007/S00500-022-06768-8/FIGURES/7
Han, S. C., Lim, T., Long, S., Burgstaller, B., & Poon, J. (2021). GLocal-K: Global and Local Kernels for Recommender Systems. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 3063-3067. https://doi.org/10.1145/3459637.3482112
Hoos, H. H. (2012). Automated Algorithm Configuration and Parameter Tuning. In E. and S. F. Hamadi Youssef and Monfroy (Ed.), Autonomous Search (pp. 37-71). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-21434-9_3
Horasan, F., Yurttakal, A. H., & Gündüz, S. (2023). A novel model based collaborative filtering recommender system via truncated ULV decomposition. Journal of King Saud University - Computer and Information Sciences, 35(8), Article 101724. https://doi.org/https://doi.org/10.1016/j.jksuci.2023.101724
Imani, M., Imani, M., & Ghoreishi, S. F. (2022). Bayesian Optimization for Expensive Smooth-Varying Functions. IEEE Intelligent Systems, 37(4), 44-55. https://doi.org/10.1109/MIS.2022.3163227
Khaledian, N., & Mardukhi, F. (2022). CFMT: a collaborative filtering approach based on the nonnegative matrix factorization technique and trust relationships. Journal of Ambient Intelligence and Humanized Computing, 13(5), 2667-2683. https://doi.org/10.1007/s12652-021-03368-6
Kim, D., & Suh, B. (2019, September). Enhancing VAEs for collaborative filtering: flexible priors & gating mechanisms. Proceedings of the 13th ACM Conference on Recommender Systems. https://doi.org/10.1145/3298689.3347015
Koren, Y. (2008). Factorization meets the neighborhood: a multifaceted collaborative filtering model. Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp.426-434). https://doi.org/10.1145/1401890.1401944
Koren, Y. (2009). The bellkor solution to the netflix grand prize. https://api.semanticscholar.org/CorpusID:6114578
Lam, C. P. (2005). Collaborative Filtering Using Associative Neural Memory. In S. S. Mobasher Bamshad and Anand (Ed.), Intelligent Techniques for Web Personalization (pp. 153-168). Springer Berlin Heidelberg.
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2017). Hyperband. The Journal of Machine Learning Research, 18, 1-52. https://doi.org/10.5555/3122009.3242042
Li, Z., Huang, J., & Zhong, N. (2017). Exploiting user and item embedding in latent factor models for recommendations. Proceedings of the International Conference on Web Intelligence (pp.1241-1245). https://doi.org/10.1145/3106426.3109437
Linden, G., Smith, B., & York, J. (2003). Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Computing, 7(1), 76-80. https://doi.org/10.1109/MIC.2003.1167344
Molaei, S., Havvaei, A., Zare, H., & Jalili, M. (2021). Collaborative Deep Forest Learning for Recommender Systems. IEEE Access, 9, 22053-22061. https://doi.org/10.1109/ACCESS.2021.3054818
Morita, Y., Rezaeiravesh, S., Tabatabaei, N., Vinuesa, R., Fukagata, K., & Schlatter, P. (2022). Applying Bayesian optimization with Gaussian process regression to computational fluid dynamics problems. Journal of Computational Physics, 449, Article 110788. https://doi.org/10.1016/j.jcp.2021.110788
Rendle, S. (2012). Factorization Machines with libFM. ACM Trans. Intell. Syst. Technol., 3(3), 1-22 https://doi.org/10.1145/2168752.2168771
Rendle, S., Zhang, L., & Koren, Y. (2019). On the Difficulty of Evaluating Baselines: A Study on Recommender Systems. ArXiv, abs/1905.01395. https://api.semanticscholar.org/CorpusID:146120960
Salakhutdinov, R., & Mnih, A. (2008). Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. Proceedings of the 25th International Conference on Machine Learning (pp. 880-887). https://doi.org/10.1145/1390156.1390267
Shenbin, I., Alekseev, A., Tutubalina, E., Malykh, V., & Nikolenko, S. I. (2020). RecVAE: A New Variational Autoencoder for Top-N Recommendations with Implicit Feedback. Proceedings of the 13th International Conference on Web Search and Data Mining (pp. 528-536). https://doi.org/10.1145/3336191.3371831
Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical Bayesian optimization of machine learning algorithms. Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 2, 2951-2959.
Srikumar, K. (2004). A Framework of Agent-Based Personalized Recommender System for E-Commerce. South Asian Journal of Management, 11, Article 66. https://api.semanticscholar.org/CorpusID:166413341
Steck, H. (2019, May). Embarrassingly Shallow Autoencoders for Sparse Data. The World Wide Web Conference. https://doi.org/10.1145/3308558.3313710
Szabó, P., & Genge, B. (2020). Hybrid Hyper-parameter Optimization for Collaborative Filtering. 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), 210-217. https://doi.org/10.1109/SYNASC51798.2020.00042
Tim, H., Manoj, K., Holger, N., Gilles, L., & Shcherbaty, I. (2021). scikit-optimize/scikit-optimize (v0.9.0). https://scikit-optimize.github.io/stable/
Xia, H., Luo, Y., & Liu, Y. (2021). Attention neural collaboration filtering based on GRU for recommender systems. Complex & Intelligent Systems, 7(3), 1367-1379. https://doi.org/10.1007/s40747-021-00274-4
Yan, S. (2014). A Collaborative Filtering Recommender Approach by Investigating Interactions of Interest and Trust. In T. and L. H. Sun Fuchun and Li (Ed.), Knowledge Engineering and Management (pp. 173-188). Springer Berlin Heidelberg.
Zhang, R., Liu, Q., Chun-Gui, Wei, J.-X., & Huiyi-Ma. (2014). Collaborative Filtering for Recommender Systems. 2014 Second International Conference on Advanced Cloud and Big Data, 301-308. https://doi.org/10.1109/CBD.2014.47
Aljunid, M. F., & Doddaghatta Huchaiah, M. (2020). Multi‐model deep learning approach for collaborative filtering recommendation system. CAAI Transactions on Intelligence Technology, 5(4), 268-275. https://doi.org/10.1049/trit.2020.0031
Bardenet, R., Brendel, M., Kégl, B., & Sebag, M. (2013). Collaborative hyperparameter tuning. Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, II–199–II–207.
Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. The Journal of Machine Learning Research, 13, 281-305. https://doi.org/10.5555/2188385.2188395
Chen, Y., Mensah, S., Ma, F., Wang, H., & Jiang, Z. (2021). Collaborative filtering grounded on knowledge graphs. Pattern Recognition Letters, 151, 55-61. https://doi.org/https://doi.org/10.1016/j.patrec.2021.07.022
Cho, Y. H., Kim, J. K., & Kim, S. H. (2002). A personalized recommender system based on web usage mining and decision tree induction. Expert Systems with Applications, 23(3), 329-342. https://doi.org/https://doi.org/10.1016/S0957-4174(02)00052-0
Frazier, P. I. (2018). A Tutorial on Bayesian Optimization. https://arxiv.org/abs/1807.02811
Garnett, R. (2023). gaussian processes. In Bayesian Optimization (pp. 15-44). Cambridge
University Press.
Guido, R., Groccia, M. C., & Conforti, D. (2023). A hyper-parameter tuning approach for cost-sensitive support vector machine classifiers. Soft Computing, 27(18), 12863-12881. https://doi.org/10.1007/S00500-022-06768-8/FIGURES/7
Han, S. C., Lim, T., Long, S., Burgstaller, B., & Poon, J. (2021). GLocal-K: Global and Local Kernels for Recommender Systems. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 3063-3067. https://doi.org/10.1145/3459637.3482112
Hoos, H. H. (2012). Automated Algorithm Configuration and Parameter Tuning. In E. and S. F. Hamadi Youssef and Monfroy (Ed.), Autonomous Search (pp. 37-71). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-21434-9_3
Horasan, F., Yurttakal, A. H., & Gündüz, S. (2023). A novel model based collaborative filtering recommender system via truncated ULV decomposition. Journal of King Saud University - Computer and Information Sciences, 35(8), Article 101724. https://doi.org/https://doi.org/10.1016/j.jksuci.2023.101724
Imani, M., Imani, M., & Ghoreishi, S. F. (2022). Bayesian Optimization for Expensive Smooth-Varying Functions. IEEE Intelligent Systems, 37(4), 44-55. https://doi.org/10.1109/MIS.2022.3163227
Khaledian, N., & Mardukhi, F. (2022). CFMT: a collaborative filtering approach based on the nonnegative matrix factorization technique and trust relationships. Journal of Ambient Intelligence and Humanized Computing, 13(5), 2667-2683. https://doi.org/10.1007/s12652-021-03368-6
Kim, D., & Suh, B. (2019, September). Enhancing VAEs for collaborative filtering: flexible priors & gating mechanisms. Proceedings of the 13th ACM Conference on Recommender Systems. https://doi.org/10.1145/3298689.3347015
Koren, Y. (2008). Factorization meets the neighborhood: a multifaceted collaborative filtering model. Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp.426-434). https://doi.org/10.1145/1401890.1401944
Koren, Y. (2009). The bellkor solution to the netflix grand prize. https://api.semanticscholar.org/CorpusID:6114578
Lam, C. P. (2005). Collaborative Filtering Using Associative Neural Memory. In S. S. Mobasher Bamshad and Anand (Ed.), Intelligent Techniques for Web Personalization (pp. 153-168). Springer Berlin Heidelberg.
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2017). Hyperband. The Journal of Machine Learning Research, 18, 1-52. https://doi.org/10.5555/3122009.3242042
Li, Z., Huang, J., & Zhong, N. (2017). Exploiting user and item embedding in latent factor models for recommendations. Proceedings of the International Conference on Web Intelligence (pp.1241-1245). https://doi.org/10.1145/3106426.3109437
Linden, G., Smith, B., & York, J. (2003). Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Computing, 7(1), 76-80. https://doi.org/10.1109/MIC.2003.1167344
Molaei, S., Havvaei, A., Zare, H., & Jalili, M. (2021). Collaborative Deep Forest Learning for Recommender Systems. IEEE Access, 9, 22053-22061. https://doi.org/10.1109/ACCESS.2021.3054818
Morita, Y., Rezaeiravesh, S., Tabatabaei, N., Vinuesa, R., Fukagata, K., & Schlatter, P. (2022). Applying Bayesian optimization with Gaussian process regression to computational fluid dynamics problems. Journal of Computational Physics, 449, Article 110788. https://doi.org/10.1016/j.jcp.2021.110788
Rendle, S. (2012). Factorization Machines with libFM. ACM Trans. Intell. Syst. Technol., 3(3), 1-22 https://doi.org/10.1145/2168752.2168771
Rendle, S., Zhang, L., & Koren, Y. (2019). On the Difficulty of Evaluating Baselines: A Study on Recommender Systems. ArXiv, abs/1905.01395. https://api.semanticscholar.org/CorpusID:146120960
Salakhutdinov, R., & Mnih, A. (2008). Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. Proceedings of the 25th International Conference on Machine Learning (pp. 880-887). https://doi.org/10.1145/1390156.1390267
Shenbin, I., Alekseev, A., Tutubalina, E., Malykh, V., & Nikolenko, S. I. (2020). RecVAE: A New Variational Autoencoder for Top-N Recommendations with Implicit Feedback. Proceedings of the 13th International Conference on Web Search and Data Mining (pp. 528-536). https://doi.org/10.1145/3336191.3371831
Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical Bayesian optimization of machine learning algorithms. Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 2, 2951-2959.
Srikumar, K. (2004). A Framework of Agent-Based Personalized Recommender System for E-Commerce. South Asian Journal of Management, 11, Article 66. https://api.semanticscholar.org/CorpusID:166413341
Steck, H. (2019, May). Embarrassingly Shallow Autoencoders for Sparse Data. The World Wide Web Conference. https://doi.org/10.1145/3308558.3313710
Szabó, P., & Genge, B. (2020). Hybrid Hyper-parameter Optimization for Collaborative Filtering. 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), 210-217. https://doi.org/10.1109/SYNASC51798.2020.00042
Tim, H., Manoj, K., Holger, N., Gilles, L., & Shcherbaty, I. (2021). scikit-optimize/scikit-optimize (v0.9.0). https://scikit-optimize.github.io/stable/
Xia, H., Luo, Y., & Liu, Y. (2021). Attention neural collaboration filtering based on GRU for recommender systems. Complex & Intelligent Systems, 7(3), 1367-1379. https://doi.org/10.1007/s40747-021-00274-4
Yan, S. (2014). A Collaborative Filtering Recommender Approach by Investigating Interactions of Interest and Trust. In T. and L. H. Sun Fuchun and Li (Ed.), Knowledge Engineering and Management (pp. 173-188). Springer Berlin Heidelberg.
Zhang, R., Liu, Q., Chun-Gui, Wei, J.-X., & Huiyi-Ma. (2014). Collaborative Filtering for Recommender Systems. 2014 Second International Conference on Advanced Cloud and Big Data, 301-308. https://doi.org/10.1109/CBD.2014.47