Knowledge-enhanced policy-guided interactive reinforcement recommendation system

Yuqi ZHANG; Xiaowen HUANG; Jitao SANG

doi:10.11959/j.issn.2096-0271.2022033

您当前的位置：

首页 >

文章列表页 >

Knowledge-enhanced policy-guided interactive reinforcement recommendation system

STUDY | 更新时间：2024-06-03

- Knowledge-enhanced policy-guided interactive reinforcement recommendation system
- Big data research Vol. 8, Issue 5, Pages: 88-105(2022)
- 作者机构：
  
  1. 北京交通大学计算机与信息技术学院，北京 100044
  2. 交通数据分析与挖掘北京市重点实验室，北京 100044
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2022033
  CLC： TP391
- Online First：2022-09，
  
  Published：15 September 2022
- 稿件说明：
移动端阅览
Yuqi ZHANG, Xiaowen HUANG, Jitao SANG. Knowledge-enhanced policy-guided interactive reinforcement recommendation system[J]. Big data research, 2022, 8(5): 88-105.
DOI：

Yuqi ZHANG, Xiaowen HUANG, Jitao SANG. Knowledge-enhanced policy-guided interactive reinforcement recommendation system[J]. Big data research, 2022, 8(5): 88-105. DOI： 10.11959/j.issn.2096-0271.2022033.

摘要

推荐系统是解决社会媒体信息过载问题的重要手段。为了解决传统推荐系统无法优化用户长期体验的问题，研究人员提出了交互式推荐系统，并尝试使用深度强化学习优化推荐策略。但是，强化推荐算法面临反馈稀疏、从零学习影响用户体验、物品空间大等问题。为了解决上述问题，提出一种改进的知识增强策略引导的交互式强化推荐模型KGP-DQN。该模型构建行为知识图谱表示模块，将用户历史行为和知识图谱结合，解决反馈稀疏问题；构建策略初始化模块，根据用户历史行为为强化推荐系统提供初始化策略，解决从零学习影响用户体验的问题；构建候选集筛选模块，根据行为知识图谱上的物品表示进行动态聚类，从而减少物品空间，解决动作空间大的问题。在3个真实数据集上进行了实验，实验结果表明，KGP-DQN可以快速有效地对强化推荐系统进行训练，其在3个数据集上的推荐准确率均超过80%。

Abstract

The recommendation system is an important means to solve the problem of information overload in social media.To solve the problem that traditional recommendation systems cannot optimize the longterm user experience

researchers have proposed the interactive recommendation system and tried to use deep reinforcement learning to optimize the strategy of recommendation.However

the reinforcement recommendation algorithm faces problems such as sparse feedback

learning from zero which damages the user experience

and large item space.To solve the above problems

an improved interactive reinforcement recommendation model KGP-DQN was proposed.The model constructed a behavioral knowledge graph representation module

which combines user historical behavior and knowledge graph to solve the problem of sparse feedback.The model constructed a strategy initialization module to provide an initialization strategy for the reinforcement recommendation system based on user historical behaviors to solve the problem of learning from zero.The model constructed the candidate select module which creates candidates by dynamic clustering based on the item representation on the behavioral knowledge graph to solve the problem of large action space.The experiments were conducted on three real-world datasets.The experimental results show that the KGP-DQN method can quickly and effectively train the reinforcement recommendation system and its recommendation accuracy on three datasets is more than 80%.

关键词

Keywords

references

WANG H Z , WU Q Y , WANG H N . Factorization bandits for interactive recommendation [C ] // Proceedings of the 31st AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2017 : 2695 - 2702 .

MAHMOOD T , RICCI F . Learning and adaptivity in interactive recommender systems [C ] // Proceedings of the 9th International Conference on Electronic Commerce . New York:ACM Press , 2007 : 75 - 84 .

ZHAO X Y , ZHANG L , DING Z Y , et al . Recommendations with negative feedback via pairwise deep reinforcement learning [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2018 : 1040 - 1048 .

ZHOU S J , DAI X Y , CHEN H K , et al . Interactive recommender system via knowledge graph-enhanced reinforcement learning [C ] // Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York:ACM Press , 2020 : 179 - 188 .

WANG P F , FAN Y , XIA L , et al . KERL:a knowledge-guided reinforcement learning model for sequential recommendation [C ] // Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York:ACM Press , 2020 : 209 - 218 .

GOLDBERG D , NICHOLS D , OKI B M , et al . Using collaborative filtering to weave an information tapestry [J ] . Communications of the ACM , 1992 , 35 ( 12 ): 61 - 70 .

RENDLE S , . Factorization machines [C ] // Proceedings of 2010 IEEE International Conference on Data Mining . Piscataway:IEEE Press , 2010 : 995 - 1000 .

HIDASI B , KARATZOGLOU A , BALTRUNAS L , et al . Session-based recommendations with recurrent neural networks [J ] . arXiv preprint,2015,arXiv:1511.06939 .

ZHOU G R , ZHU X Q , SONG C R , et al . Deep interest network for click-through rate prediction [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2018 : 1059 - 1068 .

CHEN H K , DAI X Y , CAI H , et al . Large-scale interactive recommendation with tree-structured policy gradient [C ] // Proceedings of the 33rd AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2019 : 3312 - 3320 .

ZHENG G J , ZHANG F Z , ZHENG Z H , et al . DRN:a deep reinforcement learning framework for news recommendation [C ] // Proceedings of the 2018 World Wide Web Conference .[S.l.:s.n. ] , 2018 : 167 - 176 .

ZOU L X , XIA L , DING Z Y , et al . Reinforcement learning to optimize longterm user engagement in recommender systems [C ] // Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2019 : 2810 - 2818 .

DULAC-ARNOLD G , EVANS R , SUNEHAG P , et al . Reinforcement learning in large discrete action spaces [J ] . arXiv preprint,2015,arXiv:1512.07679 .

XIAN Y K , FU Z H , MUTHUKRISHNAN S , et al . Reinforcement knowledge graph reasoning for explainable recommendation [C ] // Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York:ACM Press , 2019 : 285 - 294 .

HASSELT H V , GUEZ A , SILVER D . Deep reinforcement learning with double Q-learning [C ] // Proceedings of the 30th AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2016 .

KOREN Y , BELL R , VOLINSKY C . Matrix factorization techniques for recommender systems [J ] . Computer , 2009 , 42 ( 8 ): 30 - 37 .

DAVIES D L , BOULDIN D W . A cluster separation measure [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 1979 , PAMI-1 ( 2 ): 224 - 227 .

Views

571

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Knowledge-enhanced large language model based history subject exam question generation system

A novel quality of service-oriented Spark job scheduler

Research on database pre-aggregation method for cloud-edge-device collaboration architecture

Key technologies and application exploration of aerospace big data in the construction of new smart city

Large scale pre-trained knowledge graph model and e-commerce application

Related Author

HUANG Wei

LIU Qi

TONG Wei

ZHAO Yuze

ZHANG Zheng

JI Tianyun

HUANG Zhenya

CHEN Enhong

Related Institution

National Education Examinations Authority

State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China

College of Computer Science & Software Engineering, Shenzhen University

Guangdong Laboratory of Artificial Intelligence and Digital Economy

Faculty of Computing, Harbin Institute of Technology

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273122 Visits Today：28

⁰