学术讲座

大数据管理与分析方法研究北京市重点实验室（BDAI）研究生沙龙由中国人民大学高瓴人工智能学院师生组织定期举行。本周研讨会由Jamie Callan教授指导的学生高陆羽以及张骁、徐君老师指导的学生思子华介绍自己的研究工作。欢迎同学们积极参与研讨！

报告题目：Language Model Pre-training for Dense Retrieval

报告人：Luyu Gao（高陆羽）is a Ph.D. student at Language Technologies Institute (LTI), Carnegie Mellon University where he is supervised by professor Jamie Callan. His research focuses on information retrieval and representation learning for search with neural networks. He has published first-author papers at conferences including ACL, EMNLP, and NAACL. He has alsoauthored several open-source toolkits and IR competition top systems. Prior to his Ph.D., he received his BS from UIUC where he worked with professor Chengxiang Zhai.

报告摘要：This talk will present our recent research on pre-training methods for dense retrieval. I will first motivate the talk by describing several key differences between standard pre-trained language models and dense retrieval models. The second part of the talk will introduce Condenser/coCondenser pre-training models. I will show how to adjust BERT's pre-training architecture and loss to learn effective CLS embedding and embedding space for dense retrieval.

报告题目：A Model-Agnostic Causal Learning Framework for Recommendation using Search Data

报告人：思子华，本科四年级 导师：张骁，徐君

研究方向：序列推荐；因果学习推荐系统

报告摘要：Machine-learning based recommender systems(RSs) has become an effective means to help people automatically discover their interests. Existing models often represent the rich information for recommendation, such as items, users, and contexts, as embedding vectors and leverage them to predict users’ feedback. In the view of causal analysis, the associations between these embedding vectors and users’ feedback are a mixture of the causal part that describes why an item is preferred by a user, and the non-causal part that merely reflects the statistical dependencies between users and items, for example, the exposure mechanism, public opinions, display position, etc. However, existing RSs mostly ignored the striking differences between the causal parts and non-causal parts when using these embedding vectors. In this paper, we propose a model-agnostic framework named IV4Rec that can effectively decompose the embedding vectors into these two parts, hence enhancing recommendation results. Specifically, we jointly consider users’ behaviors in search scenarios and recommendation scenarios. Adopting the concepts in causal analysis, we embed users’ search behaviors as instrumental variables (IVs), to help decompose original embedding vectors in recommendation, i.e., treatments. IV4Rec then combines the two parts through deep neural networks and uses the combined results for recommendation. IV4Rec is model-agnostic and can be applied to a number of existing RSs such as DIN and NRHUB. Experimental results on both public and proprietary industrial datasets demonstrate that IV4Rec consistently enhances RSs and outperforms a framework that jointly considers search and recommendation.