short-paper

Rethinking Large-scale Pre-ranking System: Entire-chain Cross-domain Models

Authors:

Changping Peng,

Jingping ShaoAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 4495 - 4499

https://doi.org/10.1145/3511808.3557683

Published: 17 October 2022 Publication History

Abstract

Industrial systems such as recommender systems and online advertising, have been widely equipped with multi-stage architectures, which are divided into several cascaded modules, including matching, pre-ranking, ranking and re-ranking. As a critical bridge between matching and ranking, existing pre-ranking approaches mainly endure sample selection bias (SSB) problem owing to ignoring the entire-chain data dependence, resulting in sub-optimal performances. In this paper, we rethink pre-ranking system from the perspective of the entire sample space, and propose Entire-chain Cross-domain Models (ECM), which leverage samples from the whole cascaded stages to effectively alleviate SSB problem. Besides, we design a fine-grained neural structure named ECMM to further improve the pre-ranking accuracy. Specifically, we propose a cross-domain multi-tower neural network to comprehensively predict for each stage result, and introduce the sub-networking routing strategy with L0 regularization to reduce computational costs. Evaluations on real-world large-scale traffic logs demonstrate that our pre-ranking models outperform SOTA methods while time consumption is maintained within an acceptable level, which achieves better trade-off between efficiency and effectiveness.

Supplementary Material

MP4 File (CIKM22-sp1103.mp4)

Pre-ranking model is a vital module in large-scale recommender system. In order to solve SSB in pre-ranking models, we propose Entire-chain Cross-domain(EC) paradigm and two efficient and effective models, namely ECM and ECMM.

Download
31.31 MB

References

[1]

Qingyao Ai, Keping Bi, Jiafeng Guo, and W. Bruce Croft. 2018. Learning a Deep Listwise Context Model for Ranking Refinement. The 41st International ACM SIGIR Conference on Research Development in Information Retrieval (2018), 135--144.

[2]

Miao Fan, Jiacheng Guo, Shuai Zhu, Shuo Miao, Mingming Sun, and Ping Li. 2019. MOBIUS: Towards the Next Generation of Query-Ad Matching in Baidu's Sponsored Search. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2019), 2509--2517.

Digital Library

[3]

Siyu Gu, Xiang-Rong Sheng, Biye Jiang, Siyuan Lou, Shuguang Han, Hongbo Deng, and Bo Zheng. 2022. On Ranking Consistency of Pre-ranking Stage. arXiv preprint arXiv:2205.01289 (2022).

[4]

Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. 2020. Embeddingbased retrieval in facebook search. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining (2020), 2553--2561.

Digital Library

[5]

Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry P. Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. 22nd ACM International Conference on Information and Knowledge Management (CIKM) (2013), 2333--2338.

[6]

Shichen Liu, Fei Xiao, Wenwu Ou, and Luo Si. 2017. Cascade Ranking for Operational E-commerce Search. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2017), 1557--1565.

Digital Library

[7]

Christos Louizos, Max Welling, and Diederik P. Kingma. 2018. Learning Sparse Neural Networks through L0 Regularization. Proceedings of the 6th International Conference on Learning Representations (ICLR) (2018).

[8]

Jiaqi Ma, Zhe Zhao, Jilin Chen, Ang Li, Lichan Hong, and EdHChi. 2019. SNR: subnetwork routing for flexible parameter sharing in multi-task learning. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (2019), 216--223.

Digital Library

[9]

Xu Ma, Pengjie Wang, Hui Zhao, Shaoguo Liu, Chuhan Zhao, Wei Lin, Kuang-Chih Lee, Jian Xu, and Bo Zheng. 2021. Towards a Better Tradeoff between Effectiveness and Efficiency in Pre-Ranking: A Learnable Feature Selection Based Approach. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021), 2036--2040.

Digital Library

[10]

Xiao Ma, Liqin Zhao, Guan Huang, ZhiWang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018. Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate. (2018), 1137--1140.

[11]

Changhua Pei, Yi Zhang, Yongfeng Zhang, Fei Sun, Xiao Lin, Hanxiao Sun, Jian Wu, Peng Jiang, Junfeng Ge, Wenwu Ou, and Dan Pei. 2019. Personalized Re-Ranking for Recommendation. Proceedings of the 13th ACM Conference on Recommender Systems (2019), 3--11.

Digital Library

[12]

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as Treatments: Debiasing Learning and Evaluation. Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (2016), 1670--1679.

Digital Library

[13]

Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. 2020. Progressive layered extraction (ple): A novel multi-task learning model for personalized recommendations. Proceedings of the Fourteenth ACM Conference on Recommender Systems (2020), 269--278.

Digital Library

[14]

JizheWang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-scale Commodity Embedding for E-commerce Recommendation in Alibaba. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2018), 839--848.

[15]

Zhe Wang, Liqin Zhao, Biye Jiang, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2020. COLD: Towards the Next Generation of Pre-Ranking System. CoRR abs/2007.16122 (2020).

[16]

Wenhao Zhang, Wentian Bao, Xiao-Yang Liu, Keping Yang, Quan Lin, Hong Wen, and Ramin Ramezani. 2020. Large-scale Causal Approaches to Debiasing Post-click Conversion Rate Estimation with Multi-task Learning. Proceedings of The Web Conference (WWW) (2020), 2775--2781.

Digital Library

[17]

Guorui Zhou, Na Mou, Ying Fan, Qi Pi,Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep Interest Evolution Network for Click-through Rate Prediction. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, Article 729 (2019), 8 pages.

[18]

Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-Through Rate Prediction. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD (2018), 1059--1068.

Digital Library

Cited By

Evnine AIoannidis SKalimeris DKalyanaraman SLi WNir ISun WWeinsberg UBaeza-Yates RBonchi F(2024)Achieving a Better Tradeoff in Multi-stage Recommender Systems through PersonalizationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671593(4939-4950)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671593

Index Terms

Rethinking Large-scale Pre-ranking System: Entire-chain Cross-domain Models
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

AutoFAS: Automatic Feature and Architecture Selection for Pre-Ranking System
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Industrial search and recommendation systems mostly follow the classic multi-stage information retrieval paradigm: matching, pre-ranking, ranking, and re-ranking stages. To account for system efficiency, simple vector-product based models are commonly ...
AutoML for Large Capacity Modeling of Meta's Ranking Systems
WWW '24: Companion Proceedings of the ACM Web Conference 2024

Web-scale ranking systems at Meta serving billions of users is complex. Improving ranking models is essential but engineering heavy. Automated Machine Learning (AutoML) can potentially release engineers from labor intensive work of tuning ranking models; ...
Query-Adaptive Hash Code Ranking for Large-Scale Multi-View Visual Search

Hash-based nearest neighbor search has become attractive in many applications. However, the quantization in hashing usually degenerates the discriminative power when using Hamming distance ranking. Besides, for large-scale visual search, existing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
274
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)4

Reflects downloads up to 19 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Evnine AIoannidis SKalimeris DKalyanaraman SLi WNir ISun WWeinsberg UBaeza-Yates RBonchi F(2024)Achieving a Better Tradeoff in Multi-stage Recommender Systems through PersonalizationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671593(4939-4950)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671593

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents