Skip to main navigation menu Skip to main content Skip to site footer

Vol. 5 No. 1 (2024)
2024 Edition

Articles

PALM: Personalized Attention-based Language Model for Long-tail Query Understanding in Enterprise Search Systems

Published 2024-08-19

  • Qiwen Zhao
    • ,
  • Zhongwen Zhou
    • ,
  • Yibang Liu

Qiwen Zhao
Computer Science, University of California San Diego, CA, USA

Zhongwen Zhou
Computer Science, University of California, Berkeley, CA, USA

Yibang Liu
Financial Engineering, Baruch College, NY, USA

Abstract

Enterprise search systems face significant challenges in handling long-tail queries, which constitute a substantial portion of search traffic but often receive inadequate attention in traditional systems. This paper introduces PALM (Personalized Attention-based Language Model), a novel framework designed to enhance long-tail query understanding in enterprise search environments. PALM integrates personalization capabilities with an advanced attention mechanism to improve search accuracy for infrequent queries while maintaining high performance on common queries. The framework employs a unique hierarchical architecture that combines user context, query semantics, and organizational knowledge through a sophisticated attention mechanism. The system features an innovative query embedding approach that adapts to individual user contexts while leveraging collective organizational knowledge. Extensive experiments on a large-scale enterprise dataset, comprising over 5 million queries from 50,000 users, demonstrate PALM's superior performance compared to state-of-the-art baselines. The results show significant improvements across multiple metrics, with a 17.5% increase in MAP for ultra-rare queries and a 10.4% overall improvement in NDCG@10. The framework exhibits robust performance across different organizational units and query types, making it particularly valuable for enterprise environments where query patterns are highly diverse and context-dependent. Our ablation studies confirm the effectiveness of each component in the PALM architecture, while case analyses provide insights into the framework's practical applications.

Keywords

Enterprise Search, Long-tail Queries, Attention Mechanism, Personalized Search

View PDF

References

  1. Wang, M., Liu, J., Wang, J., Wang, Y., & Chu, X. (2023). A Topicality Relevance-Aware Intent Model for Web Search. IEEE Access, 11, 65739-65748.
  2. Dong, B., Xu, Y., Chi, S., Shi, Z., & Du, Z. (2023, December). APAM: Adaptive Pre-Training and Adaptive Meta Learning in Language Model for Noisy Labels and Long-Tailed Learning. In 2023 International Conference on Machine Learning and Applications (ICMLA) (pp. 1867-1874). IEEE.
  3. Chang, T. J., Lin, L. H. M., & Tsai, R. T. H. (2024, April). Conversational Product Recommendation using LLM. In 2024 IEEE 4th International Conference on Electronic Communications, Internet of Things and Big Data (ICEIB) (pp. 340-343). IEEE.
  4. Wang, W., Hu, B., Peng, Z., Zhong, M., Zhang, Z., Liu, Z., ... & Zhou, J. (2023, April). GARCIA: Powering Representations of Long-tail Query with Multi–granularity Contrastive Learning. In 2023 IEEE 39th International Conference on Data Engineering (ICDE) (pp. 3182-3195). IEEE.
  5. Liu, J., Dou, Z., Nie, J. Y., & Wen, J. R. (2023). Integrated Personalized and Diversified Search Based on Search Logs. IEEE Transactions on Knowledge and Data Engineering.
  6. Wang, Y., Zhou, Y., Ji, H., He, Z., & Shen, X. (2024, March). Construction and application of artificial intelligence crowdsourcing map based on multi-track GPS data. In 2024 7th International Conference on Advanced Algorithms and Control Engineering (ICAACE) (pp. 1425-1429). IEEE.
  7. Akbar, A., Peoples, N., Xie, H., Sergot, P., Hussein, H., Peacock IV, W. F., & Rafique, Z. . (2022). Thrombolytic Administration for Acute Ischemic Stroke: What Processes can be Optimized?. McGill Journal of Medicine, 20(2).
  8. Zhang, Y., Xie, H., Zhuang, S., & Zhan, X. (2024). Image Processing and Optimization Using Deep Learning-Based Generative Adversarial Networks (GANs). Journal of Artificial Intelligence General science (JAIGS) ISSN: 3006-4023, 5(1), 50-62.
  9. Lu, T., Jin, M., Yang, M., & Huang, D. (2024). Deep Learning-Based Prediction of Critical Parameters in CHO Cell Culture Process and Its Application in Monoclonal Antibody Production. International Journal of Advance in Applied Science Research, 3, 108-123.
  10. Xia, S., Zhu, Y., Zheng, S., Lu, T., & Ke, X. (2024). A Deep Learning-based Model for P2P Microloan Default Risk Prediction. International Journal of Innovative Research in Engineering and Management, 11(5), 110-120.
  11. Zheng, W., Yang, M., Huang, D., & Jin, M. (2024). A Deep Learning Approach for Optimizing Monoclonal Antibody Production Process Parameters. International Journal of Innovative Research in Computer Science & Technology, 12(6), 18-29.
  12. Ma, X., Wang, J., Ni, X., & Shi, J. (2024). Machine Learning Approaches for Enhancing Customer Retention and Sales Forecasting in the Biopharmaceutical Industry: A Case Study. International Journal of Engineering and Management Research, 14(5), 58-75.
  13. Cao, G., Zhang, Y., Lou, Q., & Wang, G. (2024). Optimization of High-Frequency Trading Strategies Using Deep Reinforcement Learning. Journal of Artificial Intelligence General science (JAIGS) ISSN: 3006-4023, 6(1), 230-257.
  14. Wang, G., Ni, X., Shen, Q., & Yang, M. (2024). Leveraging Large Language Models for Context-Aware Product Discovery in E-commerce Search Systems. Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online), 3(4).
  15. Ju, C., & Zhu, Y. (2024). Reinforcement Learning‐Based Model for Enterprise Financial Asset Risk Assessment and Intelligent Decision‐Making.
  16. Huang, D., Yang, M., & Zheng, W. (2024). Integrating AI and Deep Learning for Efficient Drug Discovery and Target Identification.
  17. Yang, M., Huang, D., & Zhan, X. (2024). Federated Learning for Privacy-Preserving Medical Data Sharing in Drug Development.
  18. Zhang, H., Pu, Y., Zheng, S., & Li, L. (2024). AI-Driven M&A Target Selection and Synergy Prediction: A Machine Learning-Based Approach.
  19. Zhang, H., Pu, Y., Zheng, S., & Li, L. (2024). AI-Driven M&A Target Selection and Synergy Prediction: A Machine Learning-Based Approach.Zhang, H., Pu, Y., Zheng, S., & Li, L. (2024). AI-Driven M&A Target Selection and Synergy Prediction: A Machine Learning-Based Approach.
  20. Li, L., Zhang, Y., Wang, J., & Ke, X. (2024). Deep Learning-Based Network Traffic Anomaly Detection: A Study in IoT Environments.
  21. Wang, J., Lu, T., Li, L., & Huang, D. (2024). Enhancing Personalized Search with AI: A Hybrid Approach Integrating Deep Learning and Cloud Computing. International Journal of Innovative Research in Computer Science & Technology, 12(5), 127-138.