Yushan Pan 潘昱杉
Portrait of Yushan Pan

Yushan Pan

潘昱杉

Associate Professor & MSc Human–Computer Interaction Programme Director, Xi'an Jiaotong–Liverpool University. Working on multimodal sentiment analysis, machine unlearning, and affordance-driven robotics.

Senior Member, IEEE Senior Member, CCF Lifelong Member, CAA Member, CAAI · ACM · AAAC
XJTLU Profile ↗ Selected Publications yushan.pan [at] liverpool.ac.uk
184
peer-reviewed papers
82
CAS Q1/Q2 (TOP) journal articles
5
edited books & 1 chapter
¥4.5M
running grants as PI
About

Professional Highlights

I am an Associate Professor at Xi'an Jiaotong–Liverpool University (XJTLU), Suzhou, where I direct the MSc Human–Computer Interaction programme. My research sits at the intersection of multimodal learning, affective computing, and human–computer interaction, with current projects on multimodal sentiment analysis, machine unlearning in multimodal data fusion, and affordance-driven robotic systems.

My work appears in venues including IEEE Transactions on Affective Computing, IEEE/ACM Transactions on Audio, Speech and Language Processing, ACM CHI, and CSCW. I co-authored two Chinese national standards for AI programming assistants and large-model quality, and hold seven Chinese patents on EEG-based fatigue monitoring, underwater detection, and blockchain crowdsourcing.

Prior to XJTLU, I was a tenured Senior Researcher at the Norwegian University of Science and Technology (NTNU), and an industry consultant with PTC Inc. in Greater China and the USA.

Research

Current Research Areas

Multimodal Sentiment Analysis

Cross-modal interaction, dynamic routing, and fusion for language, audio, and visual evidence in English and Chinese benchmarks.

Machine Unlearning

Unlearning in multimodal data fusion for embedded intelligence under data-governance requirements.

Affordance-driven Robotics

Multimodal manipulator response and robotic grasping in complex industrial environments.

HCI & CSCW

Ethnographic studies of work, VR teleoperation, and collaborative systems in maritime and industrial domains.

Face Anti-Spoofing

Ultra-lightweight architectures, anomaly metrics, and multi-semantic GAN models for secure recognition.

Embedded & AIoT Systems

Hardware–software co-design, FPGA acceleration, and edge intelligence for perception tasks.

Career

Experience & Education

Work Experience

2025 – Present
Associate Professor
Xi'an Jiaotong–Liverpool University, Suzhou, China
2022 – 2025
Assistant Professor
Xi'an Jiaotong–Liverpool University, Suzhou, China
2019 – 2021
Senior Researcher (Tenured)
Faculty of Engineering, NTNU, Trondheim, Norway
2017 – 2019
Postdoctoral Researcher
Faculty of Engineering, NTNU, Trondheim, Norway
2012 – 2013
Senior Consultant
Parametric Technology Corporation (PTC), Greater China & Boston, USA
2009 – 2010
Technical Consultant
Parametric Technology Corporation (PTC), Xi'an, Greater China

Education

2021
Habilitation — Norwegian University Pedagogy
Equivalent to FHEA, UK · NTNU, Trondheim
2018
Ph.D. in Informatics
Department of Informatics, University of Oslo, Norway
2012
M.Sc. Information Systems Engineering
Norwegian University of Science and Technology (NTNU), Trondheim

Administration & Expert Roles

  • 2022–Nationwide Expert for Random Inspection & Evaluation of Undergraduate & Postgraduate Graduation Theses, CDGDC (Ministry of Education)
  • 2025–University Academic Board Member, XJTLU
  • 2025–Programme Director, MSc HCI — IET & BCS accreditation
  • 2025–Designed dual-pathway MSc HCI curriculum (CS / non-CS)
  • Sep 2026–Member, National Standardization Technical Committee SAC/TC 159
  • Jan 2026Review Expert, National AI Application Scenario Innovation Challenge
  • 2026–Review Expert, Yangtze River Delta International AI Application Scenario Innovation Competition (CAAI)

Professional Committee Work

  • 2026–29Associate Professor, Ruijin–XJTLU Intelligent Medicine Institute, Shanghai Jiao Tong University
  • 2023–26Adjunct Faculty & External Censor, NTNU & University of Oslo
  • 2023–Visiting Researcher, China Mobile Research Institute
  • 2023–Visiting Researcher, Xidian University
Scholarship

Selected Publications

Affective Computing and Its Applications for the Caregiving Industry: Theory and Practice
Pan, Y.*, Wang, Y., Xu, Z., & Li, Z. (Accepted). Liverpool University Press.
Vision-based Large-Model Technology for Detecting Unsafe Behaviors in Industrial Production
Xu, Z., Pan, Y., Zhang, C., Xu, Y., Cao, Y., & Wang, K. (Accepted). Springer, New York.
Infrared Targeted Detection and Tracking: Method and Practice
Zhu, H., Pan, Y.*, Deng, X., & Xu, G. (2023). Springer.
Fieldwork for Design: Theory and Practice (Chinese translation)
Pan, Y., Wang, Y., Luo, Y., & Wu, D. (translators, 2023). Randall, Harper & Rouncefield (authors). Shandong University Press.
Book Chapter: Design of Digital Environments for Operations on Vessels
Pan, Y.* (2016). Chapter 8 in Design of Cooperative Systems, pp. 123–138. Springer.
CAS Q1CAMC: A Conflict-Aware Multimodal Clue Model for Sarcasm Recognition
Li, Q., Pan, Y.*, Luo, Y., et al. (2027). IEEE Transactions on Affective Computing · IF 11.3
CAS Q1TCaN: Temporal Calibration Network for Dynamic MSA with Adaptive Modality Fusion
Hu, Y., Pan, Y.*, He, X., et al. (2027). IEEE Transactions on Affective Computing · IF 11.3
CAS Q1Semantic Purification and Time-Causal Stabilization with Energy-Inspired Arbitration for MSA
Zhu, M., Pan, Y.*, Li, Z., et al. (2027). IEEE Transactions on Affective Computing · IF 11.3
CAS Q1Target-specific Attention and Sentiment-guided GNN for Target-oriented MSA
Liu, S., Lu, T., Zhu, C., & Pan, Y.* (2026). IEEE Transactions on Affective Computing · IF 9.8
CAS Q1QDRF: Quantized and Distilled Robust Fusion for Incomplete Data in MSA
Hou, J., Cheng, Z., Xing, C., et al., & Pan, Y.* (2026). Information Sciences · IF 6
CAS Q1A Text-guided Cross-hierarchical Fusion and Multi-task Learning Framework for MSA
Zhu, M., Pan, Y.*, Xiang, N., et al. (2026). Neural Networks 202, 108995 · IF 6.3
CAS Q1AL-HCL: Active Learning and Hierarchical Contrastive Learning for MSA with Fusion Guidance
He, X., Pan, Y.*, Xu, Z., Li, Z., Guo, X., & Yang, C. (2025). IEEE Transactions on Affective Computing · IF 9.8
CAS Q1An Active Learning-based Reinforcement Contextual Information Fusion Model for MSA
He, X., Pan, Y.*, Chen, Y., et al. (2025). IEEE/ACM Transactions on Audio, Speech and Language Processing · IF 6.1
CAS Q1Multi-level Correlation Mining Framework with Self-supervised Label Generation for MSA
Li, Z., Guo, Q., Pan, Y.*, Ding, W., et al. (2023). Information Fusion 99 · IF 18.4
CAS Q1DSANet: Dynamic and Structure-aware GCN for Sparse and Incomplete Point Cloud Learning
Li, Y., Baciu, G., Chen, R., Ding, W., Wang, H., Pan, Y., & Li, C. (2024). IEEE Transactions on Neural Networks and Learning Systems · IF 14.3
CAS Q2Innovative Application of LLM in Prediction of Potential Depression in the Elderly (CHARLS)
Gao, S., Gao, Y., Su, M., Liu, J., & Pan, Y.* (2026). Multimedia Systems 32, 42 · IF 3.1
CAS Q1A Multi-view and Multi-granularity Emotional Semantic Interaction Framework for MSA
Li, Z., Guo, X., Wu, H., et al., & Pan, Y.* (2025). Neurocomputing 654, 131380 · IF 6.5

* Corresponding author. A full list of 184 peer-reviewed papers is available on my XJTLU scholar profile ↗ and Google Scholar ↗.

中文核心基于可形变规范化点云的三维人脸重建
刘姝, 张宇峰, 王科选, 潘昱杉*, 王皓 (2025). 小型微型计算机系统, 46(12), 2927–2933. DOI ↗
中文核心融合多头注意力TCN网络与触地预分类模块的六足机器人步态识别模型
陈逸阳, 费世杰, 毕晓宇, 吴狄, 潘昱杉* (2025). 小型微型计算机系统, 46(10), 2345–2350. DOI ↗
中文核心生成式人工智能大模型赋能的元宇宙生命体：前瞻和挑战
王皓, 潘昱杉, 潘毅 (2023). Big Data Research (大数据), 9(3), 85–96. DOI ↗
CCF 通讯提示工程在医疗信息产业中的应用：前瞻与挑战
王皓, 许东, 潘昱杉, 潘毅 (2024). CCF Communications (中国计算机学会通讯), 20(2), 50–58.
CCF AIntrinsic vs. Extrinsic Programming Challenges in Educational Games
Chen, B., Pan, Y., Xiang, N., & Huang, S. (2026). ACM CHI 2026, Article 953.
CCF ASEVADE: Self-Evolving Multi-Agent Analysis with Decoupled Evaluation for Hallucination-resistant Sarcasm Detection
Liu, Z., Chen, Y., Zhou, Z., et al., Pan, Y., & Xu, Z. (2026). AAAI 2026 40(35), 29573.
CCF BPersona Drift Detection in Role-Playing Agents: A Multi-dimensional Consistency Framework
Gao, Y., Li, Z., Chen, Y., Zhang, C., Xu, Z., & Pan, Y.* (2026). ICASSP 2026.
CCF A“Please Be Nice”: Robot Responses to User Bullying Across Aggression Levels
Luo, Y., Liu, S., Wu, D., Wang, H., & Pan, Y.* (2024). ACM CHI 2024, Paper 582.
CCF AThe Challenges of LLMs in Financial Services
Xie, L., Jiang, W., Luo, Y., Pan, Y.*, & Wang, H. (2023). IJCAI 2023, Macao.
CCF AReflexivity of Account, Professional Vision, and CSCW: Working in the Maritime Domain
Pan, Y. (2021). ACM CSCW 2021, pp. 1–32.
CCF BWalkFormer: Point Cloud Completion via Guided Walks
Zhang, M., Li, Y., Chen, R., Pan, Y.*, Wang, J., Wang, Y., & Xiang, R. (2024). WACV 2024, pp. 3281–3290.
CCF CTeleoperation of a Fast Omnidirectional UGV via a VR Interface (Best Paper Award)
Luo, Y., Wang, J., Pan, Y., Luo, S., Irani, P., & Liang, H. N. (2022). ACM VRCAI 2022.
Funding

Research Projects

Suzhou Intelligent Manufacturing Global AI Data Collaborative Industry Innovation Consortium

2026–2028 · Governmental project
PI · ¥2,000,000

Multimodal Manipulator Response Technology in Affordance-enabled Environments

2026–2029 · Collaborative project
PI · ¥500,000

CAAE: Context-Aware and Attention-Enhanced MSA for Intelligent HMI

2024–2027 · PGRS
PI · ¥290,000

Machine Unlearning in Multimodal Data Fusion for Embedded Intelligence

2025–2030 · PGRS
PI · ¥290,000

Jiangsu Province “Double Innovation” Doctoral Talent Program

2023–2025 · Governmental talent program
PI · ¥150,000

Modeling, Optimization & Digital Twin of Mixed Traffic Flow

2024–2027 · NSFC General Program
Co-PI · ¥510,000

Interpretable System for Intelligent Multimodal Medical Diagnosis

2024–2027 · Collaborative project
Co-PI · ¥500,000

Cloud-based Intelligent Control for Flexible Manufacturing of High-end Equipment

2024–2027 · Collaborative project
Co-PI · ¥500,000
Recognition

Selected Awards

Innovation

National Standards & Patents

Chinese National Standards (Co-author)

Technical Requirements and Test Specifications for All-in-One Programming Assistants

辅助编程一体机技术要求及测试规范 · 2025 · MIIT / National Library for Standards

T/CIE 257-2024 Large Model System Quality Requirements — Part 1: Text Type

大模型系统质量要求 第一部分 文本类型 · 2025 · MIIT / National Library for Standards

Patents

Education

Teaching & Invited Talks

Courses

  • 2022–Human–Robot Interaction · Master's · Module Leader, XJTLU
  • 2022–Introduction to Object-Oriented Programming in Java · Bachelor's · Module Leader
  • 2023Multiagent Systems · Bachelor's · Co-teach, XJTLU
  • 2017–21Human-centred Computing · Ph.D. level, NTNU
  • 2019–21Computer-Supported Cooperative Work, NTNU

Guest lectures at University of Oslo, Osaka University, Xidian University, Xi'an Jiaotong University, SJTU, Google Research, and China Mobile Research Institute, among others.

Selected Talks & Panels

  • 2025 · Media interview: AI Development and the Indispensability of Critical Thinking — The Paper (澎湃新闻)
  • 2024 · Emotional Human–Robot Interaction, Xidian University Guangzhou
  • 2024 · Affective Computing Applications in HCI, UM-SJTU
  • 2023 · Emotion Computing and Smart Cultural Tourism, Xinhua News Agency
  • 2023 · Wearable Fatigue Detection Based on SSVEP-BCI, CCF Embedded Systems Conference
  • 2022 · Characterizing HCI Research in China, Xidian University
  • 2020 · Human–Machine Cohabitation, DigitalWorld Congress, Valencia
  • 2019 · Information Support for Dynamic Positioning Operations, University of Oslo
Mentorship

Students

Current Ph.D. Students

  • Ziyue Zhang (primary, 2027)
  • Yifan Gao (primary, 2028)
  • Qiuyu Li (primary, 2028)
  • Xiaosong Hu (primary, 2030)
  • Yuhan Liu (primary, 2030)
  • Minghui Zhu (co-supervisor, 2030)
  • Wanli Bai (co-supervisor, 2030)

Current MSc Students

  • Hongfei Shen
  • Xinrui Li
  • Zhaoyi Huang

Current BSc Students

  • Zhihao Jin
  • Junyao Zhou
  • Zheyan Cheng
  • Zixuan Wang
  • Yanle Chen

BSc / MSc / Ph.D. Placements

  • 30+ BSc/MSc graduates supervised
  • Ph.D. graduates now at Xidian University, DNV, Northwestern Polytechnical University (NPU), Nanjing University of Posts and Telecommunications (NJUPT), Osaka University, among others.
  • Most students pursue further study at UCL, Imperial College London, Manchester, Wisconsin–Madison, HKU, UIUC, CMU, UPenn, JHU, Columbia University, Cornell University, UMichigan–Ann Arbor, Duke University, Georgia Institute of Technology, Brown University, University of California Irvine, Case Western Reserve University, Northwestern University, University of Copenhagen, and University of Oslo.
  • Industry: Tencent, DNV, Bosch, CETC, Shandong Port Group, ICBC, ABC, BOC, CCB, CMB, SPDB, CMBC, People's Bank of China, Kongsberg, Rolls-Royce Marine, XPeng, Li Auto, NIO, China North Industries Group (NORINCO), China Aerospace Science and Technology Corporation (CASC), China Electronics Corporation (CEC), China State Shipbuilding Corporation (CSSC), and China Merchants Group, among others.
Community

Academic Service

Conference Leadership

Associate Chair for ACM CHI, CSCW, DIS, and ICMI; Vice Chair (2025–27) and Treasurer (2024–25) of the Xi'an ACM SIGCHI Chapter.

Committee Service

Expert Evaluator, European Commission (Horizon Europe); Reviewer, Open Research Europe; ACM SIGCHI CARES Committee (2022–2026); CCF and CAA professional committee member.

National Innovation Expertise

Specially-invited expert for major projects at the Yangtze River Delta National Center of Technology Innovation (NICE).

Reviewer for Nature Communications, IEEE TPAMI, IEEE TNNLS, IEEE Transactions on Affective Computing, IEEE/ACM TASLP, IEEE Transactions on Image Processing, IEEE Transactions on Industrial Informatics, IEEE Transactions on Multimedia, and many other journals, plus ACM CHI, CSCW, DIS, ICMI, and HRI conferences. Book reviewer for Cambridge University Press, CRC Press, and Routledge.