Publications | Xiaoyi Wang

2026

Prosaccades as Lightweight Workload Probes in MR Teleoperation

Xiaoyi Wang, Guangtao Zhang, Yuan Li, Jingbang Xiao, I. Scott MacKenzie, and John Paulin Hansen

International Journal of Human-Computer Studies, 2026

Accepted for publication

Abs Bib Video

Assessing mental workload is essential in safety-critical interactive systems such as mixed-reality (MR) teleoperation. Antisaccade tasks provide a validated and sensitive index of executive control and workload, but their high difficulty and intrusiveness limit practical deployment. This study investigates whether prosaccades, reflexive and lightweight eye movements, can serve as a lower-burden alternative while retaining diagnostic sensitivity. We conducted a 2 x 2 within-subjects experiment in an MR teleoperation environment, manipulating workload through concurrent backward counting and comparing prosaccade and antisaccade probes. Using linear mixed-effects modeling and pre-specified non-inferiority margins, we evaluated workload sensitivity across saccade latency, variability, accuracy, subjective workload, and operational performance. Both paradigms reflected workload increases in objective saccade measures. Prosaccades demonstrated non-inferior sensitivity on latency and accuracy within conservative tolerances relative to antisaccades. These findings position prosaccades as a viable lightweight episodic workload probe in applied HCI contexts. Methodologically, we show that non-inferiority testing provides a principled framework for evaluating lower-burden interaction alternatives in HCI. Conceptually, the results inform the design of adaptive MR systems by showing that minimally intrusive episodic behavioral probes retain sensitivity to centrally mediated cognitive demand while reducing user burden.
@article{wang2026Prosaccades, title = {Prosaccades as {{Lightweight Workload Probes}} in {{MR Teleoperation}}}, author = {Wang, Xiaoyi and Zhang, Guangtao and Li, Yuan and Xiao, Jingbang and MacKenzie, I. Scott and Hansen, John Paulin}, year = {2026}, journal = {International Journal of Human-Computer Studies}, note = {Accepted for publication}, keywords = {Mental workload,Teleoperation,Eye tracking,Antisaccade,Prosaccade,Workload sensitivity,Non-inferiority,Head-mounted display,Mixed reality} }
MIAdvisor: Enhancing Counselor Competence in Motivational Interviewing through Visual Analytics

Xiaoyi Wang, Jialong Ye, Guangtao Zhang, Yang Liu, and Honglei Guo

International Journal of Human-Computer Studies, 2026

Abs DOI Bib Video

Mental disorders pose a major global health challenge, yet many individuals face barriers to professional counseling. Online mental health communities provide volunteer support, but these volunteers often lack the training needed to apply effective therapeutic techniques. Motivational Interviewing (MI) is one such approach that requires adaptive application to clients’ needs, making it particularly challenging for trainees to master. While prior research has focused on diagnosis and treatment, less attention has been given to fostering counselor trainees’ skill development. To address this, we propose MIAdvisor, a visual analytics system designed to enhance counselor trainees’ competence in MI. It includes three key features to support skill development: (a) Enhancing the effective application of MI principles and techniques through visual analysis of their impact on clients’ behavior, cognition, and emotions; (b) Uncovering key dynamics between counselor strategies and client responses through interactive exploration of recurring patterns; and (c) Facilitating vicarious reflective practice by enabling comparative analysis of therapy sessions. MIAdvisor uses Large Language Models to analyze MI therapy conversations based on the MI Skill Code framework, tagging key MI elements to facilitate its interactive visual analysis of counselor-client interactions. We evaluate MIAdvisor through a case study with one counseling trainee and a think-aloud study involving seven psychology students and one postdoctoral researcher, analyzing a subset of 18 annotated MI sessions. Our findings suggest that MIAdvisor effectively strengthens trainees’ competence and supports ongoing skill development. These findings also offer broader design insights for advancing research on visual analytics in mental health training and practice.
@article{wang2026MIAdvisor, title = {{{MIAdvisor}}: {{Enhancing}} Counselor Competence in Motivational Interviewing through Visual Analytics}, author = {Wang, Xiaoyi and Ye, Jialong and Zhang, Guangtao and Liu, Yang and Guo, Honglei}, year = {2026}, journal = {International Journal of Human-Computer Studies}, volume = {215}, pages = {103853}, issn = {1071-5819}, doi = {10.1016/j.ijhcs.2026.103853}, keywords = {Counselor training,Large language models,Mental health,Mental health education,Motivational interviewing,Reflective learning,Skill development,Visual analytics} }
Bridging Discrepancies in Debate: Joint Debate for Enhancing Multimodal Emotion-Cause Pair Extraction in Conversation

Shoupu Wu, Jiali Lin, Xiaoyi Wang, Dazhi Jiang, and Erik Cambria

IEEE Transactions on Affective Computing, 2026

Abs DOI Bib

Multimodal Emotion-Cause Pair Extraction in Conversation (MECPEC) aims to accurately identify emotions and their causes in multimodal multiparty conversations. Existing methods primarily rely on discriminative or generative predictions from pre-trained language models, overlooking the potential to leverage the differences and complementary strengths between these two approaches. This paper proposes a novel framework, DEC-Debate, based on a multiparty-multiround debate mechanism. Concretely, the utterance emotion classification results from different models are integrated through debate, which can assign more appropriate emotion labels to utterances. Subsequently, multiple LLMs collaborate to combine their knowledge, express the causal relationships in each emotion-cause utterance pair, and then explain and update them through multiround debate, reducing the impact of erroneous model knowledge. By enabling independent generation and mutual criticism, our method gradually reaches consensus, enhancing the accuracy and reliability of the inference. It achieves state-of-the-art performance and strong generalization on three public benchmark datasets.
@article{11477004, title = {Bridging Discrepancies in Debate: {{Joint}} Debate for Enhancing Multimodal Emotion-Cause Pair Extraction in Conversation}, author = {Wu, Shoupu and Lin, Jiali and Wang, Xiaoyi and Jiang, Dazhi and Cambria, Erik}, year = {2026}, journal = {IEEE Transactions on Affective Computing}, pages = {1--15}, doi = {10.1109/TAFFC.2026.3681902}, keywords = {Affective computing,Antennas,Electronic mail,emotion analysis,emotion-cause pair extraction,Feeds,Filtering,Filters,Fuses,LLMs debate,LoRa,Product development,Video equipment,Videos} }
A Bayesian-Like Inference Framework Based on Causal Rules for Causal Emotion Entailment

Zhaogan Zeng, Zihuai Chen, Yuyao Chen, Xiaoyi Wang, and Dazhi Jiang

Knowledge-Based Systems, 2026

Abs DOI Bib

Causal Emotion Entailment (CEE) aims to identify the underlying causes behind the emotion expressed in non-neutral utterance within dialogues. Despite advancements in CEE, previous research has failed to adequately exploit the implicit causal rules in dialogues, thereby restricting the inferential performance of the model. Moreover, existing models lack sufficient logical support for their inferences, resulting in limited interpretability. To address this, we model the emotional and positional relationships between utterances to encode the causal rules. To enhance the reasoning ability and interpretability of the model, we conceptualize the CEE task as a Bayesian-like causal inference process, which consists of three steps: evidence extraction, prior evolution, and likelihood adjustment. Firstly, we design a causal perception module to obtain global and local causal evidence. Next, a Causal Inference Module (CIM) composed of Pre-inference (PI) and Secondary Inference (SI) is proposed. As CIM continuously absorbs causal evidence under the constraints of causal rules, the prior causal probability obtained by PI evolves into the Posterior Causal Probability (PCP) after SI. Finally, we propose a likelihood adjustment module to align the PCP with the causal relationship between utterances. The experimental results indicate that our proposed framework achieves State-Of-The-Art (SOTA) performance, yielding excellent Macro F1 scores of 83.30
@article{ZENG2026115917, title = {A {{Bayesian-Like}} Inference Framework Based on Causal Rules for Causal Emotion Entailment}, author = {Zeng, Zhaogan and Chen, Zihuai and Chen, Yuyao and Wang, Xiaoyi and Jiang, Dazhi}, year = {2026}, journal = {Knowledge-Based Systems}, volume = {342}, pages = {115917}, issn = {0950-7051}, doi = {10.1016/j.knosys.2026.115917}, keywords = {Bayesian-like inference framework,Causal emotion entailment,Causal rules,Conversational sentiment analysis} }

2025

Feel the Difference? A Comparative Analysis of Emotional Arcs in Real and LLM-generated CBT Sessions

Xiaoyi Wang, Jiwei Zhang, Guangtao Zhang, and Honglei Guo

In Findings of the Association for Computational Linguistics: EMNLP 2025, Nov 2025

Abs DOI Bib

Synthetic therapy dialogues generated by large language models (LLMs) are increasingly used in mental health NLP to simulate counseling scenarios, train models, and supplement limited real-world data. However, it remains unclear whether these synthetic conversations capture the nuanced emotional dynamics of real therapy. In this work, we introduce RealCBT, a dataset of authentic cognitive behavioral therapy (CBT) dialogues, and conduct the first comparative analysis of emotional arcs between real and LLM-generated CBT sessions. We adapt the Utterance Emotion Dynamics framework to analyze fine-grained affective trajectories across valence, arousal, and dominance dimensions. Our analysis spans both full dialogues and individual speaker roles (counselor and client), using real sessions from the RealCBT dataset and synthetic dialogues from the CACTUS dataset. We find that while synthetic dialogues are fluent and structurally coherent, they diverge from real conversations in key emotional properties: real sessions exhibit greater emotional variability, more emotion-laden language, and more authentic patterns of reactivity and regulation. Moreover, emotional arc similarity remains low across all pairings, with especially weak alignment between real and synthetic speakers. These findings underscore the limitations of current LLM-generated therapy data and highlight the importance of emotional fidelity in mental health applications. To support future research, our dataset RealCBT is released at https://gitlab.com/xiaoyi.wang/realcbt-dataset.
@inproceedings{wang2025feel, title = {Feel the Difference? {{A}} Comparative Analysis of Emotional Arcs in Real and {{LLM-generated CBT}} Sessions}, booktitle = {Findings of the Association for Computational Linguistics: {{EMNLP}} 2025}, author = {Wang, Xiaoyi and Zhang, Jiwei and Zhang, Guangtao and Guo, Honglei}, editor = {Christodoulopoulos, Christos and Chakraborty, Tanmoy and Rose, Carolyn and Peng, Violet}, year = {2025}, month = nov, pages = {19999--20017}, publisher = {Association for Computational Linguistics}, address = {Suzhou, China}, doi = {10.18653/v1/2025.findings-emnlp.1089}, isbn = {979-8-89176-335-7} }
PersonalityLens: Visualizing in-Depth Analysis for LLM-driven Personality Insights

Xiaoyi Wang, Jialong Ye, Guangtao Zhang, and Honglei Guo

Computers & Graphics, 2025

Abs DOI Bib Video

Large Language Models (LLMs) have demonstrated strong potential for text-based personality assessment and are increasingly adopted by domain experts as assistive tools. Rather than focusing on prediction accuracy, users now prioritize insight-driven analysis, using LLMs to explore large volumes of written and spoken language through simple verbal prompts. However, a gap remains between LLM-detected personality traits and users’ ability to contextualize these outputs within established psychological theories and mechanisms. Existing tools often lack support for multi-level insights and fail to capture the dynamic evolution of traits and facets over time, limiting deeper analysis. To address this, we propose PersonalityLens, a visual analysis tool designed to enhance insight discovery in personality analysis. Our design is informed by a comprehensive requirements analysis with domain experts and supports: (1) in-depth exploration of detected traits and their corresponding utterances, supporting insights at varying levels of granularity, (2) exploration of how personality traits and facets dynamically evolve in finer contexts over time, (3) alignment of traits and facets with psychological theories. We present two complementary case studies — one based on fictional TV dialogue and the other on therapeutic interactions — demonstrating PersonalityLens’s adaptability to diverse analytic goals and contexts. A qualitative think-aloud user study shows that PersonalityLens supports context-aware interpretation and insight discovery. Building on these findings, we outline design implications to inspire future research and enhance psychotherapy tools with integrated personality analysis for mental health support.
@article{wang2025PersonalityLens, title = {{{PersonalityLens}}: {{Visualizing}} in-Depth Analysis for {{LLM-driven}} Personality Insights}, author = {Wang, Xiaoyi and Ye, Jialong and Zhang, Guangtao and Guo, Honglei}, year = {2025}, journal = {Computers \& Graphics}, volume = {133}, pages = {104452}, issn = {0097-8493}, doi = {10.1016/j.cag.2025.104452}, keywords = {Dialogue data,Large language models,Mental health,Personality analysis,Personality visualization,Therapy dialogues,Visual analytics} }

2024

Multi-State Visualizations of Descriptive Statistics

Xiaoyi Wang and Kasper Hornbæk

In Proceedings of the 2024 International Conference on Advanced Visual Interfaces, 2024

Abs DOI Bib

Decision making often requires a series of low-level tasks involving descriptive statistics, each of them supported by a single visual representation, such as a bar chart or a violin plot. However, a single representation conveys limited information and little work has investigated using a combination of visual representations to facilitate the decision-making process. We propose multi-state visualizations, which allow users to switch easily between different visual representations (or states). Such visualizations provide users with more opportunities to explore and interpret the underlying data. We present three candidate multi-state visualizations, pairing error bars with violin plots, quantile dot plots, or hypothetical outcome plots. In a crowd-sourced study, we compare multi-state and single-state visualizations to investigate if they enhance users’ accuracy and confidence in making probability estimates. The results show that participants using multi-state visualization feel more confident and make more accurate estimations. Furthermore, we discuss the benefits of using multiple states in visualization for uncertainty visualizations.
@inproceedings{wang2024MultistateVisualizations, title = {Multi-State {{Visualizations}} of {{Descriptive Statistics}}}, booktitle = {Proceedings of the 2024 {{International Conference}} on {{Advanced Visual Interfaces}}}, author = {Wang, Xiaoyi and Hornbæk, Kasper}, year = {2024}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3656650.3656662}, isbn = {979-8-4007-1764-2}, keywords = {and Plots,Charts,Coordinated and Multiple Views,Diagrams,Human-Subjects Quantitative Studies,Uncertainty Visualization} }

2021

Argus: Interactive a Priori Power Analysis

Xiaoyi Wang, Alexander Eiselmayer, Wendy E Mackay, Kasper Hornbæk, and Chat Wacharamanotham

IEEE Transactions on Visualization and Computer Graphics, 2021

Abs DOI Bib Video

A key challenge HCl researchers face when designing a controlled experiment is choosing the appropriate number of participants, or sample size. A priori power analysis examines the relationships among multiple parameters, including the complexity associated with human participants, e.g., order and fatigue effects, to calculate the statistical power of a given experiment design. We created Argus, a tool that supports interactive exploration of statistical power: Researchers specify experiment design scenarios with varying confounds and effect sizes. Argus then simulates data and visualizes statistical power across these scenarios, which lets researchers interactively weigh various trade-offs and make informed decisions about sample size. We describe the design and implementation of Argus, a usage scenario designing a visualization experiment, and a think-aloud study.
@article{wang2021Argus, title = {Argus: {{Interactive}} a Priori {{Power Analysis}}}, author = {Wang, Xiaoyi and Eiselmayer, Alexander and Mackay, Wendy E and Hornbæk, Kasper and Wacharamanotham, Chat}, year = {2021}, journal = {IEEE Transactions on Visualization and Computer Graphics}, volume = {27}, number = {2}, pages = {432--442}, doi = {10.1109/TVCG.2020.3028894} }

2020

RegLine: Assisting Novices in Refining Linear Regression Models

Xiaoyi Wang, Luana Micallef, and Kasper Hornbæk

In Proceedings of the International Conference on Advanced Visual Interfaces, 2020

Abs DOI Bib Video

The process of verifying linear model assumptions and remedying associated violations is complex, even when dealing with simple linear regression. This process is not well supported by current tools and remains time-consuming, tedious, and error-prone. We present RegLine, a visual analytics tool supporting the iterative process of assumption verification and violation remedy for simple linear regression models. To identify the best possible model, RegLine helps novices perform data transformations, deal with extreme data points, analyze residuals, validate models by their assumptions, and compare and relate models visually. A qualitative user study indicates that these features of RegLine support the exploratory and refinement process of model building, even for those with little statistical expertise. These findings may guide visualization designs on how interactive visualizations can facilitate refining and validating more complex models.
@inproceedings{wang2020RegLine, title = {{{RegLine}}: {{Assisting Novices}} in {{Refining Linear Regression Models}}}, booktitle = {Proceedings of the {{International Conference}} on {{Advanced Visual Interfaces}}}, author = {Wang, Xiaoyi and Micallef, Luana and Hornbæk, Kasper}, year = {2020}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3399715.3399913}, isbn = {978-1-4503-7535-1}, keywords = {data transformation,exploratory data analysis,linear regression,model verification and remedy,residual analysis} }
Visual Exploration of Time-Series Forecasts Through Structured Navigation

Xiaoyi Wang and Kasper Hornbæk

In Proceedings of the International Conference on Advanced Visual Interfaces, 2020

Abs DOI Bib

Evaluating the forecasting ability of time-series involves observations of multiple charts representing different aspects of model accuracy. However, the sequence of the charts observed by users is not controlled and it is difficult for users to discover relations among charts. Therefore, we propose a method for constructing a navigation structure that shows these relations based on the syntax and semantics of the charts. An excerpt from the structure is used as a context menu that allows users to navigate through a series of charts and explore their relations in a structured way. A qualitative study is conducted to evaluate the system and the results show that our approach helps users explore the connections among charts and enhances the understanding of time-series forecasting performance.
@inproceedings{wang2020VisualExplorationTimeSeries, title = {Visual {{Exploration}} of {{Time-Series Forecasts Through Structured Navigation}}}, booktitle = {Proceedings of the {{International Conference}} on {{Advanced Visual Interfaces}}}, author = {Wang, Xiaoyi and Hornbæk, Kasper}, year = {2020}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3399715.3399906}, isbn = {978-1-4503-7535-1}, keywords = {model evaluation,navigation,time series} }

2019

Work Hard, Play Hard: How Linking Rewards in Games to Prior Exercise Performance Improves Motivation and Exercise Intensity

Jan David Smeddinck, Marc Herrlich, Xiaoyi Wang, Guangtao Zhang, and Rainer Malaka

Entertainment Computing, 2019

Abs DOI Bib

The concept of providing power-ups and other rewards to players in sedentary gaming sessions based on prior engagement in beneficial activities, such as exercising, has recently been explored under the term pervasive accumulated context exergames (PACE). Such games have less special requirements and may appeal to a broader audience than regular exergames. However, so far, little is known about the motivational potential and the impact on the targeted beneficial outcomes. To advance research on the potential of asynchronously linking physical exercises to games, we provide a discussion of related work and present a design space for further systematic exploration. Additionally, we present a study which indicates that linked rewards in gaming after an exercise session can lead to motivational benefits and to increased physical activity, when compared to playing a game that does not include such explicitly linked rewards after an exercise session.
@article{SMEDDINCK201920, title = {Work Hard, Play Hard: {{How}} Linking Rewards in Games to Prior Exercise Performance Improves Motivation and Exercise Intensity}, author = {Smeddinck, Jan David and Herrlich, Marc and Wang, Xiaoyi and Zhang, Guangtao and Malaka, Rainer}, year = {2019}, journal = {Entertainment Computing}, volume = {29}, pages = {20--30}, issn = {1875-9521}, doi = {10.1016/j.entcom.2018.10.001}, keywords = {Activity tracking,Exergames,Fitness,Games for health,Motivation,Player experience,Quantified self} }