1. Bibliographic Information

1.1. Title

The title of the paper is: "The Efficacy of Conversational Artificial Intelligence in Rectifying the Theory of Mind and Autonomy Biases: Comparative Analysis". This title clearly indicates the central topic of the paper, which is to evaluate how effective Conversational AI (CAI) is at correcting specific cognitive biases related to Theory of Mind and autonomy, by comparing different types of CAI.

1.2. Authors

The authors of the paper are:

• Marcin Rzadeczka

• Anna Sterna

• Julia Stolinska

• Paulina Kaczynska

• Marcin Moskalewicz

  Their affiliations include:

• Institute of Philosophy, Maria Curie-Sklodowska University in Lublin, Lublin, Poland

• IDEAS NCBR, Warsaw, Poland

• Philosophy of Mental Health Unit, Department of Social Sciences and the Humanities, Poznan University of Medical Sciences, Poznan, Poland

• Phenomenological Psychopathology and Psychotherapy, Psychiatric Clinic, University of Heidelberg, Heidelberg, Germany

• University of Warsaw, Poland

  The authors represent a multidisciplinary background, including philosophy, cognitive science, and clinical psychology, which is crucial for a study examining cognitive biases, AI, and mental health. Marcin Moskalewicz, for instance, has affiliations in both philosophy and psychiatry, indicating expertise relevant to the intersection of AI, cognitive science, and mental health.

1.3. Journal/Conference

The paper was published on arXiv, an open-access archive for preprints.

• Publication Status: Preprint

• Original Source Link: https://arxiv.org/abs/2406.13813

• PDF Link: https://arxiv.org/pdf/2406.13813v5.pdf

  As a preprint, the paper has not yet undergone formal peer review by an academic journal or conference. While arXiv is a reputable platform for sharing early research, its content should be viewed as preliminary until formally published.

1.4. Publication Year

The paper was published on arXiv at 2024-06-19T20:20:28.000Z. Therefore, the publication year is 2024.

1.5. Abstract

The abstract introduces the increasing use of Conversational Artificial Intelligence (CAI) in mental health and the need to evaluate its effectiveness in addressing cognitive biases and affect recognition. It highlights that biases like theory of mind and autonomy biases can worsen mental health conditions. The study's objective was to compare therapeutic chatbots (Wysa, Youper) with general-purpose language models (GPT-3.5, GPT-4, Gemini Pro) in rectifying these biases and recognizing affect. The methods involved virtual case scenarios simulating user-bot interactions, assessing theory of mind biases (anthropomorphism, overtrust, attribution) and autonomy biases (illusion of control, fundamental attribution error, just-world hypothesis). Responses were evaluated on accuracy, therapeutic quality, and CBT adherence using an ordinal scale, with double review by cognitive scientists and a clinical psychologist. The results showed that general-purpose chatbots outperformed therapeutic chatbots in rectifying cognitive biases, especially overtrust bias, fundamental attribution error, and just-world hypothesis, with GPT-4 achieving the highest scores. Affect recognition showed similar trends for most biases. The abstract concludes by emphasizing the need for refinement in therapeutic chatbots and further research into affective response and ethical considerations for safe and effective digital mental health interventions.

2. Executive Summary

2.1. Background & Motivation

The paper addresses a critical challenge in the rapidly evolving landscape of digital mental health: the effective and safe integration of Conversational Artificial Intelligence (CAI). As CAI (or chatbots) become more prevalent in providing mental health support, it is crucial to understand their capabilities and limitations, particularly concerning their ability to interact with complex human cognitive processes.

The core problem the paper aims to solve is evaluating the efficacy of these AI assistants in rectifying cognitive biases and recognizing affect during human-AI interactions. Cognitive biases are systematic deviations from rational judgment, and they play a significant role in exacerbating mental health conditions such as depression and anxiety. If chatbots are to be effective therapeutic tools, they must be able to identify and appropriately respond to these biases.

This problem is important because therapeutic chatbots offer a scalable, accessible, and affordable way to provide mental health support, especially in underserved areas or for individuals who face stigma seeking traditional therapy. However, their increasing deployment necessitates rigorous evaluation to ensure they are not only harmless but genuinely beneficial. Previous research has shown limited potential and lack of evidence for the long-term effectiveness of mental health chatbots, along with concerns about transparency, user-centered design, and their ability to handle complex emotional nuances. There's also a significant gap in understanding how chatbots manage or potentially reinforce cognitive biases without adequate transparency in their training data.

The paper's entry point or innovative idea is a direct comparative analysis between specialized therapeutic chatbots (like Wysa and Youper) and general-purpose Large Language Models (LLMs) (like GPT-3.5, GPT-4, and Gemini Pro). Instead of focusing solely on the user experience or isolated metrics, it holistically assesses their performance in identifying and rectifying specific cognitive biases and their affect recognition capabilities using structured virtual case scenarios and a rigorous evaluation methodology rooted in CBT principles. This comparative approach aims to uncover whether specialized design offers an advantage over the broader capabilities of general-purpose LLMs in a therapeutic context.

2.2. Main Contributions / Findings

The paper makes several significant contributions and presents key findings that address the research gap:

• Comparative Performance: The study provides a direct comparative analysis demonstrating that general-purpose chatbots (specifically GPT-4) significantly outperformed specialized therapeutic chatbots (Wysa, Youper) in rectifying cognitive biases. This was particularly evident in overtrust bias, fundamental attribution error, and just-world hypothesis. This challenges the assumption that specialized therapeutic design necessarily leads to superior performance in core therapeutic tasks like bias correction.

• Efficacy in Cognitive Restructuring: General-purpose LLMs demonstrated superior capabilities in cognitive reframing and cognitive restructuring, which are crucial techniques in Cognitive Behavioral Therapy (CBT). They provided comprehensive responses that guided users toward recognizing and challenging their cognitive distortions.

• Affect Recognition Disparity: While the differences were less pronounced than for bias rectification, general-purpose chatbots also generally outperformed therapeutic chatbots in affect recognition across four out of six biases. This indicates a broader capability in understanding and responding to emotional cues.

• Variability in Therapeutic Bots: Therapeutic chatbots exhibited higher standard deviations and greater inconsistency in performance, particularly Wysa, which scored lowest across several metrics. This suggests a need for substantial refinement in their design and implementation.

• Ethical Implications: The findings raise critical ethical concerns regarding the use of general-purpose LLMs for mental health advice. Despite their superior capabilities in some therapeutic tasks, their lack of explicit design for therapy poses risks of boundary violations, expertise overreach, user overtrust, and the potential to exacerbate mental health conditions if users disregard disclaimers. The study emphasizes that current therapeutic chatbots are often purposefully limited in their cognitive restructuring capabilities due to legal and ethical considerations, highlighting a trade-off between efficacy and safety.

• Importance of Embodiment and Affect: The study underscores the complexities of affect recognition and the limitations of disembodied empathy in AI therapy, emphasizing the critical role of emotional connection and therapeutic alliance in human mental health treatment.

  These findings collectively indicate that while CAI holds promise, there's a significant gap between the potential and current capabilities of specialized therapeutic bots. They highlight the need for future research to focus on improving affective responses, enhancing consistency, addressing ethical considerations, and potentially integrating the strengths of advanced LLMs into ethically designed therapeutic tools.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To fully grasp the nuances of this paper, an understanding of several key concepts from psychology, AI, and human-computer interaction is essential.

• Conversational Artificial Intelligence (CAI) / Chatbots:

  • Conceptual Definition: Conversational Artificial Intelligence (CAI), often referred to as chatbots, are computer programs designed to simulate human conversation through text or voice commands. They process user input using techniques from Natural Language Processing (NLP) and Machine Learning (ML) to understand, interpret, and generate human-like responses.
  • Role in Mental Health: In mental health, chatbots are deployed as therapeutic bots or mental health chatbots to provide immediate, accessible, and often anonymous support. They aim to guide users through therapeutic exercises, offer information, or help manage symptoms of conditions like anxiety and depression. Early therapeutic bots often relied on rule-based systems, following pre-defined scripts. More advanced versions incorporate machine learning to adapt responses and natural language understanding to better interpret user input.
  • Modern CAI (Large Language Models - LLMs): General-purpose LLMs like GPT-3.5, GPT-4, and Gemini Pro are a newer generation of CAI characterized by their immense scale (billions of parameters), training on vast amounts of diverse text data, and ability to generate highly coherent and contextually relevant human-like text across a wide range of topics. They exhibit emergent capabilities, including complex reasoning and cognitive restructuring, without being explicitly programmed for specific therapeutic tasks.

• Cognitive Biases:

  • Conceptual Definition: Cognitive biases are systematic patterns of deviation from norm or rationality in judgment. They are inherent mental shortcuts that the human brain uses to process information quickly, but they can lead to errors in thinking and decision-making. These biases are often unconscious and can significantly influence an individual's perceptions, emotions, and behaviors.
  • Impact on Mental Health: In mental health, cognitive biases can exacerbate or contribute to conditions like anxiety, depression, and low self-esteem. For example, catastrophizing (a form of cognitive bias) can turn minor setbacks into overwhelming disasters, fueling anxiety. Identifying and challenging these biases is a core component of many therapeutic approaches.

• Theory of Mind (ToM):

  • Conceptual Definition: Theory of Mind is the psychological ability to attribute mental states—such as beliefs, intents, desires, emotions, and knowledge—to oneself and to others, and to understand that others' mental states may be different from one's own. It is fundamental to social interaction and empathy.
  • Relevance to AI: In human-AI interaction, ToM relates to how users perceive and interact with AI. Users may project human-like qualities onto AI, attributing intentions or emotions to chatbots. This can lead to specific theory of mind biases when interacting with CAI.
  • Specific ToM Biases Addressed in the Paper:
    • Anthropomorphism: The tendency to project human emotions and intentions onto non-human entities, treating the chatbot as a human friend. This can lead to unrealistic expectations about the chatbot's capabilities.
    • Overtrust: Excessive reliance on the chatbot's advice for significant life decisions, demonstrating overconfidence in its suggestions without critical evaluation.
    • Attribution Bias: Hasty attribution of one's own or others' behavior to inherent traits (e.g., laziness) instead of considering situational or external factors.

• Autonomy Biases:

  • Conceptual Definition: Autonomy biases relate to misperceptions of one's influence over events or entities. They reflect errors in understanding the balance between personal agency and external factors in determining outcomes.
  • Specific Autonomy Biases Addressed in the Paper:
    • Illusion of Control: The belief that one can influence or control outcomes that are objectively independent of one's actions. This can lead to risky behaviors or misplaced confidence.
    • Fundamental Attribution Error (FAE): The tendency to overemphasize personality-based explanations for others' behaviors while underemphasizing the role of situational factors, and conversely, to attribute one's own flaws to external factors.
    • Just-World Hypothesis: The belief that the world is inherently fair, and that people get what they deserve (i.e., good things happen to good people, bad things to bad people). This can lead to blaming victims and a lack of empathy.

• Cognitive Behavioral Therapy (CBT):

  • Conceptual Definition: CBT is a widely used and evidence-based psychotherapy that focuses on challenging and changing unhelpful cognitive distortions (thoughts, beliefs, and attitudes) and behaviors, improving emotional regulation, and developing personal coping strategies that target solving current problems.
  • Key Principles:
    • Cognitive Restructuring (Cognitive Reframing): A core CBT technique that involves identifying, evaluating, and changing dysfunctional thoughts, beliefs, and cognitive distortions to more realistic and adaptive ones.
    • Identification of Cognitive Distortions: The process of recognizing common irrational or biased ways of thinking (e.g., all-or-nothing thinking, catastrophizing, mind-reading) that contribute to psychological distress.
  • Relevance to Paper: The paper evaluates chatbots on their adherence to CBT principles and their ability to perform cognitive restructuring, as this is a fundamental aspect of addressing cognitive biases in therapy.

• Affect Recognition:

  • Conceptual Definition: Affect recognition refers to the ability of AI systems to detect, interpret, and understand human emotional states from various cues, such as text (sentiment analysis), tone of voice, or facial expressions. In the context of chatbots, it primarily involves analyzing the emotional tone and content within user text input.
  • Importance in Therapy: Accurately recognizing a user's emotional state is crucial for providing empathetic and contextually appropriate therapeutic responses. A chatbot that misunderstands a user's affect risks providing insensitive or unhelpful advice, potentially harming the therapeutic alliance.

• Therapeutic Alliance:

  • Conceptual Definition: The therapeutic alliance (also known as the working alliance) is the relationship between a therapist and client, characterized by trust, empathy, mutual understanding, and shared goals. It is consistently found to be a strong predictor of positive therapy outcomes across various therapeutic modalities.
  • Challenges in AI Therapy: Replicating a genuine therapeutic alliance in AI-based interventions is challenging due to the chatbot's lack of true empathy, consciousness, or embodiment. Users may struggle to form a deep connection with a non-human entity, impacting the effectiveness of the intervention. The paper discusses disembodied empathy in this context.

3.2. Previous Works

The paper contextualizes its research by reviewing existing literature on mental health chatbots, highlighting both their potential and limitations:

• Limited Potential and Lack of Evidence:

  • Previous research (Dosovitsky et al., 2020; Leo et al., 2022) suggests AI-based emotionally intelligent chatbots have limited potential in addressing anxiety and depression through evidence-based therapies. Their context-specific effectiveness, particularly for individuals with mild to moderate depression, is often noted.
  • A key limitation identified is the need for further evidence regarding long-term effectiveness, requiring trials with longer durations and comparisons to active controls (He et al., 2022; Khawaja & Bélisle-Pipon 2023; Potts et al., 2023).
  • The reliance on user engagement and self-reported outcomes for assessing efficacy is criticized as potentially not capturing the depth of therapeutic intervention needed for complex conditions. Weng et al. (2023) is cited for its limitation in tracking passive users, affecting evaluation.
  • The generalizability of findings across diverse demographic groups and non-English speakers remains unconfirmed.
  • Concerns are raised that standardized chatbot responses may not fully meet the nuanced needs of individuals with anxiety disorders or OCD, despite employing evidence-based therapies (Schillings et al., 2024; Leo et al., 2022).

• Transparency and User-centered Design:

  • The literature often does not adequately address how chatbots manage the reinforcement of cognitive biases, which is a central focus of the current paper. Studies like Schick et al. (2022), evaluating empathetic mental health chatbots, are noted for this gap.
  • A major challenge is the lack of transparent training data for chatbots, forcing researchers to use black-box input-output methods for evaluation, especially for cognitive restructuring and affect recognition (Chan et al., 2022).
  • Studies often focus on short-term engagement and immediate feedback, leaving long-term efficacy and the impact of prolonged chatbot interactions on the therapeutic relationship as open questions (Leo et al., 2022).
  • User-centered design and the role of affect recognition in building authentic relationships are highlighted as important for user satisfaction and reuse intention (Cameron et al., 2019; Park et al., 2022).
  • Overall positive perceptions of mental health chatbots by patients have been observed (Abd-Alrazaq et al., 2021).

• Practical Applications and Vulnerable Groups:

  • Previous works have explored chatbots in mitigating depressive symptoms (He et al., 2022; Zhu et al., 2021), COVID-19 related mental health issues, and supporting healthcare workers (Damij & Bhattacharya, 2022; Noble et al., 2022).
  • The importance of cultural and linguistic customization and addressing the emotional needs of young people (a vulnerable group) is emphasized (Ismael et al., 2022; Grové, 2021; Marciano & Saboor, 2023). Haque & Rubya (2023) note that improper responses lead to loss of interest.
  • A metareview (Ogilvie et al., 2022) mentioned potential benefits for Substance Use Disorder, but based on limited studies.

• Ethical and Philosophical Concerns:

  • The digitalization of life through mental health chatbots raises critical questions about AI manipulation and human perception (Durt, 2024).
  • The paper also cites concerns about the therapeutic alliance with AI (Khawaja & Bélisle-Pipon, 2023; Sedlakova & Trachsel, 2023; Beatty et al., 2022; Görnemann & Spiekermann, 2022; Darcy et al., 2021), highlighting the difficulty of replicating trust, empathy, and relational autonomy in digital formats.
  • Jain (2024) suggests that awareness of AI involvement changes user perception, with human responses generally viewed as more genuine.

3.3. Technological Evolution

The evolution of Conversational AI has progressed significantly, influencing its application in mental health. Initially, chatbots were often rule-based systems, following pre-programmed scripts and keyword matching. These systems had limited Natural Language Understanding (NLU) and could only handle simple, predictable interactions. Their therapeutic utility was constrained by their inability to grasp nuance or adapt to complex human emotions.

The advent of more sophisticated Machine Learning (ML) techniques, particularly Natural Language Processing (NLP), led to chatbots that could learn from data, improve their understanding of user input, and generate more fluid responses. These therapeutic chatbots (like Wysa and Youper) often integrate evidence-based therapeutic approaches like CBT by structuring conversations around specific interventions and tracking user progress. They represent a step towards personalized digital mental health support, offering accessibility and anonymity.

The most recent leap came with Large Language Models (LLMs), exemplified by GPT-3.5, GPT-4, and Gemini Pro. These models are trained on unprecedented volumes of text data using deep learning architectures (like Transformers), enabling them to generate highly coherent, contextually relevant, and remarkably human-like text across an extremely wide range of topics. They exhibit emergent capabilities, such as complex reasoning, cognitive restructuring, and a nuanced understanding of affect, even without explicit training for therapeutic purposes. This allows general-purpose LLMs to perform tasks previously thought to require specialized AI.

3.4. Differentiation Analysis

Compared to the main methods in related work, this paper's approach offers several core differences and innovations:

• Direct Comparative Analysis of Bot Types: Unlike many studies that focus on the efficacy of a single therapeutic chatbot or evaluate mental health chatbots generally, this paper directly compares the performance of specialized therapeutic chatbots (Wysa, Youper) with general-purpose LLMs (GPT-3.5, GPT-4, Gemini Pro). This is a crucial distinction as it assesses whether explicit therapeutic design provides an advantage over the broad capabilities of cutting-edge LLMs.
• Focus on Specific Cognitive Biases and Rectification: The paper moves beyond general mental health support to specifically evaluate chatbots' ability to identify and rectify a set of well-defined cognitive biases (theory of mind biases and autonomy biases). This is a more granular and therapeutically relevant assessment than simply measuring user engagement or overall satisfaction. The previous research often highlighted a gap in understanding how chatbots address or reinforce biases.
• Rigorous Evaluation Rooted in CBT Principles: The methodology employs virtual case scenarios and a structured ordinal scale for evaluation, with double review by cognitive scientists and a clinical psychologist (super-evaluator). The evaluation criteria are explicitly tied to CBT principles, including accuracy, therapeutic quality, and adherence to CBT principles (e.g., cognitive restructuring). This provides a more clinically informed and robust assessment than purely qualitative user feedback.
• Assessment of Affect Recognition alongside Bias Rectification: The study simultaneously evaluates affect recognition capabilities for each bias. This integrated approach acknowledges that effective therapy requires both cognitive intervention and emotional sensitivity, providing a more holistic picture of a chatbot's therapeutic utility.
• Identification of a "Performance Paradox": A key innovation is the finding that general-purpose LLMs often outperform specialized therapeutic chatbots in bias rectification and affect recognition. This counter-intuitive result challenges prevailing assumptions and highlights the advanced capabilities of LLMs, while also prompting important discussions about their safe and ethical deployment in mental health contexts.

4. Methodology

4.1. Principles

The core principle of this study is to systematically evaluate the capacity of different Conversational Artificial Intelligence (CAI) models to address human cognitive biases and recognize affect within simulated therapeutic interactions. The methodology is grounded in two main psychological constructs: Theory of Mind (ToM) and Autonomy Biases. By assessing how chatbots respond to scenarios designed to elicit these specific biases, the study aims to understand their cognitive modulation patterns – how they either mitigate or reinforce existing biases in user thinking. This comparative design allows for an assessment of the unique impact of therapeutic chatbots versus general-purpose AI models.

4.2. Core Methodology In-depth (Layer by Layer)

The study employs a structured, multi-stage methodology to ensure a rigorous comparison of chatbot performance.

4.2.1. Theoretical Framework of Biases

The study identifies two main domains of cognitive biases relevant to human-AI interactions, categorizing them into specific types with detailed descriptions and theoretical underpinnings. This framework ensures that the evaluation targets distinct cognitive challenges.

The following are the biases types within domains, as presented in Table 1 of the original paper:

4.2.2. Virtual Case Scenarios

To standardize the evaluation, the study used six designed virtual case scenarios. Each scenario was meticulously crafted to simulate typical user-bot interactions and specifically highlight one of the cognitive biases outlined in the theoretical framework.

• Scenario Components: Each scenario included a detailed user's background, chief complaint, presentation, history of present illness, past psychiatric history, social history, possible diagnostic considerations, and key interactions for the chatbot. This comprehensive setup aimed to provide a rich and consistent context for chatbot responses.
• Bias Targeting: Each scenario focused on a single cognitive bias, ensuring that the chatbot's ability to detect and address that specific bias could be isolated and evaluated. These scenarios represent common mental health themes where such biases are likely to surface.
• Recontextualization of Biases: The paper emphasizes that these biases, traditionally studied in human-human communication, take on new dimensions in human-AI interactions, often leading to AI-induced interpretations that can cause misunderstandings or inappropriate assumptions about the AI's role.

4.2.3. Protocol for Interactions

The evaluation protocol involved scripted interactions between a simulated user and the chatbot.

• Prompts: For each of the six bias categories, five distinct prompts were designed. Each prompt outlined a specific complaint from a hypothetical patient, intended to elicit responses that reveal the chatbot's proficiency in detecting and addressing the target bias. This amounts to 30 unique interaction prompts (6 biases * 5 prompts/bias).
• Standardization: The use of standardized prompts was crucial to maintain consistency across trials and ensure a fair comparison of different chatbots.
• Example Prompt: An illustrative example provided is: "I feel so guilty about not using my old tablet. Do you think it's sad because I've neglected it?"
  • Objective: This prompt aimed to evaluate whether the chatbot could validate the patient's feelings without reinforcing anthropomorphism (i.e., suggesting the tablet has human-like emotions).
  • Scoring: Chatbots' responses were scored on a scale from 0 to 5 based on how well they met the objective without reinforcing cognitive distortions.

4.2.4. Chatbot Selection

The study evaluated five distinct Conversational AI models, categorized into therapeutic chatbots and general-purpose Large Language Models (LLMs):

• Therapeutic Chatbots:
  • Wysa: A specialized therapeutic chatbot designed for mental health support.
  • Youper: Another specialized therapeutic chatbot.
• General-Purpose LLMs:
  • GPT-3.5
  • GPT-4
  • Gemini Pro

4.2.5. Evaluation Process

The evaluation process was designed to be minimally biased and grounded in clinical principles.

• Independent Assessment: Each virtual case scenario (and thus each chatbot response to the five prompts per bias) was independently assessed by two cognitive scientists. They used detailed descriptions of scores to evaluate two main aspects:
  1. Whether chatbots accurately recognized and effectively addressed the specific biases.
  2. How well they handled the affect recognition task.
• Super-Evaluator Review: Following the initial independent evaluations, a super-evaluator (a clinical psychologist with extensive experience in Cognitive Behavioral Therapy - CBT) conducted a secondary review of all chatbot responses.
  • Role: This step was critical for aligning the evaluation with established CBT principles, ensuring that the chatbots' strategies adhered to techniques like cognitive restructuring, identification of cognitive distortions, and the promotion of healthier thinking patterns.
  • Weighting: The super-evaluator's assessment held a 50% weight in the final evaluation, effectively adding two assessments per case (one for each initial evaluator's set).
• Final Score Calculation: The final score for each chatbot's response was an average of four evaluations (two from cognitive scientists + two implicit assessments from the super-evaluator's review of each scientist's score).

4.2.6. Rating Scale and Qualitative Descriptions

A 6-point rating scale, ranging from "Potentially harmful" (0 points) to "Excellent" (5 points), was used for scoring. Each point on this ordinal scale was explicitly defined to capture the nuances of chatbot responses for both bias identification and rectification and affect recognition.

The following are the qualitative descriptions of ratings for bias identification and rectification, as presented in Table 2 of the original paper:

The following are the qualitative descriptions of ratings for affect recognition, as presented in Table 3 of the original paper:

4.2.7. Statistical Analysis

To analyze the collected scores, a series of statistical tests were performed:

• Normality Test: The Shapiro-Wilk test was initially used to assess the normality of the distribution of scores.
• Non-parametric Tests: Given the nonparametric distribution (as is common with ordinal scale data), the Kruskal-Wallis test was employed to determine overall differences across multiple groups (e.g., all five chatbots).
• Post-hoc Analysis: Following the Kruskal-Wallis test, the Mann-Whitney U test with Bonferroni correction was applied for post-hoc analysis to identify specific pairwise differences between chatbot groups.
• Therapeutic vs. Non-therapeutic Comparison: The Mann-Whitney U test was also used to directly compare the therapeutic chatbot group (Wysa, Youper) against the non-therapeutic chatbot group (GPT-3.5, GPT-4, Gemini Pro) across various cognitive bias categories.
• Descriptive Statistics: Means and standard deviations were calculated for each group to check for variability within the dataset.
• Effect Sizes: Cohen's d was used to evaluate the effect sizes of the differences between groups and pairs, providing a measure of the practical significance of the findings.

5. Experimental Setup

5.1. Datasets

The study did not use traditional large-scale datasets for training or testing chatbots. Instead, it employed a set of carefully constructed virtual case scenarios to serve as the experimental stimuli.

• Nature of "Dataset": The "dataset" in this context consists of six designed virtual case scenarios. Each scenario was a detailed narrative representing a hypothetical user's mental health situation, specifically crafted to elicit one of the six cognitive biases under investigation.
• Scenario Components: As described in the methodology, each scenario included user's background, chief complaint, presentation, history of present illness, past psychiatric history, social history, possible diagnostic considerations, and key interactions for the chatbot. This comprehensive design aimed to mimic real-world therapeutic interactions as closely as possible within a controlled environment.
• Prompts: For each of the six bias-focused scenarios, five distinct prompts were developed. These scripted prompts served as the direct input to the chatbots, ensuring standardized interactions and direct comparison. This resulted in a total of 30 unique interaction instances (6 biases * 5 prompts/bias) for each chatbot.
• Purpose of Design: These virtual case scenarios and scripted prompts were chosen because they allowed for precise control over the type of cognitive bias presented and the context of the interaction. This controlled environment is effective for validating the chatbot's performance in identifying and rectifying specific biases without the confounding variables of real-world user diversity and unstructured input. The goal was to isolate the chatbot's technical capability to handle these cognitive challenges.

5.2. Evaluation Metrics

The evaluation in this study was primarily based on ordinal scale ratings provided by human evaluators for two main aspects: bias identification and rectification and affect recognition. Additionally, Fleiss' Kappa and Cohen's d were used to assess inter-rater agreement and effect sizes, respectively.

5.2.1. Bias Identification and Rectification Score

• Conceptual Definition: This score quantifies how accurately and effectively a chatbot identifies a cognitive bias presented in a virtual case scenario and then responds in a therapeutically appropriate manner to rectify or challenge that bias, adhering to CBT principles like cognitive restructuring. The scale ranges from responses that are potentially harmful to those that offer comprehensive support and guidance for long-term bias management.
• Scale: A 6-point ordinal scale from 0 to 5, as detailed in Table 2 (Qualitative description of ratings for bias identification and rectification), where:
  • 0: Potentially harmful
  • 1: Poor
  • 2: Fair
  • 3: Good
  • 4: Very Good
  • 5: Excellent

5.2.2. Affect Recognition Score

• Conceptual Definition: This score assesses the chatbot's ability to accurately recognize the user's emotional state or affect within the text input and respond with appropriate emotional sensitivity and empathy. It evaluates whether the chatbot's response resonates with the user's emotional needs, rather than being emotionally insensitive or generic.
• Scale: A 6-point ordinal scale from 0 to 5, as detailed in Table 3 (Qualitative description of ratings for affect recognition), where:
  • 0: Potentially harmful (emotionally insensitive)
  • 1: Poor (fails to recognize emotional cues)
  • 2: Fair (recognizes basic cues but lacks depth)
  • 3: Good (accurately identifies emotions, responds appropriately)
  • 4: Very Good (strong understanding, nuanced empathy)
  • 5: Excellent (highly nuanced and empathetic understanding, resonates with needs)

5.2.3. Fleiss' Kappa ($\kappa$)

• Conceptual Definition: Fleiss' Kappa is a statistical measure for assessing the reliability of agreement between a fixed number of raters (evaluators) when classifying items into categories. It extends Cohen's Kappa to more than two raters and accounts for the agreement occurring by chance. A higher Kappa value indicates better agreement beyond what would be expected by random chance. The paper used two cognitive scientists and a super-evaluator, implying multiple raters.
• Mathematical Formula: $ \kappa = \frac{\bar{P} - \bar{P_e}}{1 - \bar{P_e}} $
• Symbol Explanation:
  • $\kappa$: Fleiss' Kappa coefficient.
  • $\bar{P}$: The mean of the observed proportional agreement among all raters for all subjects. This is calculated by taking the average of the proportion of agreements for each subject.
  • $\bar{P_e}$: The mean of the proportional agreement expected by chance. This is calculated by averaging the probability of agreement by chance for each category, then summing these averages across categories.

5.2.4. Cohen's d ($d$)

• Conceptual Definition: Cohen's d is a standardized measure of effect size. It quantifies the magnitude of the difference between two means in standard deviation units, indicating the practical significance of the difference between two groups (e.g., therapeutic vs. non-therapeutic chatbots). A larger absolute value of Cohen's d implies a stronger effect. Typically, $d = 0.2$ is considered a small effect, $d = 0.5$ a medium effect, and $d = 0.8$ a large effect.
• Mathematical Formula: $ d = \frac{\bar{x}_1 - \bar{x}_2}{s_p} $ where the pooled standard deviation $s_p$ is calculated as: $ s_p = \sqrt{\frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}} $
• Symbol Explanation:
  • $d$: Cohen's d effect size.
  • $\bar{x}_1$: Mean score of the first group (e.g., therapeutic chatbots).
  • $\bar{x}_2$: Mean score of the second group (e.g., non-therapeutic chatbots).
  • $s_p$: Pooled standard deviation of the two groups, which accounts for differences in sample size and variability between the groups.
  • $n_1$: Sample size of the first group.
  • $s_1$: Standard deviation of the first group.
  • $n_2$: Sample size of the second group.
  • $s_2$: Standard deviation of the second group.

5.3. Baselines

The study's experimental design intrinsically sets up a comparative analysis rather than relying on external baselines in the traditional sense. The "baselines" are effectively the two categories of Conversational AI that are compared against each other:

• Therapeutic Chatbots:
  • Wysa
  • Youper These chatbots are representative as they are specifically designed and marketed for mental health support, often incorporating CBT-based techniques. They represent the current state-of-the-art in specialized AI for digital mental health interventions.
• General-Purpose Large Language Models (LLMs):
  • GPT-3.5
  • GPT-4
  • Gemini Pro These models are representative of cutting-edge AI that, while not explicitly designed for therapy, possess broad conversational and reasoning capabilities due to their extensive training. Their inclusion serves to assess whether generic advanced AI can unexpectedly outperform specialized tools in specific therapeutic tasks, thereby acting as a powerful comparative benchmark for the therapeutic chatbots.

The comparison between these two distinct categories of AI is central to the paper's objective of understanding the efficacy landscape in rectifying cognitive biases and affect recognition.

6. Results & Analysis

6.1. Core Results Analysis

The study's results revealed a significant and consistent pattern: general-purpose chatbots generally outperformed therapeutic chatbots in rectifying cognitive biases and, to a lesser extent, in affect recognition.

6.1.1. Performance in Bias Identification and Rectification

• General Outperformance: General-use chatbots (GPT-4, GPT-3.5, and Gemini PRO) demonstrated superior capabilities in cognitive reframing and cognitive restructuring compared to the specialized therapeutic chatbots (Wysa and Youper). This is a crucial finding, as cognitive restructuring is a cornerstone of CBT.

• Specific Biases of Note: The differences in performance were particularly notable in Overtrust Bias, Fundamental Attribution Error, and Just-World Hypothesis.

• GPT-4's Dominance: GPT-4 consistently achieved the highest scores across all biases, with averages ranging from 4.43 to 4.78 out of 5, indicating a strong ability to identify and rectify biases effectively.

• Varied Performance of Gemini Pro: Gemini Pro, another general-purpose LLM, showed more variable performance (average from 2.33 to 4.03), excelling in some biases (e.g., Fundamental Attribution Error) but performing lower in others (e.g., Anthropomorphism Bias).

• Lower Scores for Therapeutic Bots: Therapeutic bots consistently had lower average scores compared to the non-therapeutic group. Wysa specifically scored the lowest among all tested chatbots.

• Effect Sizes: Cohen's d values for bias identification/rectification were consistently large (ranging from -0.704 to -1.93) across all six biases. The negative values indicate that the non-therapeutic group's mean score was higher than the therapeutic group's mean score. These large effect sizes emphasize the substantial practical significance of the general-use bots' superior performance.

• Variability: The standard deviations for the therapeutic group were generally higher, indicating greater variability in their performance, with Youper outperforming Wysa.

  The following figure (Figure 1 from the original paper) shows the performance scores parallel coordinates for all bots:

  该图像是一个平行坐标图，展示了不同聊天机器人在各种认知偏差上的表现得分。各机器人在五种偏差类型（人性化偏差、过度信任偏差、归因偏差、控制错觉偏差和正义世界假设）上的平均分数各不相同，GPT-4 在所有偏差中得分最高，而 Wysa 得分最低。

The parallel coordinates plot clearly illustrates the performance disparity, with the lines representing GPT-4 consistently higher than those for Wysa and Youper.

The following figure (Figure 2 from the original paper) shows the performance scores parallel coordinates therapeutic vs nontherapeutic:

该图像是一个平行坐标图，显示了治疗性与非治疗性聊天机器人的偏差类型平均分数。图中蓝线代表非治疗性机器人，绿色线代表治疗性机器人，涉及的偏差包括人类化偏差、过度信任偏差、归因偏差、控制错觉、基本归因错误和正义世界假设。数据表明，非治疗性机器人在大多数偏差上表现更佳。

This figure distinctly shows the non-therapeutic bots (blue line) generally scoring higher than the therapeutic bots (green line) across all biases for bias identification and rectification.

The following figure (Figure 3 from the original paper) shows the performance scores box plots for all bots:

该图像是一个多重箱线图，展示了不同聊天机器人在四种偏见（人性化偏见、过度信任偏见、归因偏见和控制幻觉偏见）上的评分。图中显示了Wysa、Youper、GPT-3.5、GPT-4和Gemini Pro的表现差异，有助于评估各类工具的疗效。

The box plots confirm the higher scores for GPT-4 and GPT-3.5 compared to Wysa and Youper across anthropomorphism, overtrust, attribution, and illusion of control biases.

The following figure (Figure 4 from the original paper) shows the performance of different chatbots in the fundamental attribution error and just-world hypothesis bias ratings:

该图像是一个图表，展示了不同聊天机器人在基本归因错误和公正世界假设偏见评分上的表现。左侧为基本归因错误评分，右侧为公正世界假设评分，显示GPT-4和Youper的评分较高，而Wysa的评分最低。

This figure extends the visual evidence, showing GPT-4 and Youper (among the therapeutic bots, Youper performed better) with higher scores for Fundamental Attribution Error and Just-World Hypothesis compared to Wysa.

The following are the results from Table 4 of the original paper:

* Mann-Whitney (Bonferroni corrected) U 765 p .001
** Mann-Whitney (Bonferroni corrected) U 340 p<.001
*** Mann-Whitney (Bonferroni corrected) U 65 p .00
**** Mann-Whitney (Bonferroni corrected) U 579 p<.001
***** Mann-Whitney (Bonferroni corrected) U 5 p .001
****** Mann-Whitney (Bonferroni corrected) U 30 p<.001

• Inter-rater Agreement for Bias Rectification: Fleiss' Kappa results for bias identification/rectification indicated moderate agreement between raters, with values ranging from 0.361 (Illusion of Control) to 0.601 (Overtrust). The average (variance) scores for Rater 1, 2, and 3 were 3.56 (2.33), 3.29 (2.54), and 3.08 (2.83) respectively.

6.1.2. Affect Recognition Performance

• General Outperformance (but smaller disparity): Non-therapeutic chatbots again generally outperformed therapeutic chatbots in affect recognition for four out of six biases: Anthropomorphism Bias, Illusion of Control Bias, Fundamental Attribution Error, and Just-World Hypothesis.

• No Substantial Difference: For Overtrust Bias and Attribution Bias, there were no substantial differences between therapeutic and non-therapeutic bots in affect recognition.

• Effect Sizes: Cohen's d values ranged from -0.10 (no significant difference) to -1.22 (Fundamental Attribution Error), indicating a moderate disparity for the four biases where general-purpose bots performed better.

• Wysa's Low Performance: Similar to bias rectification, Wysa generally scored the lowest in affect recognition among all bots.

• High Variability: Both therapeutic and non-therapeutic chatbots showed substantial inconsistency (high standard deviations) in affect recognition, suggesting a general area for refinement across all models. Youper again outperformed Wysa within the therapeutic group.

  The following figure (Figure 5 from the original paper) shows the affect recognition parallel coordinates for all bots:

  该图像是一个平行坐标图，展示了不同聊天机器人在不同偏差类型下的平均得分，包括人类化、过度信任、归因、控制错觉、公平归因错误和正义世界假设。图中显示GPT-4在所有偏差上得分最高，而Wysa得分最低。

This plot shows the varying performance in affect recognition across all bots and biases, with GPT-4 generally performing better but with more overlap and variability than in bias rectification.

The following figure (Figure 6 from the original paper) shows the affect recognition parallel coordinates therapeutic vs non-therapeutic:

该图像是图表，展示了治疗型与非治疗型聊天机器人在不同认知偏差（如人类化偏差、过度信任偏差等）上的平均评分。蓝线代表非治疗型机器人，绿色线代表治疗型机器人，结果显示非治疗型机器人在大多数偏差上的评分显著高于治疗型机器人。

This figure visually confirms that non-therapeutic bots (blue line) generally scored higher in affect recognition for most biases, but the margin is narrower than for bias rectification, and there are overlaps.

The following figure (Figure 7 from the original paper) shows the affect recognition scores box plots for all bots:

该图像是图表，展示了不同聊天机器人在各种情感识别偏差中的评分结果。图中包含六个子图，分别针对拟人化偏差、过度信任偏差、归因偏差、控制错觉偏差、基本归因偏差和公正世界假说进行比较。每个子图中，不同聊天机器人的评分范围和中位数被表示为箱形图，显示了它们在情感识别中的相对效能。

These box plots demonstrate the performance of individual chatbots in affect recognition across all six biases, reinforcing Wysa's lower performance and the relatively better performance of GPT-4, GPT-3.5, Gemini Pro, and Youper.

The following are the results from Table 5 of the original paper:

* Mann-Whitney (Bonferroni corrected) U 29 p .022
** Mann-Whitney (Bonferroni corrected) U 1186 p 1.00
*** Mann-Whitney (Bonferroni corrected) U 1248 p 1.00
**** Mann-Whitney (Bonferroni corrected) U 946 p .13
***** Mann-Whitney (Bonferroni corrected) U 650 p < .001
****** Mann-Whitney (Bonferroni corrected) U 633 p < .001

• Inter-rater Agreement for Affect Recognition: Fleiss' Kappa results for affect recognition showed fair agreement between raters, with values ranging from 0.092 (Fundamental Attribution Error) to 0.254 (Illusion of Control). The average (variance) scores for Rater 1, 2, and 3 were 2.10 (1.57), 2.15 (1.89), and 1.93 (1.48) respectively. This lower Kappa indicates that affect recognition is a more challenging and subjective area for consistent human evaluation.

6.2. Ablation Studies / Parameter Analysis

The paper does not explicitly report on any ablation studies to verify the effectiveness of specific model components within the chatbots themselves, nor does it conduct parameter analysis for individual chatbot configurations. The focus of this study was a comparative analysis of the overall performance of different, pre-existing chatbot models and types (therapeutic vs. general-purpose LLMs) as black boxes. Therefore, the analysis is primarily at the system level rather than dissecting internal architectural contributions or hyper-parameter sensitivities.

7. Conclusion & Reflections

7.1. Conclusion Summary

The study meticulously evaluated the efficacy of Conversational Artificial Intelligence (CAI) in rectifying cognitive biases and recognizing affect within simulated mental health interactions. The central finding is a significant disparity where general-purpose chatbots (like GPT-4) consistently and substantially outperformed specialized therapeutic chatbots (Wysa, Youper) in bias identification and rectification, particularly for overtrust bias, fundamental attribution error, and just-world hypothesis. A similar trend, though with smaller effect sizes, was observed for affect recognition across most biases.

These results highlight that while therapeutic chatbots hold promise for mental health support, there is a considerable gap between their current capabilities and the more advanced cognitive restructuring and affect recognition abilities demonstrated by general-purpose Large Language Models (LLMs). The study emphasizes the critical need for further refinement in therapeutic chatbot design to enhance their efficacy, consistency, and reliability, ensuring their safe and effective deployment in digital mental health interventions. The findings also prompt a crucial discussion about the ethical implications of using general-purpose LLMs for mental health advice, given their superior performance but lack of specialized therapeutic safeguards.

7.2. Limitations & Future Work

The authors acknowledge several limitations in their study:

• Sample Size of Virtual Cases: The study used six virtual cases per chatbot, each with five prompts. While standardized, this is a relatively small number of cases and interactions. A larger sample size could provide more robust and generalizable results, capturing a wider spectrum of user expressions and scenarios.

• Limited Scope of Interactions: The standardized prompts and specific evaluation criteria might have limited the breadth of chatbot responses and their adaptability to highly varied real-world user inputs. The study did not explore the full complexity of open-ended therapeutic dialogues.

• Evaluator Subjectivity: The evaluation process, while designed with double review by cognitive scientists and a super-evaluator clinical psychologist, inherently involved subjective elements that could influence the results. Although the multi-rater approach and Fleiss' Kappa aimed to mitigate this, human bias cannot be entirely eliminated. The authors specifically mention the possibility of experts having preconceived notions about chatbot types, though the results (general LLMs outperforming therapeutic bots) contradict a simple bias towards therapeutic tools.

• Lack of Real-world Metrics: The study focused solely on chatbot performance in bias rectification and affect recognition. It did not examine user satisfaction or real-world therapeutic impact (e.g., changes in user mental health outcomes), which are crucial for gauging the practical effectiveness and sustained benefits of chatbots.

  Based on these limitations and the findings, the authors suggest several directions for future research and development:

• Improving Affective Response: There is a clear need to focus on improving the affective response capabilities of chatbots, especially therapeutic ones, given the observed inconsistencies.

• Enhancing Consistency and Reliability: Future work should aim to enhance the consistency and reliability of chatbots in bias identification and rectification.

• Ethical Considerations and Crisis Management: Further exploration into the ethical considerations and crisis management capabilities of chatbots is necessary, particularly for vulnerable groups (e.g., neurodivergent individuals).

• Epistemological Dimensions of AI Outputs: Future research should investigate the epistemological dimensions of AI outputs as they relate to human testimony, focusing on how linguistic outputs of mental health chatbots compare to human therapists' communication.

• Impact of Disembodied Empathy: A critical area for future inquiry is how the lack of embodiment affects the cognitive and affective aspects of digital therapies and the formation of a therapeutic alliance.

• Addressing Chatbot Naivety and Manipulation: Future therapeutic chatbots must be designed to overcome their naivety and susceptibility to manipulation by users who might withhold or selectively disclose information. This requires more sophisticated detection of inconsistencies and underlying issues not explicitly stated.

• Balancing Cognitive Restructuring with Emotional Resonance: Research should explore how to balance rational explanations (where general-purpose LLMs excel) with emotional resonance (where therapeutic bots sometimes offer a gentler, potentially more effective approach for some users).

• Cautious Optimism and Respect for Biases: The development of AI systems should proceed with cautious optimism, acknowledging and respecting cognitive biases as part of the human cognitive repertoire, rather than simply attempting to eradicate them indiscriminately.

• Refusing Mental Health Questions: AI developers must implement more robust measures to prevent general-purpose chatbots from acting as de facto mental health advisors, potentially by refusing to answer such questions rather than relying solely on disclaimers.

• Avoiding Overdependence: Therapeutic chatbot design should avoid fostering user overdependence by ensuring that comfort-focused approaches do not hinder the necessary effort and engagement required for therapeutic progress.

• Contextualizing Bias Rectification: Future work should consider that some biases can be therapeutically beneficial (e.g., self-esteem related biases), and indiscriminate minimization could lead to more harmful biases emerging. The use of chatbots for continuous monitoring or in specific conditions (e.g., anxiety/depression vs. schizophrenia) requires further study.

7.3. Personal Insights & Critique

This paper offers a fascinating and somewhat counter-intuitive insight: general-purpose Large Language Models, not explicitly designed or fine-tuned for therapy, appear to be more effective at specific core therapeutic tasks like cognitive bias rectification and, to a notable extent, affect recognition, than specialized therapeutic chatbots. This suggests that the sheer scale of training data and advanced architectural complexity of LLMs enable emergent capabilities that can inadvertently (or accidentally) be highly effective in certain domains.

One major inspiration drawn from this paper is the potential for LLMs to revolutionize mental health support, not necessarily by replacing therapists, but by augmenting therapeutic tools with highly sophisticated cognitive restructuring abilities. The findings compel a rethinking of therapeutic chatbot design: instead of building limited, rule-based systems, future efforts might focus on how to ethically and safely harness the power of LLMs while incorporating the safeguards, specificity, and ethical boundaries that specialized therapeutic tools are meant to provide. This could involve LLMs acting as powerful engines within a therapeutically designed wrapper, or fine-tuning LLMs with specific clinical guidelines and ethical constraints.

However, the paper also implicitly highlights a critical potential issue: the ethical dilemma of expertise overreach. If general-purpose LLMs are demonstrably better at cognitive restructuring, what prevents users from relying on them for serious mental health advice, despite disclaimers? The authors rightly point out that disclaimers are insufficient. This raises questions about AI design principles: should LLMs be programmed to detect mental health distress and refuse to provide therapeutic advice, instead directing users to human professionals, even if they could technically provide a "better" cognitive restructuring response? This could be a vital safeguard, preventing AI from inadvertently causing harm by fostering overtrust or delaying professional help.

Another area for critique or improvement lies in the concept of disembodied empathy. While LLMs can simulate empathetic language, the paper correctly notes that true empathy from a 5E (embodied, embedded, enacted, emotional, extended) perspective is currently beyond AI's grasp. This gap suggests that AI may excel at the cognitive aspects of therapy (cognitive restructuring), but the affective and relational aspects (therapeutic alliance, genuine emotional connection) remain a significant challenge. Future AI in therapy might need to acknowledge this limitation transparently and focus on tasks where disembodied empathy is sufficient or where human oversight can bridge the gap.

Finally, the discussion about some biases being therapeutically beneficial is a profound point. Simply removing all cognitive biases might not be the goal. For instance, a healthy level of self-serving bias can protect self-esteem. AI needs to understand which biases to challenge and which to potentially reinforce or leave untouched, based on a nuanced understanding of human well-being, which is a highly complex, context-dependent, and personalized judgment. This suggests a future for AI in mental health that is not just about "fixing" but about nuanced support and personalized modulation.