Skip to main content

2024 | OriginalPaper | Buchkapitel

Untangling Emotional Threads: Hallucination Networks of Large Language Models

verfasst von : Mahsa Goodarzi, Radhakrishnan Venkatakrishnan, M. Abdullah Canbaz

Erschienen in: Complex Networks & Their Applications XII

Verlag: Springer Nature Switzerland

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Generative AI models, known for their capacity to generate intricate and realistic data, have rapidly found applications across various domains. Yet, doubts linger regarding the full scope of their hallucinatory capabilities and the reliability of their outcomes. These concerns underscore the need for rigorous analysis and validation of generative AI models. This study employs network analysis to explore the inherent characteristics of generative AI models, focusing on their deviations and disparities between generated and actual content. Using GPT3.5 and RoBERTa, we analyze tweets, vocabulary, and emotion networks from their outputs. Although network comparison demonstrated hallucination, non-classification, and instability patterns in GPT-3.5 compared to RoBERTa as a baseline, both models exhibit promise and room for improvement.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat Acheampong, F.A., Nunoo-Mensah, H., Chen, W.: Transformer models for text-based emotion detection: a review of BERT-based approaches (2021) Acheampong, F.A., Nunoo-Mensah, H., Chen, W.: Transformer models for text-based emotion detection: a review of BERT-based approaches (2021)
3.
Zurück zum Zitat Alkaissi, H., McFarlane, S.I.: Artificial hallucinations in chatgpt: implications in scientific writing. Cureus (2023) Alkaissi, H., McFarlane, S.I.: Artificial hallucinations in chatgpt: implications in scientific writing. Cureus (2023)
4.
Zurück zum Zitat Amidi, F.: Hashtags, a viral song and memes empower iran’s protesters (2022) Amidi, F.: Hashtags, a viral song and memes empower iran’s protesters (2022)
5.
Zurück zum Zitat Athaluri, S.A., Manthena, S.V. Kesapragada, V.S.R., Yarlagadda, V., Dave, T., Duddumpudi, R.T.S.: Exploring the boundaries of reality: investigating the phenomenon of artificial intelligence hallucination in scientific writing through chatgpt references (2023) Athaluri, S.A., Manthena, S.V. Kesapragada, V.S.R., Yarlagadda, V., Dave, T., Duddumpudi, R.T.S.: Exploring the boundaries of reality: investigating the phenomenon of artificial intelligence hallucination in scientific writing through chatgpt references (2023)
6.
Zurück zum Zitat Bar-Kalifa, E., Sened, H.: Using network analysis for examining interpersonal emotion dynamics. Multivariate Behav. Res. (2020) Bar-Kalifa, E., Sened, H.: Using network analysis for examining interpersonal emotion dynamics. Multivariate Behav. Res. (2020)
7.
Zurück zum Zitat Barnes, J.: Twitter ends its free API: Here’s who will be affected Barnes, J.: Twitter ends its free API: Here’s who will be affected
8.
Zurück zum Zitat Bird, S., Klein, E., Loper, E.: Natural language processing with Python. O’Reilly (2009) Bird, S., Klein, E., Loper, E.: Natural language processing with Python. O’Reilly (2009)
9.
Zurück zum Zitat Biswas, S.S.: Potential use of chat gpt in global warming. Annals Biomed. Eng. (2023) Biswas, S.S.: Potential use of chat gpt in global warming. Annals Biomed. Eng. (2023)
10.
Zurück zum Zitat Blondel, V.D., Guillaume, J.-L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks (2008) Blondel, V.D., Guillaume, J.-L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks (2008)
11.
Zurück zum Zitat Camacho-Collados, J., et al.: TweetNLP: Cutting-Edge Natural Language Processing for Social Media (2022) Camacho-Collados, J., et al.: TweetNLP: Cutting-Edge Natural Language Processing for Social Media (2022)
12.
Zurück zum Zitat Cowen, A.S., Keltner, D.: Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proc. Nat. Acad. Sci. (2017) Cowen, A.S., Keltner, D.: Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proc. Nat. Acad. Sci. (2017)
13.
Zurück zum Zitat Davis, W.: Twitter is being rebranded as x - the verge (2023) Davis, W.: Twitter is being rebranded as x - the verge (2023)
14.
Zurück zum Zitat Fuchs, K.: Exploring the opportunities and challenges of nlp models in higher education: is chat gpt a blessing or a curse? (2023) Fuchs, K.: Exploring the opportunities and challenges of nlp models in higher education: is chat gpt a blessing or a curse? (2023)
16.
Zurück zum Zitat Hasmi, L., et al.: Network approach to understanding emotion dynamics in relation to childhood trauma and genetic liability to psychopathology: replication of a prospective experience sampling analysis (2017) Hasmi, L., et al.: Network approach to understanding emotion dynamics in relation to childhood trauma and genetic liability to psychopathology: replication of a prospective experience sampling analysis (2017)
17.
Zurück zum Zitat Huang, F., Kwak, H., An, J.: Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech. arXiv:2302.07736 (2023) Huang, F., Kwak, H., An, J.: Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech. arXiv:​2302.​07736 (2023)
18.
Zurück zum Zitat Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., et al.: Assessing the accuracy and reliability of ai-generated medical responses: an evaluation of the chat-gpt model (2023) Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., et al.: Assessing the accuracy and reliability of ai-generated medical responses: an evaluation of the chat-gpt model (2023)
19.
Zurück zum Zitat Kinnison, J., Padmala, S., Choi, J-M., Pessoa, L.: Network analysis reveals increased integration during emotional and motivational processing. J. Neurosci. (2012) Kinnison, J., Padmala, S., Choi, J-M., Pessoa, L.: Network analysis reveals increased integration during emotional and motivational processing. J. Neurosci. (2012)
20.
Zurück zum Zitat Lange, J., Zickfeld, J.H.: Emotions as overlapping causal networks of emotion components: implications and methodological approaches (2021) Lange, J., Zickfeld, J.H.: Emotions as overlapping causal networks of emotion components: implications and methodological approaches (2021)
21.
Zurück zum Zitat Liang, S., et al.: The relations between emotion regulation, depression and anxiety among medical staff during the late stage of covid-19 pandemic: a network analysis. Psych. Res. (2022) Liang, S., et al.: The relations between emotion regulation, depression and anxiety among medical staff during the late stage of covid-19 pandemic: a network analysis. Psych. Res. (2022)
22.
Zurück zum Zitat Loureiro, D., et al.: Tweet insights: a visualization platform to extract temporal insights from twitter arXiv:2308.02142 (2023) Loureiro, D., et al.: Tweet insights: a visualization platform to extract temporal insights from twitter arXiv:​2308.​02142 (2023)
23.
Zurück zum Zitat Martín-Brufau, R., Suso-Ribera, C., Corbalán, J.: Emotion network analysis during covid-19 quarantine-a longitudinal study. Front. Psychol. (2020) Martín-Brufau, R., Suso-Ribera, C., Corbalán, J.: Emotion network analysis during covid-19 quarantine-a longitudinal study. Front. Psychol. (2020)
24.
Zurück zum Zitat Matsumoto, D., Keltner, D., Shiota, M.N., O’Sullivan, M., Frank, M.: Facial expressions of emotion (2008) Matsumoto, D., Keltner, D., Shiota, M.N., O’Sullivan, M., Frank, M.: Facial expressions of emotion (2008)
25.
Zurück zum Zitat Mauss, I.B., Levenson, R.W., McCarter, L., Wilhelm, F.H., Gross, J.J.: The tie that binds? coherence among emotion experience, behavior, and physiology (2005) Mauss, I.B., Levenson, R.W., McCarter, L., Wilhelm, F.H., Gross, J.J.: The tie that binds? coherence among emotion experience, behavior, and physiology (2005)
26.
Zurück zum Zitat Ouyang, L., et al.: Training language models to follow instructions with human feedback (2022) Ouyang, L., et al.: Training language models to follow instructions with human feedback (2022)
27.
Zurück zum Zitat Parrott, W.G.: Emotions in social psychology: Essential readings. Psychology Press (2001) Parrott, W.G.: Emotions in social psychology: Essential readings. Psychology Press (2001)
28.
Zurück zum Zitat Pessoa, L.: Understanding emotion with brain networks. Current Opinion Behav. Sci. (2018) Pessoa, L.: Understanding emotion with brain networks. Current Opinion Behav. Sci. (2018)
29.
Zurück zum Zitat Rudolph, J., Tan, S., Tan, S.: Chatgpt: bullshit spewer or the end of traditional assessments in higher education? J. Appl. Learn. Teach. (2023) Rudolph, J., Tan, S., Tan, S.: Chatgpt: bullshit spewer or the end of traditional assessments in higher education? J. Appl. Learn. Teach. (2023)
30.
Zurück zum Zitat Sailunaz, K., Alhajj, R.: Emotion and sentiment analysis from twitter text. J. Comput. Sci. (2019) Sailunaz, K., Alhajj, R.: Emotion and sentiment analysis from twitter text. J. Comput. Sci. (2019)
31.
Zurück zum Zitat Si, C., et al.: Prompting GPT-3 to be reliable (2023) Si, C., et al.: Prompting GPT-3 to be reliable (2023)
32.
Zurück zum Zitat Siegel, E.H., et al.: Emotion fingerprints or emotion populations? a meta-analytic investigation of autonomic features of emotion categories (2018) Siegel, E.H., et al.: Emotion fingerprints or emotion populations? a meta-analytic investigation of autonomic features of emotion categories (2018)
33.
Zurück zum Zitat Sohail, S.S., et al.: Decoding chatgpt: A taxonomy of existing research, current challenges, and possible future directions (2023) Sohail, S.S., et al.: Decoding chatgpt: A taxonomy of existing research, current challenges, and possible future directions (2023)
34.
Zurück zum Zitat Tantardini, M., Ieva, F., Tajoli, L., Piccardi, C.: Comparing methods for comparing networks. 9, 17557 (2019) Tantardini, M., Ieva, F., Tajoli, L., Piccardi, C.: Comparing methods for comparing networks. 9, 17557 (2019)
35.
Zurück zum Zitat Trampe, D., Quoidbach, J., Taquet, M.: Emotions in everyday life. PloS one (2015) Trampe, D., Quoidbach, J., Taquet, M.: Emotions in everyday life. PloS one (2015)
36.
Zurück zum Zitat Tunstall, L., von Werra, L., Wolf, T.: Natural language processing with transformers: building language applications with Hugging Face (2022) Tunstall, L., von Werra, L., Wolf, T.: Natural language processing with transformers: building language applications with Hugging Face (2022)
37.
Zurück zum Zitat Vaira, L.A., et al.: Accuracy of chatgpt-generated information on head and neck and oromaxillofacial surgery: a multicenter collaborative analysis (2023) Vaira, L.A., et al.: Accuracy of chatgpt-generated information on head and neck and oromaxillofacial surgery: a multicenter collaborative analysis (2023)
38.
Zurück zum Zitat Vaswani, A., et al.: Attention is all you need (2017) Vaswani, A., et al.: Attention is all you need (2017)
39.
Zurück zum Zitat Venkatakrishnan, R., Goodarzi, M.,Canbaz, M.A.: Use cases from the middle east, Exploring large language models’ emotion detection abilities (2023) Venkatakrishnan, R., Goodarzi, M.,Canbaz, M.A.: Use cases from the middle east, Exploring large language models’ emotion detection abilities (2023)
40.
Zurück zum Zitat Witkower, Z., Tracy, J.L.: A facial-action imposter: How head tilt influences perceptions of dominance from a neutral face (2019) Witkower, Z., Tracy, J.L.: A facial-action imposter: How head tilt influences perceptions of dominance from a neutral face (2019)
Metadaten
Titel
Untangling Emotional Threads: Hallucination Networks of Large Language Models
verfasst von
Mahsa Goodarzi
Radhakrishnan Venkatakrishnan
M. Abdullah Canbaz
Copyright-Jahr
2024
DOI
https://doi.org/10.1007/978-3-031-53468-3_17

Premium Partner