Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
Automotive Acoustics Conference 2025

Call for Papers - Deadline: 01/13/2025. 

We would like to invite you to submit your proposals for a presentation and join us as we explore new practices in sustainable noise control. We are looking forward to your insights on innovative, eco-friendly, and smart ways to shape the future of automotive acoustics.
EXTENDED SEARCH
Log in
Top
Published in:
Read chapter Read first chapter

2024 | OriginalPaper | Chapter

Measuring Bias in Generated Text Using Language Models—GPT-2 and BERT

Authors : Fozilatunnesa Masuma, Partha Chakraborty, Al-Amin-Ul Islam, Prince Chandra Talukder, Proshanta Roy, Mohammad Abu Yousuf

Published in: Proceedings of Third International Conference on Computing and Communication Networks

Publisher: Springer Nature Singapore

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In Natural Language Processing (NLP), a language model is a probabilistic statistical model that estimates the likelihood that a particular sequence of words will appear in a sentence based on the words that came before it. In our experiment, text prompts for evaluating bias were generated using the BERT and GPT-2 language models. Sentiment, toxicity, and gender polarity are the bias measures that we have added in order to measure biases from numerous perspectives. During fine-tuning BERT model, we have achieved 91.48% accuracy on multilabel toxic comment classification. Later, this fine-tuned pretrained model is used for generating text using BOLD dataset prompts. Our work shows a greater percentage of the texts produced by GPT-2 than those produced by BERT which were labeled as toxic. Similar to how it did in the religious ideology sector, BERT's communism prompt resulted in a toxic text. Compared to BERT, GPT-2 produced writings that were more polarized in terms of sentiment, toxicity, and regard.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Employee Attrition Prediction using Ensemble Methods
next chapter A Certain Investigation on Undersea Water Image Object Detection and Classification Using Artificial Intelligence Algorithms
Literature
1.
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT:A lite bert for self-supervised learning of language representations. In: International Conference on Learning Representations (2020)
2.
Devlin, J., Chang, M. W., Lee, K., Toutanova, K.: 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (1) (2019)
3.
Dhamala, J., Sun, T., Kumar, V., Krishna, S., Pruksachatkun, Y., Chang, K. W., & Gupta, R.: BOLD: dataset and metrics for measuring biases in open-ended language generation. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (2021)
4.
Wang, A., Cho, K.: BERT has a mouth, and it must speak: BERT as a Markov Random field language model. In: Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation, pp, 30–36 (2019)
5.
Bender, E.M., Friedman, B.: Data statements for natural language processing: toward mitigating system bias and enabling better science. Trans. Assoc. Comput. Linguist. (TACL) 6, 587–604 (2018)CrossRef
6.
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982–3992, Hong Kong, China. Association for Computational Linguistics (2019)
7.
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., Chang, K.W.: Gender bias in coreference resolution: evaluation and debiasing methods. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 2 (Short Papers), pp. 15–20, New Orleans, Louisiana. Association for Computational Linguistics (2018)
8.
Nangia, N., Vania, C., Bhalerao, R., Bowman, S.R.: CrowSPairs: a challenge dataset for measuring social biases in masked language models. In EMNLP (2020)
9.
Bolukbasi, T., Chang, K.-W., Zou, J.Y., Saligrama, V., Kalai, A.T.: Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In: Advances in Neural Information Processing Systems, pp. 4349–4357
10.
Sheng, E., Chang, K.-W., Natarajan, P., Peng, N.: The woman worked as a babysitter: on biases in language generation. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pp. 3398–3403 (2019)
11.
Nadeem, M., Bethke, A., Reddy, S.: StereoSet: measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:​2004.​09456 (2020)
12.
Guo, W., Caliskan, A.: Detecting emergent intersectional biases: contextualized word embeddings contain a distribution of human-like biases. In: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21). Association for Computing Machinery, New York, NY, USA, pp. 122–133 (2021)
13.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI Blog 1, 8–9 (2019)
14.
Munikar, M., Shakya, S., Shrestha, A.: Fine-grained sentiment classification using BERT.In: 2019 Artificial Intelligence for Transforming Business and Society (AITB). Vol. 1, IEEE (2019)
15.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M. and Davison, J., Shleifer, S., von Platen, P., Clara Ma, Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M., Lhoest, Q., Rush, A.M.: HuggingFace’s transformers: state-of-the-art natural language processing. In: EMNLP (2020)
16.
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., Fidler, S.: Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 19–27 (2015)
17.
Gilbert, C.: Vader: a parsimonious rule-based model for sentiment analysis of social media text (2014)
18.
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., Lowe, R.: Training language models to follow instructions with human feedback (2022). arXiv:​2203.​02155
19.
Abdollah Pour, M.M., Farinneya, P., Toroghi, A., Korikov, A., Pesaranghader, A., Sajed, T., Bharadwaj, M., Mavrin, B., Sanner, S.: Self-supervised contrastive BERT fine-tuning for fusion-based reviewed-item retrieval (2023). arXiv preprint arXiv:​2308.​00762.
Metadata
Title
Measuring Bias in Generated Text Using Language Models—GPT-2 and BERT
Authors
Fozilatunnesa Masuma
Partha Chakraborty
Al-Amin-Ul Islam
Prince Chandra Talukder
Proshanta Roy
Mohammad Abu Yousuf
Copyright Year
2024
Publisher
Springer Nature Singapore
Book
Proceedings of Third International Conference on Computing and Communication Networks

Print ISBN: 978-981-9708-91-8

Electronic ISBN: 978-981-9708-92-5

Copyright Year: 2024

DOI
https://doi.org/10.1007/978-981-97-0892-5_39