Improving Commonsense Bias Classification by Mitigating the Influence of Demographic Terms

Jinkyu Lee, Jihie Kim

Research output: Contribution to journalArticlepeer-review

Abstract

Understanding commonsense knowledge is crucial in the field of Natural Language Processing (NLP). However, the presence of demographic terms in commonsense knowledge poses a potential risk of compromising the performance of NLP models. This study aims to investigate and propose methods for enhancing the performance and effectiveness of a commonsense polarization classifier by mitigating the influence of demographic terms. Three methods are introduced in this paper: (1) hierarchical generalization of demographic terms (2) threshold-based augmentation and (3) integration of hierarchical generalization and threshold-based augmentation methods(IHTA). The first method involves replacing demographic terms with more general ones based on a term hierarchy ontology, aiming to mitigate the influence of specific terms. To address the limited bias-related information, the second method measures the polarization of demographic terms by comparing the changes in the model's predictions when these terms are masked versus unmasked. This method augments commonsense sentences containing terms with high polarization values by replacing their predicates with synonyms generated by ChatGPT. The third method combines the two approaches, starting with threshold-based augmentation followed by hierarchical generalization. The experiments show that the first method increases the accuracy over the baseline by 2.33%, and the second one by 0.96% over standard augmentation methods. The IHTA techniques yielded an 8.82% and 9.96% higher accuracy than threshold-based and standard augmentation methods, respectively.

Original languageEnglish
Pages (from-to)161480-161489
Number of pages10
JournalIEEE Access
Volume12
DOIs
StatePublished - 2024

Keywords

  • Commonsense bias
  • bias mitigation
  • demographic term
  • hierarchical generalization
  • threshold-based augmentation

Fingerprint

Dive into the research topics of 'Improving Commonsense Bias Classification by Mitigating the Influence of Demographic Terms'. Together they form a unique fingerprint.

Cite this