알림 및 행사

  • 김태균 교수 논문, "Is External Information Useful for Stance Detection with LLMs?" ACL 출판
  • 관리자
  • 2025-06-05 09:52:23
  • 13

김태균 교수의 논문, "Is External Information Useful for Stance Detection with LLMs?"이 ACL(Association for Computational Linguistics) Findings에 게재되었습니다. 관련하여 아래 내용 및 링크를 참고해 주시기 바랍니다.

Is External Information Useful for Stance Detection with LLMs? ACL (Association for Computational Linguistics) Findings, with Quang Minh Nguyen
- In the stance detection task, a text is classified as either favorable, opposing, or neutral towards a target. Prior work suggests that the use of external information, e.g., excerpts from Wikipedia, improves stance detection performance. However, whether or not such information can benefit large language models (LLMs) remains an unanswered question, despite their wide adoption in many reasoning tasks. In this study, we conduct a systematic evaluation on how Wikipedia and web search external information can affect stance detection across eight LLMs and in three datasets with 12 targets. Surprisingly, we find that such information degrades performance in most cases, with macro F1 scores dropping by up to 27.9%. We explain this through experiments showing LLMs’ tendency to align their predictions with the stance and sentiment of the provided information rather than the ground truth stance of the given text. We also find that performance degradation persists with chain-of-thought prompting, while fine-tuning mitigates but does not fully eliminate it. Our findings, in contrast to previous literature on BERT-based systems which suggests that external information enhances performance, highlight the risks of information biases in LLM-based stance classifiers.

* https://openreview.net/pdf?id=0jbnsY5ouv

첨부파일