CALL: 대규모 언어 모델 기반 온라인 정치 커뮤니티 특성 추출 및 평가 전략

초록

Online political communities are expanding as influential spaces for political discourse in modern society and influence individuals’ political understanding and decision-making. As online community opinions and values evolve in unpredictable directions through user interactions, accurately capturing and interpreting them is becoming increasingly important in research. However, previous research on online community analysis has faced limitations, often relying on labor-intensive manual analysis or rule-based approaches, both of which constrain the scalability and comprehensiveness of data analysis. To address these limitations, we propose the CALL (Community-Agent LLM-based Representation) framework, which leverages the knowledge and data processing capabilities of Large Language Model (LLM) to generate and evaluate AI agents that reflect online community characteristics. CALL filters community posts on specific topics, extracts key arguments, and injects them into LLM agents. To rigorously validate the effectiveness of the CALL framework, we evaluated it using three general-purpose foundation models (i.e., GPT, Gemini, Claude) across two dimensions (i.e., opinion consistency and value alignment). Experimental results show that CALL captures the political values of Korean online communities more effectively compared to baseline methods and demonstrates generalizability. Our study highlights the importance of feature extraction in developing LLM-based agents for online political communities and provides key considerations for adapting the framework to the unique characteristics of diverse communities.

키워드