A Review of Human-Centric Evaluation of Cultural Biases in Indic Languages in LLMs: Rethinking Research Directions

초록

Indic languages, spoken by more than 1.3 billion people in South Asia, are often underrepresented in AI systems due to their diversity, limited resources, and evaluation challenges. This region, known for its many coexisting developed and endangered cultures, faces significant cultural bias risks as these cultures are not equally represented. These biases can significantly impact the design, development, and deployment of AI in real-life scenarios. Our survey explores cultural bias in Indic languages from a human-centered perspective, combining computational social science and NLP. We interpret different types of bias, critically review different evaluation and mitigation works, and identify research gaps and priorities to focus on to improve AI systems. Based on our review findings, we argue that we need to rethink and reprioritize our research efforts to support other Indic languages except Hindi, and focus on different evaluation approaches for cultural biases. We discuss human-centered perspectives to enhance LLMs, focusing on how they can operate more effectively and equitably within South Asia’s diverse linguistic and rich cultural landscape, thereby creating fairer and more inclusive AI systems for real-world applications.

키워드