Hindi Text Summarization Comparison Model using LLMs

Abstract

Text summarization is the process of condensing long documents into concise summaries while retaining their essential information and context. While this task has seen significant progress in English, it remains relatively underexplored in Indian regional languages such as Hindi due to limited resources and tools. This research focuses on abstractive summarization of Hindi news articles using advanced transformer-based Large Language Models (LLMs). We evaluate and compare the performance of three models: IndicBART, mT5, and meta llama/Llama-3.2-1B-Instruct, all fine-tuned on a curated Hindi news dataset. The models are assessed based on standard evaluation metrics including Precision, Recall, and F1-score, computed using BERT-based semantic similarity. Experimental results indicate that meta-llama/Llama-3.2-1B-Instruct significantly out performs the other two models in generating fluent, coherent, and context-aware summaries. While IndicBART demonstrates better performance than mT5, it still lags behind the LLaMA-based model. Our findings underscore the effectiveness of fine-tuning LLMs for low-resource languages and emphasize the potential of domain-specific adaptation. Additionally, we explore the challenges faced in Hindi text summarization and propose directions for future work, such as the integration of reinforcement learning techniques or hybrid summarization methods. This study contributes to the growing body of research on Indian language processing and demonstrates the viability of LLMs in enhancing the quality of Hindi text summarization.