LLM On Mobile Devices

Abstract

Large Language Models (LLMs) have revolutionized human-AI interaction but were initially restricted to cloud platforms due to high computational needs. This limited their accessibility on mobile devices, especially in offline scenarios. However, recent advancements such as Retrieval Augmented Generation (RAG) have enabled LLMs to function efficiently on resource-constrained devices. These techniques reduce model size and improve speed without compromising accuracy. Our project leverages this progress to assist sales professionals who often struggle to retrieve product information quickly from bulky PDF catalogs during customer interactions. We introduce a mobile-based AI assistant that allows users to upload multiple PDF catalogs and interact with them through a chatbot-style interface. Powered by a lightweight LLM like Qwen2-0.5B and Gemma-2B, the system delivers accurate, context-aware answers to user queries. A key feature of this solution is its dual-mode functionality. In online mode, the system can access real-time data or APIs for enhanced responses. In offline mode, all computations occur locally, ensuring privacy, low latency, and independence from internet connectivity. This is particularly beneficial for field workers operating in remote or network-restricted areas. Using model optimization and embedding caching, the assistant ensures smooth, energy-efficient performance on mobile devices. Experimental results confirm high accuracy and fast response times, making it a practical solution for real-world use. This project significantly improves sales efficiency by replacing manual searches with instant AI-powered assistance. It enhances customer interactions and empowers sales professionals with reliable, on-the-go support. Overall, this work contributes to the growing field of mobile edge AI by demonstrating how advanced LLM capabilities can be successfully brought to smartphones for practical, offline, domain-specific applications.