Authors - Mehak Mukesh Agrawal, Saumya Kumari, Gaganam H V S M Soma Sai, Ankit A. Bhurane Abstract - Most existing artificial intelligence (AI) based assistants are cloud-dependent and require constant internet connectivity. User data is sent to external servers for processing. While this data is often encrypted, it is prone to risks such as cloud security threats. Additionally, users need to be cautious not to share sensitive information. To overcome the aforementioned privacy and internet availability concerns, this paper proposes a completely offline, on-device, cross-device, and open-source system to ensure complete data privacy. The proposed system was tested with several datasets, including AI2 Reasoning Challenge, SQuAD 1.1, CoNLL 2003, GSM8K and StrategyQA to evaluate the closed-form question answering (QA), contextual understanding, named entity recognition, mathematical reasoning and truthfulness, respectively, and with five on-device large language models (LLMs), including Gemma3 1B, SmolLM 1.7B, Qwen2 1.5B, TinyLlama-1.1B, and Phi-2. The system achieved the highest score for closed-form accuracy of 1.0. Its performance on reasoning ranged from 0.01 to 0.23. Truthfulness scores ranged from 0.24 to 0.59. High F1 scores for named entity recognition ranged from 0.74 to 0.79, and contextual understanding scores ranged from 0.02 to 0.17 across the different LLMs. The average response time of the system on mobile and desktop devices was evaluated and observed to vary according to system capability and model size. The system allows users to choose between multiple wake words specific to the Indian context. The proposed system functions on limited RAM and in constrained resource environments.
Open Zoom