Edge AI: Small Language Models on the Go

Small Language Models on the Go: The proliferation of edge devices, ranging from smartphones and various wearable devices, up to industrial sensors or autonomous vehicles, gives  an opportunity to leverage the power of AI-based methods directly at the point where data is acquired or generated. However, deploying traditional Large Language Models (LLMs) on resource-constrained edge devices becomes impractical due to substantial computational and memory requirements which are usually needed. In this contribution, the rapidly evolving  field of Edge AI is explored, specifically focusing on the development and deployment of Small Language Models (SLMs), optimized for edge environments. The various challenges and opportunities associated with SLMs are indicated, together with a review of the current techniques for model compression and optimization. An outline of future research and development is also given.