We’ve trained a study model to establish the foundation for future on-device language model applications. Built on a first-of-its-kind German dataset and an end-to-end pipeline we developed from scratch, the model has 2.7B parameters and was trained on 3.8T tokens across a mix of German, English, and code. While it’s not yet a fully optimized, production-grade system, it lays important groundwork for future work and industry-relevant models.
This video shows the model running directly on a smartphone chip. Mobile- or edge LLMs are still in their infancy, but they promise lower latency, improved privacy, offline capability and improved performance for specialized use cases.
The demo highlights a simple persona setting to showcase how the model can be adapted to different roles on resource-constrained hardware. It’s an early prototype, not a polished product, but it demonstrates what’s possible as mobile LLM inference rapidly evolves.
We will share more details about the training setup, conversion pipeline, and performance in upcoming articles.
Author: Alexander Schwirjow, Valentina Ciardini and Joel Schlotthauer from Fraunhofer IIS
*The audio overlay was generated using Allinga TTS by Fraunhofer IIS
Header image c
