On-Device Generative AI APIs: ML Kit and Gemini Nano

Caren Chang, Chengji Yan, and Taj Darra from Google just made on-device AI dramatically easier for Android developers. Four new ML Kit APIs let you integrate Gemini Nano without any prompt engineering or fine-tuning.

Google released on-device GenAI APIs as part of ML Kit, bringing Gemini Nano capabilities to Android apps through high-level, ready-to-use interfaces. The APIs handle summarization, proofreading, rewriting, and image description entirely on-device, with no server costs or internet dependency.

Key Takeaways

• Summarization API jumped from 77.2 to 92.1 benchmark score with LoRA fine-tuning
• Processes 510 tokens/second input and generates 11 tokens/second output on Pixel 9 Pro
• Works offline with zero API costs and local data processing for privacy
• Envision app now summarizes documents for blind users in production

The article reveals the four-layer architecture that makes these APIs work without prompt engineering, plus specific token limits you need to know.