Google Taj Darra Oct 1, 2024

Gemini Nano Experimental Access Available on Android

Article Summary

Taj Darra just announced that Google is opening Gemini Nano to all Android developers. On-device AI just became accessible for production-ready mobile apps.

Google's Product Manager Taj Darra reveals experimental access to Gemini Nano, their most efficient on-device AI model, through the AI Edge SDK via AICore. Initially available for text-to-text prompts on Pixel 9 devices, this marks a shift from limited partner access to broad developer availability.

Key Takeaways

Critical Insight

Gemini Nano 2 nearly doubles its predecessor's size while delivering 2x quality improvements on real-world tasks, now available to all Android developers through a simple SDK integration.

The article reveals specific inference parameters that dramatically change output quality, plus code examples showing how to structure multi-string prompts for best results.

About This Article

Problem

Getting generative AI models onto mobile devices is tough because they need a lot of computing power and storage. Most developers don't have the specialized infrastructure to make this work.

Solution

Google built AICore, an Android system service that handles runtimes and models so developers don't have to ship them separately. The AI Edge SDK lets developers adjust inference settings like temperature, top K, and max output tokens to fit their needs.

Impact

Gemini Nano 2 scores 56% accuracy on MMLU benchmarks and 23% on MATH tasks, up from Nano 1's 46% and 14%. It also powers smart reply with 82% accuracy.