Chrome is adding `window​.ai` — a Gemini Nano AI model right inside your browser.

AI Services and Gemini Nano

When I first heard about AI services being costly, images of companies hauling wheelbarrows of cash sprang to mind. It turns out, that's not too far from the truth. AI services like ChatGPT are indeed burning holes in many company wallets due to the massive GPU servers they need. Picture this: AI companies are pouring cash ten times faster into these services than they're scooping from customers. Ouch.

Running these mega AI models locally? Forget it. Not unless you enjoy your computer doubling as a space heater. Enter the hero of our tale: Gemini Nano.

Google is shaking up the game by introducing a spiffy new API to Chrome called window.ai. This API is essentially a Gemini Nano AI model living rent-free right inside your browser. This means developers can now run AI models directly within the browser, sparing those precious server expenses and possibly saving a few CPU fans too.

Why Run AI Models in the Browser?

Latency and privacy – the dynamic duo of dilemmas when it comes to running AI models on servers. Every millisecond counts, and so does data security. Running AI models locally slashes latency since there's no need for data to play fetch with a distant server. Your AI-driven web applications just got a turbo boost.

Privacy? Checked. No more sending sensitive data on cross-country trips. Everything stays right on the user's device.

Now, let’s manage those expectations. Don't expect your browser to turn into a quantum AI powerhouse. Gemini Nano is trim and streamlined – not as beefy as ChatGPT's large models but definitely robust enough for most tasks. Think of it as the scrappy, efficient sidekick rather than the hulking, resource-hungry superhero.

How to Use It

Step 1: Update Chrome

Ensure your Chrome is version 127 or higher. Time for a quick pit stop.

Step 2: Enable Experimental Flags

1. Navigate to chrome://flags.
2. Enable chrome://flags/#prompt-api-for-gemini-nano.
3. Also enable chrome://flags/#optimization-guide-on-device-model, specifically under BypassPrefRequirement.
4. Visit chrome://components and click Check for Update on Optimization Guide On Device Model to download the needed model. If you don't see the Optimization Guide, double-check those flags, relaunch Chrome, and try again.

Step 3: Witness the Magic

Visit this demo site to see the model strut its stuff or have fun in the console:

How It Works

The snippet below shows how to stream text with Chrome AI and the Vercel AI SDK. Yes, it's that easy.

Code.ts
import { streamText } from "ai";
import { chromeai } from "chrome-ai";

const { textStream } = await streamText({
  model: chromeai(),
  prompt: "What is a large language model?",
});

Behind the curtains, Chrome-AI implements a Provider that slyly leverages window.ai.

Conclusion

Developers, rejoice! Chrome's new window.ai API marks a turning point, injecting AI models directly into your browser. Say hello to faster, more responsive, and privacy-friendly AI-powered applications. The web development playground just got a whole lot more exciting – and I, for one, am eager to see the next blockbuster creations you cook up.

Happy coding and may your browsers be ever AI-empowered! 🚀