An AI concierge for every department — without the API bill

Every team wants an assistant: support wants one that knows the macros, HR wants one that knows the handbook, sales wants one that knows the catalog. What kills most of these projects isn’t the model — it’s the meter. Cloud AI charges for every question, and a concierge that’s genuinely useful gets asked thousands of them.

There’s now a third option between “pay per token forever” and “no assistant at all”: the model your users already have.

The model inside the browser

Chrome ships Gemini Nano — a small, capable model built into the browser itself (Chrome 138+, via the Prompt API). It downloads once, runs entirely on the user’s machine, and keeps working even offline. Your web app talks to it in a few lines of JavaScript; no keys, no endpoints, no vendor account.

The economics flip

With a cloud concierge, cost scales with curiosity: more questions, bigger bill. On-device, the marginal cost of a question is zero. Every employee’s laptop brings its own compute, so the “inference bill” scales itself — a thousand users means a thousand small GPUs you didn’t have to provision. No rate limits, no quotas, no surprise invoice at month end.

Security by architecture, not by policy

The strongest data-handling clause is the one you don’t need. With Gemini Nano, prompts and context are processed on the device — per Google’s own documentation, “no data is sent to Google or any third party.” The HR question about a medical leave, the support ticket with an account number, the sales note about an unannounced deal: none of it crosses the wire. For teams handling PII or working under compliance regimes, that’s not a smaller attack surface — it’s a missing one.

Unplug it — it still works

Here’s the part that surprises people: once the model has downloaded, the network is optional. Turn on airplane mode and the assistant keeps answering. On a flight, on a train through a dead zone, in a warehouse with no coverage, at a customer site where the guest Wi-Fi never cooperates — the concierge is exactly as capable as it was at your desk. A cloud assistant becomes a spinner the moment connectivity drops; an on-device one doesn’t even notice. For anyone who travels with their work, that’s not a nice-to-have — it’s hours back.

One pattern, every department

A concierge is a system prompt plus your content. Point the same on-device model at different context and you get a different specialist: the support desk gets ticket triage, HR gets policy Q&A, operations gets runbook lookup, sales gets product answers. And it isn’t text-only — the Prompt API accepts images today (paste a screenshot into a support triage flow), with audio input emerging. Any department, more than one modality, one architecture.

Public assistants, internal assistants — same engine

The pattern faces both ways. Outward: every visitor to your website is likely a Chrome user, which means you can put a product concierge on your public site that answers questions about your products and services — plans, capabilities, “which option fits me” — straight from your own catalog and docs. Because each visitor’s browser brings its own model, the widget costs the same whether ten people ask or ten thousand. Public traffic is exactly where per-token pricing hurts most, and exactly where on-device shines.

Inward: your authenticated teams get the deeper version. Load the assistant with product internals, department playbooks, and the tribal knowledge that never makes it into the public docs. New hires stop interrupting the one person who knows; the answer to “how do we handle X” is a question away — and it never left the building to be answered.

The honest limitations

This is a Chrome story, and only a Chrome story — no Safari, no Firefox, no iOS or Android today. The hardware bar is real too: desktop-class machines with at least 22 GB free disk and either a GPU (4 GB+ VRAM) or a modern CPU with 16 GB RAM, plus a one-time model download. In practice that describes most corporate laptops — but not all of them, and none of the phones. The right design treats on-device as an upgrade, not a requirement: check availability, use Gemini Nano when it’s there, and fall back to a managed model when it isn’t. Nobody gets a broken assistant; the users who can run locally simply cost you nothing.

How we build it

Webend Creator ships this pattern out of the box: an on-device assistant component that uses Gemini Nano when the browser supports it, backed by our managed AI when it doesn’t — same interface, same conversation, your data and permissions either way.

And when static content isn’t enough, the assistant can be given hands: we provide the means for it to reach out to your APIs — including asking your own company database — so an authenticated user can get live answers, not just documented ones. That’s a security posture you opt into, not a default: the model still runs on the device, the calls go through your existing endpoints, and the data access follows the permissions the signed-in user already has. Nothing is exposed that the user couldn’t already see — the assistant just saves them the clicks.

The concierge your departments keep asking for doesn’t need a new line item to exist.

Photos via Unsplash.