---
title: "On-Device AI: What Phones Really Run Offline and Privately"
url: https://syntaxradar.com/radar/on-device-phone-ai-local-models-apple-android-privacy
author: "Abdul Rehman, Lead Systems Architect and Technical Director"
publishedAt: 2026-10-25T08:30:00.000Z
dateModified: 2026-10-29T10:33:58.900Z
category: "#HARDWARE-CHIPS"
topic: "on device ai"
primarySource: https://machinelearning.apple.com/research/apple-foundation-models-tech-report-2025
summary: "On-device AI runs small models on your phone. See what Apple and Google say works offline, what still goes to the cloud, and how private it really is."
---


# On-Device AI: What Phones Really Run Offline and Privately

**Short answer:** Apple and Google both ship small AI models that run on the phone itself. Each company documents offline use for supported tasks, and each also keeps a cloud route for requests the local model cannot handle.

**Takeaway:** On-device AI keeps short tasks local, but both Apple and Google keep a cloud path for bigger requests, so privacy depends on which path a request takes.

**Key figures (as reported by the source):**
- Apple on-device model size: ~3B parameters
- Apple on-device quantization: 2-bit quantization-aware training
- Gemini Nano first shipped on: Pixel 8 Pro, December 2023

On-device AI means the model that answers your request runs on the phone in your hand instead of on a company's server. Apple and Google both ship small language models built this way, and both describe them as working without a network connection. Neither company says everything stays local.

### What is on-device AI?

On-device AI is a language model stored on your phone and run by your phone's own chips. The prompt (your text, audio or photo) is processed locally, and the answer is produced there too. No round trip to a data center is needed for that step. Apple says its model is optimized for Apple silicon, and Google says AICore uses device hardware to keep latency low.

The two big examples are Apple's on-device foundation model and Google's Gemini Nano. In its 2025 technical report ([arXiv](https://arxiv.org/abs/2507.13575)), Apple describes "a ~3B-parameter on-device model optimized for Apple silicon." A parameter is one learned number inside the model, so 3B means roughly three billion of them. Cloud models are generally far larger, which is why phone models are called small.

Google describes Gemini Nano as its on-device foundation model. According to [Android's developer documentation](https://developer.android.com/ai/gemini-nano), it lets apps "deliver rich generative AI experiences without needing a network connection or sending data to the cloud."

### How does a phone fit an AI model in its memory?

Apple's report names two techniques. The first is quantization-aware training at 2 bits. Quantization stores each model number with fewer bits, which shrinks the file and speeds up the math, and "aware training" means the model is trained while accounting for that rounding so quality holds up. The second is KV-cache sharing. The KV cache is the short-term memory a model builds while reading your prompt, and sharing it between parts of the model saves memory ([Apple Machine Learning Research](https://machinelearning.apple.com/research/apple-foundation-models-tech-report-2025)).

Google's side runs through AICore, an Android system service. The documentation says it "leverages device hardware to enable low inference latency and keeps the model up-to-date." Inference means the model producing an answer from a prompt.

Both companies tune for dedicated phone silicon, often called a neural engine or NPU, which is a chip block built for the matrix math AI uses. Google's page does not list supported devices, so check your phone's settings or spec sheet.

For the chip side of the story, see our explainer on [on-device NPUs and edge computing](/radar/next-gen-silicon-edge-computing-on-device-npus).

![A person holding a smartphone on a train platform, reading a short summary on the screen](https://images.unsplash.com/photo-1550751827-4bd374c3f58b?w=1600&auto=format&fit=crop&q=80)

### What can on-device AI actually do on a phone?

Google lists the tasks its ML Kit GenAI APIs cover with Gemini Nano: summarizing articles or conversations as a bulleted list, proofreading short chat messages, rewriting short messages in different tones, describing an image in a short sentence, transcribing spoken audio, and answering a custom text or multimodal prompt.

Apple's report adds that its models support more languages, understand images and can call tools. It also describes a Swift framework that gives app developers access to the on-device model, with guided generation and tool calling.

The pattern is short, bounded jobs: a short text, a single image or a short audio clip handled on the spot, not a long research session. Google's own documentation points to the limits: Gemini Nano has "on-device constraints such as token limits, device hardware limitations, and model capability boundaries," and "inference speed depends on device hardware." A token is a chunk of text, roughly a short word or word fragment, so a token limit caps how much text the model can handle at once.

Apple reports that both of its models match or beat comparably sized open models in public benchmarks and human evaluations. That comparison is against similar small models, so it says little about how a 3B model stacks up against a large cloud assistant.

### Is on-device AI private?

Running a prompt locally removes one exposure: the prompt does not have to travel to a server. Google's documentation says on-device generative AI "executes prompts locally, eliminating server calls."

In plain terms, local processing keeps the prompt on the phone for that step, and Google says AICore keeps no record of inputs or outputs afterward. Google adds architectural details. AICore does not have direct internet access, and its network requests, including model downloads, go through the open-source Private Compute Services companion app. The page also says AICore "is built to isolate each request and doesn't store any record of the input data or the resulting outputs after processing them."

On-device AI covers the model run, but a third-party app may still handle your text before or after it calls the model. Apple's and Google's pages describe only their own system layers.

### Does on-device AI work without internet?

For the tasks the local model handles, yes. Google says Gemini Nano works "without needing a network connection," and Apple's model is built to run on the device.

When a request is too big for the local model, Apple says its Private Cloud Compute (PCC) system handles requests Apple Intelligence cannot complete on the device, specifically more complex tasks that need larger foundation models. In its [security blog post on PCC](https://security.apple.com/blog/private-cloud-compute/), Apple says the user's device encrypts the request only to PCC servers it has verified, and that "no user data is retained in any form after the response is returned."

[MIT Technology Review's 2024 coverage](https://www.technologyreview.com/2024/06/11/1093577/apple-is-promising-personalized-ai-in-a-private-cloud-heres-how-that-will-work/) relays Apple's account that the system tries each task on the device first. The same article reports security experts saying any data that leaves the device becomes more exposed, that Apple's promises stay unproven until outside researchers verify them, and that it was unclear whether users would be told when a request goes to the cloud.

Android has its own version. Google's [hybrid inference page](https://developer.android.com/ai/hybrid) describes developer modes, including one that prefers the on-device model and falls back to a cloud model "if the on-device model is unavailable or unsupported for the request," and one that throws an error instead of using the cloud. Which mode an app picks is the developer's decision.

![A traveler using a phone with no signal bars in a quiet airport lounge](https://images.unsplash.com/photo-1550751827-4bd374c3f58b?w=1600&auto=format&fit=crop&q=80)

### Which phones have on-device AI?

Gemini Nano began on the Pixel 8 Pro in December 2023, [9to5Google reported](https://9to5google.com/2023/12/06/pixel-8-pro-gemini-nano/), powering Summarize in Recorder and Gboard Smart Reply.

Support depends on the chip, the operating system version and, per Google, device hardware in general. The practical way to check is the maker's current feature page for your exact model. When Gemini Nano is unavailable on a device, apps can route requests to the cloud through hybrid inference, per Google's documentation, so an app that "has AI" may be running it remotely on a phone that cannot host the model.

### What to watch next

- Independent checks. Apple said it would let researchers inspect PCC software. Published findings from outside security teams would turn marketing claims into evidence.
- Clear labels. Whether phones tell you when a request leaves the device is still the user-facing question MIT Technology Review raised in 2024.
- Model size creep. Apple's 3B model is small by design. If phone memory grows, the local share of work could grow with it, and cloud fallbacks could shrink.
- App behavior. App-level privacy labels and developer choices on fallback modes deserve attention, since the app around the model sets its own rules.

For a related look at models you can download and run yourself, see our explainer on [open-weight models](/radar/open-weight-models-explained-ibm).

## Frequently asked questions

### What is on-device AI?
On-device AI is a language or vision model that runs on your phone's own chips instead of a remote server. Apple describes a roughly 3-billion-parameter model optimized for Apple silicon, and Google describes Gemini Nano running in Android's AICore service. Both companies say it can work without a network connection for supported tasks.

### Is on-device AI more private than cloud AI?
It removes the step of sending the prompt to a server, which Google's documentation says eliminates server calls for local requests. That claim comes from the companies themselves. Privacy also depends on the app using the model and on whether a request falls back to the cloud.

### Can on-device AI work offline?
For tasks the local model can handle, yes. Google says Gemini Nano works without a network connection, and Apple's model runs on the device. Larger requests may be sent elsewhere: Apple routes tasks it cannot finish locally to Private Cloud Compute, and Android developers can choose a cloud fallback.

### What are the limits of on-device AI?
Google's documentation lists token limits, device hardware limits and model capability boundaries, and notes that speed depends on the device. Local models suit short jobs such as summarizing, proofreading, rewriting and transcription. Complex requests may need a larger cloud model, depending on the app and the phone.

## Source
Based on Apple's 2025 tech report and Android docs. https://machinelearning.apple.com/research/apple-foundation-models-tech-report-2025

---
Published on SyntaxRadar (https://syntaxradar.com). Canonical URL: https://syntaxradar.com/radar/on-device-phone-ai-local-models-apple-android-privacy. Last modified 2026-10-29T10:33:58.900Z.