Practical On-Device AI Patterns
·1 min read
aion-devicecore-mlpcc
When building AI-powered features for iOS and macOS, the first decision is always: where does inference run?
Apple gives you three tiers:
- On-device — Core ML, Create ML, Foundation Models framework. Zero latency, full privacy, works offline.
- Private Cloud Compute (PCC) — Apple silicon in the cloud, end-to-end encrypted, no data retention.
- Third-party cloud — OpenAI, Anthropic, your own endpoints. Maximum capability, least privacy.
When to stay local
Stay on-device when:
- The task is latency-sensitive (autocomplete, real-time classification)
- The data is privacy-critical (health, finance, personal notes)
- You need offline capability
- The model fits in memory (most classification and NLP tasks under ~500MB)
When to escalate
Escalate to PCC or cloud when:
- The task requires reasoning over large context (document summarization, multi-turn conversation)
- You need generative output beyond what the on-device model supports
- The accuracy gap between local and cloud models is material to the user experience
The hybrid pattern
The most practical pattern is a tiered approach: attempt on-device first, fall back to PCC, then to cloud. The Foundation Models framework on iOS 26+ makes this straightforward — you declare the model requirement and the system routes accordingly.
let session = LanguageModelSession()
let response = try await session.respond(to: prompt)
The key insight: design your prompts for the smallest capable model. If you need 200 words of output, don't send a prompt that assumes GPT-4-class reasoning. Constrain the task, and the on-device model often suffices.