← Back to blog

Practical On-Device AI Patterns

·1 min read
aion-devicecore-mlpcc

When building AI-powered features for iOS and macOS, the first decision is always: where does inference run?

Apple gives you three tiers:

  1. On-device — Core ML, Create ML, Foundation Models framework. Zero latency, full privacy, works offline.
  2. Private Cloud Compute (PCC) — Apple silicon in the cloud, end-to-end encrypted, no data retention.
  3. Third-party cloud — OpenAI, Anthropic, your own endpoints. Maximum capability, least privacy.

When to stay local

Stay on-device when:

  • The task is latency-sensitive (autocomplete, real-time classification)
  • The data is privacy-critical (health, finance, personal notes)
  • You need offline capability
  • The model fits in memory (most classification and NLP tasks under ~500MB)

When to escalate

Escalate to PCC or cloud when:

  • The task requires reasoning over large context (document summarization, multi-turn conversation)
  • You need generative output beyond what the on-device model supports
  • The accuracy gap between local and cloud models is material to the user experience

The hybrid pattern

The most practical pattern is a tiered approach: attempt on-device first, fall back to PCC, then to cloud. The Foundation Models framework on iOS 26+ makes this straightforward — you declare the model requirement and the system routes accordingly.

let session = LanguageModelSession()
let response = try await session.respond(to: prompt)

The key insight: design your prompts for the smallest capable model. If you need 200 words of output, don't send a prompt that assumes GPT-4-class reasoning. Constrain the task, and the on-device model often suffices.