Edge AI
The pattern of running ML inference on the end-user's device rather than in a centralized data center.
Edge AI describes the architectural choice to run model inference on the device that captured the data, rather than uploading the data to a centralized GPU cluster. The motivations are latency (no round-trip), cost (no per-call inference bill), privacy (data does not leave the device), and offline behavior (it works without connectivity).
Mac Note Taker is an edge-AI app by construction. ASR, diarization, speaker embeddings, and (with Ollama) LLM summaries all run on the user's Mac, on the Neural Engine where possible. The Mac is the edge node; there is no centralized inference component to scale, secure, or audit.
Related terms
- On-device ↗Processing that happens entirely on the user's hardware - no cloud, no network round-trip, no third-party data processor.
- Apple Neural Engine (ANE) ↗The dedicated ML accelerator inside every Apple Silicon chip, used to run ASR and diarization without burning the CPU or GPU.
- ANE (Apple Neural Engine) ↗Shorter name for the Apple Neural Engine - the per-chip ML accelerator that runs Mac Note Taker's transcription and diarization models.
- M-series (Apple Silicon) ↗Apple's family of ARM-based system-on-chip designs (M1 through M4) that power every modern Mac and ship with a Neural Engine.