The Hallway Track
56 tracked signals on on-device-ai.
Apple rebuilt Apple Intelligence on Google Gemini technology to power its new Foundation Models.
“Working together with Google and leveraging the technologies behind their Gemini family of models, we created the latest Apple Foundation Models for our integrated Apple Intelligence experiences.”
Apple's latest Foundation Models are now built with Google, leveraging Gemini technology to power Apple Intelligence.
“Working together with Google and leveraging the technologies behind their Gemini family of models, we created the latest Apple Foundation models to power our Apple intelligence experiences”
Sonnet 5.5 is 5x cheaper than Fable while matching frontier-level quality
“similar intelligence keeps getting possible in smaller and smaller systems”
Apple launches Core AI framework for on-device model inference across iOS and macOS.
Apple unveils integrated AI platform at WWDC26 with Xcode 27 agentic coding and new on-device frameworks.
“At Apple, AI isn't a layer you bolt on. It's integrated into our platforms.”
Apple opens its Foundation Models framework to Private Cloud Compute and third-party models like Gemini and Claude.
“This year, we're offering developers access to our Private Cloud Compute as well. So you'll have access to a very powerful server model, the one that actually powers our Apple Intelligence features, as well as access to third-party models like Gemini and Claude from Anthropic.”
Apple's new Core AI framework lets developers run advanced AI models entirely on-device in their apps.
“With Core AI, you can build app experiences where user's data never leaves their device.”
Apple opens its Foundation Models framework to nearly any LLM via a new open-source Language Model protocol.
“Anthropic and Google will soon extend the Foundation Models framework with Swift packages of their own, making state-of-the-art Claude and Gemini models available to all Swift developers.”
Apple opens Core AI, its on-device inference framework powering Apple Intelligence, to all developers.
“Core AI is the inference framework powering on-device Apple Intelligence. And now, it's available for you to use, bringing that same power to your app's own intelligence.”
Apple is open-sourcing the Foundation Models framework and adding cloud reasoning models plus third-party model access from Anthropic and Google.
“The Foundation Models framework, including many of the brand new APIs that we're announcing today, is going open source!”
Apple opens its Private Cloud Compute server LLM to third-party developers via the Foundation Models framework at WWDC26.
“And now, by changing just one line of code, you can switch to the new server model on PCC.”
Apple launches zero-cost server LLM for developers via Private Cloud Compute with no API keys
“there are no token costs to you, the developer”
Qwen 3.8B runs on consumer laptops while matching frontier model performance
“if we wait a bit, we might get frontier level systems running on our laptops”
Memory harnesses for local models can solve context rot in long-horizon research agents.
“that makes this issue of dealing with context rot a priority”
Google launched Gemini 3.5 Live Translate, a speech-to-speech model interpreting 70+ languages in near real time, now available via API.
“Developers can now build experiences that preserve speaker's tone, pitch, and pacing, resulting in more natural-sounding experiences than voice translation apps of the past.”
Google's on-device Gemma 4 models hit 5 million AI Edge Gallery downloads within one month of launch.
“Within one month of the launch, we see more than 5 million downloads”
Google's Gemma models run entirely on-device on phones, enabling multimodal AI, agent skills, and offline use.
“And what's also important to remember is that this is running entirely on the device. So, it will work offline or in areas of low connectivity.”
Apple's Core AI lets developers run cutting-edge AI models on-device for private, offline, zero-cost inference.
“Inference happens on device so data stays private, AI features can be readily available and work offline, and there is no per inference cost to you or your users.”
Apple announced new Core AI and evaluations frameworks, foundation model updates, and open-sourced Core ML models at WWDC26.
Apple's Foundation Models framework adds a language model protocol letting developers plug in third-party inference backends.
“We've got a Google package that just went live. Anthropic released theirs this morning.”
Apple's Foundation Models framework now offers frictionless frontier-model access via Private Cloud Compute, plus agentic coding in Xcode.
“the ability to now just access one of these frontier models in Private Cloud Compute with basically no setup or friction at all”
Apple's WWDC26 Foundation Models framework adds DynamicProfile APIs and an open-source Utilities package for building agentic app experiences.
“You can think of this as swapping hats, or switching agents.”
Apple is adding an fm command-line tool and Python SDK to access on-device Foundation Models at WWDC26.
“We're introducing a new command line tool called fm, and a new Foundation Models SDK for Python.”
Apple's new Evaluations framework lets developers hill-climb and align model judges to reduce drift in AI features.
“This discrepancy between model and human is known as drift, and it is a problem faced by all developers trying to evaluate intelligent features.”
Apple's new Evaluations framework lets developers measure quality of generative-AI features powered by on-device models.
“These models break a contract that is fundamental to software testing.”
Apple's MLX now supports distributed LLM inference and training across multiple Macs via RDMA over Thunderbolt 5 and the new JACCL library.
“JACCL is an open-source collective communication library built by Apple. It leverages RDMA over Thunderbolt and gives you collective communication primitives for sending data between machines and combining results across the group”
Apple's iOS 27 Siri, powered by Apple Intelligence, can access app entities, take actions via App Intents, and understand on-screen context.
“This year, Siri becomes more powerful in three key ways. Siri can now access your app's entities, the real meaningful content inside your app.”
Apple introduces Core AI, a suite for optimizing, converting, and deploying models including LLMs on Apple Silicon.
“It includes a Swift package for running LLMs in your app. But at its core, it's an open-source repository of models that are ready to go, including generative architectures like cutting-edge large language models.”
On-device AI inference on Intel Panther Lake NPUs eliminates per-token cloud cost and latency for agentic systems.
“every token has a round trip. It has latency. It is a cost item associated with it.”
Google launches Gemma 4 open model family with four sizes from 2B to 31B parameters
“Our Gemma model, here evaluated on L M Arena, are scoring as well as model 20X the size.”
Gemma 4's per-layer embeddings boost model performance without increasing compute parameters
“It's a very nice way to increase the performance of the model without actually increasing parameters because these parameters are stored on flash storage.”
Qwen 3.8 27B defaults to extreme overthinking, consuming 22K tokens for simple tasks
“This is a hilarious default. It's absolutely not a good way to run the model, especially on consumer hardware.”
Panel from NVIDIA, Unsloth, HuggingFace, and Ollama frames quantization as AI democratization at the edge.
“same cost more intelligence”
Tiny LLMs are essential to scale AI intelligence beyond expensive robots to mass-market devices
“If we want intelligence to get into lots and lots and lots of devices and not just really expensive robots, we are going to need tiny models.”
Local on-device models can replace frontier models like GPT-5 and Claude to cut inference costs, latency, and security risks.
“Every time you reach for foundation models like GPT-5 or Claude, it's costing you, your users, and the environment.”
Apple introduces a new Evaluations Framework at WWDC26 to measure model outputs, including model-judge evaluators.
“if models are judging other models, who's judging the model judges?”
Google demos Gemma running on-device on Hugging Face's open-source Reachy Mini robot for local AI interaction.
“Faster processing means quicker reactions, which is great for robots navigating the real world.”
NVIDIA launched the ACE Game Agent SDK and Unreal Engine 5 plugins for building on-device AI companions.
MLX on Apple Silicon runs the entire agentic AI loop locally on a Mac, with M5 neural accelerators making prompt processing up to 4x faster.
“What if your AI agent never needed the internet?”
Gemma 4 E2B runs fully on-device for multimodal inference on robot ducks via Raspberry Pi 5 and Jetson Orin Nano.
“And through multimodal inputs of Gemma 4, they're able to process and understand their environment like never before.”
Apple introduces SpotlightSearchTool, letting on-device LLMs search app content via Core Spotlight for grounded responses.
“today, we're introducing SpotlightSearchTool. It's a tool that adopts the tool protocol, to let a language model directly search your app's content in Core Spotlight for contextual response generation.”
Apple's Shortcuts now integrates more capable Apple Intelligence models with web access, transcript debugging, and persistent storage.
“The Use Model action lets you tap into the power of large language models, directly within your shortcuts.”
Apple's new Evaluations framework in Xcode 27 lets Swift developers test and scale agentic AI app quality.
“The quality of your evaluation results is only as good as the data behind them.”
Apple's Instruments now profiles Foundation Models apps, giving developers observability into on-device and server LLM agent loops.
“Traditional code is predictable. LLMs are non-deterministic - the same input can produce different outputs.”
Apple's WWDC26 adds image inputs to Foundation Models and a new Tap to Segment Vision API.
“this year Foundation Models is supporting image inputs”
Apple revised its Developer Program License Agreement to formalize AI/ML terms, including Foundation Models framework and access to Apple models.
Qualcomm VP argues image generation needs architectural innovation, not just scale
Nativ is a new macOS desktop app for running AI models locally via MLX
Apple's WWDC26 adds on-device AI-generated subtitles via speech transcription and translation for video playback.
“Apple AI generated subtitles can be created live locally on the device as the media plays.”
Microsoft demos building offline AI-powered Windows apps using GitHub Copilot, WinUI, and the Windows AI stack.
“whether you're bringing your own model, whether you're looking for models on Hugging Face, or use some of these primitives uh primitives that we provide as part of the Windows SDK, we're trying to make it easy for you to effectively uh run AI models on Windows.”
LLMware Model HQ runs scheduled, multi-step SLM agent workflows locally on Snapdragon NPUs using AI Foundry-optimized models.
“what if also without having to prompt you could run automated workflows that are scheduled driven”
Microsoft and NVIDIA release new tools to build on-device personal AI agents on Windows PCs.
Apple is deprecating the ImageCreator class, removing programmatic on-device image generation in OS version 27.
“the ImageCreator class is being discontinued and will no longer work in iOS 27, iPadOS 27, macOS 27, and visionOS 27 or later”
Qualcomm argues small on-device tasks like docstrings shouldn't be routed to large cloud models.
“if you want to predict the future, you have to go and create it”
On-device AI on Snapdragon can offload lightweight tasks from large cloud models to cut token costs.
“if you want to predict the future, you have to go and create it”
Google I/O skit highlights PM-engineer tension behind Android feature development
“I'm a PM and I want a notification summaries so on device AI can automatically summarize 30 plus unread group chat messages into a single haik coup.”