01
New on-device model:
Focuses on iOS 26.4.
AI & Machine Learning / 21:13
What’s new in the Foundation Models framework
AI & Machine Learningiosipadosmacostvosvisionos
Key Points
- Covers what's new in the Foundation Models framework.
- Explains how to access Private Cloud Compute, integrate third-party and open source models, and work with vision capabilities.
- Covers context management APIs, built-in semantic search, and powerful primitives for creating agentic experiences in apps.
- The session moves through New on-device model, Vision: image understanding, Private Cloud Compute, Model abstraction layer, Partner model integrations, System tools: Vision and Spotlight, and related wrap-up guidance.
- Key concepts include Foundation Models, Private Cloud Compute, Vision framework, Evaluations framework, iOS 26.4, UIImage, NSImage, CGImage.
- Platform coverage: ios, ipados, macos, tvos, visionos.
Condensed Flow
02
Vision: image understanding:
Focuses on UIImage, NSImage, CGImage, Core Image.
03
Model abstraction layer:
Focuses on Private Cloud Compute, CoreAILanguageModel, MLXLanguageModel.
04
Partner model integrations:
Focuses on FoundationModels, Vision framework.
05
System tools: Vision and Spotlight:
Focuses on BarcodeReaderTool, OCRTool, Core Spotlight.
06
Dynamic Profiles for agentic apps:
Focuses on Dynamic Profiles, Private Cloud Compute, Foundation Models.
07
Composing models and configurations:
Focuses on LanguageModelSession, Dynamic Profiles.
08
The fm command line tool:
Focuses on Evaluations framework, fm CLI, Foundation Models.
More Detail
Additional details
- Detailed flow: New on-device model -> Vision: image understanding -> Private Cloud Compute -> Model abstraction layer -> Partner model integrations -> System tools: Vision and Spotlight -> Dynamic Profiles for agentic apps -> Composing models and configurations -> ...
- APIs and concepts to recognize: Foundation Models, Private Cloud Compute, Vision framework, Evaluations framework, iOS 26.4, UIImage, NSImage, CGImage, Core Image, CoreVideo, URLs, CoreAILanguageModel.
- Version and support notes focus on iOS 26.4.
- Implementation focus: iOS 26.4, Private Cloud Compute, SystemLanguageModel, PrivateCloudComputeLanguageModel, CoreAILanguageModel, MLXLanguageModel, Keychain, Foundation Models, Dynamic Profiles, DynamicProfile.