AI & Machine Learning / 21:43

Build agentic app experiences with the Foundation Models framework

AI & Machine Learningiosipadosmacosvisionos

Key Points

  • Explains how to take the intelligence features further with Foundation Models framework primitives for dynamic context and agentic workflows.
  • Walks through engineering shared context, setting up privacy boundaries, and managing key value caching.
  • Shows how to orchestrate smooth handoffs between local and server models.
  • The session moves through The example app and agents, Declaring a dynamic profile, Dynamic instructions, Configuring models per phase, Transcript management and history transforms, Custom modifiers, and related wrap-up guidance.
  • Key concepts include Foundation Models, DynamicProfile, LanguageModelSession, OrigamiExpert, PrivateCloudComputeLanguageModel, CraftOrchestrator, DynamicInstructions, SessionPropertyEntry.
  • Platform coverage: ios, ipados, macos, visionos.

Condensed Flow

01

The example app and agents:

Focuses on DynamicProfile, LanguageModelSession.

02

Declaring a dynamic profile:

Focuses on CraftOrchestrator, OrigamiExpert, DynamicInstructions.

03

Dynamic instructions:

Focuses on OrigamiExpert, PrivateCloudComputeLanguageModel, Foundation Models.

04

Configuring models per phase:

Focuses on DynamicProfile, LanguageModelSession.

05

Transcript management and history transforms:

Focuses on LanguageModelSession.

06

Custom modifiers:

Focuses on DynamicProfile, Foundation Models.

07

Lifecycle modifiers and session properties:

Focuses on SessionPropertyEntry, SessionPropertyValues.

08

Orchestration: phone-a-friend and skills:

Focuses on Foundation Models.

More Detail

Additional details

  • Detailed flow: The example app and agents -> Declaring a dynamic profile -> Dynamic instructions -> Configuring models per phase -> Transcript management and history transforms -> Custom modifiers -> Lifecycle modifiers and session properties -> Orchestration: baton-pass -> ...
  • APIs and concepts to recognize: Foundation Models, DynamicProfile, LanguageModelSession, OrigamiExpert, PrivateCloudComputeLanguageModel, CraftOrchestrator, DynamicInstructions, SessionPropertyEntry, SessionPropertyValues, LanguageModelsSession, Evaluations framework, Dynamic Profiles.
  • Version and support notes focus on PrivateCloudComputeLanguageModel, Foundation Models, PCCLanguageModel, DynamicProfile.
  • Implementation focus: Dynamic Profiles, DynamicProfile, LanguageModelSession, PrivateCloudComputeLanguageModel, Foundation Models, PCCLanguageModel, DynamicProfileModifier, SessionPropertyEntry, SessionPropertyValues, LanguageModelsSession.

Resources