Graphics & Games / 14:22

Use foveated streaming to bring immersive content to visionOS

Graphics & GamesSpatial Computingvisionos

Key Points

  • Find out how foveated streaming delivers remotely rendered scenes to Apple Vision Pro in full fidelity.
  • Covers how this framework combines native visionOS capabilities with third-party streaming technologies completely wirelessly, demonstrated using an OpenXR scene and NVIDIA CloudXR.
  • Covers the FoveatedStreaming framework, integration with the NVIDIA CloudXR SDK, and how dynamically foveated streaming provides benefits while still preserving privacy.
  • The session moves through How Foveated Streaming works, Set up the streaming endpoint, Create a visionOS receiver app, Integrate with the streaming endpoint, Measure performance, Enhance with visionOS features.
  • Key concepts include foveated streaming, OpenXR, NVIDIA, CloudXR, FoveatedStreaming, VisionOS, ARKit, SwiftUI.
  • Platform coverage: visionos.

Condensed Flow

01

How Foveated Streaming works:

Focuses on foveated streaming, OpenXR, NVIDIA, CloudXR.

02

Set up the streaming endpoint:

Focuses on OpenXR, NVIDIA, CloudXR.

03

Create a visionOS receiver app:

Focuses on SwiftUI.

04

Integrate with the streaming endpoint:

Focuses on OpenXR, NVIDIA, CloudXR.

05

Enhance with visionOS features:

Focuses on SwiftUI, OpenXR, ARKit.

More Detail

Additional details

  • Detailed flow: How Foveated Streaming works -> Set up the streaming endpoint -> Create a visionOS receiver app -> Integrate with the streaming endpoint -> Measure performance -> Enhance with visionOS features.
  • APIs and concepts to recognize: foveated streaming, OpenXR, NVIDIA, CloudXR, FoveatedStreaming, VisionOS, ARKit, SwiftUI, RealityKit, FoveatedStreamingSession, ImmersiveSpace, RealityView.
  • Implementation focus: SwiftUI, OpenXR.

Resources