AI & Machine Learning / 22:16

Build real-time neural rendering pipelines with Metal

AI & Machine LearningGraphics & Gamesiosipadosmacos

Key Points

  • Shows how to integrate machine learning into the real-time rendering pipeline using Metal 4.
  • Covers practical adoption patterns and best practices for achieving production-quality results with MetalFX neural denoising, featuring real-world insights from Maxon Redshift Live.
  • Explains how to train and deploy a neural tone mapper using the ML command encoder inline with the graphics work.
  • Finally, dive into the new tensor API to build and evaluate small, specialized neural networks directly within the shaders.
  • The session moves through MetalFX Denoising, Deploy custom ML networks with Metal 4, Inline neural networks with tensorOps.
  • Key concepts include Metal, MetalFX, TensorOps, GPUs, MTLPackage, PyTorch, HDRNet.

Condensed Flow

01

MetalFX Denoising:

Focuses on MetalFX.

02

Deploy custom ML networks with Metal 4:

Focuses on Metal, TensorOps.

03

Inline neural networks with tensorOps:

Focuses on TensorOps.

More Detail

Additional details

  • Detailed flow: MetalFX Denoising -> Deploy custom ML networks with Metal 4 -> Inline neural networks with tensorOps.
  • APIs and concepts to recognize: Metal, MetalFX, TensorOps, GPUs, MTLPackage, PyTorch, HDRNet.
  • Implementation focus: TensorOps, GPUs, MetalFX, HDRNet, MTLPackage.

Resources