The Depth Output

The CameraDepthFrameOutput allows streaming Depth frames in realtime, making them accessible via a JS worklet function.

Creating a Depth Frame Output

function App() {
  const device = useCameraDevice('back')
  const depthOutput = useDepthOutput({
    // ...options
    onDepth(depth) {
      'worklet'
      console.log(`Received ${depth.width}x${depth.height} Depth!`)
      depth.dispose()
    }
  })

  return (
    <Camera
      style={StyleSheet.absoluteFill}
      isActive={true}
      device={device}
      outputs={[depthOutput]}
    />
  )
}

function App() {
  const device = useCameraDevice('back')
  const depthOutput = useDepthOutput({
    // ...options
    onDepth(depth) {
      'worklet'
      console.log(`Received ${depth.width}x${depth.height} Depth!`)
      depth.dispose()
    }
  })

  const camera = useCamera({
    isActive: true,
    device: device,
    outputs: [depthOutput],
  })
}

const session = await HybridCameraFactory.createCameraSession(false)
const device = await getDefaultCameraDevice('back')
const depthOutput = HybridCameraFactory.createDepthFrameOutput({ /* options */ })
const workletRuntime = createWorkletRuntimeForThread(depthOutput.thread)
scheduleOnRuntime(workletRuntime, () => {
  'worklet'
  depthOutput.setOnDepthFrameCallback((depth) => {
    console.log(`Received ${depth.width}x${depth.height} Depth!`)
    depth.dispose()
  })
})

await session.configure([
  {
    input: device,
    outputs: [
      { output: depthOutput, mirrorMode: 'auto' }
    ],
    config: {}
  }
], {})
await session.start()

See DepthFrameOutputOptions for a full list of configuration options for the Depth Output.

Dependency Required

The CameraDepthFrameOutput requires react-native-worklets to be installed to synchronously run the onDepth(...) function on a parallel JS Worklet Runtime.

A Depth frame is a GPU-backed buffer, streamed at high resolution and frame rate. The Camera pipeline keeps a small pool to re-use buffers, and if that pool is full, the pipeline stalls and subsequent frames will be dropped. To prevent frame drops, you need to dispose a Depth frame once you are done using it via dispose():

const depthOutput = useDepthOutput({
  onDepth(depth) {
    'worklet'
    try {
      // processing...
    } finally {
      depth.dispose()
    }
  }
})

Creating a Depth Frame Output

Disposing a Depth Frame

On this page