Why does my OpenXR hand tracking visibly lag behind the controller pose?

OpenXR predicts a display time for each frame and asks the runtime for poses at that time. If your hand tracker updates poses in _physics_process (typically 60 Hz) while controllers update in _process (display rate, 72–120 Hz), the hand skeleton visibly trails the controller. Moving hand skeleton updates into _process fixes this for most headsets.

Does OpenXR runtime setting quality affect hand tracking latency?

Yes. Some runtimes (notably Meta) expose a Hand Tracking Fidelity or frequency setting. Low fidelity updates joints at a lower rate to save CPU but introduces lag. Setting the hand tracking to High or enabling high-frequency hand tracking reduces the perceived delay.

Should I smooth the hand joints to hide latency?

Smoothing makes lag worse by adding its own delay. Instead, reduce latency at the source: query poses at the correct display time, run updates in the frame hook (_process), and use the predicted display time that OpenXR provides. Smoothing should be reserved for jitter, not lag.

Fix: Godot OpenXR Hand Tracking Lag

Quick answer: Hand tracking lag in Godot OpenXR is almost always a timing mismatch. The XR runtime provides poses for the predicted display time; if you read them in _physics_process at 60 Hz and apply them to a skeleton used at 90 Hz, you will always be one frame behind. Move hand updates to _process, enable high-fidelity tracking in the runtime, and leave the joint transforms alone after Godot’s OpenXRHand node writes them.

You boot your Godot 4 VR prototype. Hold up a controller — rock steady. Put the controllers down and use hand tracking — fingers visibly lag about 30–80 ms behind your actual hand. Recording a video shows the delay is not perception; it really is one display frame or more. This is a fixable problem but it takes understanding how OpenXR delivers poses.

The Symptom

With controllers, Godot nodes driven by XRController3D track perfectly — no visible latency beyond the inherent display pipeline. When you switch to hand tracking (using OpenXRHand node or directly reading hand joints via XRServer.get_tracker(”/user/hand_tracker/left”)), the wrist follows your real hand but with a noticeable lag. Fast gestures — finger snaps, quick opens and closes — feel rubbery.

In extreme cases the user sees “double hands” when moving fast: their proprioception says the hand is in one place, their eyes see the virtual hand arrive 50 ms later.

What Causes This

1. Update in physics, render in process. Godot runs _physics_process at a fixed 60 Hz by default. The display runs at 72, 90, or 120 Hz on modern headsets. If your hand skeleton is updated in physics, 30–60% of frames show stale joint data.

2. Pose queried for the wrong time. OpenXR poses are time-specific. Asking for “the current hand pose” returns whatever the runtime last tracked — typically 5–15 ms old. Asking for “the pose at predicted display time” returns an extrapolated pose aligned with the frame about to render. The latter is what Godot’s XR trackers should deliver, but custom code often gets this wrong.

3. Low runtime fidelity. Runtime-specific settings dictate how often the hand tracker publishes updates. Meta’s default on battery-saving mode is 30 Hz. Some SteamVR OpenXR runtimes idle at half-rate.

4. Skeleton apply happens after view matrix is captured. If you manually set joint transforms late in the frame, after the renderer has already laid out draw commands for the view, your changes show up next frame.

5. Well-meaning smoothing. Exponential smoothing at lerp(old, new, 0.5) adds roughly one frame of extra lag. People apply it to “stabilize” hands and make the lag worse.

The Fix

Step 1: Put hand skeleton logic in _process.

extends Node3D
# Hand visualization driven by OpenXRHand

@onready var hand_skeleton: Skeleton3D = $Hand/Skeleton3D

func _process(_delta: float) -> void:
    # Visual updates only — render at display rate
    _sync_grab_visuals()

func _physics_process(_delta: float) -> void:
    # Physics-facing grab checks only — use hand_skeleton’s current bones
    _update_interaction_queries()

Step 2: Let OpenXRHand do the pose work. The built-in node queries poses at the predicted display time and writes the skeleton for you. Avoid re-reading joint data elsewhere.

# Scene tree:
# XROrigin3D
#   XRCamera3D
#   LeftHand  (OpenXRHand, hand=0, motion_range=UNOBSTRUCTED)
#     Skeleton3D
#       ...joints...
#   RightHand (OpenXRHand, hand=1, motion_range=UNOBSTRUCTED)

# DO NOT do this in _physics_process:
#   var xf = XRServer.get_tracker("/user/hand_tracker/left")\
#              .get_pose("default").transform
#   skeleton.set_bone_pose_position(0, xf.origin)
# You will be one frame behind.

Step 3: Enable high-fidelity hand tracking in the runtime manifest. For Meta Quest, this lives in the OpenXR extension and in the Meta Horizon OS developer settings. In Godot, enable the XR_FB_hand_tracking_aim and XR_META_hand_tracking_wide_motion_mode extensions (available via OpenXR Vendor Plugins).

# Enable vendor extensions in Project Settings
# xr/openxr/extensions/hand_tracking = true
# xr/openxr/extensions/meta_hand_tracking_wide = true
# xr/openxr/extensions/fb_hand_tracking_aim = true

# In code, prefer querying the fastest available tracking source:
func _ready() -> void:
    var iface := XRServer.find_interface("OpenXR")
    if iface and iface.is_initialized():
        print("OpenXR display FPS: ", iface.get_display_refresh_rate())

Step 4: Remove any smoothing on joint data. If you have code that lerps between last-frame and current-frame bone transforms, delete it unless you can measure that raw poses are jittery.

Step 5: Gate interactions on current joint positions. When physics grabs should use the latest joint positions, sample the skeleton transforms at the start of physics, not separately from OpenXR.

func _physics_process(_delta: float) -> void:
    # Use the skeleton that OpenXRHand already updated in _process.
    # These transforms are ~one frame fresh, which is fine for physics.
    var index_tip_local: Transform3D = hand_skeleton.get_bone_global_pose(
        hand_skeleton.find_bone("IndexFingerTip")
    )
    var index_tip_world := hand_skeleton.global_transform * index_tip_local

    if _pinch_distance(index_tip_world, thumb_tip_world) < 0.02:
        _try_grab(index_tip_world.origin)

Step 6: Measure actual latency. Record at 240 fps with your phone, count frames between real-hand motion and virtual-hand response. Each frame at 90 Hz is 11 ms. If you are at three frames, there is more to fix than hand tracking alone.

Why This Works

OpenXR’s whole design centers on the “predicted display time”: each frame, the runtime tells the application the timestamp at which the next image will actually be visible on the headset. The application passes this timestamp to every pose query, and the runtime returns a pose extrapolated to that moment. This cancels out the rendering pipeline latency.

When you update the skeleton in _physics_process, you are reading a pose from some earlier predicted display time (60 Hz means the physics frame happened 16.67 ms ago at most, but could be older). The renderer then samples the skeleton for the current display time and finds it stale. Moving hand logic to _process aligns both ends of the pipeline to the same frame.

Runtime fidelity matters because the runtime is what actually talks to the camera hardware. A 30 Hz hand tracker cannot respond to finger motion faster than every 33 ms regardless of how well-written your game is. Enabling high-frequency mode halves that intrinsic delay.

"Hand tracking lag is usually your frame budget telling on you. Fix the sample timing before reaching for filters."

Related Issues

For general XR performance tuning, see Fix: Godot XR Low Framerate on Quest. If controller input feels delayed rather than hand tracking, check Fix: Godot XR Controller Input Delay.

Predicted display time is the whole game. Query it once, apply it in _process, don’t smooth it.