Fix: Unreal SteamVR Controller Drift After Game Minimize

A player alt-tabs out of your Unreal VR game to check Discord, comes back, puts the headset back on, and their sword is floating 3 meters away pointing at the floor. The hand mesh tracks their controller fine, but the sword attached to it is way off. Run for a few seconds and the sword pops back into place — or worse, it stays offset forever until they drop and re-grab. This is the canonical focus-loss pose caching bug.

How Focus Loss Corrupts Pose Caches

OpenXR (and SteamVR through OpenXR) has a session lifecycle: SYNCHRONIZED, VISIBLE, FOCUSED. When the user alt-tabs, the game drops from FOCUSED to VISIBLE or lower. Input may stop, controller tracking may pause, and the runtime may recenter the tracking origin (especially if the headset sleeps and wakes).

The drift comes from anything that cached a pose-derived transform while focus was active. For example, when the player grabs a sword, you typically compute:

FTransform GripToSword = Sword->GetActorTransform() *
                         Controller->GetComponentTransform().Inverse();

You store GripToSword and keep the sword at Controller.GetComponentTransform() * GripToSword every tick. When focus returns, Controller.GetComponentTransform() reflects the new tracking origin, but GripToSword was computed relative to the old origin. Multiply them together and the sword lands at a nonsensical place.

Step 1: Listen for Focus Events

Bind to both the engine-level and OpenXR-level focus transitions:

void AMyVRPawn::BeginPlay() {
    Super::BeginPlay();
    FCoreDelegates::ApplicationHasReactivatedDelegate.AddUObject(
        this, &AMyVRPawn::OnAppReactivated);
    FCoreDelegates::ApplicationWillDeactivateDelegate.AddUObject(
        this, &AMyVRPawn::OnAppDeactivated);
}

void AMyVRPawn::OnAppDeactivated() {
    bFocusLost = true;
    InvalidatePoseCaches();
}

void AMyVRPawn::OnAppReactivated() {
    bFocusLost = false;
    // Wait one tick for tracking to stabilize, then rebuild
    GetWorld()->GetTimerManager().SetTimerForNextTick(
        FTimerDelegate::CreateUObject(this, &AMyVRPawn::RebuildPoseCaches));
}

ApplicationHasReactivatedDelegate fires when the OS window regains focus. For VR-specific state, also hook UHeadMountedDisplayFunctionLibrary::GetXRSystemFlags or the IXR module’s session state listener.

Step 2: Invalidate Pose Caches

Any transform computed from a pose should be treated as perishable. Either recompute every tick or explicitly invalidate when focus is lost. The simplest pattern is to skip rendering/logic on pose-dependent actors for one tick after focus returns:

void AGrabbable::Tick(float DeltaTime) {
    Super::Tick(DeltaTime);
    if (!AttachedController) return;
    if (Pawn->bFocusLost) return;  // freeze while minimized

    FTransform ControllerTransform = AttachedController->GetComponentTransform();
    if (bCacheDirty) {
        // Recompute offset from current pose; previous cache is invalid
        GripToObject = GetActorTransform() * ControllerTransform.Inverse();
        bCacheDirty = false;
    }
    SetActorTransform(GripToObject * ControllerTransform);
}

Step 3: Rebuild Grabbed Objects

The exact strategy depends on whether you want the object to stay in the player’s hand on resume or return to a rest state:

• Stick to hand: On focus return, re-snap the object’s world transform to the current controller and recompute GripToObject as identity-ish. This gives the feel of “the sword never left my hand.”
• Drop on focus loss: More conservative — on focus loss, trigger the release logic and let the object fall. Players returning do not find phantom objects.

Stick-to-hand is generally expected in VR and avoids the “my sword is on the floor now” player frustration.

void AMyVRPawn::RebuildPoseCaches() {
    for (AGrabbable* G : HeldObjects) {
        if (!G) continue;
        FTransform CtrlT = G->AttachedController->GetComponentTransform();
        // Re-snap to current pose: object jumps to hand
        G->SetActorTransform(G->GripToObjectRest * CtrlT);
        G->GripToObject = G->GripToObjectRest;
        G->bCacheDirty = false;
    }
}

Step 4: Avoid ResetOrientationAndPosition

Some tutorials suggest calling UHeadMountedDisplayFunctionLibrary::ResetOrientationAndPosition on resume. Do not. That call physically moves the play area relative to the headset, which breaks room-scale calibration — the player’s real-world chair is no longer where the game thinks it is. Let the OpenXR runtime own recentering (it recenters correctly on its own when it needs to), and fix your own caches in response.

Step 5: Test the Race

Drift only appears if the user alt-tabs at a specific moment, so write a test checklist and run it every milestone:

1. Grab a sword with the right hand. Hold the trigger.
2. Alt-tab to the desktop for 5 seconds.
3. Alt-tab back into the game. Sword should be in your hand at the expected orientation within 1 frame.
4. Now repeat with the headset off. Put it on a table face down for 10 seconds. Many headsets sleep during this and recenter the play space on wake.
5. Put the headset back on. Sword should still be in your hand.

“Every pose you cache is a bet that tracking will not discontinuity. Minimize events break that bet every time.”

Verifying the Fix

Add a debug overlay showing the most recent focus event and the distance between held-object-world-position and controller-world-position. Drift shows up as that distance growing after resume. After the fix, the distance should stay constant within a centimeter at all times.

Related Issues

If the headset itself drifts (not the controllers), see VR Camera Tilt After Recenter. For broken haptics after minimize, read VR Haptics Silent After Alt-Tab.

Focus events + cache invalidation + re-snap on resume = no more phantom swords after minimizing.