How to Set Up Smoke Tests for Game Builds

Nothing wastes a QA tester’s time like a build that crashes on startup. They pull the latest build, wait for it to download, launch it, and immediately hit a black screen or a crash dialog. That’s thirty minutes lost before any actual testing begins. Smoke tests exist to catch these catastrophic failures automatically, within minutes of the build completing, so broken builds never leave the pipeline.

What Smoke Tests Should Cover

A smoke test is not a comprehensive test suite. It answers one question: “Does this build work at the most basic level?” Keep the scope narrow and the execution fast. A good game smoke test covers four things:

• Boot test — the executable launches without crashing and reaches the main menu within a reasonable timeout (30-60 seconds).
• Scene load test — the game can transition from the main menu to the first gameplay scene without errors.
• Basic interaction test — the player character spawns and can perform a basic action (move, jump, or interact). This catches broken input bindings, missing player prefabs, and initialization failures.
• Error log check — no error-level log messages appear during the test run. Warnings are acceptable; errors are not.

Anything beyond this belongs in a dedicated integration test or manual QA pass. If your smoke test takes more than five minutes, you’re testing too much. The value of a smoke test is speed—you want feedback within minutes of a build completing.

Building a Headless Boot Test

Most game engines support a headless or null-renderer mode that runs game logic without rendering to a screen. This is ideal for CI environments that lack a GPU.

In Godot, use the --headless flag with a dedicated test scene:

# smoke_test.gd - Attached to a Node in a test scene
extends Node

var _timeout := 30.0
var _elapsed := 0.0

func _ready():
    print("Smoke test: Boot successful")
    # Try loading the main menu scene
    var err = get_tree().change_scene_to_file("res://scenes/main_menu.tscn")
    if err != OK:
        push_error("Smoke test FAILED: Could not load main menu")
        get_tree().quit(1)
        return
    print("Smoke test: Main menu scene change initiated")

func _process(delta):
    _elapsed += delta
    if _elapsed > _timeout:
        push_error("Smoke test FAILED: Timed out after %d seconds" % _timeout)
        get_tree().quit(1)

func _on_main_menu_ready():
    print("Smoke test: Main menu loaded successfully")
    # Verify critical nodes exist
    var play_button = get_node_or_null("/root/MainMenu/PlayButton")
    if play_button == null:
        push_error("Smoke test FAILED: PlayButton not found")
        get_tree().quit(1)
        return
    print("Smoke test PASSED")
    get_tree().quit(0)

Run it from the command line:

godot --headless --path /path/to/project res://tests/smoke_test.tscn

For Unreal Engine, use the -nullrhi flag to skip rendering and the automation framework for test commands:

UnrealEditor-Cmd.exe MyProject.uproject -nullrhi -ExecCmds="Automation RunTests SmokeTest" -unattended -nopause -log

For Unity, use -batchmode -nographics with a test runner script:

Unity.exe -batchmode -nographics -projectPath /path/to/project -executeMethod SmokeTest.Run -logFile smoke_test.log -quit

Adding Gameplay Verification

A boot test catches missing assets and initialization crashes, but it won’t catch a broken player controller or a misconfigured spawn point. Extend your smoke test with a scripted gameplay sequence that simulates a few seconds of basic play.

The simplest approach is an input replay. Record a short sequence of inputs (move forward, jump, interact) and play it back during the smoke test. Verify that the player’s position changes, that no errors are logged, and that the frame time stays below a threshold (to catch infinite loops or performance regressions).

# Godot - Simple scripted gameplay verification
func _run_gameplay_test():
    # Simulate pressing "move forward" for 2 seconds
    var action = InputEventAction.new()
    action.action = "move_forward"
    action.pressed = true
    Input.parse_input_event(action)

    await get_tree().create_timer(2.0).timeout

    action.pressed = false
    Input.parse_input_event(action)

    # Verify the player moved
    var player = get_node("/root/Game/Player")
    if player.global_position.distance_to(Vector3.ZERO) < 0.1:
        push_error("Smoke test FAILED: Player did not move")
        get_tree().quit(1)
        return

    print("Smoke test: Gameplay verification passed")
    get_tree().quit(0)

Keep the gameplay sequence short and deterministic. Avoid tests that depend on physics simulation settling or AI behavior, as these introduce flakiness. The goal is to verify that the core game loop runs, not to test specific mechanics.

CI Pipeline Integration

The real value of smoke tests comes from running them automatically. Integrate them into your CI pipeline so every build is validated before it’s distributed to QA or deployed to players.

A typical pipeline looks like this:

1. Build — compile the game for the target platform(s).
2. Smoke test — launch the built executable with the smoke test scene. Set a timeout of 5 minutes.
3. Log analysis — scan the output log for error-level messages. Fail if any are found.
4. Notification — send results to your team’s Discord or Slack channel.

In a GitHub Actions workflow:

- name: Run smoke test
  run: |
    timeout 300 ./build/MyGame.x86_64 --headless --scene smoke_test
    RESULT=$?
    if [ $RESULT -ne 0 ]; then
      echo "Smoke test failed with exit code $RESULT"
      exit 1
    fi

- name: Check logs for errors
  run: |
    if grep -i "error" build/logs/game.log; then
      echo "Errors found in game log"
      exit 1
    fi

For CI environments without GPU access, headless mode is sufficient for boot and logic tests. If you need rendering verification (checking for black screens, missing textures, or shader compilation failures), use a CI service with GPU support or run visual smoke tests on a dedicated machine with a virtual framebuffer like Xvfb on Linux.

Handling Smoke Test Failures

When a smoke test fails, the pipeline should block the build from being distributed and immediately notify the team. Include the following in the failure notification: the commit hash that triggered the build, the specific test that failed, the relevant log output, and a link to the full CI run. This gives the responsible developer enough context to investigate without digging through CI dashboards.

Treat smoke test failures as high-priority interrupts. A broken build blocks every QA tester, playtester, and developer who depends on the latest build. Establish a team norm that the developer whose commit broke the build fixes it within the hour or reverts the change.

Avoid disabling smoke tests when they fail repeatedly. If a test is flaky (passing sometimes and failing others), fix the flakiness rather than skipping the test. Common causes of flaky game smoke tests include race conditions during scene loading, non-deterministic physics, and hardcoded timeouts that are too tight for slower CI machines. Increase timeouts, add explicit ready checks, and make the test deterministic.

Five minutes of automated smoke testing saves hours of wasted QA time on broken builds.