What causes a test on random logic that is not reproducible?

The system under test uses a global or time-seeded random generator, so each run produces different values and the test cannot assert on a specific outcome.

How do I catch this when it only happens for some players?

Capture it automatically from the player's device with the full stack trace, the device and OS, the build, and a breadcrumb trail of what they did. That turns a vague complaint into a specific, reproducible issue you can fix without owning the hardware it happens on.

How to Fix a Test on Random Logic That Is Not Reproducible

Quick answer: Inject the random source so the test seeds it deterministically, then assert on the exact sequence that seed produces.

A test cannot assert on output that changes every run. Injecting the RNG and seeding it in the test makes the sequence fixed and the assertions stable.

How to fix it

1. Inject the random source

Have the system take a seedable generator as a dependency instead of calling a global Random.value or rand(). The test then supplies a generator seeded to a constant.

2. Seed to a constant in the test

Construct the generator with a fixed seed (for example new System.Random(12345)) so the produced sequence is identical every run and the expected values are knowable.

3. Assert on the full sequence

Pull several values and assert the whole sequence, not just one, so a regression in the algorithm shows up rather than coincidentally matching a single expected number.

Catching the ones you can't reproduce

The hardest version of this to fix is the one you can't reproduce — it only happens on a player's hardware, OS, driver, or save state, under conditions that simply aren't present on your machine. A report that says “it crashed” or “it froze” gives you nothing to act on, so the bug survives release after release while quietly costing you players.

Automatic error capture closes that gap. Each failure arrives with its full stack trace, the device and OS, the build number, and a breadcrumb trail of what the player did right before it broke, so even a failure you have never seen becomes a specific, reproducible issue. Fold identical failures into one signature ranked by how many players each hits, and your worklist sorts itself worst-first instead of arriving as a stream of vague complaints.

This is where a tool like Bugnet earns its place. Its SDK captures every error automatically with the full stack trace plus device, OS, memory, build, and game-state context, folds duplicates into one grouped issue with an occurrence count, and ties each to the build it first appeared on — so you fix the problem that hurts the most players first and confirm it is gone when its signature disappears from the next release.

The bug you can't reproduce isn't gone — it's just invisible until you capture it from the player's device.