Render bench

These tools drive the real WebGPU game with a scripted camera at fixed simulation steps. Use production bundles for timing, a fresh output directory for every build and run (build, bench, ABBA, report, review and GPU-check runs refuse to reuse one), and one renderer process at a time.

Each script launches headless Chrome, Edge or Chromium from its standard install path; set RENDER_BENCH_BROWSER to use another Chromium build. The browser helper (browser.mjs) deletes the profile it created after closing the browser, and never touches other profiles or runs. Reports and screenshots stay in the run directory. A script can pass keepProfile: true to launchBrowser for a storage investigation.

Timed runs

npm run test:graphics
npm run render:build -- scratch/bench/before-build
npm run render:serve -- --dist scratch/bench/before-build --port 5191
# In a second terminal:
npm run render:bench -- --url http://127.0.0.1:5191/ --out scratch/bench/before --environment exclusive

test:graphics (check.mjs) runs tsc --noEmit and the renderer, streaming, density-parity and octree-bake tests the benchmark depends on.

render:build runs a Vite production build into the given directory (default scratch/render-bench/build) without copying public/. It writes source-state.json with the git HEAD, the binary git diff, SHA-256 hashes of every tracked and untracked source file outside public/, and the hash of the physics WASM, which it copies beside the bundle. It resolves @dimforge/rapier3d-compat from this checkout’s vendor/rapier3d, so worktrees that share node_modules cannot contaminate each other.

render:serve serves the build (--dist, default dist) on --port (default 5191) and falls back to the checkout’s public/ for assets. Keep those assets unchanged between arms, or pass --assets pointing at a preserved copy.

render:bench (run.mjs) appends diag, renderbench, noenter and nointro to the URL (default http://127.0.0.1:5190/). It waits, untimed, for the loading overlay to go and for pipeline creation to settle, then runs each scene. Scenes come from --profile (default scenes.json: ground-terrain, dense-settlement, water-shore, combat-effects, forest-traversal, planetary-horizon, orbit, orbit-to-ground, ground-to-orbit); --scenes a,b selects a subset.

Scenes with discovery helpers (an anchor such as nearest-tree or open-water, or a radial descent/ascent path) are resolved into numeric camera points and saved to resolved-scenes.json in the run directory. Pass that exact file as --profile for every later arm; never let each arm discover its own anchor. trees.json adds wind and rain fixtures anchored on a resident tree; a discovered tree camera still needs a visual check for terrain occlusion. The combat scene is a scripted effects workload, not a deterministic AI battle or an input-latency measurement.

--environment exclusive records that you stopped other GPU workloads; it cannot enforce it. Use shared while another game runs (the default is unspecified). Only runs where both arms are exclusive can be marked as controlled comparisons.

Defaults

1920x1080, device scale 1, render scale 1, uncapped frame rate (vsync and the frame-rate limit disabled), stochastic seed 0x51f15e, 1/60 s simulation steps, 600 warmup frames and 600 measured frames. The two orbit transitions in scenes.json measure 900 frames. Each scene sets an epoch (default 10000 + index * 1000 s) and a solar hour (default 13); epochs must increase through a run. Browser, loading and pipeline readiness are untimed, and idle waits do not advance actors. --capped keeps the display cap as a separate experiment. --width and --height must match in both arms.

ABBA comparison

npm run render:ab -- --a http://127.0.0.1:5191/ --b http://127.0.0.1:5192/ --profile scratch/bench/before/resolved-scenes.json --out scratch/bench/abba --environment exclusive

This runs A, B, B, A, each arm in its own fresh browser. --blocks N (1 to 10) repeats the sequence; --scenes, --width and --height are forwarded to every arm. render:ab defaults to --environment shared. Each pair’s spread is reported explicitly: opposing small deltas are inconclusive and are not a demonstrated speedup. CPU submission time, GPU elapsed time and GPU completion period are kept separate. npm run render:compare -- BEFORE AFTER [OUT.json] compares two single runs directly.

Reading results

• metadata.json: hardware, browser and arguments, git HEAD and status, the served build’s source-state.json, harness hashes and the contention declaration.
• <scene>.json: raw frame series, median/mean/p95/p99/max, start and end settings and streaming state, GPU resource counters, pipelines created during the scene, JS heap and errors.
• summary.json: compact CPU, submission-period, GPU elapsed/period and streaming GPU rows per scene.
• ab.json: pair comparisons and repeatability. A timing comparison still needs separate visual acceptance.
• gpu-telemetry.csv: read-only 1 Hz nvidia-smi samples (utilisation, clocks, temperature, power, memory) when available; gpu-telemetry-status.json records availability and errors.

npm run render:report -- BEFORE AFTER OUTPUT_DIR writes a portable HTML report with CPU/GPU medians and tails, frame plots, streaming and memory counters and source provenance. Serve it with npm run render:serve -- --dist OUTPUT_DIR. The report keeps failed comparability checks and never treats one pair as a confirmed speedup.

A scene is invalid (and the run stops) if it logged errors, lost a GPU sample, measured the wrong frame count or resolution, or ended with the camera inside terrain. Missing GPU samples stay missing rather than becoming zero. Warmup, loading and new shader compilations are kept apart from the measured phase; a scene that is still streaming should be read as a streaming workload. The end-camera terrain check alone does not prove a whole transition is visually valid.

--profile scripts/render-bench/travel.json exercises 30, 120 and 480 m/s terrain travel plus orbital descent and ascent. arrivalHoldFrames keeps the camera at the final point for a measured refinement tail; those frames count in frames, not in warmup, so travel lasts (frames-arrivalHoldFrames-1)*stepSeconds. Read the gauges stream.speed, stream.lodDrop, stream.queued, stream.inFlight, stream.visibleQueued, stream.visibleInFlight, stream.regenQueued and stream.fallback alongside frame time. stream.visibleQueued is a 0/1 demand flag, not a count.

Fixed-step playback feeds the same step to the streaming motion estimator, while congestion budgets keep wall time. The start state records this as streamingMotionClock: simulation-v1; builds without it used wall time for motion too, so rendering speed could change LOD decisions. compare.mjs flags arms with different motion clocks. The travel paths are stress fixtures: a surface-safe endpoint does not certify every intermediate position.

--geometry counts direct vertex/index submissions, instances and cached bundle replays by pipeline and pass. Indirect draw counts are reported, but their GPU-generated geometry is invisible to this observer, and the totals are submitted elements, not unique visible triangles. --allocations enables V8 heap allocation sampling. Both add overhead, and compare.mjs excludes such runs from final FPS claims. Resource byte counters describe API-visible allocations, not driver VRAM residency or whole-process memory.

Matched visual review

npm run render:bench -- --url http://127.0.0.1:5191/ --profile scratch/bench/before/resolved-scenes.json --out scratch/bench/before-visual --visual --shot-every 2 --environment exclusive
npm run render:bench -- --url http://127.0.0.1:5192/ --profile scratch/bench/before/resolved-scenes.json --out scratch/bench/after-visual --visual --shot-every 2 --environment exclusive
npm run render:review -- scratch/bench/before-visual scratch/bench/after-visual scratch/bench/review --video path/to/ffmpeg.exe
npm run render:serve -- --dist scratch/bench/review --port 5193

Open http://127.0.0.1:5193/. The divider, frame timeline and held-before control compare matched frames; the optional videos (--video takes the path to an ffmpeg executable) keep simulation cadence. --shot-every defaults to 10; a stride of 2 samples motion at 30 fps, and stride 1 is for frame-to-frame shimmer. These paused screenshot runs are untimed: their wall times are not frame-rate measurements. Review disocclusion, thin geometry, foliage, transparency, shadows and distance transitions, including the first and last frame.

Add --shadow-check to a visual run (it is rejected otherwise) to compare original and culled tree shadows on captured frames whose index is a multiple of 60 and on the final frame, through __treeShadowCullCheck(). The renderer stays paused while both arms use identical live geometry, weather, alpha textures and projection matrices. Results go to *.shadow-depth.json beside the screenshots, and any changed texel fails the run. A check with zero occupied pixels is vacuous and is not visual validation.

GPU and density diagnostics

Start the Vite dev server on port 5190 (npx vite --port 5190; Vite’s own default is 5173, and noise-kernel.mjs hard-codes 5190):

npm run render:gpu-check -- http://127.0.0.1:5190/scripts/render-bench/tree-environment-check.html __treeEnvironmentCheck scratch/bench/tree-parity
npm run render:gpu-check -- http://127.0.0.1:5190/scripts/render-bench/tree-bounds-check.html __treeBoundsCheck scratch/bench/tree-bounds
npm run render:gpu-check -- http://127.0.0.1:5190/scripts/render-bench/ocean-mip-check.html __oceanMipCheck scratch/bench/ocean-mips
npm run render:gpu-check -- http://127.0.0.1:5190/scripts/render-bench/tree-hidden-check.html __treeHiddenCheck scratch/bench/tree-hidden
npm run render:gpu-check -- http://127.0.0.1:5190/scripts/render-bench/cloud-blur-check.html __cloudBlurCheck scratch/bench/cloud-blur
npm run render:gpu-check -- http://127.0.0.1:5190/scripts/render-bench/terrain-index-check.html __terrainIndexCheck scratch/bench/terrain-index
node scripts/render-bench/noise-kernel.mjs scratch/bench/density-gpu
npx tsx scripts/render-bench/physics-ab.ts BASELINE_PACKAGE CANDIDATE_PACKAGE scratch/bench/physics-ab.json

render:gpu-check loads the page, calls the named entry point and writes result.json.

• Tree environment: compares cached and inline branch and leaf deformation under calm, gust, rain, slot changes and fallback.
• Tree bounds: runs the wind/rain shader on randomised vertices and checks both displacement and final world-space containment.
• Ocean mips: compares every output texel of the separate and paired mip shaders, including odd dimensions.
• Hidden trees: compares the tree cull shader against its previous version, checking surviving IDs and signed fades across every template and both LOD overlaps.
• Cloud blur: compares the blur with and without shared spatial weights, bit for bit on its half-float colour and full-float depth outputs, for radii 0 to 4, strides 1, 2 and 4, several far and guard settings, and image sizes that leave incomplete workgroups.
• Terrain index: exercises GPU index packing and decoding (odd tails, nonzero offsets) and compiles every terrain-index shader without a populated game.
• Noise kernel: compares all density output bits with the reference and times the tile kernel. These are density microbenchmarks, not game FPS; use ABBA in the game for frame-rate claims.
• Physics A/B: a CPU-only WASM density benchmark. Each package directory must contain rapier_wasm3d.js and rapier_wasm3d_bg.wasm and support the versioned tile parameter blob. It samples 8,192 points for home and Luna, in hash and tile modes, over 12 ABBA blocks, and fails if the two builds produce different density bytes. The third argument (output JSON) is optional.

Runtime switches for one-bundle A/B

In a diag boot these console hooks toggle one optimisation, so a single production bundle can be compared ABBA by putting the same setup expression in both scene profiles with a different value:

• __treeShadowInstanceCull(boolean): near tree-shadow instance culling.
• __cloudSpatialWeights(boolean): shared workgroup weights in the cloud blur (changes repeated arithmetic, not filter quality).
• __oceanMipPair(boolean): the two-layer ocean mip path.
• __fpBroadphase(boolean): the local footprint index.

URL switches, read at boot before pipelines are created:

• ?diag&referenceWaterHash selects the reference water density hash. The default shares integer corner products without changing density values.
• ?diag&renderbench&tileDensity=<body> gives one landable, non-home body the experimental tile-v1 noise model before chunks, placement, probes and colliders are built. The home planet is rejected. This is a different world: reuse a camera path only if it stays above the changed surface, and report the result as a changed world, not an equivalent-scene win. compare.mjs flags a density-revision mismatch.

water.json covers open water and storm. Its discovery samples density below, at and above sea level along the view corridor and saves fixed numeric camera points; reuse that resolved profile for both arms, including old builds without the discovery helper. Sampling does not replace looking at the screenshots.

Keep both URLs in the run metadata. Changes below the timing noise floor can still be worth keeping; label their savings as unmeasured rather than reporting them as FPS gains.

Contamination

Do not compile, run CPU tests, encode video or run another GPU workload during a timed block. If it happens anyway, keep the data and put a contamination.json (reason and timestamp) in the run or ABBA directory. compare.mjs then marks those results ineligible for performance claims. Repeat the complete block under comparable conditions.

Written by release:site-sync from scripts/render-bench/README.md at v2026.09.22. Edit the engine document, not this page.