Skip to content

Acceptance gates

The bridge is correct when it reproduces a native computation. That claim is checked, not asserted: a set of example executables compare bridge-computed results against native references and print loud PASS sentinels, exiting nonzero on failure. There is no ctest wiring (Known limitations) – the gates are run by hand, and this page is the runner’s manual: what each gate checks, how to run it, and the one way the whole procedure silently lies to you (a version-skewed baseline).

What the gates mean physically is told on the Validation page; here we keep the mechanics and the maintenance discipline.

The gate inventory

Sentinel Example Checks
SOD_VALIDATION_PASS 07_capstone/1_reacting_flow.f90 (sod mode) Sod tube vs the exact Riemann solution
DETONATION_VALIDATION_PASS 07_capstone/1_reacting_flow.f90 (detonation mode) CJ detonation speed vs the analytic Chapman–Jouguet value
EX_CAPSTONE_GRI_IGNITION_PASS 07_capstone/2_gri_ignition.f90 four gated quantities vs the native Julia run (below)
DEMO_PASS 06_sciml/3_sciml_demo.f90 the SciML callback demo end to end

The detonation gate is the headline physics check: at nx = 2700 the computed front speed is 7.12391 against the analytic CJ speed 7.12470 – a deviation of 0.011%, with the burned state within 0.025% of CJ (u 0.025%, p 0.023%, rho 0.018%). (At nx = 400 the same gate fails loudly at +8.53% and says so: under-resolution is an error, not a pass.)

The four-quantity GRI gate

2_gri_ignition.f90 gates not one number but four, all at the same tolerance – the ignition delay t_ign (1.1075 ms), the end temperature T_end, and two mole-fraction spot values chosen because they pin the layout-sensitive 4 x 2001 column-major matrix extraction (CH4 at t = 0, CO2 at t_end). The verdict block, verbatim from examples/07_capstone/2_gri_ignition.f90:

    rel = abs(t_ign_f - t_ign_nat) / t_ign_nat
    rel_t = abs(T(NSAMPLE) - t_end_nat) / t_end_nat
    rel_x1 = abs(x_at(1, 1) - x_ch4_nat) / x_ch4_nat
    rel_x2 = abs(x_at(3, NSAMPLE) - x_co2_nat) / x_co2_nat
    ...
    if (rel < REL_TOL .and. rel_t < REL_TOL .and. rel_x1 < REL_TOL &
        .and. rel_x2 < REL_TOL) then
        print *, 'EX_CAPSTONE_GRI_IGNITION_PASS'

with REL_TOL = 1.0e-8_c_double. Under a correctly pinned runtime the four deviations are exactly 0, and the full-trajectory overlay audit puts the worst series residual below 1e-12 (measured ~8.3e-13, on the steepest CO2 rise). The two X gates are the layout tripwire: if the column-major flattening were ever off, X(1,1) reads the wrong species and the gate fails by orders of magnitude, not marginally.

Note what is gated: converged scalars, not pointwise trajectories. Two independent solves of the same stiff system differ in last-bit fashion across a handful of transition points even on one runtime, so a pointwise gate would flake; the four scalars are stable. The gates are tight enough that a wrong runtime cannot sneak through – which is the next section.

Running the gates

Build with examples on, then generate the native baseline before running the bridge side. The recipe from the header of examples/07_capstone/2_gri_ignition.f90 (the Julia path below is a placeholder – use your pinned 1.12.7 binary, not a version shim):

    cd build/examples
    <pinned-julia-1.12.7>/bin/julia --project=../../scripts/sysimage_env \
        ../../examples/07_capstone/run_gri_native.jl
    ./ex_capstone_gri_ignition

The baseline script writes gri_native.dat (the four reference values) and the full native trajectory output/gri_native_series.csv; the executable reads the .dat, runs the identical FjKinetics call path through the bridge, extracts the arrays, and prints the four relative differences plus the sentinel. The other gates need no baseline step – their references are analytic or built in.

A stale baseline is caught loudly: the executable refuses a .dat whose format does not match the current script, with a “regenerate” hint.

The trap: a drifting baseline fails the gates and blames the bridge

The acceptance comparison is only meaningful if both sides run the same Julia. The bridge executable embeds the build-pinned libjulia 1.12.7; the native baseline is generated by whatever julia is on the path. On a machine where that julia is an auto-updating version shim, the pair silently skews – and the failure mode is nasty: the gates report the bridge’s extraction as the suspect, because the bridge is the side being tested.

This is not hypothetical; it happened. A baseline regenerated under a shim pointing at 1.13.2 left two of the four gates showing nonzero margins (0 / 1.74e-9 / 0 / 6.75e-9, the worst 6.75e-9) – apparently passing, actually poisoned – and on the transition window the pointwise drift reached T rel 1.5e-5 and CH4 rel 0.54. The 1e-8 door was being crossed by a baseline skew, while the investigation initially chased solver nondeterminism and thread ordering. Re-generating the baseline with the pinned binary turned all four deviations into exact zeros overnight. The full story is on Lessons; the engineering residue is two guards.

The runtime guard, verbatim from examples/07_capstone/run_gri_native.jl (the original error string names the developer’s install path; shortened here to the placeholder):

const BRIDGE_JULIA = v"1.12.7"   # the libjulia the bridge executable links
VERSION == BRIDGE_JULIA || error(
    "native baseline must be generated with the bridge-pinned Julia " *
    "$BRIDGE_JULIA (this is $(VERSION)) -- use the pinned 1.12.7 binary, " *
    "not the juliaup shim. A different runtime drifts at the ignition " *
    "transition and fails the 1e-8 gates while wrongly indicting the " *
    "bridge extraction.")

The build-side twin is the FJ_BUILD_SYSIMAGE guard in CMakeLists.txt, which rejects any Julia outside the 1.12.x line at configure time and says so (“do not use the juliaup shim”).

Maintenance discipline

  1. Regenerate the baseline with the pinned binary, every time. The guard makes any other choice loud.
  2. After any bridge change, run all four gates, not just the one nearest your change. A past review fix introduced a regression that only the newly added X gates caught – the t_ign gate alone had been passing over a doubly-indirect species indexing bug.
  3. Watch the margins, not just the sentinel. The gate prints the four deviations; exact zeros are the healthy steady state. Anything consistently at 1e-9 means the baseline pair is skewed again – find the runtime before touching the bridge.

Where to go next

  • Lessons – the drift incident in full, plus the other rules this page assumes.
  • Rebuild from zero – regenerating the sysimage and AOT artifacts the gates run against.
  • Validation – the physics behind each gate.