The direct answer
Fix the trusted reference, observed value, unit and explicit tolerances before comparing a numerical regression indicator. Approximate agreement can depend on which argument is the reference and on the absolute allowance near zero. It is not symmetric identity, model-quality evidence or permission to relax an exact business or security rule.
Define a numerical agreement question before choosing a test
A fictional AI evaluation pipeline emits one already-defined scalar indicator for one permitted case. A reviewer wants to compare that observation with an independently approved reference. The missing policy is not which record belongs in which array; it is which numerical deviations this particular regression check permits.
Select one finite, bounded scalar observation and one fixed finite reference in the same invented unit. Exclude arrays, broadcasting, custom numeric objects, coercion, booleans and special values. The actual representation and adapter still require independent support. This worksheet constructs no scalar, metric or library call.
The existing shape contract keeps its ordinary-integer, per-record scope. The exact budget contract also stays exact. Neither identifiers, monetary allowances nor authorization decisions gain an approximate-equality exception.
Record the reference side rather than assuming symmetry
The inspected NumPy v2.5 isclose reference gives the finite-value rule: absolute(a − b) ≤ atol + rtol × absolute(b). It treats b as the reference, so swapping inputs can change the answer. For scalar inputs it returns a single boolean. The page warns that default atol can be inappropriate for small reference magnitudes, especially near zero.
Our original application policy assigns the observation to role A and the approved reference to role B before any candidate result is seen. Record those roles alongside their case identity, revisions and unit. A function argument’s convenient position must not silently decide what the evaluator meant.
There is no rule here to try both directions and retain whichever passes. That would define a different acceptance policy. A reference update likewise creates a changed comparison proposal; a familiar metric label cannot preserve the old decision automatically.
Work the orientation counterexample on paper
Invent observation 9 and reference 10, with relative allowance 0.1 and absolute allowance zero. In elementary real-number arithmetic, the difference is 1 and the reference-scaled allowance is 1, so the proposed inequality fits at its boundary. Swap only the reference role: the allowance becomes 0.9 and the difference no longer fits.
| Observation / reference | Paper difference | Paper allowance | Paper disposition |
|---|---|---|---|
| 9 / 10 | 1 | 1 | Fits the invented oriented rule. |
| 10 / 9 | 1 | 0.9 | Does not fit that rule. |
This is a teaching derivation in real-number arithmetic, not a guaranteed or captured NumPy result. The equality-boundary case does not establish how an actual dtype, rounding path or adapter represents and compares these values. Those prerequisites remain unknown and block a real positive.
The fictional allowance is deliberately visible, not recommended for production. No observed model score, accuracy improvement or empirical regression threshold exists. A team must justify its actual criterion for its task independently.
Give absolute and relative allowances separate meanings
Both tolerance values must be finite and nonnegative; zero absolute tolerance is permitted in this chosen application policy. Write down the unit, operating scale and acceptable-deviation rationale together. Our application policy gives the absolute allowance the indicator’s unit and the relative allowance a scale role. Changing from one unit to another without reviewing the absolute value changes the proposed decision.
For a reference of zero, elementary substitution removes the reference-scaled term. The remaining absolute allowance therefore needs its own justified policy. A tolerance copied from an unrelated test cannot establish what a small but meaningful nonzero observation should mean in this task.
Do not tune the allowance after seeing a candidate until it passes, replace missing values with zero, or round the displayed observation into an apparently acceptable result. Each changes the evidence or acceptance rule. Hold an unresolved policy rather than asking the model whether two numbers look close enough.
Keep an original comparison receipt
| Field | Independent evidence required |
|---|---|
| Meaning | Fixed indicator definition, case identity and common unit. |
| Roles | Trusted observed/reference association and exact admitted revisions. |
| Acceptance | Explicit relative/absolute values, rationale and policy fixed before results. |
| Implementation | Supported finite bounded numeric representation, operations and complete result capture. |
| Release | Result association, independent evaluation interpretation and current audience rights. |
The useful positive conditionally admits only this fixed comparison plan when every field is independently established. A later boolean result must remain associated with that plan. It cannot certify a different metric, record, reference or tolerance policy.
Actual runtime, dtype, extraction, rounding, overflow behavior, bounds, adapter, unit policy and current rights are unimplemented or unknown. Any missing prerequisite blocks a real positive. The source title identifies documentation, not an installed numerical environment.
Walk eight inert acceptance cards
| Condition | Proposed disposition |
|---|---|
| 1. Complete fixed oriented plan and every independent prerequisite hold | Conditionally admit only that scalar comparison plan. |
| 2. Arguments are reversed to obtain an acceptable result | Reject the changed evidence policy. |
| 3. Reference is near zero but absolute policy is unreviewed | Hold tolerance applicability. |
| 4. Tolerances are omitted or inherited without approval | Require explicit owned values and rationale. |
| 5. Units or indicator meanings differ | Reject before comparison. |
| 6. Special values, unsupported coercion, arrays or bounds appear | Outside this finite scalar profile. |
| 7. Tolerance was selected after viewing the candidate | Withhold the predeclared regression-check claim. |
| 8. Agreement fits but record association, quality evidence or current rights is missing | Withhold those separate claims. |
Keep approximate agreement inside its declared scope
This guide provides no numerical accuracy certificate, statistical inference, model ranking or security comparison. It supplies no matcher implementation, automatic tolerance selector or general conversion policy. Existing task-evaluation evidence still needs its own rubric and applicable observations.
No NumPy import, numeric value, array, dtype, comparison, rounding experiment, metric, model, plot, fixture or API operation was constructed or executed. Ordinary editorial file handling is separate. Boundary remains prelaunch with planned request-path checks; its existing synthetic policy evidence does not implement this numerical evaluator.
Primary sources
Official references supporting the guidance. Examples in this guide are synthetic and are not reports of a production incident.