The direct answer
For a paired comparison, preserve each case’s candidate-score tuple throughout the resampling plan. Independent column draws change that association even when the original rows matched. Freeze the case manifest, statistic and failure rules first; computational eligibility does not establish valid uncertainty or model superiority.
Ask what a resampled observation represents
Two frozen AI candidates receive scores on the same evaluation cases. A report proposes bootstrap analysis of their mean score difference. The original score columns line up correctly, but the resampling adapter draws a different collection of case positions for each candidate. The new numbers no longer preserve the intended paired experiment.
The one-task comparison design still governs the task, rubric, failures and permitted data. This narrower worksheet reviews the resampling unit after those decisions. It supplies no model outputs, confidence interval, statistical ranking or release recommendation.
Select two ordinary one-dimensional score vectors, one scheduled score per candidate per admitted case. Their ordered case identities, units and frozen revisions must agree before this plan is considered. A meaningful pairing comes from the evaluation design, not the fact that two lists have equal lengths.
Read the selected interface at its actual scope
The actual SciPy 1.18.0 bootstrap reference describes replacement sampling with the original sample size. With paired=True, it draws indices shared across the data arrays; otherwise each array is resampled independently. The statistic accepts the samples separately; axis identifies the sample dimension. The returned bootstrap_distribution holds statistic values for the resamples.
For this proposed computational profile declare nonvectorized one-dimensional inputs, the paired branch, axis zero and the percentile method explicitly. This is not a recommendation that this method is appropriate for an actual evaluation. The same reference marks BCa as the default, discusses degenerate-distribution problems for that method, and describes percentile as rarely used in practice.
No installed SciPy or numeric adapter was inspected. The examples on the page were not run or copied into this worksheet. Method selection, statistical applicability and result interpretation need independent review; a recognizable API argument cannot settle them.
Work one case-association example on paper
Invent case R-A with A-score 0 and B-score 1, and case R-B with A-score 100 and B-score 101. The numbers are assigned teaching labels in one fictional unit, not measured predictions or an empirical benchmark. Define the paper statistic as the mean B-minus-A difference. Each original case contributes 1.
| Case | A | B | Same-case difference |
|---|---|---|---|
| R-A | 0 | 1 | 1 |
| R-B | 100 | 101 | 1 |
A hypothetical draw naming R-B twice produces the two intact tuples (100,101) and (100,101). Its teaching mean difference is 1. Repetition is part of this paper illustration, not a duplicate-ID repair or a second independently observed evaluation case.
Contrast a different proposal that selects R-A twice for A and R-B twice for B. It combines (0,101) twice and produces a teaching difference of 101. The arithmetic is elementary; neither value was captured from SciPy. The contrast identifies different case association, not a measured bias or a claim that independent resampling is universally invalid for another design.
Freeze the planned analysis and its evidence receipt
Our original admission policy records the corpus and case manifest, candidate revisions, scoring rubric, units, expected scheduled outcomes and missing-result treatment. It then binds the statistic, sample dimension, pairing decision, method, requested work bound and complete result-capture contract to that same analysis revision.
Keep input association and resampling association as separate fields. The existing array-shape worksheet establishes a different prerequisite. Correct geometry cannot restore wrongly matched case identities; correct source pairs cannot repair a later changed resampling plan.
For the ordinary conditional positive, independently supported inputs and every actual prerequisite permit only the declared paired analysis plan. A future output must still be captured under the same receipt and reviewed for its intended audience. No figure or interval is eligible for publication merely because a numerical routine returned normally.
Freeze failure handling before looking at results. Dropping a missing B-score while retaining A, excluding an outage after scoring, or substituting a different rubric changes the planned comparison. Hold that proposal instead of silently shortening the manifest until the adapter accepts it.
Review eight inert resampling decisions
| Invented condition | Proposed disposition |
|---|---|
| 1. Complete actual prerequisites and the frozen paired receipt hold | Conditionally admit only this analysis plan; no result is claimed. |
| 2. The independently recombined paper proposal above is selected | Reject it for this paired-case design. |
| 3. Source IDs are reversed on one side | Reject the original association before analysis. |
| 4. A required score is missing or an error was silently omitted | Hold the complete planned-outcome contract. |
| 5. Repeated trials or related paraphrases are presented as independent cases | Hold statistical applicability; no automatic alternative sampling recipe. |
| 6. The two-row teaching example appears to give a perfectly stable difference | Use it only to explain association, never as empirical uncertainty evidence. |
| 7. Statistic, method, work bound or result-capture behavior is undeclared | Hold the actual adapter review. |
| 8. Numerical prerequisites fit but data-use rights or current release permission fail | Withhold the affected computation or disclosure independently. |
Keep uncertainty and representativeness unresolved
The two invented rows cannot validate a population claim. They were deliberately written with matching differences, so their tidy arithmetic is a feature of the teaching construction. It is not evidence that a real model comparison has zero uncertainty, adequate sample size or reliable coverage.
Actual case independence, sampling and task-population suitability, candidate comparability, bounded finite score extraction, numeric behavior, statistic implementation, method applicability, resource limits and current rights are unknown or unimplemented here. Missing evidence blocks a real positive. Related issue variants and repeated trials require a separately justified design; this article supplies no cluster, temporal or nested resampling procedure.
The holdout lifecycle also remains effective. Resampling cannot remove prior answer-key exposure or restore independence to selection decisions made with final data. Do not tune a candidate against this published example and call it an untouched final test.
No library import, array, seed, random draw, statistic, score, confidence interval, plot, model call or runtime fixture was constructed or executed. Boundary remains prelaunch with planned request checks, not a shipped statistical evaluator. Its synthetic policy evidence covers different behavior.
Primary sources
Official references supporting the guidance. Examples in this guide are synthetic and are not reports of a production incident.