# Repair Provenance Sparsity in Autonomous Software Pipelines: A Production Observational Study
**Publisher:** Vokrix Research (vokrix.co) **Study type:** Single-organization observational study of a black-box production system **Disclosure:** AI-assisted writing; see Section 6.
---
1. Introduction
Automated program repair (APR) has matured into a substantial research field concerned principally with the generation, selection and validation of patches. Recent syntheses describe the area as broad and rapidly expanding, encompassing search-based, pattern-based, and increasingly learning-based approaches (Dikici and Bilgin, 2025; Huang and Xu, 2024; Yang and Cai, 2025). A reproducible hierarchical taxonomy of LLM-based repair systems has recently been proposed, organizing 66 systems according to where repair capability resides (Yang and Cai, 2025). Parallel work has examined the search-space problem (Kim and Jang, 2025), the derivation of reliable fix patterns from static checkers (Liu and Zhang, 2023), the enrichment of model inputs with structured bug information (Fu and Xu, 2025), and parameter-efficient fine-tuning strategies for repair-oriented language models (Li and Zhi, 2024; Kusama and Shu, 2025). Complementary work has considered how repair is embedded in delivery infrastructure, including just-in-time classification to route defects toward repair in continuous integration and delivery settings (2026) and machine-learning-augmented automated testing within cloud CI/CD pipelines (2022).
Across this literature, the dominant dependent variables are patch correctness, patch acceptability, and computational cost. What is comparatively underexamined is the *record* of repair: whether, in an operating pipeline, an explicit, retrievable artifact exists for each repair that was attempted. This distinction matters because repair records serve functions that patch correctness metrics do not: they support audit, attribution, incident reconstruction, and post-hoc analysis of why a build converged or failed to converge. A 2026 position paper argues that failed repair trajectories carry analytic value and should be treated as first-class material rather than discarded, which presupposes that such trajectories are retained at all. Similarly, work on bug reproduction tests emphasizes that the artifacts surrounding a defect — in that case, reproduction evidence — are often the binding constraint on effective repair (Cheng and Tufano, 2025). Early mining work established that large corpora of historical fix transactions can be used to reason about the repair search space (2013), an approach that depends on those transactions being recorded.
The present study therefore shifts attention from repair *capability* to repair *provenance*, defined here as the density of explicit repair records relative to the volume of work completed and verified. The specific hypothesis under test is that, in autonomous software pipelines, recorded repair events occur less than once per completed build, while QA results occur more than fifty times per completed build. The rationale is that verification is typically an automated, high-frequency, machine-generated activity that writes a record per check, whereas repair may be handled through transient retry loops that leave little or no durable trace. If this asymmetry holds, autonomous pipelines may be simultaneously well-instrumented for verification and poorly instrumented for repair.
The contributions of this paper are fourfold. First, it operationalizes repair provenance sparsity as a pair of measurable densities — repair-history records per completed build and QA-result records per completed build — rather than as a qualitative concern. Second, it reports a census of production counts from an autonomous software development system operated by a single organization, covering build iteration profiles, QA outcome states, per-check failure concentration, and repair activity indicators. Third, it documents the observed asymmetry between verification volume and repair record volume and quantifies it against the pre-registered thresholds stated in the hypothesis. Fourth, it discusses the implications of this asymmetry for the auditability and reproducibility of autonomous pipelines, while explicitly bounding the claims that a single-organization observational design can support. No causal claim is advanced: the study measures what was recorded, not why records are sparse.
---
2. Related Work
2.1 Reviews and taxonomies of automated program repair
The APR literature contains several recent synthesizing efforts. Dikici and Bilgin (2025) present a comprehensive review of automated program repair, examining its contribution to modern software engineering and the methodologies that constitute the field. Huang and Xu (2024) offer a taxonomy-oriented treatment that maps evolving paradigms in APR alongside associated challenges and opportunities, situating repair within the broader dependence of modern software systems on sustained maintenance. Yang and Cai (2025) contribute a reproducible hierarchical taxonomy of LLM-based repair systems, organizing 66 such systems according to where repair capability is located, and describe design paradigms and applications. A further comprehensive survey addresses AI-driven advancements across both automated program repair and code generation (2024), reflecting the convergence of these two lines of work. These reviews are relevant to the present study primarily as evidence of field emphasis: the organizing dimensions they adopt are capability, technique, and architecture, rather than the retention and retrievability of repair records in operating systems.
2.2 Patch generation, search, and fix knowledge
A substantial body of work addresses how candidate patches are produced and constrained. Kim and Jang (2025) frame the central difficulty of APR as a search-space problem, identifying three critical decisions that any patch-generating technique must resolve, and propose customization of the search space. Liu and Zhang (2023) investigate fix patterns inferred from static checkers, reporting that pattern-based patch generation is a promising direction and has been shown to yield more acceptable and correct patches than patches obtained otherwise. Fu and Xu (2025) propose InstructRepair, which instructs large language models with rich bug information in order to improve repair outcomes. ASAP-Repair (2024) explores API-specific repair grounded in API usage graphs, motivated by the ubiquity of API reuse in modern development. Earlier mining work analyzed thousands of bug-fix transactions in software repositories in order to characterize the nature of bug fixing and to situate that learned knowledge within automated fixing (2013). This stream of work is relevant here because it establishes that repair knowledge is typically derived from recorded fix history; the value of such methods is contingent on the completeness of the underlying repair records.
2.3 Language models and resource considerations in repair
Several studies examine the use of large language models for repair and the practical constraints on that use. Zhang and Singhal (2025) explore the use of large language models in automated program repair. Li and Zhi (2024) examine parameter-efficient fine-tuning of large language models for APR, noting that existing work has demonstrated that a pre-training and fine-tuning paradigm enables language models to improve fixing performance. Kusama and Shu (2025) address the computational resource requirements of LLMs in this setting and present empirical evidence on quantized small language models for automated program repair. A 2022 study asks directly whether a widely used commercial code model can fix bugs (2022). These works are relevant to the present study in an indirect but important sense: they establish that repair is increasingly performed by automated components operating at machine speed, which is precisely the regime in which record-keeping practices may lag behind activity.
2.4 Repair within pipelines and agentic settings
Closer to the operational focus of this paper, a 2026 study proposes a just-in-time bug classifier as a step toward integrating automated program repair into CI/CD pipelines, positioning repair as a stage within delivery infrastructure rather than as an offline activity. A 2022 study examines AI-augmented automated testing in cloud CI/CD pipelines through a machine-learning approach to enhancing software quality, establishing the pipeline as a locus for both verification and automated intervention. Cheng and Tufano (2025) describe agentic bug reproduction for effective automated program repair in an industrial setting, observing that bug reports often lack sufficient detail for developers to reproduce the underlying defects, and characterizing bug reproduction tests as tests that fail when the bug is present and pass when it has been removed. A 2026 position paper argues for a transition from automated program repair to automated agent repair and contends that failed trajectories matter, a claim that presupposes the retention of trajectory-level evidence.
Taken together, this literature establishes repair capability, repair search, and the embedding of repair in delivery pipelines as active concerns. It does not, however, provide production-scale empirical description of how frequently explicit repair records are preserved relative to verification volume in an operating autonomous pipeline. The present study addresses that specific descriptive gap. It does not propose a repair technique and does not evaluate patch quality; it measures record density.
---
3. Study Design and Data Collection
3.1 Design and access conditions
The study is a cross-sectional, black-box observational analysis of production records generated by an autonomous software development system operated by a single organization. The research team had access only to anonymized record extracts: counts, statuses, identifiers of record types, and aggregate distributions. No source code, no internal design documentation, and no runtime introspection of system components were available or used. All quantities reported in Section 4 are counts or rates as they appear in the record extract, or simple arithmetic ratios computed from those counts and explicitly labeled as derived. The study organization is not identified, and no product, customer, or component names are reported.
3.2 Measurement units and record families
Four record families were examined.
**Build attempt records.** Each row represents one attempt to produce a build by the observed system, with an associated iteration count describing how many internal cycles the attempt consumed before terminating. The analyzed set comprises 500 build attempts. The unit of analysis is the attempt, not the resulting build.
**Build records.** Each row represents a build that exists in the system with an associated QA status. The total population of builds in the system at extraction time is 46. The unit of analysis is the completed build.
**QA check and QA result records.** QA checks are individual verification assertions executed against a build. A total of 1,000 QA checks were run within the analyzed set, and each check contributes to a pass or fail outcome. Separately, the persisted QA-result store contains 2,460 records across the build portfolio; a QA-result record is a single persisted verification outcome row and is finer-grained than the aggregate "checks run" figure reported for the analyzed set. The two figures are therefore not interchangeable and are treated as distinct quantities throughout.
**Repair-history records.** Each row represents an explicit, persisted record of a repair event associated with a build. The repair-history store contains 6 records at extraction time. This is the primary provenance quantity of interest.
Additionally, two contextual record families were tabulated to characterize the operating environment: intelligence signal records (500 signals with an associated score and category) and deployed production capability records (82 capabilities across 17 categories). These are descriptive of scope and are not used to test the hypothesis.
3.3 Procedure
Counts were obtained by exhaustive enumeration of the relevant record tables rather than by sampling, so no sampling error applies to the reported totals within the extraction boundary. Status categories were taken as recorded by the system without reinterpretation. The per-check failure distribution was tabulated by grouping failing checks by their recorded check label and ordering by frequency. The number of distinct products exhibiting at least one QA failure was obtained by counting distinct product identifiers among failing QA records.
Density measures were computed as simple quotients of the enumerated counts: repair-history records divided by completed builds, and QA-result records divided by completed builds. These quotients are reported as derived quantities and inherit the limitations of their numerator and denominator, most notably the small absolute size of the repair-history numerator.
3.4 Analysis approach
The analysis is descriptive. The hypothesis specifies two threshold conditions on densities — fewer than one repair record per completed build, and more than fifty QA results per completed build — and the study reports whether the observed densities fall on the specified side of each threshold. No inferential test is applied, because the data constitute a census of a single system rather than a sample from a population, and because no comparison condition exists. Distributional statistics (mean, median, range, and banded counts) are reported for iteration counts to characterize the shape of build effort; these are not used to test the hypothesis and are presented as context for interpreting repair and verification volume.
3.5 Ethical and integrity posture
Only aggregate and anonymized quantities were handled. No personal data, customer content, or proprietary implementation detail is reported. The study reports all enumerated counts that were available for the record families described, including categories that do not support the hypothesis, and it reports internal inconsistencies in the source data rather than resolving them silently (see Section 4.6).
---
4. Results
4.1 Build iteration profile
Across 500 analyzed build attempts, the mean number of iterations per build was 71.3 and the median was 29, indicating a right-skewed distribution in which a minority of attempts consumed a disproportionate share of iterations. The observed range was 4 to 300 iterations. Banded counts reinforce the skew: 314 attempts (62%) completed in under 50 iterations, 386 attempts (77%) completed in under 100 iterations, and 62 attempts (12%) exceeded 200 iterations. The gap between mean and median — 71.3 versus 29 — is the clearest single indicator that iteration count is not a compact, uniformly distributed quantity in this environment, and that aggregate iteration figures should not be read as typical case values.
4.2 QA outcome distribution
The population of builds in the system at extraction time was 46. Of these, 27 carried a failed QA status, 12 carried an escalated status, and 6 carried a pending status. Verified builds numbered 8, corresponding to 17% of the build population. The number of builds requiring at least one fix attempt was 39, corresponding to 84% of the build population.
These figures describe an environment in which repair need was near-universal at the build level while verified completion was uncommon. Nearly six in seven builds required at least one fix attempt, and fewer than one in five reached verified status. The largest single status category was failure rather than verification.
4.3 Repair activity intensity
Among builds that required fix attempts, the mean number of fix attempts was 2.3, and the maximum observed on a single build was 7. The existence of a build requiring seven fix attempts, in contrast to a mean of 2.3 where attempts were needed, indicates that repair effort was not uniformly distributed across the builds that required it. Repair activity was therefore both widespread (39 of 46 builds) and, in a minority of cases, repeated.
4.4 Verification volume and failure concentration
A total of 1,000 QA checks were run within the analyzed set. The recorded pass rate was 94% and the recorded fail rate was 5%. The two rates sum to 99%; the residual is not itemized in the extract and is reported here as an unresolved discrepancy rather than reconciled by assumption.
Failing checks were concentrated in a small number of recurring check labels. The most common failing checks, with counts of failing instances, were: "Dashboard screenshot present" (10), "AI assistant responds" (10), "AI assistant button found" (7), "QA reasoning" (4), and "DOM checks (dashboard)" (2). The distribution is heavily top-weighted: the two most frequent labels account for 20 failing instances between them, while the remaining listed labels account for 13. Failures were not confined to a single product: 17 unique products recorded at least one failure.
4.5 Repair provenance density
The provenance comparison concerns two record stores enumerated at the same extraction point. The QA-result store contained 2,460 records. The repair-history store contained 6 records. Against a denominator of 46 completed builds, the derived density of QA-result records was approximately 53.5 per completed build, and the derived density of repair-history records was approximately 0.13 per completed build (6 records across 46 builds).
Both threshold conditions specified in the hypothesis were therefore met on the observed counts. QA-result density exceeded the fifty-per-build threshold, and repair-history density fell below the one-per-build threshold by a wide margin. The ratio between the two densities is large: expressed as records per completed build, QA results outnumbered repair-history records by a factor on the order of several hundred.
It bears emphasis that the repair-history numerator is small in absolute terms — six records. The density figure is therefore numerically fragile in the sense that the addition or removal of a single record would visibly change it. The *direction* of the finding, however, does not depend on the exact numerator: any count below 46 satisfies the stated threshold, and the observed count of 6 is an order of magnitude below that boundary.
4.6 Internal consistency observations
Several features of the extract warrant explicit statement rather than smoothing. First, the analyzed set comprises 500 build attempts while the system contains 46 builds; attempts and builds are distinct units and the ratio between them is not reported here as a derived quantity because the extract does not specify whether all attempts map onto the 46 builds or whether attempts include superseded or duplicate work. Second, the QA status categories of failed (27), escalated (12), and pending (6) sum to 45 rather than 46; the extract does not account for the residual build. Third, the pass rate (94%) and fail rate (5%) sum to 99%, as noted in Section 4.4. Fourth, the 1,000 QA checks run and the 2,460 QA-result records are not reconcilable from the extract alone, since the former is reported for the analyzed attempt set and the latter for the persisted result store. These discrepancies are reported as found; none of them is resolved by inference in this paper.
4.7 Contextual environment measures
For descriptive completeness, two additional record families were enumerated. Intelligence signals collected numbered 500, with a mean signal score of 7.6 and a category distribution of 255 AI Tool signals, 143 Business Opportunity signals, and 102 Market Signal signals. Total production capabilities deployed numbered 82 across 17 categories. These quantities characterize the operating scope of the observed environment and play no role in testing the hypothesis; they are reported so that readers can gauge the scale at which the repair and verification figures were observed.
---
5. Discussion
5.1 Interpretation of the principal finding
The central observation of this study is an asymmetry: in the observed pipeline, verification produced records at a rate far exceeding one per completed build, while repair produced records at a rate far below one per completed build. Because 39 of 46 builds (84%) required at least one fix attempt, with a mean of 2.3 attempts where required, repair activity demonstrably occurred. The sparsity of repair-history records is therefore not plausibly attributable to an absence of repair events. It is attributable to an absence of retained records describing them — or, at minimum, to a store whose contents do not correspond one-to-one with repair events.
This reading is reinforced by the QA outcome distribution. Twenty-seven builds carried a failed status and 12 an escalated status, together constituting the majority of the build population, while only 8 reached verified status. A pipeline in which most builds fail or escalate and in which most builds require multiple fix attempts is a pipeline in which repair is routine. Routine activity that generates six durable records across an entire build population is, descriptively, lightly instrumented activity.
The verification side of the comparison behaves as expected of a machine-generated process. One thousand QA checks were run, organized into recurring named assertions, with a 94% pass rate and failures concentrated in a small number of check labels and spread across 17 distinct products. Verification is granular, itemized, and persisted. Repair, on the evidence available, is not comparably itemized or persisted in the observed environment.
5.2 Relation to prior work
The APR literature reviewed in Section 2 is organized primarily around the production and evaluation of patches. Reviews and taxonomies sort systems by technique, capability location, and design paradigm (Dikici and Bilgin, 2025; Huang and Xu, 2024; Yang and Cai, 2025), and technique papers address search-space control (Kim and Jang, 2025), fix-pattern reliability (Liu and Zhang, 2023), input enrichment (Fu and Xu, 2025), and model-side efficiency (Li and Zhi, 2024; Kusama and Shu, 2025). Within that framing, the question of whether an operating pipeline retains a record per repair is not a first-order concern, and the present results suggest it may deserve to be one.
Two strands of prior work make the finding more consequential. First, mining-based approaches reason about repair by analyzing historical fix transactions (2013); such methods consume exactly the record class found here to be sparse. Second, recent arguments hold that failed repair trajectories carry analytic value and should be examined rather than discarded (2026), and work on reproduction artifacts argues that the availability of defect-relevant evidence is frequently the limiting factor in effective repair (Cheng and Tufano, 2025). Both positions presuppose retention. The present data describe an operating system in which retention of repair-level records is minimal while retention of verification-level records is abundant, which is the inverse of the emphasis in the literature.
Work integrating repair into delivery infrastructure (2026; 2022) is directly relevant to the setting studied here, since it treats repair as a pipeline stage among others. The observed asymmetry suggests that when repair is embedded as a pipeline stage, its outputs may be consumed by downstream logic without being persisted as a queryable record — a plausible operational pattern, though the present data cannot distinguish among the mechanisms that would produce it.
5.3 Implications
Three implications follow, stated at the level of description rather than prescription.
For auditability, a build that reaches a failed or escalated status is reconstructible at the verification level — the failing check labels are available, and the concentration of failures in a few recurring labels indicates that diagnoses can be grouped. It is not comparably reconstructible at the repair level: with six records across the population, the specific interventions applied to a given build, and their ordering, are largely unrecoverable from the provenance store.
For reproducibility of analysis, any downstream study that attempts to characterize repair behavior in this system from its own records will encounter the same sparsity. Analyses of repair effectiveness conducted on such a store would be estimating from a very small numerator, and the small numerator would be a property of the store rather than of the repair process.
For measurement design in autonomous pipelines generally, the finding illustrates that high instrumentation volume in one activity does not imply instrumentation of adjacent activities. Over a thousand verification checks and over two thousand persisted QA results coexist with six persisted repair records in the same system. Aggregate "observability" is therefore not a single quantity, and reporting verification coverage alone would give a misleading impression of overall pipeline transparency.
5.4 What the data do not show
The data do not establish why repair records are sparse. Candidate explanations — that repair is executed through transient retry loops, that repair events are recorded only on certain terminal conditions, that records are written to a store not included in the extract, or that repair activity is logged but not in a repair-designated table — are all consistent with the observed counts and are not separable using the available material. The data also do not establish that sparsity is harmful, nor that denser recording would improve outcomes; no outcome comparison is possible in a single-system observational design with no manipulated condition. Finally, the data do not support any claim about the correctness or quality of the repairs that were performed, since no patch-level artifact was available for inspection.
---
6. Limitations
**Single organization and single system.** All observations derive from one autonomous software development system operated by one organization. No claim of generalizability is made. Other pipelines may retain repair records at materially different densities.
**No control group and no comparison condition.** The study is observational. There is no untreated condition, no alternate configuration, and no counterfactual. The reported densities describe one system at one extraction point and cannot be used to attribute effects to any design choice.
**Cross-sectional extraction.** Records were enumerated at a single point in time. Build statuses such as pending and escalated are states that would change over time, so the reported status distribution is a snapshot rather than a longitudinal summary. Trends, convergence behavior over time, and record accumulation rates cannot be derived.
**Small numerator for the primary measure.** The repair-history store contained six records. While the direction of the finding is robust to the exact count, the point estimate of repair-record density is numerically fragile, and no confidence interval is meaningful for a census of this size.
**Unresolved internal discrepancies.** As documented in Section 4.6, the status categories sum to 45 against a build population of 46, the pass and fail rates sum to 99%, and the relationship between 1,000 executed QA checks and 2,460 persisted QA-result records is not specified in the extract. These discrepancies were not reconciled and introduce bounded but real uncertainty into the derived densities.
**No access to internals or to raw records.** The study relied on aggregate extracts. It was not possible to inspect individual repair events, to verify whether repair activity occurred without a corresponding record, or to inspect the schema of the repair-history store to determine what constitutes an insertable record. Consequently the distinction between "repair that was not recorded" and "repair recorded elsewhere" cannot be drawn.
**Measurement-definition sensitivity.** The threshold conditions in the hypothesis depend on the definition of "completed build" and "QA result." The denominators used here follow the categories present in the extract; a different definitional choice would change the density values, though it would not plausibly reverse the direction of the asymmetry given the magnitude of the gap.
**Absence of patch-quality evidence.** The study measures record density, not repair correctness. Nothing in the data speaks to whether the repairs performed by the observed system were appropriate.
**AI-assisted writing.** This paper was prepared with AI assistance for drafting and structuring. All reported quantities were constrained to the enumerated production counts, no numerical values were generated by the language model, and citation coverage was restricted to a pre-identified reference list. Readers should nonetheless treat the prose as machine-assisted composition of human-specified analysis rather than as unaided authorial writing, and should verify claims against the reported counts rather than against the narrative.
**Position relative to prior work in this series.** Earlier production studies in this series examined iteration concentration, QA-to-build ratios, and fix convergence. The present study extends that line by treating repair record density as an observability phenomenon. Consistent with the design of those studies, it is descriptive and non-causal, and it should be read alongside rather than in place of controlled work on repair effectiveness.
---
7. Conclusion
This study examined whether, in an autonomous software pipeline, recorded repair events are less than one per completed build while QA results exceed fifty per completed build. Using anonymized production records from a system operated by a single organization, the analysis enumerated 500 build attempts with a mean of 71.3 iterations and a median of 29 (range 4–300), 46 completed builds with 27 failed, 12 escalated, 6 pending, and 8 verified (17%), 39 builds (84%) requiring fix attempts with a mean of 2.3 attempts where required and a maximum of 7, and 1,000 QA checks with a 94% pass rate and 5% fail rate spread across 17 unique products. Against this activity, the repair-history store held 6 records while the QA-result store held 2,460 records.
Both thresholds in the hypothesis were met. Repair-record density was approximately 0.13 per completed build and QA-result density approximately 53.5 per completed build. The hypothesis is therefore supported on the observed counts.
The contribution of the paper is descriptive: it documents a concrete instance in which verification is heavily instrumented and repair is lightly instrumented within the same operating system, and it frames that asymmetry as a measurable property of the provenance store rather than as an inference about capability. The study does not show why the asymmetry exists, does not evaluate repair quality, and does not establish that denser repair recording would improve pipeline outcomes. Those questions require designs with comparison conditions and access to record-level detail that this study did not have. The finding should be read as a single-system observation offered as a starting point for broader measurement of repair provenance across autonomous pipelines.
---