# Iteration Concentration in Autonomous Software Generation: A Production Observational Study
**Prepared by:** Independent Research Institute **Study type:** Single-organization observational study of production telemetry **Publication status:** Technical report
---
1. Introduction
Autonomous and semi-autonomous software generation systems have moved from demonstration settings into production environments over a comparatively short period. The literature returned by the search conducted for this study describes agents that perform code synthesis, program repair, and test generation (Xia and Deng, 2025), agents that diagnose and repair build failures in fragmented platform ecosystems (Kim and Rui, 2025), and multi-agent frameworks that search interdependent hardware and software design spaces (Jiang and Sun, 2025). A doctoral-level study of agent-driven software improvement observes that software maintenance accounts for 50% of the cost of developing software, framing reliability and maintainability as central economic concerns rather than secondary quality attributes (Agent-Driven Automatic Software Improvement, 2024).
These contributions establish that autonomous agents can execute meaningful software engineering work. They also establish, in aggregate, that the work is iterative: agents attempt, evaluate, revise, and attempt again. What the literature returned by the present search does not appear to establish is how iteration effort is distributed *across* the units of work an autonomous system produces. The dominant reporting convention identified in the abstracts reviewed is outcome-oriented. Systems are characterised by whether a task succeeded, how many tasks succeeded, or how a workflow performed on average. The empirical question of whether iteration effort is spread evenly across builds, or whether a small number of builds absorb a disproportionate share of the total iteration budget, is not answered by success rates or per-task means.
That question has practical consequence. If iteration effort is concentrated, then average-case reporting describes a system that does not exist for most of its operating life: the typical build finishes quickly while an atypical minority consumes the bulk of the loop budget, the retry allowance, and the accompanying evaluation overhead. Capacity planning, escalation thresholds, and human review allocation all depend on which of these two shapes is true. Concentration also affects how engineers should read summary statistics. A reported mean iteration count that is more than twice its median is not a neutral description; it is evidence about the shape of the underlying distribution.
This study addresses that question using a production extract from a system that was not instrumented for the purpose. The extract was supplied as anonymised production records by a single organization operating an autonomous software development system. The system is treated throughout as a black box: only externally observable inputs, outputs, counts, and rates are reported here. No internal architecture, component naming, or mechanism is described, and none was inspected. The observed system is not named, in accordance with the anonymisation conditions under which the data were made available.
The study makes the following contributions:
- It reports the distribution of iteration counts across 500 build attempts in a production autonomous software generation setting, including the range, mean, median, and the proportions of attempts falling below and above specified thresholds.
- It quantifies the concentration implied by the relationship between the extreme and central values of that distribution, and evaluates the stated hypothesis against that evidence.
- It reports build-level operational outcomes for 41 builds, specifically the proportion requiring fix attempts, the mean and maximum fix-attempt counts, and the proportion verified.
- It reports the observed distribution of QA check outcomes across 1000 checks, including the most frequently failing check identities.
- It provides a documented, reproducible description of the extract and its internal consistency properties, so that subsequent studies can compare against it.
The hypothesis under test was directional and binary-answerable: that the maximum single-build iteration count exceeds the mean across completed builds by a factor greater than four, implying that a minority of builds consume a disproportionate share of total iteration budget. The extract supports an evaluation of that hypothesis and is reported in full below, including the respects in which the extract does not neatly reconcile with itself.
---
2. Related Work
The literature reviewed for this study clusters into four areas: agentic software engineering broadly, build-focused self-correction, iteration and convergence behaviour in agent loops, and architectural or orchestration framing for multi-agent systems.
**Agentic software engineering.** Xia and Deng (2025) survey and analyse the use of large language models in automating software development tasks, identifying code synthesis, program repair, and test generation as the principal areas of advancement. Their abstract positions recent agentic systems as building on earlier automation efforts. The relevance to the present study is definitional: the tasks they enumerate are the tasks that produce iteration. Program repair and test generation are inherently loop-generating activities, yet the framing in their abstract is capability- and task-oriented rather than distribution-oriented. This study does not contradict that framing; it supplements it with a description of how loop volume is distributed across produced units.
**Build-focused self-correction.** Kim and Rui (2025) present a self-correcting agent for automated Android builds, motivated by the fragmentation and churn of the Android ecosystem, which they describe as making builds from open-source sources difficult to reproduce reliably. Their work establishes build repair as a distinct sub-problem within agentic software engineering and confirms that iteration against a build is a real and necessary activity in practice. Their abstract describes the problem and the agent; it does not report a per-build iteration distribution. The present study addresses a related but distinct quantity: not whether build repair succeeds, but how the volume of iteration is spread across the builds a production system generates.
**Iteration and convergence in agent-managed systems.** MACO (Jiang and Sun, 2025) presents a multi-agent framework for coarse-grained reconfigurable array hardware and software co-design, explicitly motivated by the observation that the design space is vast, interdependent, and bottlenecked by costly manual iteration. The abstract thus identifies iteration cost as a binding constraint in agent-driven design work. Iterative Audit Convergence in LLM-Managed Multi-Agent Systems (2026) examines convergence behaviour in a prompt-engineering quality assurance case study, addressing iteration as a process that terminates or fails to terminate. The reference entry titled "fixed-count iteration" (DOI:10.1007/springerreference_14624) concerns fixed versus variable iteration counts as a concept; no abstract was available during this review and its specific findings are therefore not characterised here. Together these works establish that iteration count is a recognised variable of interest in agentic systems. The literature search conducted for this study did not return a paper reporting an empirical distribution of per-build iteration counts from a production autonomous software development system.
**Orchestration, reliability, and escalation.** Several works address the surrounding structure rather than the inner loop. Dai (2026) analyses architectural convergence across agent harnesses, defining a harness as the code that builds context, mediates tools, runs the loop, and persists state — that is, the scaffolding that determines how many loop iterations occur and what terminates them. Sakhinana and Runkana (2026) describe agentic cloud engineering with a zero-trust harness in which agents reason over operational state, invoke authorised tools, modify software and infrastructure, deploy services, and verify execution. The Wheel of Intelligence (2026) proposes contract-enforced closed-loop architectures intended to make LLM-agent systems more reliable. ProgRouter (2026) addresses online progress-guided orchestration for multi-agent workflows and explicitly frames the problem in terms of quality–cost tradeoffs, which places iteration volume on the cost side of the ledger. Before You Build a Multi-Agent System: An Escalation Framework for LLM Adaptation (2026) concerns the decision of when an LLM-based approach should be escalated to an alternative — directly relevant to the escalation category observed in the present extract. Multi-Artifact Versioning for LLM-Agent-Based Software (2026) addresses the management of the multiple artefacts such systems generate.
What these works share is a concern with structure, reliability, orchestration cost, and escalation policy. What the search did not return, across any of these areas, is a study reporting the concentration of iteration effort across builds in a live production system. The contribution of the present study is therefore narrow and empirical: a documented distribution, not a new architecture.
---
3. Study Design and Data Collection
3.1 Design
This is a retrospective observational study of production telemetry from a single organization. No intervention was applied, no system configuration was altered, and no data were generated for research purposes. All observations derive from records produced by the routine operation of the observed system before and independently of this analysis. The study therefore supports description and hypothesis evaluation only; it cannot support causal inference or comparison against a counterfactual.
3.2 Black-box scope
The observed system was made available to the analysts exclusively through anonymised production records describing inputs and outputs. Internal implementation detail was neither provided nor inspected. The analysis is accordingly restricted to externally observable quantities: counts of attempts and builds, iteration counts, fix-attempt counts, QA check outcomes, status categories, signal counts and scores, and a count of deployed production capabilities. No claim in this report describes how any result was produced internally. Where a capability is acknowledged as existing, it is acknowledged only as an observable count.
3.3 Unit definitions and measured quantities
Six distinct units of observation appear in the extract. They are not interchangeable, and the analysis treats them separately.
- **Build attempt.** A single attempt by the observed system to produce a build. The primary measured quantity is the **iteration count** for that attempt, expressed as a count of iterations. 500 build attempts were analysed.
- **Build.** A build in the system, considered at the level of its operational status. 41 builds were recorded in the system. The measured quantities are status category, whether fix attempts were required, the number of fix attempts when required, and whether the build was verified.
- **QA check.** A single automated quality check execution. 1000 QA checks were run. Measured quantities are pass or fail, and, for failing checks, the check identity.
- **QA status outcome.** The terminal status of a build with respect to quality assurance (failed, escalated, pending) or its verification state.
- **Intelligence signal.** An external item ingested by the system, characterised by a category and a numeric score. 500 signals were collected.
- **Production capability.** A capability recorded as deployed, aggregated into categories. 82 capabilities across 17 categories were recorded.
3.4 Collection procedure
Anonymised production records were exported by the study organization and provided to the analysts as a single extract. The analysts performed no instrumentation, no sampling, and no filtering. All figures reported in Section 4 are taken verbatim from the extract. The observation window corresponds to the production period covered by the retained records; explicit calendar dates were not included in the extract and are therefore not reported.
3.5 Data integrity checks performed
Before reporting, the following internal checks were carried out on the extract:
- **Percentage recomputation.** Category percentages were recomputed from the stated numerator and denominator. The proportion of build attempts under 50 iterations recomputes to 65.0% (325/500). The proportion under 100 iterations recomputes to 79.6%, reported in the extract as 79%. The proportion over 200 iterations recomputes to 11.6%, reported in the extract as 11%. Both discrepancies are consistent with downward rounding to whole percentages.
- **Category summation.** The build-attempt threshold categories are non-exhaustive and non-overlapping as reported, and no total is claimed for them. The signal category counts sum exactly to 500 (258 + 142 + 100). The QA status categories sum to 40 against a stated total of 41 builds, leaving one build unaccounted for by the listed status categories.
- **Rate summation.** The reported QA pass rate (94%) and fail rate (5%) sum to 99%, not 100%.
- **Derived ratios.** Two ratios are reported in this paper as arithmetic derived from extract values: the ratio of maximum to mean iteration count, and the ratio of mean to median iteration count. Both are labelled as derived where they appear.
3.6 Reconciliation note
The hypothesis statement supplied with this study referenced 23 completed builds, 2147 QA results, and a mean of 71.0 iterations per build. Those figures do not appear in the production extract analysed here, which reports 500 build attempts, 1000 QA checks, and a mean of 68.3 iterations per build attempt. In accordance with the reporting standard adopted for this study, only values present in the extract are used in the Results section. The hypothesis is therefore evaluated against the extract, not against the figures in the original statement. This discrepancy is discussed further in Section 6.
3.7 Analysis approach
Analysis was descriptive. For the build-attempt unit, the range, mean, and median iteration counts were reported, together with the proportions of attempts falling below and above the stated thresholds. For the build unit, status category frequencies, fix-attempt statistics, and verification proportion were reported. For QA, the overall pass and fail rates and the frequency of each failing check identity were reported. Signal and capability counts were reported as collected. No inferential testing was performed, and no modelling was undertaken, because the extract constitutes a census of the retained records rather than a sample drawn from a defined population.
---
4. Results
4.1 Iteration distribution across build attempts
Across 500 analysed build attempts, the mean iteration count was 68.3 and the median was 29. Iteration counts ranged from a minimum of 4 to a maximum of 300.
Two derived relationships characterise the shape of the distribution. The maximum observed iteration count is approximately 4.4 times the mean (300 / 68.3). The mean is approximately 2.4 times the median (68.3 / 29). Both ratios are arithmetic derivations from extract values.
The threshold proportions are reported in Table 1.
**Table 1. Distribution of iteration counts across 500 build attempts.**
| Threshold | Build attempts | Proportion | |---|---|---| | Under 50 iterations | 325 | 65% | | Under 100 iterations | 398 | 79% | | Over 200 iterations | 58 | 11% |
The reported categories are not exhaustive of the range. The under-50 and under-100 categories overlap by construction (all attempts under 50 are also under 100), and 102 attempts fall at or above 100 iterations, of which 58 exceed 200.
4.2 Concentration of iteration effort
The stated hypothesis was that the maximum single-build iteration count would exceed the observed mean across completed builds by a factor greater than four, implying concentration of iteration effort in a minority of builds. Against the extract, the maximum is 300 and the mean is 68.3, giving a ratio of approximately 4.4. The directional component of the hypothesis is therefore supported at the stated threshold.
The supporting distributional evidence is consistent with concentration. More than half of all build attempts (325 of 500, 65%) completed in fewer than 50 iterations, while 58 attempts (11%) exceeded 200 iterations. Because the extract does not provide the sum of iteration counts, the exact share of total iteration budget consumed by the highest-iteration attempts cannot be computed from the available data. The concentration claim is therefore supported by the relationship between the extreme and central values, and by the spread between the reported threshold proportions, rather than by a directly measured budget share.
4.3 Build-level outcomes
Build-level records are reported for 41 builds in the system. The distributions of QA status and verification are shown in Table 2.
**Table 2. Build-level status and verification (41 builds).**
| Category | Count | Proportion of 41 | |---|---|---| | Failed | 22 | — | | Escalated | 12 | — | | Pending | 6 | — | | Verified | 8 | 19% |
The three status categories sum to 40, leaving one build not represented by the listed status values. The verified count of 8 corresponds to 19% of 41 as reported.
Fix-attempt behaviour is reported in Table 3.
**Table 3. Fix-attempt statistics (41 builds).**
| Measure | Value | |---|---| | Builds requiring fix attempts | 34 (82%) | | Mean fix attempts when needed | 2.4 | | Maximum fix attempts on a single build | 7 |
The majority of builds therefore entered at least one repair cycle, and among those that did, the mean number of fix attempts was 2.4, with a maximum of 7 on a single build.
4.4 QA check outcomes
A total of 1000 QA checks were run. The reported pass rate was 94% and the reported fail rate was 5%. These figures do not sum to 100%; no residual category was specified in the extract.
The most frequently failing check identities are reported in Table 4.
**Table 4. Most common failing QA checks.**
| Failing check | Count | |---|---| | Dashboard screenshot present | 10 | | AI assistant responds | 10 | | AI assistant button found | 7 | | QA reasoning | 4 | | DOM checks (dashboard) | 2 |
Failures were distributed across 17 unique products. The five check identities listed account for 33 failing check instances in total. The reported fail rate implies a larger number of failing checks across the full run of 1000 checks than these five identities represent; the extract lists the most common failing checks without stating whether the list is exhaustive.
4.5 Inputs and deployed capabilities
The system collected 500 intelligence signals with a mean signal score of 7.3. The category distribution was: AI Tool, 258; Business Opportunity, 142; Market Signal, 100. These categories sum exactly to 500.
A total of 82 production capabilities were recorded as deployed, across 17 categories. No per-category breakdown of capabilities was provided in the extract.
---
5. Discussion
5.1 Interpretation of the iteration distribution
The principal finding is that iteration counts in the observed production system are not uniformly distributed. The mean of 68.3 against a median of 29, a spread confirmed by the threshold proportions in Table 1, indicates a strongly right-skewed distribution in which a large body of build attempts terminates relatively quickly and a smaller body extends far beyond the central tendency. The ratio of maximum to mean of approximately 4.4 satisfies the directional threshold specified in the hypothesis.
The practical implication of this shape concerns how the system's operating behaviour is summarised. A mean of 68.3 iterations per build attempt is arithmetically accurate but describes no typical build attempt: the median attempt completes in 29 iterations, less than half the mean. A stakeholder reading only the mean would form an expectation that is wrong for the majority of attempts and materially too low for the small number of attempts in the upper tail. The 58 attempts exceeding 200 iterations — 11% of the population — sit at the opposite extreme, and any decision about retry allowances, timeout configuration, or human review triage that is calibrated to the mean will be poorly calibrated to both the median and the tail.
This observation connects directly to the escalation category reported in Table 2. Twelve of 41 builds were recorded as escalated, and the related literature treats escalation as a policy decision about when an approach should be replaced rather than continued (Before You Build a Multi-Agent System, 2026). The concentration observed here supplies a description of the condition under which escalation policy becomes consequential: the tail of the iteration distribution is where the budget is spent, and therefore where escalation thresholds have the greatest effect. The present data cannot establish what determines membership in the tail, and no attempt is made to do so.
5.2 Fix attempts and verification
The build-level results show that 82% of builds required at least one fix attempt, with a mean of 2.4 fix attempts when required and a maximum of 7. Two observations follow. First, repair is the normal case rather than the exception in this system, which is consistent with the framing of build repair as a first-class problem in agentic software engineering (Kim and Rui, 2025). Second, the mean fix-attempt count of 2.4 sits close to the median iteration count of 29 divided by an order of magnitude, and no relationship between fix attempts and iteration counts is derivable from the extract; the two are reported as separate measurements. Eight builds, or 19%, were verified, against 22 failed and 12 escalated. The extract does not permit inference about whether escalation is a route to eventual verification.
5.3 QA failure modes
The failing check identities in Table 4 are informative about where quality assurance pressure falls. The two most frequent failures — "Dashboard screenshot present" and "AI assistant responds" — each occurred 10 times, followed by "AI assistant button found" at 7. Three of the five most common failures concern the presence, responsiveness, or discoverability of an interactive interface element rather than the correctness of generated logic. This pattern suggests that in this system, automated verification most often fails at the boundary between generated software and its user-facing surface. Interpretation beyond this is not supported: the extract does not report the total number of checks per identity, so failure *rates* per check cannot be computed, only failure *counts*. The "QA reasoning" failure (4) and the "DOM checks (dashboard)" failure (2) are consistent with the same boundary, but the extract does not supply enough detail to characterise them further.
The gap between the reported fail rate (5% of 1000 checks) and the failing instances listed in Table 4 is a reporting limitation rather than a finding. It is noted here so that subsequent users of these figures do not treat the five listed identities as a complete inventory.
5.4 Relationship to prior work
The literature reviewed in Section 2 is oriented toward capability, architecture, reliability, and cost. Xia and Deng (2025) frame progress in terms of what agents can automate. Dai (2026) frames the harness as the determinant of loop behaviour. Sakhinana and Runkana (2026) frame verification as part of a broader operational loop. ProgRouter (2026) places orchestration explicitly in a quality–cost tradeoff. MACO (Jiang and Sun, 2025) identifies iteration cost as the bottleneck that motivates agentic assistance in the first place. The present study does not dispute any of these framings. It supplies a specific empirical shape for one quantity that those framings treat abstractly: the distribution of iteration volume across produced units. The observation that a large majority of units terminate quickly while a small minority extend to multiples of the mean is consistent with the general premise that iteration cost is a binding constraint, and it locates that constraint in a minority of units rather than across the population evenly.
5.5 What the results do not show
The results do not show why some build attempts require 300 iterations. They do not show that high-iteration attempts are more likely to fail, escalate, or remain unverified, because the extract does not link the build-attempt record to the build-status record. They do not show that concentration is stable over time, that it is characteristic of autonomous systems generally, or that it differs from any alternative process. They do not establish a causal relationship between any observed input and any observed outcome. The findings are descriptive of one system over one retention period.
---
6. Limitations
**Single organization.** All data derive from one organization operating one production system. Nothing in this design supports generalisation to other systems, other organizations, or autonomous software generation as a class. The observed values should be treated as one documented case.
**No control group.** There is no comparison condition. No baseline, no alternative configuration, and no non-autonomous comparator was available. The study therefore cannot attribute any observed outcome to the system under observation, to its configuration, or to the task distribution it encountered.
**Observational and retrospective.** The analysis uses records produced by routine operation before this study was conceived. No variable was manipulated, and no sampling frame was applied. The extract constitutes a census of retained records rather than a probability sample, and standard inferential statistics are consequently not reported.
**Unit ambiguity between datasets.** The iteration dataset concerns 500 build attempts, while the status, fix-attempt, and verification dataset concerns 41 builds. The extract does not state a mapping between the two. No analysis in this paper assumes one, and no cross-dataset relationship is claimed. This is the single most important structural limitation of the study, because it prevents linking iteration concentration to build outcomes.
**Internal inconsistencies in the extract.** As documented in Section 3.5: the QA pass and fail rates sum to 99%; the build status categories sum to 40 against a stated total of 41 builds; two threshold proportions reflect downward rounding; and the failing-check list may be non-exhaustive relative to the reported fail rate. These are reported rather than corrected, because correcting them would require assumptions not supported by the extract.
**Discrepancy with the hypothesis statement.** The figures in the original hypothesis statement (23 completed builds, 2147 QA results, mean 71.0 iterations) do not appear in the extract analysed here. The hypothesis was evaluated against the extract. Should the original figures pertain to a different extract or a different observation window, the evaluation reported here applies only to the extract described in Section 3.
**Derived quantities.** Two ratios reported in Results are arithmetic derivations from extract values and are labelled as such. No ratio, percentage, or comparison outside those derivations has been introduced.
**Truncated literature records.** Several abstracts in the reviewed literature were truncated in the source records available to the analysts. Characterisations in Section 2 are restricted to what the available abstract text states, and no inference has been drawn about unreviewed content. One reference entry, concerning fixed-count iteration, had no abstract available and its findings are explicitly not characterised.
**AI-assisted writing.** This paper was produced with AI assistance. The analysis, structure, and drafting were carried out with the support of an AI system. All quantitative values were taken from the production extract supplied by the study organization, and all citations were restricted to the literature records made available to the analysts. Readers should weight interpretive passages accordingly and are encouraged to verify the reported figures directly against the extract conditions described in Section 3.
**Anonymisation.** The observed system and the study organization are not named. Capability-level detail was deliberately excluded from this report to maintain the black-box framing conditions under which data access was granted. This reduces contextual information that would ordinarily aid interpretation.
**No reliability assessment.** No second coder, inter-rater reliability measure, or independent recomputation of the extract was performed. The integrity checks in Section 3.5 were arithmetic.
---
7. Conclusion
This study set out to test whether iteration effort in a production autonomous software generation system is concentrated rather than uniform. Against an extract of 500 build attempts, iteration counts ranged from 4 to 300, with a mean of 68.3 and a median of 29. The maximum exceeded the mean by approximately a factor of 4.4, satisfying the stated directional hypothesis. A total of 325 build attempts (65%) completed in fewer than 50 iterations, while 58 (11%) exceeded 200, and 398 (79%) completed in fewer than 100. The distribution is right-skewed, and the reported threshold proportions are consistent with a pattern in which a minority of attempts occupies the upper portion of the iteration range.
At build level, 34 of 41 builds (82%) required fix attempts, with a mean of 2.4 fix attempts when needed and a maximum of 7; 8 builds (19%) were verified; 22 were recorded as failed, 12 as escalated, and 6 as pending. Of 1000 QA checks run, the reported pass rate was 94% and the reported fail rate 5%, with the most frequent failing check identities concerning the presence or responsiveness of interactive interface elements and 17 unique products affected by failures. The system collected 500 intelligence signals with a mean score of 7.3, and 82 production capabilities were recorded across 17 categories.
The study is observational, single-organization, and uncontrolled, and it does not establish causation, generalisability, or a mechanism. Its contribution is a documented distribution and a set of reported measurements that other studies can compare against. The central empirical claim — that iteration effort is concentrated rather than uniform in the observed system — is supported to the extent that the extract permits, and the extract's own internal inconsistencies and unit ambiguities are reported rather than resolved. Whether the concentration observed here is a general property of autonomous software generation, a property of this system, or a property of the task distribution it encountered cannot be determined from these data.
---