AI Is Moving Software Engineering's Bottleneck Beyond Code

01The value chain is a loop, not a coding station

An internet product begins with discovery: observing a user problem, a commercial opportunity or an operational constraint. Product managers, researchers, designers and domain specialists turn that observation into a proposed outcome. Architects and engineers then translate the outcome into system changes. Developers implement those changes; reviewers, security specialists and automated pipelines inspect them; release systems deploy them; site-reliability and support teams observe their behaviour; analytics and customer feedback inform the next decision.

This is a loop because production use changes the next specification. A payment failure, confusing interface or slow query is not merely an operations problem. It is evidence about the product and the system. Good software organisations shorten the time between that evidence and a safe improvement.

AI can enter almost every stage. It can cluster support tickets, draft a product brief, explain an unfamiliar module, suggest code, generate test cases, review a change, summarise an incident and propose documentation. GitHub's current Copilot feature documentation ↗ describes suggestions, chat, code review and an agent mode that can choose files, propose edits and terminal commands, then iterate. OpenAI's original Codex introduction ↗ similarly described repository editing and command or test execution in isolated environments. These are evidence of product capability, not proof of business benefit.

Value appears only when a useful output survives the rest of the chain. Generated code that passes a narrow unit test but violates a data-retention rule has negative value. A fast prototype that reveals a mistaken assumption before a quarterly planning decision can have substantial value even if none of its code ships. The correct unit of analysis is therefore a verified product outcome, not an accepted suggestion or a completed prompt.

That framing also explains why AI adoption can feel fast to an individual while leaving a team unchanged. One engineer finishes a patch sooner, but two colleagues spend longer reconstructing its intent. More pull requests arrive, but the staging environment remains unreliable. A support summary identifies a bug, but ownership is unclear. Local acceleration is consumed by a downstream queue.

02Where AI is already useful

The strongest current uses share three properties: the task is bounded, relevant context can be supplied, and correctness can be checked cheaply.

First, assistants are useful for mechanical transformations. Examples include converting a repetitive API client to a new interface, drafting a database query from a known schema, adding routine validation, updating configuration or translating a small function between languages. A compiler, formatter, type checker or existing test suite can reject many mistakes. The human still decides whether the transformation belongs in the system.

Second, AI can reduce the cost of orientation. An engineer entering an unfamiliar repository can ask for a map of modules, likely call paths or the tests associated with a feature. The answer may be incomplete, but it supplies search hypotheses. This is especially helpful when combined with direct inspection of source files and version history. The safe question is not “tell me how the whole system works,” but “show me the files and evidence I should verify to understand this behaviour.”

Third, it can expand the first draft of verification work. An assistant can propose edge cases, property-based tests, fixtures, threat scenarios or failure-injection ideas. This does not establish that the system is correct. It helps a human tester explore a larger possibility space. The highest value comes when generated tests are reviewed against an independent requirement, rather than generated from the same implementation they are meant to assess.

Fourth, it can compress low-risk documentation work. Release-note drafts, code comments, migration instructions and incident timelines are all plausible uses when a person checks the source material. The benefit is not literary polish. It is making operational knowledge easier to retrieve and keeping documentation closer to the code.

Fifth, AI can support investigation. It can summarise logs, compare stack traces, construct a query or list competing hypotheses. During an incident, however, a suggestion should remain a hypothesis until confirmed by telemetry or a controlled test. Granting a model broad production permissions because it writes a plausible diagnosis would confuse fluency with authority.

These uses apply to internet products as well as internal systems. A marketplace might use an assistant to draft an experiment-analysis query; a software-as-a-service company might generate a compatibility test; a mobile team might explain an unfamiliar build error. In each case the useful output is close to observable evidence.

03Where it remains unreliable, and why

AI is least reliable where a task depends on tacit history, ambiguous intent, rare operational conditions or consequences that cannot be cheaply reversed.

A mature system contains decisions that are not visible in its latest files. A strange-looking branch may protect a legacy customer. A duplicated service may reflect a regulatory boundary. A timeout may be coordinated with a downstream provider. Unless those constraints are documented and available as context, an agent can produce locally elegant code that removes an important safeguard.

Requirements are another weak point. Models are good at making an instruction look complete, but they cannot decide whose interests should prevail. “Make checkout faster” can conflict with fraud checks, accessibility, consent and financial reconciliation. Those are product and governance choices. Generating implementation before resolving them merely encodes an unexamined decision more quickly.

Security failures can arise at two levels. The model may suggest vulnerable code, outdated dependencies or unsafe defaults. The agent system may also be attacked through its context or tools. The maintained OWASP GenAI/LLM risk project ↗ treats risks such as prompt injection, unsafe handling of model output and excessive agency as design concerns. In software work, an untrusted issue description, repository document or test fixture could try to redirect an agent. Tool permissions and output validation therefore matter as much as prompt wording.

Privacy and intellectual-property boundaries are similarly specific to the deployed surface. GitHub documents repository, organisation and enterprise content-exclusion controls ↗, but also states that Copilot Chat agent mode in integrated development environments does not support that exclusion. The broader lesson is not about one vendor: a policy toggle should never be assumed to cover every editor, chat, agent, plug-in and command path. Controls must be tested where work actually happens.

Operations remain difficult because rare events matter. A change can satisfy its acceptance tests while degrading a regional dependency, exhausting a connection pool or corrupting an unusual data state. An agent does not carry the pager, explain the incident to customers or own the remediation budget. The person approving a change must be able to understand its blast radius and reverse it.

04Productivity evidence does not support a universal percentage

Claims that AI makes every developer dramatically faster are not justified by the available evidence. Neither are claims that it makes every developer slower.

In a 2025 randomised controlled trial ↗, the research group METR studied 16 experienced open-source developers working on 246 tasks in large repositories they knew well. When allowed to use early-2025 AI tools, participants took 19 per cent longer. Before the work, they expected AI to make them faster; after it, they still perceived a speed-up.

The result is valuable because it challenges self-reported productivity. It is not a population estimate. The participants, repositories, task lengths and tools represented a narrow setting. METR explicitly warned against generalising the finding to most developers or other work. In February 2026, the group said a follow-up experiment could not provide a reliable current estimate because selection effects and time measurement created bias; its method update ↗ called any apparent later improvement weak evidence.

This uncertainty is commercially important. A benchmark can show that an agent solves a defined task without showing the review time, maintainability or incident risk of the result. An adoption survey can show enthusiasm without measuring delivered value. A vendor study can illuminate a product under selected conditions without settling how an organisation should restructure work.

Google's DORA research programme ↗ is useful here because it treats software delivery as a sociotechnical system of capabilities, metrics and outcomes. Its current framing says AI acts as an amplifier and that returns depend on the underlying system. The prudent management question is therefore local and testable: for this class of change, in this repository, does assisted work improve end-to-end outcomes after review and production effects are counted?

05Occupations will change unevenly

For entry-level developers, AI removes some traditional practice tasks while creating a new need for deliberate learning. Boilerplate, syntax recall and routine debugging were never the whole job, but they gave juniors repeated exposure to program behaviour. If an assistant supplies every first draft, a new engineer can appear productive without building the mental models needed to detect a subtle error.

The remedy is not to ban tools. It is to change apprenticeship. Junior engineers should be asked to explain generated changes, predict test outcomes, trace data across boundaries and diagnose failures without immediately requesting another answer. They can use AI to generate alternatives, then compare those alternatives against documentation and runtime evidence. Their performance should be assessed on verified understanding and improvement, not prompt volume.

Mid-career engineers are likely to see the largest expansion in span. They can delegate bounded implementation, test generation, refactoring and documentation while concentrating on interfaces and integration. But the role also acquires a review burden. Someone must detect when a change is correct in isolation but wrong for the architecture. Mid-career value shifts toward decomposition, codebase literacy, evaluation design and the ability to make changes observable and reversible.

Senior engineers and architects become constraint designers. Their leverage comes from creating clear service boundaries, paved deployment paths, reliable tests and decision records that both humans and tools can use. An architecture that exists only in a few people's memories will limit agents as surely as it limits new employees. Yet senior staff must resist becoming a permanent approval queue. They should encode recurring judgement in checks and templates while reserving human attention for novel risk.

Quality engineers move from executing scripted cases toward designing evidence. They will need to distinguish between tests that confirm an implementation and tests that challenge the requirement. They can build adversarial suites, representative datasets, accessibility checks and failure scenarios. Security engineers will increasingly define permission boundaries for agent tools, scan dependencies and outputs, test prompt-injection paths and investigate provenance.

Site-reliability and platform engineers become central to the economics. If code volume rises, standardised build systems, preview environments, telemetry, gradual rollout and rapid rollback become capacity multipliers. These roles should not be treated as overhead behind “productive” feature teams. They supply the feedback and containment that make faster change tolerable.

Product managers and designers also face a shift. Drafting a requirements document becomes cheaper; deciding what should happen remains hard. Their work should move toward sharper acceptance criteria, research evidence, policy constraints and explicit non-goals. A beautifully generated specification that avoids the contested decision is still a poor specification.

Engineering managers must redesign flow rather than rank people by AI activity. Counting accepted suggestions, prompts, generated lines or pull requests encourages output and can punish careful reviewers. Managers should examine queues, rework, defects, incidents, customer effects and learning. Tool telemetry may help diagnose a workflow, but it is an unsafe automatic employment score because it cannot represent task difficulty, mentoring, judgement or avoided risk.

06The capabilities that become more valuable

Specification is the first premium skill. A strong engineer can turn a vague request into observable behaviour, constraints, examples, non-goals and rollback conditions. To demonstrate it, publish a short design record that links each requirement to a test or metric and records the trade-offs considered.

System reading is the second. Workers need to trace behaviour through code, data stores, queues, APIs and deployment configuration. A useful proof is a repository map or incident analysis that cites files, logs and runtime measurements, then shows where an initial hypothesis was wrong.

Evaluation design is the third. This includes choosing test cases that are independent of the generated implementation, building representative fixtures and defining a baseline. A worker can demonstrate it with an evaluation suite containing ordinary cases, boundary cases, adversarial inputs and known failures, together with an explanation of what the suite cannot establish.

Operational judgement is the fourth. It means estimating blast radius, protecting data, selecting a rollout strategy and knowing when to stop. Evidence can include a release plan with feature flags, canary criteria, dashboards, rollback steps and named ownership.

Review communication is the fifth. A reviewer who merely rewrites generated code becomes a bottleneck. A reviewer who identifies the violated invariant, improves the shared check and teaches the pattern increases future capacity. Good artefacts include concise review notes, reusable lint rules, architecture tests and updated runbooks.

Domain knowledge is the sixth. In a billing system, knowing how credits, tax and reconciliation interact can matter more than producing a syntactically correct service. Domain expertise can be demonstrated through accurate decision tables, examples grounded in customer cases and collaboration with operations, legal or finance specialists.

None of these capabilities is new. AI raises their relative value because it makes plausible implementation abundant.

07An illustrative implementation scenario

Illustrative example — not a reported case: consider a business-to-business software company whose customers export monthly billing records. The export service is slow, support receives recurring complaints, and the code is old enough that few engineers understand every path. Management proposes an AI coding agent to accelerate a rewrite.

The team does not ask the agent to “modernise exports.” It first defines a bounded outcome: reduce the time to produce a representative large export without changing totals, row ordering, access controls or audit records. The team constructs a sanitised fixture set, records current latency and error rates, identifies data-residency constraints and documents a rollback path.

The agent is allowed to inspect the export module and its tests in an isolated branch. It can propose code and run approved development commands, but it cannot access production credentials, deploy, merge, change infrastructure permissions or retrieve live customer data. Network access is restricted. Every change must identify the requirement it serves.

The agent finds repeated queries and proposes batching them. A developer checks the database plan and discovers that one customer-specific permission filter is applied in a helper the agent initially overlooked. That is the characteristic failure: the optimisation is locally reasonable but incomplete in context. The developer revises the specification, adds a cross-tenant isolation test and asks for new alternatives.

The selected change passes unit, integration, security and performance checks. A human reviewer inspects the query, permission boundary and migration risk. The team deploys behind a feature flag to internal accounts, then a small customer cohort. Dashboards compare latency, export totals, failure rates and permission-denial behaviour with the baseline. Automatic rollback triggers if correctness or error thresholds are breached.

Success is not “the agent wrote most of the patch.” Success is a verified improvement with no loss of correctness or security, delivered with less total effort than the established process. Even if elapsed implementation time falls, the team should record specification, review and remediation hours. Otherwise it will hide work moved downstream.

The lasting asset is not the generated code. It is the improved fixture set, explicit access-control invariant, performance benchmark, rollout dashboard and runbook. Those artefacts make the next human or agent change safer.

08KPIs that expose the whole system

A responsible pilot uses a balanced scorecard. No single metric should decide success or individual performance.

| Dimension | KPI | How to read it | |---|---|---| | Flow | Median and 85th-percentile lead time from accepted requirement to production | Separate waiting, implementation, review and deployment time so a shifted queue is visible. | | Review | Time to first substantive review; review hours per change; open review queue | Faster authoring with a growing queue is not end-to-end improvement. | | Quality | Escaped defects and post-review rework per change | Classify severity and normalise by comparable change type, not raw code volume. | | Reliability | Change-failure rate, rollback rate and incidents linked to changes | Track assisted and baseline cohorts under the same definitions. | | Recovery | Time to detect and restore service after a failed change | Faster recovery can make a higher change rate safer, but does not excuse avoidable failures. | | Security | High-severity findings, secret exposures, vulnerable dependencies and policy violations | Include findings caught before merge and those discovered later. | | Product | Task success, latency, conversion or support contacts tied to the intended outcome | Select the measure before implementation and guard against harming adjacent outcomes. | | Experience | Developer-reported cognitive load, perceived usefulness and review confidence | Treat perception as important qualitative evidence, not a substitute for measured results. | | Economics | Tool, compute, review and remediation cost per verified outcome | Include failed attempts and senior attention, not only licence price. | | Learning | Percentage of pilot changes with updated tests, documentation or runbooks | This indicates whether adoption leaves the system easier to change. |

Teams should compare like with like. A password reset, schema migration and novel recommendation feature have different risk. Random assignment is often impractical, but matched task categories and a stable pre-pilot baseline are better than comparing a busy launch month with a quiet quarter. Small samples should be presented as directional, with raw counts and uncertainty, not converted into dramatic percentages.

09Failure modes and governance controls

The first failure mode is review saturation. The control is a work-in-progress limit: do not start more agent tasks when the review queue exceeds an agreed threshold. Require small changes, a clear intent statement and evidence from checks. Measure reviewer time.

The second is false confidence from passing tests. The control is test independence. Acceptance cases should come from product rules, incident history and adversarial analysis, not only from the generated patch. High-risk changes require a reviewer who understands the affected domain.

The third is context leakage. The control is data classification mapped to exact tool surfaces. Exclude secrets and restricted repositories, use synthetic or minimised data, restrict network destinations, retain audit evidence and test that policies actually apply in each mode. Never place production credentials in prompts or agent workspaces.

The fourth is excessive agency. The control is least privilege and separation of duties. An agent may draft a change and run a constrained test without receiving permission to merge or deploy it. Destructive commands, production access and permission changes should require explicit human action and independent safeguards.

The fifth is supply-chain contamination. Generated code can introduce a package that is abandoned, misspelled or incompatible with licensing policy. Controls include approved registries, dependency scanning, lockfiles, software bills of materials and provenance checks. The NIST Secure Software Development Framework ↗ organises practices around preparing the organisation, protecting software, producing well-secured software and responding to vulnerabilities; it includes review, analysis and release-provenance practices that remain relevant regardless of who or what drafted the code.

The sixth is automation bias. NIST's Generative AI Profile ↗ describes confabulation and warns about excessive deference to automated output. It recommends empirically evaluating capability claims, documenting limits and applying test, evaluation, validation and verification before deployment. For engineering teams, that means keeping model output visibly provisional until independent evidence promotes it.

The seventh is accountability diffusion. “The model did it” is not an owner. Each assisted change needs a named human accountable for its intent, review and rollout. Incident processes should record tool involvement without treating it as an excuse or assuming it caused the failure.

The eighth is regulatory surprise. Software offered in Europe may fall within product-security, privacy, sector or employment rules depending on what it does and how it is supplied. The EU Cyber Resilience Act ↗ establishes cybersecurity requirements for products with digital elements on a phased timetable. Applicability requires legal analysis; an AI-generated component does not remove the provider's obligations. Teams should map relevant requirements before a pilot touches regulated products or personal data.

10A practical 90-day plan

### Days 1–30: Baseline and bound the experiment

Select one team, one or two repositories and two low-to-moderate-risk task classes. Good candidates include test expansion, small defect fixes, internal tooling or documentation-backed refactors. Exclude authentication, payment authorisation, destructive data migrations and safety-critical logic from the first pilot.

Record at least several weeks of baseline data using existing definitions: lead time, review time, rework, escaped defects, change failures, incidents and developer experience. Do not wait for perfect measurement, but document missing fields.

Map data and permissions. Identify which repositories, files, commands, networks and environments the tool may access. Verify vendor retention and administrative settings. Test exclusion rules on each enabled surface. Establish that the agent cannot merge, deploy or reach production secrets.

Write a one-page usage policy. It should name allowed tasks, prohibited data, required human approvals, logging expectations, incident reporting and the person who can pause the pilot. Create a standard task template with the outcome, context, constraints, acceptance tests, non-goals and rollback plan.

Milestone at day 30: the team can show a baseline dashboard, an approved risk boundary, a working isolated environment and five representative evaluation tasks with known expected results.

### Days 31–60: Run controlled, observable work

Assign comparable tasks to assisted and normal workflows where practical. Keep changes small. Record total human time across specification, implementation, waiting, review and remediation. Label assisted changes for aggregate analysis, not employee ranking.

Require independent tests and a human owner for every change. Add static analysis, dependency checks, secret scanning and policy gates to the pipeline if they are missing. Use feature flags or internal-only releases so failures remain reversible.

Hold a weekly review of both successes and near misses. For each failure, ask whether the missing control belongs in the task template, repository documentation, test suite, architecture or permission system. Improve the shared environment rather than relying on everyone to remember a clever prompt.

Interview developers and reviewers separately. Authors may experience speed while reviewers absorb hidden work. Ask about cognitive load, learning, trust, interruption and the kinds of tasks where the assistant was abandoned.

Milestone at day 60: at least one task cohort has reached production or an equivalent internal environment, every change has an audit trail, and the team can compare end-to-end flow and quality with the baseline without relying on generated-line counts.

### Days 61–90: Decide, harden and scale selectively

Analyse results by task class and risk, including raw counts. Identify where AI improved a verified outcome, where it only moved work, and where evidence is inconclusive. Price the full workflow: licences, compute, platform work, review and remediation.

Convert recurring lessons into durable controls. Add repository instructions, architecture tests, evaluation fixtures, permission profiles, runbooks and dashboards. Red-team the agent path using untrusted issue text or documentation in a safe environment. Confirm that output cannot bypass code review or deployment policy.

Choose one of three decisions for each task class: expand, continue with changes, or stop. Expansion should mean a slightly broader bounded cohort, not immediate organisation-wide default access. High-risk repositories require their own evidence and governance review.

Publish an internal pilot report that separates measurements, participant observations and management analysis. State sample limitations and unresolved questions. Define a re-evaluation date because tools and product controls change quickly.

Milestone at day 90: leadership has an evidence-based decision, named owners, an updated control set and a scale plan tied to thresholds for quality, reliability and review capacity.

11Counterarguments and limits

One counterargument is that these controls will erase the speed benefit. For a throwaway prototype, some production controls would indeed be disproportionate. The answer is risk tiering: constrain effort to the consequence of failure. A local experiment using synthetic data should not face the same gate as a payment migration. But code intended for customers eventually enters a system of responsibility, however quickly it was drafted.

Another is that models will improve so quickly that today's caution will become obsolete. Capability will change, but the need to express intent, verify behaviour and operate services is durable. Better agents may complete larger tasks and reduce supervision. That increases, rather than removes, the importance of knowing what permissions they have, how success is measured and how failure is contained.

A third is that teams with weak documentation cannot pause to prepare. They do not need a perfect knowledge base. The pilot itself can target one bounded area and turn hidden knowledge into tests and decision records. The mistake is pretending missing context has disappeared because a model produced an answer.

This analysis also has evidence limits. Vendor documentation establishes available functions and controls, not independent performance. DORA offers system-level research and practitioner framing, but local causal effects still require local measurement. METR's controlled result is unusually concrete yet narrow and tied to early-2025 tools. Occupational effects will differ by company, country, product risk and labour market. Predictions here are analysis, not reported certainties.

12What software workers and leaders should watch next

Watch whether organisations report end-to-end outcomes rather than code acceptance. Credible evidence will include review and remediation time, production quality, task mix and comparison methods. A single percentage without those details should not drive a staffing plan.

Watch the review queue. If assisted output expands while senior attention remains fixed, lead time and burnout may worsen. Better automated checks, smaller changes and clearer ownership are more meaningful signs of maturity than the number of agent licences.

Watch learning at the bottom of the career ladder. Teams need evidence that junior engineers are acquiring debugging, system and domain judgement, not merely shipping opaque patches. Apprenticeship design may become a competitive advantage.

Watch tool permissions and control coverage. Agent features are converging with terminals, issue trackers, browsers and deployment systems. Every new connection creates useful context and a possible path for untrusted instructions or excessive action. Administrative policy must follow the actual path.

Finally, watch whether companies reinvest saved effort. The durable opportunity is not to maximise software output or automatically reduce headcount. It is to solve previously uneconomic maintenance, accessibility, reliability and customer problems while preserving accountability. When code becomes cheaper, judgement about which code deserves to exist becomes the differentiator.

13A Nietzschean closing note

Nietzsche’s *Thus Spoke Zarathustra*, “On Self-Overcoming,” presents commanding as harder than obeying because the commander bears the burden of what follows. This is a philosophical image, not software-management advice, but it maps surprisingly well onto agentic development. Asking a system to produce code is easy; specifying the right constraints, judging the result and owning its consequences is the heavier task.

The hopeful implication is that engineering value moves upward rather than disappears. Treat every generated patch as an invitation to strengthen your capacity to define, test and take responsibility. The engineer who can command themselves—who refuses a plausible shortcut when the system is not understood—will remain more valuable than the person who merely accepts fluent output. Nietzsche, *Thus Spoke Zarathustra* II, “On Self-Overcoming,” eKGWB/Za-II-Ueberwindung, https://www.nietzschesource.org/eKGWB/Za-II-Ueberwindung.

14Sources

1. NIST, Secure Software Development Framework Version 1.1 ↗ 2. NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile ↗ 3. GitHub Docs, Copilot features ↗ 4. GitHub Docs, Excluding content from GitHub Copilot ↗ 5. OpenAI, Introducing Codex ↗ 6. DORA research ↗ 7. OWASP GenAI/LLM Top 10 2026 ↗ 8. EUR-Lex, Regulation (EU) 2024/2847, Cyber Resilience Act ↗ 9. METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity ↗ 10. METR, We Are Changing Our Developer Productivity Experiment Design ↗