Runtime placement criteria for software-engineering agents in regulated…

Runtime placement criteria for software-engineering agents in regulated organisations

2026-10-17 · agentic-ai governance-policy security-risk tools-infrastructure regulatory-compliance · medium · source → · wiki →

key claims

Copilot cloud agent attributes commits to Copilot with the requesting developer as co-author, signs them, links each commit to its session log, restricts pushes to one branch, and requires a human to merge, which together give the vendor-hosted stage a documented attribution and merge-gating designGithub (n.d.)Github (n.d.)
Agent audit-log events carry the action, an actor_is_agent flag, an agent_session_id, and the initiating user, and the enterprise audit log retains events for 180 days, so initiator and session hops are recorded for the vendor-hosted stage while longer retention needs log streamingGithub (n.d.)Github (n.d.)
The Copilot cloud agent pages consulted describe its token only behaviourally, as single-branch push without direct Git commands, and state no token lifetime, scope list, or revocation trigger, so credential revocation for the vendor-hosted stage is undocumented in the sources usedGithub (n.d.)Github (n.d.)
Copilot attributes an unattended automation's pull requests to its human creator and blocks that creator from approving them, which differs from the identity item's recommendation that unattended work run under the agent's own identity and shows vendors choosing creator-attribution plus approval separationGithub (n.d.)Enterprise (n.d.)
Copilot's firewall can be enabled, disabled, or delegated at organisation level with repository custom rules lockable, yet it covers only processes the agent starts through Bash and not MCP server processes or setup steps, so organisation-level enforceability exists but with documented coverage gapsGithub (n.d.)
Running Copilot cloud agent on self-hosted or Azure-private-network runners gives an organisation-owned intermediate option with an organisation-wide default runner, but the built-in firewall must be disabled for self-hosted runners and the organisation must supply network controlsGithub (n.d.)Github (n.d.)
Local agents have opt-in, partial controls from independent vendors: the Claude Code sandbox is off by default and covers shell commands only, and Copilot CLI states that its permission scoping is heuristic and unguaranteed outside trusted directoriesClaude (n.d.)Github (n.d.)
Organisation-deployed managed settings can enforce local Claude Code sandboxing, but only server-managed settings reach cloud sessions, so moving a task to a cloud stage leaves a locally deployed policy behindClaude (n.d.)Claude (n.d.)

Research Question

For software-engineering agent workloads in a regulated organisation, what evidence-based conditions justify placing a workload in local execution on a developer workstation, a vendor-hosted coding-agent runtime, or an organisation-owned managed agent runtime? How do scaffolding maturity, identity and attribution, raw versus sanctioned reach, model choice, triggering, egress controls, risk, parallelism, and volume affect that placement, and what measurable criteria support moving forward, skipping a stage, or moving back?

Findings

Executive Summary

A workload should move from local execution to a vendor-hosted or organisation-owned runtime only when the target stage enforces input and output gates, complete attribution, and egress control that the workload's risk class requires, and the consulted sources support this as a decision rule while publishing no numeric thresholds for it. Vendor-hosted Copilot cloud agent documents write-access gating, single-branch push, human merge, an extra approval for agent-identity pull requests, and audit events with initiator and session identifiers, while leaving credential lifetime and revocation undocumented in the pages consulted. Its firewall covers only processes the agent starts through its Bash tool and is incompatible with self-hosted runners, so organisation-owned runners are an intermediate stage whose egress control becomes the organisation's own responsibility. The hypothesis that unattended trust tracks scaffolding and validation more than supervision has convergent support from agent research, vendor engineering, and the 2025 DevOps Research and Assessment (DORA) report, but no consulted source isolates that effect from model capability.

Key Findings

Copilot cloud agent attributes commits to Copilot with the requesting developer as co-author, signs them, links each commit to its session log, restricts pushes to one branch, and requires a human to merge, which together give the vendor-hosted stage a documented attribution and merge-gating design.
Agent audit-log events carry the action, an actor_is_agent flag, an agent_session_id, and the initiating user, and the enterprise audit log retains events for 180 days, so initiator and session hops are recorded for the vendor-hosted stage while longer retention needs log streaming.
The Copilot cloud agent pages consulted describe its token only behaviourally, as single-branch push without direct Git commands, and state no token lifetime, scope list, or revocation trigger, so credential revocation for the vendor-hosted stage is undocumented in the sources used.
Copilot attributes an unattended automation's pull requests to its human creator and blocks that creator from approving them, which differs from the identity item's recommendation that unattended work run under the agent's own identity and shows vendors choosing creator-attribution plus approval separation.
Copilot's firewall can be enabled, disabled, or delegated at organisation level with repository custom rules lockable, yet it covers only processes the agent starts through Bash and not MCP server processes or setup steps, so organisation-level enforceability exists but with documented coverage gaps.
Running Copilot cloud agent on self-hosted or Azure-private-network runners gives an organisation-owned intermediate option with an organisation-wide default runner, but the built-in firewall must be disabled for self-hosted runners and the organisation must supply network controls.
Local agents have opt-in, partial controls from independent vendors: the Claude Code sandbox is off by default and covers shell commands only, and Copilot CLI states that its permission scoping is heuristic and unguaranteed outside trusted directories.
Organisation-deployed managed settings can enforce local Claude Code sandboxing, but only server-managed settings reach cloud sessions, so moving a task to a cloud stage leaves a locally deployed policy behind.
Model choice widens with each stage, from bring-your-own-key models locally, to a plan-dependent picker or Auto on the vendor-hosted agent, to any framework and foundation model on an organisation-owned runtime, and triggers widen from human start through issue, schedule, and event starts in the vendor-hosted stage.
The vendor-hosted Copilot agent works in one repository, on one branch, with one pull request per task and a 59-minute session limit documented as non-extendable, so tasks that span repositories or run longer are candidates to skip to an organisation-owned runtime.
Agent research, vendor engineering, and the 2025 DORA report agree that scaffolding, validation, and surrounding organisational systems change agent outcomes, but no consulted source separates that effect from model capability across stages, so the scaffolding-over-supervision hypothesis remains supported without being tested.
No consulted source publishes numeric placement thresholds for scaffolding maturity, risk class, parallelism, or volume, so every threshold in the placement matrix is a proposed decision rule that an organisation must calibrate against its own baselines.

Placement Matrix

Each row is a proposed decision rule, not an evidence-derived threshold, and each carries its own label and sources.

Decision	Proposed rule	Label and sources
Stay local	Keep a workload local when a developer watches the session, the task has weak or absent automated checks, and one agent runs per developer, with the sandbox and managed settings enforced where the organisation deploys them.	[inference; source: code.claude.com sandboxing docs.github.com
Move to vendor-hosted	Move a bounded single-repository task to the vendor-hosted agent when required status checks and a second human approval already gate merges, the firewall is enabled with an organisation allowlist, and the task fits the session limit.	[inference; source: docs.github.com risks-and-mitigations docs.github.com customize-the-agent-firewall docs.github.com
Use organisation-owned runners with a vendor harness	Choose this intermediate stage when the task needs internal network resources or private networking and the organisation already operates egress controls stronger than the built-in allowlist.	[inference; source: docs.github.com customize-the-agent-environment docs.github.com
Move to an organisation-owned managed runtime	Choose this stage when the work needs multiple repositories, sessions beyond the vendor limit, identity tied to the organisation's identity provider, invocation records in the organisation's own stack, or a model or harness the vendor stage does not offer.	[inference; source: aws.amazon.com its-safe-to-close-your-laptop-now-hosting-coding-agents-on-amazon-bedrock-agentcore docs.github.com
Skip a stage	Go from local directly to an organisation-owned stage when the vendor-hosted stage is disabled by policy, lacks a required control such as credential-revocation documentation, or cannot meet data-handling constraints.	[assumption; source: docs.github.com access-management docs.github.com
Move back	Return a workload to a more supervised stage when gate coverage drops below the entry condition, an agent-authored change cannot be traced to initiator and session, egress blocks reveal unsanctioned destinations, or post-merge defects traced to agent changes exceed the human baseline.	[assumption; source: Enterprise Agent Identity and Access Management docs.github.com

Measurable criteria for the matrix, each a metric whose cut-off an organisation sets from its own baseline because no consulted source supplies a number. Gate coverage is the share of agent-initiated changes that pass required checks and a second human approval before merge. Attribution completeness is the share of agent-authored changes traceable to initiator, machine identity, and session. Egress coverage is the share of agent-started processes under an enforced allowlist, together with the count of blocked-request warnings. Validation escape rate is the count of post-merge defects traced to agent changes relative to the human baseline. Session fit is the share of tasks that hit the session limit or need more than one repository. Concurrency collision count is the number of shared-host conflicts (ports, databases, credentials) among parallel local agents.

Assumptions

Organisation-owned runtimes are assumed to start unattended work through the organisation's own scheduling and event services because the consulted AgentCore sources describe no built-in trigger. That assumption is reasonable because the post presents invocation, identity, and observability as platform services while showing interactive and parallel use only. The proposed criteria are assumed to be measurable from existing repository, audit-log, and firewall data without new instrumentation. The audit-log fields, firewall warnings, and pull request records documented by the vendor make that assumption plausible for the vendor-hosted stage, though it is untested for the other stages. The absence of a numeric threshold in the consulted documentation is assumed to mean none is published, not that none exists in other sources. Reading only the pages cited here is the item's stated evidence base, so the claim is scoped to it.

Analysis

The evidence on scaffolding is consistent in direction across a benchmark paper, an interface-design paper, a vendor harness post, and an industry report, yet each measures a different thing, so the item treats the hypothesis as supported and untested. The vendor documentation is the strongest source for what each stage does, because it describes controls the vendor ships, and the weakest for how well they work, because it contains no measured outcomes. The documented unattended controls in Copilot are gates and attribution rules rather than live supervision, which supports placing unattended work where those gates are enforced. Moving a workload forward trades local convenience for governed attribution, and the matrix above makes each move conditional on a named control being present in the target stage. Strengthening scaffolding in place is a rival to changing stage, and the task-selection item's finding that verifier strength is the primary enabling condition supports doing that first. Adding reviewers to keep per-item human review is a second rival, and the documented Copilot controls already require a human merge, so the remaining question is which items need a live watcher in addition. The organisation-owned runner stage is the only documented way to combine the vendor harness with the organisation's own network boundary, and its value depends on the organisation's existing controls. The runtime item reports that a 39-paper systematisation found denylist enforcement failing at 69% to 98% in reviewed studies, which argues against relying on any one egress control, which is why the matrix pairs egress with gates and attribution. The access-control item's finding that autonomous triggers can run with maker credentials explains why creator attribution plus approval separation matters for scheduled work. The harness item's observation that local tools favour developer sovereignty and cloud tools favour organisational governance matches the pattern in the matrix.

Risks, Gaps, and Uncertainties

Credential lifetime, scope list, and revocation behaviour for the vendor-hosted agent are absent from the pages consulted, which leaves a gap in any regulatory attribution argument for that stage. The evidence base is one vendor family per stage, with Copilot for vendor-hosted, Claude Code for local and cloud examples, and AWS for the organisation-owned stage, so conclusions about "the stage" generalise from worked examples. No controlled study separates scaffolding effect from model capability or compares unattended reliability across the three stages. The consulted AgentCore sources do not document a built-in schedule or event trigger, so the organisation-owned trigger row rests on an assumption. Vendor documentation changes quickly, including the model list and automation features, so the findings carry recency risk. The proposed matrix thresholds are untested against real workloads.

Open Questions

Which controlled comparison could isolate scaffolding and validation effects from model capability for unattended coding agents across stages?
What are the documented lifetime and revocation properties of the credentials that vendor-hosted agents use, and can an administrator revoke them mid-session?
How should the credential-exchange hop be recorded for vendor-hosted agents so the linked audit chain in the identity item is complete?