Your CI pipeline automates everything except the decisions

AI-native SDLC, part 1. What to test, why a build failed, who owns it, whether a release is safe: people still decide, and the test suite never learns from what they decide. Five maturity levels, five layers, and the guardrails and evals that keep agents safe.

Share

Share on LinkedInShare on FacebookShare on X
The same pipeline, two operating models
Change
Build
Review
Test
Triage
Release
Deploy
Monitor
Traditionalscripts execute, people judge
Engineer writes, often with AI tools
Script runs
Peer reads the diff
Full suite runs
Engineer reads logs
Manual go or no-go
Script deploys
On-call notices
AI-nativeagents judge, people govern
Agents write code to a specification
Agent fixes breaks
Agent review with proof
Agent selects and adds tests
Agent classifies and routes
Person approves on evidence
Agent watches canary
Incidents become tests
Scripted stepPerson decides, and work waitsAgent acts, with evidencePerson approves at a gate

An AI-native pipeline moves people from making every call to governing how calls are made.

This part sets out the framework: five maturity levels, five layers, and how each layer is built. Part 2 shows it at work in one enterprise's pipeline; part 3 compares it with the other ways to get there.

01 · Maturity model

The five maturity levels

Most enterprise pipelines sit at level 1 or 2; the five questions below tell you which. Level 3 is the turning point, where agents start to do the work and people move to approving it.

L1Scripted

People make every judgement call.

L2AI-assisted

AI suggests. A person reads every output.

L3Agent-augmented

Agents propose. People check every output.

L4Agent-operated, human-gated

Agents run full stages with evidence. People approve at gates.

L5Closed loop

Outcomes update knowledge and tests. The pipeline improves itself.

L3 to L4Agents act on their own within limits

Requires evidence on every action, explicit permissions and accurate context.

L4 to L5The system learns from outcomes

Requires every miss, rejection and incident to flow back into knowledge and tests.

LevelWho decidesWhat it looks like when a build fails
L1 ScriptedPeople, on everythingAn engineer reads the logs and works out the cause
L2 AI-assistedPeople, helped by AIAn AI summary of the logs; the engineer still decides
L3 Agent-augmentedAgents propose, people check each oneThe agent suggests "flaky, re-run"; the engineer checks and re-runs
L4 Agent-operated, human-gatedAgents act within limitsThe agent confirms it is flaky, re-runs, quarantines and notifies the owner
L5 Closed loopPeople set policyThe same, and the pattern updates the knowledge store so similar failures are caught earlier

02 · Where you stand

Five questions your pipeline should be able to answer

Each one is a decision people make by hand today. If you can't answer two of them from your own records, your pipeline is at level 1 or 2 of the maturity model above, and that is where an audit starts.

  1. What share of last quarter's red builds were re-run without anyone finding the cause?Re-run until green is a decision; nobody records why it was taken.
  2. How many of last quarter's escaped defects repeated a failure pattern you had seen before?Fixes target the single defect; the same mistake in neighbouring code stays untested.
  3. How long does a failing build take to reach the person who can fix it?CODEOWNERS says who owns a file, not who fixed this kind of failure last time.
  4. Do your riskiest areas, where one feature meets another system, have a test for the last thing that broke there?In our case study, three in four of the riskiest areas sat exactly there.
  5. Which of your last ten release decisions rested on evidence, and which on a meeting?Go or no-go is usually the most expensive judgement in the pipeline, and the least recorded.

03 · Reference architecture

Five layers that make the pipeline AI-native

Agents are one layer. Without the others, they produce noise that people learn to ignore.

05Measurement and learning
Delivery metricsPipeline metricsEscaped defectsFeedback into knowledge
04Verification and control
Proof on every callAgent permissionsApproval gatesAudit trail
03Agents
Review and testTriage and fixRelease evidenceDeploy watch
02Workflow and data flow
Pipeline eventsRouting to ownersIssue tracker linkChat and review tools
01 · FoundationKnowledge store
Product mapRisk areasFailure patternsTest and failure history

Outcomes from Layer 05 continuously update the knowledge store in Layer 01.

01 · Knowledge storeWhat agents know

Product structure, where defects cluster, rules that must hold, and test and failure history.

02 · WorkflowHow agents plug in

Triggered by pipeline events; findings reach the right owner in the tools teams already use.

03 · AgentsWhat agents do

Narrow agents for review, test selection, triage, fixes, release evidence and deployment checks.

04 · Verification and controlWhat keeps agents safe

Evidence on every decision, explicit permissions, approval gates and an audit trail.

05 · MeasurementHow the pipeline improves

Metrics against a baseline, with every outcome fed back into the knowledge store.

Read onHow each layer is built

Sections 04 to 07 cover the knowledge store, agents, guardrails and evals. What we build and what we reuse is in part 3.

Examples in each layer are illustrative, not an exhaustive list.

04 · Layer 01 · Knowledge store

How the knowledge store is built

It combines four kinds of knowledge, each stored in the form that suits it. Agents query each one through tools when a task needs it.

StoreHoldsBuilt with
Code graphFiles, symbols, dependencies, build targets, test coverageCode indexers and build-system queries; updated every merge
HistoryRuns, failures, flaky record, defects, releasesTables in your database; ingested continuously
Curated rulesProduct map, risk areas, failure patterns, intended behaviourVersioned files in git, owner-reviewed each release
DocumentsTickets, design docs, runbooks, incidentsKeyword plus embedding search; indexed nightly

Mapping code to product intent and signal flow

Requirementse.g. DOORS, Polarion
→
Signalse.g. AUTOSAR, CAN
→
Componentscode graph
→
Tests and defectsverification history

Signals come from interface definitions, not inferred from code. Agents can then answer which requirements a change affects, who consumes a modified signal, and which paths have no test, and cite requirement IDs as traceability evidence.

From our engagementsDefect history is a better starting point than documentation. For code, the build graph and navigation tools are more reliable than similarity search. Curated rules work best when they are few and each has an owner.

05 · Layers 02 and 03 · Workflow and agents

How agents are orchestrated and routed

Routine steps such as fetching logs or bisecting commits are written as ordinary code. Agents handle only the decisions that need reasoning, which keeps runs cheaper and easier to debug.

How agents communicate

Orchestrator and workers

Agents never call each other directly; the orchestrator assigns and collects.

Typed artifacts

Findings, evidence and verdicts in fixed schemas; malformed output is rejected.

Producer and checker

A separate agent, without the first one's reasoning, tries to disprove each finding.

Bounded fan-out

Parallel sub-agents within a fixed cap per run.

Orchestration choice

FrameworksLangChain, LangGraph

Fast to prototype; extra abstraction is harder to debug.

Provider SDKse.g. Claude Agent SDK

Tool use and sub-agents built in; tied to one provider unless wrapped.

Our defaultThin orchestrator we own

Calls a provider SDK behind a model-agnostic interface.

How building this compares with buying a product is in part 3.

Routing beyond CODEOWNERS

Who fixed similar defects

Usually the strongest signal for a given failure pattern.

Who changed and reviewed the code

Recent changes to the files and functions involved.

Declared ownership

CODEOWNERS, tracker components and on-call, as the fallback.

Both sides of an interface

So boundary failures do not bounce between teams.

Routing recommends an owner and team with evidence; it does not score individuals. Where works council rules apply, person-level data stays within agreed bounds.

06 · Layer 04 · Guardrails and kill switches

Guardrails and kill switches

Every agent action passes through a single gateway between the agents and your systems. Permissions, limits and kill switches are enforced there, in code.

Agentproposes an action
→
Kill switchenabled?
→
Policyallowed for this tier?
→
Budget and schemawithin limits?
→
Execute and logreversible action ID
Permission tierAgent mayMoves up when
T0 ObserveDecide silently, in shadow modeAccuracy against people meets the agreed bar
T1 SuggestComment, open draft changesEngineers act on most suggestions
T2 Act, reversibleRe-run, quarantine, revert on mainPeople rarely reverse its actions
T3 IrreversibleNever alone: release, production deployStays with a named person

Stopping agents instantly

Global stop

A flag the gateway checks before every action; in-flight runs are cancelled.

Per agent or action type

Same mechanism, finer grain; the rest keeps running.

Automatic circuit breakers

Rising reversals, errors or cost drop an agent to suggest-only.

Undo and revoke

Batch rollback by action ID; revoking service accounts as last resort.

Also enforced in code: per-agent tool allow-lists, short-lived scoped credentials, and untrusted text (logs, PRs, tickets) treated as data, never as instructions.

07 · Layer 05 · KPIs and evals

Measuring agents and their outcomes

We track pipeline outcomes each release, test every agent before it ships, and monitor it while it runs.

LevelExamplesWhen
OutcomesDORA metrics, time to triage, pipeline time, escaped defects, cost per changeEach release, against baseline
Task evalsAccuracy on labelled past cases from your history, per agentEvery change to prompts, models or tools
OperationalAcceptance with reasons, reversals, evidence completeness, latency, costContinuously, with drift alerts

Generic tests every agent passes

Contract

Valid schema, valid tool calls, within budget.

Permissions

Out-of-scope actions are blocked.

Bad input

Missing or broken data gives "cannot decide".

Injection

Planted instructions change nothing.

Regression

Known cases keep their answers.

Consistency

Repeat runs reach the same decision.

Measuring hallucination

A hallucination is any claim its evidence does not support. Every finding must cite evidence, so it can be checked:

Citations resolve

Cited files, lines, logs and requirements exist and say what is claimed.

Claims reproduce

Twice from a clean environment, with a failing test.

Independent challenge

A checker agent tries to disprove it.

Human sampling

Regular audits calibrate automated checks.

Learning loop in v1Recording outcomes, proposing new rules and tests, and running them through evals are automated. Applying changes needs an owner's approval until each type of change has a measured track record.

Read next

Find out where your pipeline stands

Most enterprise pipelines sit at level 1 or 2. Fast Code AI takes frontier AI into production for enterprises, with teams in India, Germany and the UAE. We will audit your pipeline inside your environment, place each stage on these five levels from your own records, and show you where agents would pay off first. See what an audit finds, or ask for one.

Want to know more about AI ML Technology

Incorporate AI ML into your workflows to boost efficiency, accuracy, and productivity. Discover our artificial intelligence services.

Read More Blogs

View All

arrow right
logologo
Request an AI summary of Fast Code AI
  • Head Office
  • Jbr Tech Park, Plot No. 77,
    6th Rd, EPIP Zone,
    Whitefield, Bengaluru,
    Karnataka 560066
  • UAE Office
  • First Floor, Incubator Building,
    Masdar City SE4505,
    Abu Dhabi
  • German Office
  • Königstraße 26
    Stuttgart 70173
  • Email
  • arjun@fastcode.ai
  • Phone
  • +91 85530 38132

© Copyright Fast Code AI 2026. All Rights Reserved