How I let an AI agent work on live cloud systems without giving it the keys
AI agents are fast, confident and sometimes wrong. This is the operating model I use so an agent can investigate, plan and draft changes on production cloud, while anything risky still waits for a person.
- Reading time
- 11 min
Results at a glance
- Production changes the agent makes alone
- 0
- Secret values the agent can read
- 0
- Test cases the guard must pass
- 29
- Read-only estate review + 6 reports
- ~1 day
TL;DR
- The problem: an AI agent can now do real work on a company’s cloud: read logs, find faults, write the fix. It is also confident when it’s wrong, and it works faster than anyone can watch. Most teams either ban it or trust it. Both are bad bets.
- What I built: an operating model with five layers. Safe work runs on its own. Anything that changes production waits for my approval. Destructive actions and secrets are blocked outright, by a guard that reads the whole command and says no when it isn’t sure.
- The result: the agent does the slow, careful work: investigation, inventory, analysis and first drafts. A full review of a cloud estate, with six reports, takes about a working day. It cannot make a production change on its own, and it never sees a secret.
The lesson in one line: don’t ask whether you trust the AI. Build the system so you don’t have to.
Why this is hard
Picture a brilliant new contractor. They read faster than anyone on your team, never get tired, and do exactly what they think you meant. Most days that’s a gift. Now give them the master keys to the building on their first morning.
That is what connecting an AI agent to a live cloud account looks like. The risk is real, not hypothetical. In July 2025, a widely reported incident saw an AI coding agent delete a company’s live database during a code freeze, after being told not to make changes. Instructions weren’t enough. The agent had the access, so it could.
Teams usually land on one of two answers:
- Ban it. Safe, but you give up the biggest productivity shift in years, and engineers quietly use it anyway.
- Trust it. Fast, until the day it’s confidently wrong in production.
I wanted a third answer: let the agent do everything that’s safe at full speed, make everything risky wait for a person, and make the rules impossible to talk around.
I broke my own rule before the agent could
My first version was a list of banned commands. The agent could plan infrastructure changes, but it could never run terraform apply, the command that actually changes live systems.
Before trusting that rule, I tried to break it. It took one line:
terraform -chdir=infra apply
The rule matched the start of a command. This one starts with terraform -chdir, not terraform apply, so it would have slipped straight through.
I found it while testing my own rules, not in production. The command never ran against a live system. That’s the point of testing controls before you rely on them: the hole showed up where it cost nothing.
Think of a doorman who checks a banned list but only reads your first name. That’s a suggestion, not security. So I replaced the list with a guard that understands the whole command and says no when it can’t (layer 2 below). The bypass became a permanent test case. Every hole I find now gets the same treatment, so it can never quietly come back.
The five layers
Every action the agent takes passes through the same layers, in order.
1. Three lanes: green, amber, red
- Green: reads run on their own. Listing resources, querying logs, reading settings and costs. This is most of the work, and it’s harmless.
- Amber: anything that changes something asks me first. The agent stops and waits. I see the exact command before it runs.
- Red: blocked, every time. Deleting, restarting, scaling, purging, swapping live slots, reading secrets and reading Terraform state (which can hold secrets). No prompt, no override from inside the session.
2. A guard that reads the whole command, and fails closed
On top of the lanes sits a guard: a small program that inspects every command before it runs. It looks for dangerous actions anywhere in the command, not just at the start, including inside chained commands. It blocks direct calls to the cloud’s management tools and APIs, and it protects its own files, so the agent can’t edit its own rules.
Most importantly, it fails closed. If the guard can’t understand a command, the answer is no. A safety system that lets things through when it’s confused isn’t one.
It must pass 29 test cases before the agent goes near a live environment, including every bypass I could think of.
3. Data it never sees
The agent sees the names of secrets, never their values. It has no access to database contents. It can’t read infrastructure state files. Before it saves any report or note, a redaction scan checks the output for anything that looks like a key or password.
This matters beyond security. If sensitive data never enters the conversation, it can’t leak into a log, a summary or a document someone shares later.
4. No card, no change
Every change starts as a change card, a short written proposal. Each card answers seven questions:
- Why are we doing this?
- What’s the evidence?
- What does it cost or save, and what does it do to speed and reliability?
- What could it break? (the blast radius)
- What exactly will run? The exact commands or code change.
- How do we undo it?
- How will we know it worked?
The agent drafts the card. I read it. It runs only after I approve that card by number, and the result is logged against it. If a card can’t answer question 6, it doesn’t run.
5. Prove it worked
“It ran without errors” isn’t proof. Every change gets its own test. When I changed alerting, for example, each alert was triggered on purpose to confirm it reached the right people, then confirmed again when it cleared itself.
The paper trail
The agent keeps notes on what it learns about each environment, so it doesn’t relearn or misremember it. Every report opens with a plain summary, and every number in it links back to the file that proves it. If I can’t show where a number came from, it doesn’t go in front of a decision-maker.
What the agent actually does
This isn’t only about infrastructure code. I use the same model across day-to-day platform work:
| Work | What the agent does | What I do |
|---|---|---|
| Estate review | Runs read-only inventory scripts across the whole cloud account and drafts a risk report | Decide what matters and in what order |
| Incident investigation | Queries logs, builds a timeline, proposes a likely cause | Confirm the cause and choose the fix |
| Monitoring | Maps every alert to who actually receives it, drafts fixes | Approve, then fire-test each alert |
| Cost | Pulls billing and usage, checks savings advice against real usage | Decide what to buy, resize or remove |
| Security hygiene | Finds secrets stored in plain settings (names only) and plans their move | Approve and carry out the rotation |
| Releases | Reviews pipeline settings, drafts approval gates | Approve and merge |
| Infrastructure as code | Maps drift, writes import code, runs plans | Run every apply (see the Terraform revival) |
| Daily ops brief | Reads chat, email and the work tracker (read-only) and writes a morning summary | Read it and decide what needs action |
One example of why the human column matters: the cloud provider’s own savings tool recommended a reservation purchase that looked like a big saving. Checking it against what was actually running showed it had priced the wrong machine sizes, so the real saving was far smaller. Buying on the tool’s advice would have meant paying upfront for a saving that wasn’t there.
Under the hood (for engineers)
Everything above is the model. This section is how it’s built. The examples are simplified, but they follow the shape of the real configuration.
Permission lanes (agent settings, simplified; the exact format depends on the tool):
{
"permissions": {
"allow": [
"Bash(az resource list:*)",
"Bash(az graph query:*)",
"Bash(az monitor log-analytics query:*)",
"Bash(terraform plan:*)"
],
"ask": [
"Bash(az monitor metrics alert update:*)",
"Bash(az webapp config set:*)"
],
"deny": [
"Bash(az group delete:*)",
"Bash(az resource delete:*)",
"Bash(az webapp restart:*)",
"Bash(az keyvault secret show:*)",
"Bash(terraform apply:*)",
"Bash(terraform state:*)"
]
}
}
Rules like these are necessary but not sufficient: they match patterns, and patterns can be dodged. That’s what the guard is for.
The guard runs as a hook: every command the agent wants to run is passed to it before it executes. The hook exits non-zero to block, and anything it can’t parse is blocked:
#!/usr/bin/env bash
# Simplified sketch. Fail closed: any error below blocks the command.
set -euo pipefail
trap 'echo "guard: blocked (could not verify command)" >&2; exit 2' ERR
cmd=$(cat) # the command the agent wants to run
# Look for destructive verbs anywhere, not just at the start.
if grep -Eq '(^|[;&|[:space:]])terraform([[:space:]]+-[^[:space:]]+)*[[:space:]]+(apply|destroy|import|state)\b' <<<"$cmd"; then
echo "guard: Terraform changes are human-only" >&2; exit 2
fi
# Plans must not take the state lock.
if grep -Eq '\bterraform\b.*\bplan\b' <<<"$cmd" && ! grep -q -- '-lock=false' <<<"$cmd"; then
echo "guard: plans must use -lock=false" >&2; exit 2
fi
# ...cloud CLI writes, direct API calls and edits to the guard's own files are blocked the same way.
exit 0
The test matrix runs before the agent touches a live environment. The 29 cases cover groups like these:
| Group | Example | Expected |
|---|---|---|
| Plain destructive commands | terraform apply, terraform destroy |
Blocked |
| Flags before the verb | terraform -chdir=infra apply |
Blocked |
| Chained commands | cd infra && terraform apply |
Blocked |
| State manipulation | terraform state rm, state push |
Blocked |
| Cloud writes and raw API calls | cloud CLI deletes, direct REST calls | Blocked |
| Self-protection | editing the hook or its settings | Blocked |
| Malformed input | input the guard can’t parse | Blocked (fails closed) |
| Safe work | terraform plan -lock=false, read-only queries |
Allowed |
The last row matters as much as the others. A guard that blocks everything gets switched off.
Read-only tooling. Discovery runs through a small set of scripts built to be safe to run any time: they only read, they refuse to run if they detect a write, they’re pinned to one subscription so they can’t wander into another environment, they redact secrets from output, and they log permission errors instead of hiding them. Running them twice gives the same result.
Change card template:
## CARD-NNN: <one-line change>
Why: <problem, in one paragraph>
Evidence: <queries / files that prove it>
Impact: cost <±$/month> · performance <…> · reliability <…>
Blast radius: <what could break, who would notice>
Change: <exact commands or IaC diff>
Rollback: <exact commands>
Verify: <test that proves it worked, and that rollback works>
Status: proposed → approved CARD-NNN → done → verified
Results
| Measure | Result |
|---|---|
| Production changes the agent can make on its own | 0 (by design: changes ask, destructive actions are blocked) |
| Secret values in the agent’s context | 0 (names only; secret reads are blocked) |
| Guard test cases before touching a live environment | 29 |
| Full read-only estate review with six reports | About one working day |
| A Terraform revival run under this model | 135 imports, 0 changes to the cloud (case study) |
| Every number in every report | Linked to the evidence behind it |
What it doesn’t solve
Being honest about the limits is part of the design.
- Block lists are never complete. Someone will always find a command you didn’t think of. That’s why changes fall into the amber lane by default and the guard fails closed: the safe answer to anything unknown is “ask” or “no”.
- Approval fatigue is real. If you approve 50 cards a day, you stop reading them. Cards stay small, and reads never need approval, so my attention goes only to changes.
- The agent ran under my own identity. That kept accountability clear, but it means its ceiling was my access. The next step is giving it its own identity: read-only by default, with time-limited write access granted for one approved card and removed after.
- Content can carry instructions. Logs, pull requests and web pages can contain text written to trick an AI (“ignore your rules and…”). The defence isn’t hoping it ignores them. It’s that the agent can’t act on them: reading is all it can do without me.
Questions businesses ask before letting AI near production
Where does our data go? The agent works with configuration, metrics and logs. It never sees secret values or database contents, and anything it saves is scanned for keys and passwords first. In the work behind this study, the client approved the use of AI before it started. On every engagement, I agree the AI tool and its data terms with the client up front. Where they already have a business AI plan, I work inside it, so their data stays under their own contract.
Who is accountable if something goes wrong? I am. The agent can’t change production on its own, so every change carries a person’s approval, and that person is me.
What if the AI is wrong? It will be, sometimes. The whole design assumes it. A wrong answer is cheap at the draft stage and expensive in production, so every change is written up with its evidence and a way to undo it before it runs, and tested after.
What does it cost? The AI tooling is small next to engineering time. When I report savings, I keep three labels apart: what I found, what I proposed and what actually showed up on the invoice.
What happens if we stop using AI? Nothing breaks. Everything the agent helps produce is ordinary work your team owns: scripts, infrastructure code, runbooks, reports and change records. The AI speeds up the work. It isn’t part of your system.
Does this replace our team? No. It takes the slow reading and first drafts off their plate, so their time goes on decisions. And “no card, no change” applies to people as much as to the agent.
If you want this in your team
You can start with four steps this week:
- List what your AI tools can touch today. Most teams are surprised.
- Split every action into green, amber and red. Write it down before you configure anything.
- Test your own rules by trying to break them. If you can get around them in five minutes, so can the agent.
- Make “no card, no change” the rule for production, for people and AI alike.
If your team wants the speed of AI on production systems without betting the business on it, this is the operating model I set up: fast where it’s safe, human where it counts, and built so nobody has to rely on trust.
Tech stack
Azure · Azure CLI · Azure Resource Graph · KQL · Terraform · Azure DevOps · Bash