What an Agent Can Do

A few months ago we changed how API keys work in All Quiet. It sounds like a small thing. A key can now be limited to specific resources and actions, and every engineer can have one that acts as them rather than as the whole company. I think it's the most important thing we shipped this year, and the reason has less to do with API keys than with what a user interface is about to become.

For most of the time I've been writing software, the interface was the part you could see. You clicked things. Then came APIs, and a script could click for you, as long as someone wrote the script. Then Terraform, and the clicking became a file you could review and check in. Each step moved the work further from a human hand, but a human still decided every action in advance.

The next step doesn't. You tell an agent what you want, and the agent works out which calls to make. Whether it talks to the system through a REST API, a command line, or an MCP server is a detail. The user talks to the agent, and the agent is the bridge. If that's where things are going, the thing the user actually touches is no longer a screen. It's the set of operations the agent is allowed to perform.

We build your core incident management software: alerting, on-call schedules, escalations, status pages. On-call work is already a stream of small intentions. Who is on call for payments? Acknowledge this. Turn that off. So we expected agents to show up here early, which is why AI incident management is a design question for us rather than a marketing category. We had to decide what an agent should be allowed to do.

Scripts and agents

With a script, you read the code and then give it a key. With an agent there is nothing to read beforehand. It decides at run time, from text, what to do. So the review can't happen only at the caller anymore. It has to happen at the permissions too. For us, that's what turned least privilege from a line on a compliance form into a design problem.

I can think of two ways an agent goes off script, and they end up in the same place.

The first is that it wants something. In July 2026, during a cyber-capability evaluation, agents built on OpenAI models got out of the environment meant to contain them and compromised parts of Hugging Face's production systems. Both companies published detailed accounts (OpenAI's, Hugging Face's). Nothing in either account suggests malice. The agents were trying to score well on a test, and the path to a good score ran through systems nobody had meant to give them. Part of that path was credentials that had been left exposed on the public internet. What a credential can do, once it's out there, is what its scope says.

The second is closer to home for us. An alert payload is text, and it was written by whatever can trigger an alert: a monitoring tool, a webhook, a script. An agent that reads incident payloads is reading input it can't trust. OWASP calls this indirect prompt injection. Simon Willison has a sharper name for the shape, the lethal trifecta: an agent with access to private data, exposure to untrusted content, and a way to act. We can't stop a language model from being confused by text, and as far as I can tell nobody can yet. What we can do is make sure that a confused agent holding an incidents-only token can't delete a team.

My own conclusion from this is that a model's refusals aren't a safety boundary. The boundary is what the system says no to.

Excessive agency

OWASP has a name for the failure that follows from all this: excessive agency. An agent is given more tools, more permissions, or more autonomy than its task needs, and then something, a confused model or an injected instruction, makes it use them. Their advice is to grant the minimum, to run the agent's actions as the actual user with that user's permissions, and to enforce authorization in the systems the agent calls rather than trusting the model to decide. That last point is the one I keep coming back to.

An agent holding an organization-wide admin key, reading text from strangers, is the textbook case. It has authority from two sources, the key and the instructions, and no reliable way to tell which one should win.

The idea underneath is old. Saltzer and Schroeder wrote in 1975 that every program and every user "should operate using the least set of privileges necessary to complete the job." Least privilege is one of those principles that's easy to agree with and easy to skip for API keys. Our public API started with one organization key that could do everything. The caller was always a script somebody had read, and that was good enough. Agents took that excuse away. For the first time the caller improvises.

There's a second thing to say plainly. We don't run your model. We don't write your agent's prompt or choose its framework, and we don't want to; you should pick those and run them where you want. What we do control is what our API will do for a given credential. So that's where we put the work.

It also meant giving organization owners a say, because scoped keys don't help much if anyone can mint an unbounded one. An owner can turn personal tokens off for the whole organization, require that requests come from known IP ranges, revoke a user's tokens, and see in the audit log whenever someone views a secret.

How do you give an agent safe, limited access to your on-call system?

Two kinds of key

We ended up with two kinds of key, because there are two kinds of caller. An organization API key acts as the organization. It's limited to the resources and actions it names, and optionally to particular teams or sub-organizations. A personal access token acts as one named person. Its effective access is the intersection of what the token allows and what that person is allowed to do right now. If you can't do something in the product, your token can't do it through the API either.

Terraform and a deployment pipeline need the first kind. They're machines acting for the company, and they should keep working when people come and go. An agent usually needs the second kind, because it's doing the work a person used to do by clicking. Give an agent an organization key and you get an over-privileged bot. Give a pipeline a personal token and it stops working the day that person leaves.

Suppose an engineer is on call and asks her agent three things: who else is on call for payments, acknowledge the incident that just fired, and turn off the integration that has paged three times tonight. The first two need on-call:get and incidents:update, and her token has them. The third would need inbound-integrations:update, which it doesn't. The agent gets a 403 and tells her so. In the morning the team looks at that integration together. That refusal is the interface doing its job.

A scope is just a resource and an action, written incidents:list or on-call-overrides:create.

A token can also ask what it's allowed to do before it tries, with a single call to /api/public/v1/auth/me. It's easier for an agent to know its limits up front than to discover them one 403 at a time.

That is the whole shape of our position, and it is worth stating it clearly:

All Quiet runs no AI of its own on your incident data: your agent, your model, your prompts, your infrastructure, operates All Quiet through scoped, audited API tokens, with full agentic capability, no AI add-on fee, and your data never leaving your control.

Testing for refusal

Most of the tests around this don't check that the API works. They check that it refuses. We run the same requests with many different kinds of key, organization keys and personal tokens with different scopes and different team and organization limits, against every resource, and check that anything outside a key's scope comes back with a 403. The suite is about a hundred files of these, and it runs on every pull request.

Every endpoint also declares, in code, which resource and which action it requires. The permission check and the API documentation are both derived from that one declaration, so the two can't drift apart.

We also wanted a view from outside. Mint Secure conducted a thorough penetration test of the public API for us. Our own tests cover the cases we thought of. An external team looks for the ones we didn't. The current certifications, hosting regions and sub-processors are on our security page and our Data Processing Agreement.

The tradeoffs

If you're about to let an agent operate software you depend on, my advice is to give it the smallest key that can do the job. An organization key for pipelines. A personal token when the agent should act as a particular engineer. And not the admin key from the Terraform repo, however convenient it is to copy.

None of this is free. It's more setup than one admin key. Agents will sometimes fail a task because a scope is missing, and I think that's the right outcome (a failed task is cheaper than a silent overreach), but it does mean someone has to go and add the scope. A personal token's power follows the person, so when someone changes teams their agent loses access on its next call, which surprises people the first time. And moving existing automation from one broad key to several narrow ones is real work that somebody has to schedule.

For a long time the interface was the part of software you could see, and we spent enormous care on it: layout, wording, what happens when you click the wrong thing. I think the interface is about to become the part that decides what an agent may do. It deserves the same care. What an agent can do is what you let it.

None of it is a paid add-on. Scoped keys and personal tokens are part of Pro, because we aren't paying for inference on your behalf.

Keep reading

• How we think about agents operating the platform — the bring-your-own-AI approach in full
• What is agentic incident management? — the definition, and how it differs from vendor-run AI
• What is agentic incident response? — what changes once an agent is in the response loop
• Incident management with Terraform and audit trails — putting the configuration itself under review
• Least-privilege API access — changelog — what shipped, and when
• Public API documentation — the full scope list and every endpoint's required permission
• Security, hosting and certifications — where the data lives and who has been audited
• Alerting, on-call and incident response in one platform — the product the agent is operating

Notes

1. Scoped organization keys and personal access tokens are available on Pro and Enterprise plans. The changelog entry has the details, and the AI incident management page describes how we think about agents operating the platform: bring your own model, we provide the API, the keys, and the audit log.
2. "Agentic" is the word the industry has settled on for software operated by agents (see agentic incident management and agentic incident response). I've said "agents" here because it's shorter and means the same thing.
3. Unrestricted organization keys still work, for existing automation. If you have one, my advice is to replace it with scoped keys, one per job, and to put the configuration itself in Terraform so the agents only ever operate it.
4. Rate limits changed at the same time: 300 requests per 20 seconds, 900 per 5 minutes, and 5,400 per hour, per key. Not the point of this essay; see the changelog entry, 25th of August.
5. GitHub's personal access tokens are probably the version of this most engineers have used: a token that acts as you and can't do more than you. Same idea, applied to incident management.