Skip to content

Life. Adventure. Consulting. Technology.

Technology

Who Authorised That? What Happens When Software Acts Rather Than Advises

Software that recommends fits inside every control an organisation already has. Software that acts on its own authority does not, and it arrived quietly inside products that had already been bought and approved years earlier.

Chris Cooper 4 min read
A row of access barriers in a bright building lobby with daylight through tall windows

Software that recommends has been part of organisations for years. Software that acts on its own authority is new. It arrived quietly, inside products already bought.

Singapore’s regulator published a governance framework for agentic systems in January — the first to address directly what happens when software stops advising and starts doing.

1. Every Control Assumes a Person Decided

Delegated authority, segregation of duties, approval thresholds, two signatures on a payment run. All of it rests on one assumption. At the moment something happened, a person chose to do it.

That assumption is old enough that nobody states it — and it sits underneath job descriptions, audit procedures, insurance policies and the shape of every approval workflow in the building.

It has worked well for a long time, and there was never much reason to write it down.

2. Recommendation Fits. Action Does Not.

A system producing a recommendation sits comfortably inside all of that. Somebody reads it. They agree or they do not, and their name goes against the outcome.

A system that issues the credit, cancels the order, reschedules the engineer or amends the record has not made a recommendation. It has exercised authority — somebody’s authority, delegated at some point by somebody.

Nothing in the control environment was designed for the second case. Until fairly recently, the second case did not arise.

3. The Question Is Authority, Not Accuracy

Most of the discussion is about whether the thing is right — accuracy, hallucination, evaluation, confidence thresholds.

That is a real question, and it is not the governance one. Human employees are wrong sometimes too, and organisations cope — because they know what that person was permitted to do and who was supervising them.

The useful question is not whether the agent is correct. It is what it may commit the organisation to, and whose name is against the decision when it turns out badly.

4. It Arrived as a Feature, Not a Procurement

Ordinarily a capability carrying this much authority would go through something. A business case, a security review, a data protection assessment, a signature at the end of it.

Agentic features have largely not. They appeared inside software already licensed — switched on by default, or by an administrator configuring a product rather than setting policy.

So a new class of actor entered the organisation without a decision anywhere in the record.

5. The Chain Is Where It Becomes Difficult

One agent calls another, which calls an interface, which alters a record in a third system holding permissions of its own.

Traditional security models answer who authenticated. They do not answer who decided — and the account that authenticated increasingly belongs to somebody with no knowledge of the action taken on their behalf.

By the time anyone reconstructs it, the trail crosses four systems and stops at a service account.

6. Not All of It Needs This

None of which argues for treating every agent as a hazard. A great deal of what they do is low consequence and easily undone — drafting, summarising, sorting, suggesting.

Governing all of it identically produces much the same result as governing none of it — because a process applied to everything becomes a process people quietly route around.

The distinction is not how capable the agent is. It is whether its actions can be reversed, and how fast somebody would notice a wrong one.

7. What Has to Be Written Down

The practical answer is unglamorous and largely clerical. Four things, recorded for every agent running in production:

  • what it does, and which systems and data it can reach
  • what it may commit the organisation to, and what it must not
  • the named person accountable for its behaviour
  • how an action is reversed, and who would notice if one were wrong

An agent with no name against it should not be running at all. That single rule does most of the work, and it needs no tooling to enforce.

8. This Time the Question Arrived Early

Unusually, the governance question has turned up at roughly the same moment as the capability, rather than several years behind it as it normally does.

That is an advantage, and a short-lived one. The controls are straightforward to define now, while the number of agents in production is still small and somebody can remember switching each of them on.

The organisations that struggle will be the ones finding out in two years how many they have.

Final Thought

Autonomy moves accountability around the organisation. It does not remove it — responsibility still sits with the people who chose to deploy the thing and decided what it was allowed to reach.

Which makes the question in the title literal rather than rhetorical. Somebody authorised it. The only issue is whether the organisation knows who, and can say so before it is asked.

Stay in touch

Occasional writing, straight to your inbox

A short note when I publish something worth reading. No noise, and easy to leave whenever you like.