Your agent can stay within its permissions and still leak a patient’s history
Your agent can stay within its permissions and still leak a patient’s history
AI agents need real access to do real work. Access controls check whether each action is allowed. They don’t check whether it fits the task the agent was given.
AI agents need real access to do real work. Access controls check whether each action is allowed. They don’t check whether it fits the task the agent was given.
A claims agent, step by step
A claims agent, step by step
It can read patient records, and it’s allowed to send documents to insurers.
It can read patient records, and it’s allowed to send documents to insurers.
One lab result
One lab result
An insurer asks for one lab result to process a patient’s claim. A normal request.
An insurer asks for one lab result to process a patient’s claim. A normal request.
Permitted
Open the record
Open the record
The agent opens the patient’s record. It needs that access to do the job.
The agent opens the patient’s record. It needs that access to do the job.
open patient record
Permitted
Keep reading
Keep reading
It reads diagnoses, medications and clinical notes. Each read is allowed. None of it is needed.
It reads diagnoses, medications and clinical notes. Each read is allowed. None of it is needed.
read diagnoses
read medications
read clinical notes
Permitted
Send it all
Send it all
It sends the full medical history to the insurer. It’s allowed to send documents there.
It sends the full medical history to the insurer. It’s allowed to send documents there.
send to insurer:
lab result
diagnoses
medications
clinical notes
Every step was permitted. The insurer now has the patient’s whole history.
Every step was permitted. The insurer now has the patient’s whole history.
Money works the same way. An accounts payable agent paid a new bank account because a line of white text in the invoice told it to.
Money works the same way. An accounts payable agent paid a new bank account because a line of white text in the invoice told it to.
Access controls ask the wrong question
Access controls ask the wrong question
They check each action on its own, so they can’t see what the actions add up to.
They check each action on its own, so they can’t see what the actions add up to.
Access controls ask
Is this agent allowed to do this?
Is this agent allowed to do this?
For both agents above, the answer was yes at every step.
For both agents above, the answer was yes at every step.
A supervisor asks
Does this action fit the task?
Does this action fit the task?
Sending a full history doesn’t fit a request for one lab result. Paying a new account doesn’t fit an approved invoice, and the order came from hidden text in the invoice. A supervisor sees both because it looks at the task, the conversation so far, the data involved and where each instruction came from.
Sending a full history doesn’t fit a request for one lab result. Paying a new account doesn’t fit an approved invoice, and the order came from hidden text in the invoice. A supervisor sees both because it looks at the task, the conversation so far, the data involved and where each instruction came from.