The One Question to Ask Before an Agent Touches Production
Yesterday you set permissions on your own agent. You can't do that to a vendor's — all you get is questions. Ask the right five.
Yesterday's lesson had a satisfying ending: open your desktop agent's settings, turn off what shouldn't be on, walk away safer. You will never get that ending with a vendor's agent. No settings menu, no source — just a demo, a slide, and a founder who badly wants to move on to pricing. With your own agent you configure. With somebody else's, all you can do is interrogate.
The five questions
I sat through a demo last week where the founder kept tapping the same slide. "The agent authenticates with a signed token," he said, in the tone of a man who considered the matter closed.
That tells you an agent logged in. It says nothing about what it did for the next six hours. For a person, the login is most of the story. For an agent it's the least interesting part — everything expensive happens after.
And you're not buying one agent. Counts roughly doubled in a single quarter, with nearly 38% of companies now past a hundred, and the Cloud Security Alliance found 65% of organizations have already had an incident caused by their own AI agents. You're buying a population — and its worst day.
So before any agent touches production or regulated data, ask these five. Out loud, in the room.
Who is this agent? Its own identity, or is it borrowing an employee's login? If it's borrowing, you can never separate what the agent did from what your employee did.
What was it told? Show me the exact instructions it was running under last Tuesday at 4pm. Not the template. The real ones.
Which model was it using? And how would I know if you swapped in a cheaper one last month?
What was it allowed to touch — and what did it actually touch? Two different lists. I want both.
Who says so? Can my auditor confirm all of it from a signed record, without logging into your dashboard and taking your word for it?
Number five is the clincher. The first four, a good vendor answers with a screenshot. Only the last asks whether the answer survives you not being in the room.
How to hear the answer
Asking is easy. Grading is the hard part, because a confident non-answer sounds a lot like an answer. Same question, two replies:
You: Show me the exact instructions this agent was running under last Tuesday at 4pm.
What you want: "Every run is versioned. Here's the run ID, the prompt hash for that build, and the diff from the version before — we changed it on the 9th. I'll export the record; your auditor can verify the signature without an account on our platform."
What should worry you: "The prompt is proprietary, but I can assure you it's rigorously tested, and our platform gives you full observability into agent behavior."
The second reply isn't a lie. It's a different subject — it answers how carefully we built it when you asked what actually ran. Listen for that swap. Two other tells: "observability" doing the work of "evidence," and any proof that stops being true the day you stop paying them.
What changes: You stop grading vendors on what their agent is prevented from doing and start grading them on what they can prove it did. Nearly every vendor prepared for the first. Very few prepared for the second — and the ones who did will light up when you ask.
Where else this works: This isn't a security team's question. A sales leader asks it before an AI SDR touches a prospect list. A CFO before an agent reconciles a ledger. An HR lead before an agent screens a résumé. Same questions, different blast radius.
Nobody passes on day one
And don't demand it. Most vendors will get three of five and be straight about the other two — that's a vendor I'd keep talking to. What you're buying isn't a perfect score. It's the same answer format from everyone, so you can finally line them up side by side. Today you can't, because every deck defines "secure" differently.
Run these five on your next agent purchase. If the room goes quiet, you have your answer.
Your AI Sherpa,
Mark R. Hinkle
Founding Publisher, The AIE Network
Follow me on LinkedIn
