16. The question toolkit

Fluency in security is mostly the ability to ask the right question at the right moment, and to tell a strong answer from a weak one. This part collects the universal questions, shows how to sharpen vague worries, and gives checklists for judging claims, papers, advisories and products.

The ten universal questions

These apply to any system, at any layer. Every “Questions to ask” list in this guide is a specialization of them.

1. What are we protecting? (assets, and which of confidentiality, integrity and availability matters for each)
2. From whom? (the adversary’s goal, knowledge, capability and resources)
3. What can they touch? (attack surface and entry points)
4. Where are the trust boundaries, and what crosses them?
5. What does this component trust, and why?
6. If it were fully compromised, what is the blast radius?
7. Which assumption, if false, breaks everything?
8. Is this control deterministic or probabilistic? (a policy that always holds, or a classifier that usually works)
9. How would we know? (detection and logging)
10. How do we recover? (revocation, rollback, kill switch)

Sharpening vague questions

Vague questions get vague answers. A sharp question names an asset, an attacker capability and a consequence.

Vague	Sharp
“Is our chatbot secure?”	“If an attacker controls one document in our index, what is the worst action the assistant can take, and what stops it?”
“Can it be jailbroken?”	“What success rate do adaptive jailbreaks reach against our deployed configuration, and what is the impact when one works?”
“Is this model safe to use?”	“What format is the file, who published this exact revision, and did we verify its hash?”
“Are we GDPR compliant?”	“Which personal data enters prompts, logs and indexes, on what legal basis, and can we delete it on request?”
“Is this agent safe?”	“Does it combine private data, untrusted input and external communication? Which of those can we remove?”
“Do we have guardrails?”	“Which controls are deterministic policies, which are classifiers, and what is each classifier’s bypass rate?”
“Is our data private?”	“Could the model or API reveal whether a specific person’s record was used, or repeat it verbatim?”

Interrogating any security claim

When someone says a system is “secure”, “robust” or “private”, ask:

• Against what threat model? Which attacker, goal, access and budget?
• Measured how? Proof, benchmark or red team? How many trials? Were the attacks adaptive?
• What is the residual rate, and what happens on failure?
• What does it cost? Accuracy, latency, false positives, usability.
• Who verified it independently?
• What would change the answer? A new model version, a new tool, new data.

Red flags: “unhackable”; “100% detection”; “our AI firewall stops all prompt injection”; evaluation only against known attacks; security that depends on a secret prompt; no threat model stated at all.

Reading a research paper

1. Read the threat model first. Is it realistic for your setting? A white-box attack still matters for API-only models, because attacks often transfer.
2. Check what access the attack needs: queries, gradients, training data, a document in the retrieval index?
3. For defense papers: were adaptive attacks used? Is code released? Were strong baselines compared?
4. Check the metrics: success rate at what query budget; false-positive rate; cost to clean accuracy.
5. Check generality: one model or many, one language or several?
6. Look for follow-ups: search for later papers that cite it with words like “bypass” or “adaptive”. Defenses often have a public half-life.

Reading a vulnerability advisory

• Identify it: CVE ID, affected products and versions.
• Decode the CVSS vector. For example, AV:N/AC:L/PR:N/UI:N means exploitable over the network, with low complexity, no privileges and no user interaction: the urgent kind.
• Is it exploited in the wild? Check CISA’s Known Exploited Vulnerabilities catalogue and the EPSS probability score.
• Are we affected? Do we run that version, use that feature, and is it reachable?
• What is the fix? Patch, workaround or detection - and how fast can we deploy it?

Evaluating an AI security product

• What does it inspect: inputs, outputs, retrieved content, tool calls?
• Is it deterministic or model-based? What bypass rate does it show under adaptive testing?
• What latency and false-positive rate does it add?
• Where does it send our data?
• Can we test it ourselves, on our system, before buying?

What fluent answers sound like

In a design review, a paper discussion or a job interview, a strong answer about AI security usually does five things. It names the asset and the adversary. It locates the trust boundaries. It separates deterministic controls from probabilistic ones. It quantifies, with rates and uncertainty. And it states the residual risk and who owns it. Practise answering every question in this guide in that shape.