"State the instructions-are-not-data principle and explain why it fails architecturally for LLMs." "Classical principle: data cannot be executed as instruction without an explicit control-flow handoff (SQL parameterization, NX bit, DEP enforce this). FAILS FOR LLMS because an autoregressive model is a single function consuming one token sequence — system prompt, user input, tool output are ALL concatenated and processed uniformly by attention. No parser separates instruction from data. Greshake et al.: the model cannot distinguish developer instruction from untrusted data it was told to process. This is the ARCHITECTURE, not a bug. Safety RLHF moves the success rate (60%→4%); it does not build the boundary. The boundary must move into the harness (B2)." c2b::b2::recall "Enumerate the five prompt-injection attack classes and state which is most dangerous and why." "(1) DIRECT — user→model; crudest, narrow surface (one user, one session). (2) INDIRECT — tool output/web/doc→model; MOST DANGEROUS: crosses a trust boundary invisibly, scales (one poisoned page hits every agent), attacker needs no account, and is the InjecAgent ~50% vector. (3) MULTI-STEP — across turns, each step benign; defeats per-turn defenses. (4) ENCODED/OBFUSCATED — base64, homoglyphs, split; defeats pattern matchers; RedAgent's mutation class. (5) CONTEXT FLOODING — saturate the window so system-prompt attention degrades; heap-spray analogue." c2b::b2::recall "What two empirical findings make prompt injection an engineering priority, and what does each establish?" "(1) INJECAGENT — ~50% of agentic tasks vulnerable to indirect injection via tool outputs. Not a sophisticated attack that sometimes works; the DEFAULT state of an undefended agent on a random task. The bridge benchmark from Course 2A. (2) REDAGENT — most black-box LLMs jailbreakable within 5 queries via automated mutation. Sets the offensive floor: an attacker with automation finds a working payload fast, so a defense relying on 'the attacker won't think of the right phrasing' defends against humans, not automation. Together: delivery works ~50%, payload found in ~5 queries — the ship-to-hijack window is short." c2b::b2::recall "What is the difference between direct and indirect injection, and why is indirect more dangerous?" "DIRECT: the user types the injection ('ignore your instructions'). The attacker IS the user, is authenticated, and the action is plausibly attributable to them — narrow surface. INDIRECT: the attacker plants content where the agent will RETRIEVE it (web page, email, doc, API response, memory). The agent's tool faithfully returns attacker-controlled string; harness inserts it into context; if untagged, the model may comply. MORE DANGEROUS because: (a) crosses a trust boundary invisibly (agent sought data, got instructions); (b) scales (one page hits every fetching agent); (c) attacker needs no account on the target; (d) it is the InjecAgent ~50% vector — the default vulnerability, not an edge case." c2b::b2::recall "Why does multi-step injection defeat per-turn defenses, and what is the required defense shape?" "Multi-step injection pursues the injected goal across multiple tool calls, EACH INDIVIDUALLY BENIGN (step 1 reads a file, step 2 reads another, step 3 combines and exfiltrates). No single turn looks malicious — the attack is the SEQUENCE/compound trajectory. A judge model evaluating each call in isolation sees benign calls. An allowlist permitting read_file permits all three reads. This is the Microsoft zero-click HITL bypass chain: per-step approvals fail because no single step is the attack. REQUIRED DEFENSE SHAPE: SESSION-LEVEL — track the agent's intent across turns, compare the emerging trajectory against the user's original goal, halt on divergence. This is why B2's Layer 3 taint tracking propagates ACROSS turns, not just within one." c2b::b2::recall "Enumerate the five defense layers (B2.2), and for each state what it stops and what bypasses it." "L1 UNTRUSTED TAGGING — wrap inbound content in tags. Stops naive direct/indirect. Bypassed: model ignores tag (tag is just tokens). L2 INSTRUCTION ISOLATION — system prompt in privileged role. Stops override attempts. Bypassed: attention domination. L3 TAINT TRACKING + GATE — mark untrusted, propagate, block high-impact tool calls with tainted args (DETERMINISTIC). Stops impact even if model fooled. Bypassed: taint laundering (memory/files) → B3. L4 SECONDARY-MODEL DETECTION — separate model judges override intent, decodes+analyzes. Stops encoded class. Bypassed: obfuscation fools detector; ADVISORY not sole gate. L5 CAPABILITY MINIMIZATION — tools scoped to minimum, sandboxed. An injected agent cannot call what does not exist (DETERMINISTIC). Bypassed: sandbox escape (B7) / over-privilege (B5)." c2b::b2::recall "What is taint tracking (Layer 3), and why is it the layer that does the real work?" "Taint tracking is a classical information-flow-control technique: mark untrusted data with a label, PROPAGATE the label through every computation that touches it, and GATE any computation that would let tainted data influence a privileged action. Applied to agents: untrusted content (tool outputs, memory, peer messages) enters tagged tainted; when the model emits a tool call, the harness checks whether the call's arguments DERIVE FROM tainted content; if tainted + high-impact tool → BLOCK. It does the real work because it stops the attack at the MOMENT OF IMPACT (the tool call), even if the model was fooled, even if the tag was ignored, even if isolation was weak. This is where InjecAgent's 50% drops toward zero. It is DETERMINISTIC — no model in the loop, no bypass rate. Literal containment catches obvious cases; a semantic check (Layer 4 hook) catches paraphrasing." c2b::b2::recall "Why is Layer 4 (secondary-model detection) advisory rather than the sole gate, and what would go wrong if it were the sole gate?" "Layer 4 is a separate model call judging 'does this content contain an override attempt?' It is ADVISORY (raises score, triggers approval) rather than the sole gate because it is PROBABILISTIC — a sufficiently clever injection can fool the detector too (RedAgent finds bypasses in ~5 queries). If it were the sole gate, the defense would have a nonzero bypass rate findable by automation. The design: Layer 3 (taint gate) is DETERMINISTIC and gates high-impact calls regardless of what Layer 4 thinks; Layer 4 catches the encoded class that Layer 3 (which checks derivation, not content) misses, and raises the score for review. Determinism on the structural boundary (taint); probability on the semantic boundary (content). That is B2.3's resolution." c2b::b2::analysis "What is the B2.3 resolution of the probabilistic-vs-deterministic tension, and why is each assigned to its boundary?" "RESOLUTION: determinism on the TAINT boundary (L3 gate + L5 capability) and probability on the CONTENT boundary (L4 detector). The taint boundary is STRUCTURAL and enumerable: tainted? high-impact? in the allowlist? The rule is simple, the compile is trivial, no model in the loop, no bypass rate — IronCurtain's strength where it works. The content boundary is SEMANTIC and open: 'is this an override attempt?' No deterministic rule covers the obfuscation space — CrabTrap's strength where it works. MISAPPLICATIONS: probability where determinism is possible → unneeded bypass rate; determinism where space is open → unaffordable coverage gaps. THE HYBRID: deterministic gating on taint (L3) + probabilistic detection on content (L4) + capability minimization as floor (L5). Attack must defeat all three KINDS of defense." c2b::b2::analysis "Name three specific failure modes of CrabTrap (DD-19, probabilistic) under offensive pressure." "(1) THE JUDGE IS A MODEL — the same indirect-injection techniques that hijack the primary agent can fool the judge; obfuscated/encoded/multi-step-assembled content reads as benign to a per-call judge. Probabilistic defense against probabilistic attack; RedAgent finds bypasses in ~5 queries. (2) RESPONSE-SIDE GAP — CrabTrap filters the outbound request (what the agent calls) but historically NOT the inbound tool return value; attacker-controlled returns enter the context unfiltered — the indirect-injection vector. (3) LATENCY BUDGET — a judge model adds per-call latency; under load, agents skip the check or run it on a cheaper/weaker model — the defense degrades exactly when the system is busy (when attackers strike)." c2b::b2::analysis "Name three specific failure modes of IronCurtain (DD-20, deterministic) under offensive pressure." "(1) COMPILATION FIDELITY — policies are compiled by a build-time LLM; if an attacker influences the policy text or build-time context, the COMPILED rule may not match the INTENDED rule, and the deterministic enforcer faithfully enforces the WRONG thing. Determinism AMPLIFIES a poisoned compile (no probabilistic second opinion). (2) BRITTLENESS — 'block email to external recipients' compiles to a domain check; laundering the recipient through a trusted alias the compile didn't foresee defeats the rule, and the enforcer (no judgment) permits it. (3) COVERAGE — rules cover only what was enumerated; novel attacks (a tool used in a way the policy didn't contemplate) pass because NO RULE MATCHES. Certain on the enumerated set, zero coverage on the rest." c2b::b2::analysis "Why does safety RLHF not resolve prompt injection, and what does it actually change?" "Safety RLHF changes the SUCCESS RATE of a given injection (e.g., 60%→4%); it does NOT change the CATEGORY. The problem is architectural: an LLM processes instructions and data as one token stream with no parser separating them. A model trained to refuse 'ignore your instructions' will still comply with a sufficiently OBFUSCATED or CONTEXTUALLY-DISGUISED version, because it has no mechanism to know the obfuscated instruction is DATA rather than INTENT. The model cannot enforce the boundary because the model is where the boundary should be enforced AND where it is violated. Moving the rate is valuable (and B2's defenses do better), but it is a probabilistic improvement on an architectural problem. The architecture is the harness's to fix — that is why B2 builds Layers 1-5 in the harness, not in the model." c2b::b2::analysis "Explain the taint-laundering bypass of Layer 3 and which module closes it." "Layer 3 gates tool calls whose arguments DERIVE FROM tainted content. BYPASS: an injection causes the agent to WRITE a value to a TRUSTED store (a memory entry the harness does not taint, or a file a later turn reads as trusted), and a LATER turn uses the laundered value in a privileged call. The per-call taint check sees a trusted-sourced argument and passes it — the taint was laundered through the store. This is why per-call taint tracking is insufficient: taint must propagate ACROSS TURNS and ACROSS STORES. B3 (Memory and Context Poisoning) closes this by making memory writes HARNESS-MANAGED (the agent proposes, the harness decides what to persist; never let the model write raw memory) and RETRIEVAL-TIME-TAGGED (every retrieved fact re-enters tainted), so the laundered value re-enters with its taint intact. The sleeper attack (poison in session 1, activate in session 2) is exactly this bypass." c2b::b2::analysis "Trace the indirect-injection attack path and name the five defense layers that could halt it, each at its point of action." "PATH: (1) attacker content on a page; (2) fetch_url returns it, untagged; (3) enters context, model reads as instruction; (4) loop selects action, goal hijacked; (5) tool executes with real creds; (6) impact (SSH key exfiltrated). DEFENSE POINTS: (2) L1 TAG — wrap tool output in . (3) L2 ISOLATE — system prompt privileged, model taught to obey tag. (4) L3 TAINT GATE — args derived from tainted content + high-impact tool → BLOCK (deterministic). (4) L4 DETECT — secondary model judges override intent (advisory). (5) L5 CAPABILITY — read_file allowlisted to /var/agent/safe/ only. Any ONE halts the chain. The attack reaches impact only if ALL FIVE fail. Most common real failure: step 2→3, the missing tag (the ~50% rate). L3 is the deterministic backstop when the model is fooled." c2b::b2::analysis "What does it mean to say 'the five layers are a conjunction, not alternatives,' and what is the defense-in-depth claim?" "The layers are a CONJUNCTION: an attack reaches impact only if it simultaneously (1) gets past the tag (L1), (2) overcomes isolation (L2), (3) defeats or launders the taint gate (L3), (4) fools/evades the detector (L4), and (5) stays within the capability ceiling (L5). Each bypass condition is DIFFERENT (tag-resistance, attention-domination, taint-laundering, detector-fooling, capability-conformance). THE CLAIM: the conjunction of bypasses is HARD — an attacker must find a payload that defeats all five at once, and each layer's bypass is a different kind of challenge. This is NOT a proof of security; it is an ARGUMENT ABOUT ATTACKER WORK FACTOR. The work factor is what you measure (InjecAgent-style residual rate). No single layer suffices; the conjunction is the defense." c2b::b2::analysis "Why is 'we added prompt injection defenses, the agent is secure' not a deliverable, and what is the correct deliverable?" "Declaring 'secure' after deploying defenses repeats B0's anti-pattern of treating an AI finding as binary fixed/unfixed. Prompt injection defenses REDUCE the success rate; they do not eliminate it (the architecture cannot be perfectly patched in the harness either — residuals like taint laundering remain until B3, sandbox escape until B7). THE CORRECT DELIVERABLE: measure the residual injection success rate under an InjecAgent-style harness (agentic tasks with indirect injections planted in tool outputs; run the defended agent; count successes) and report the NUMBER. E.g., 'injection success dropped from 50% to 2% under InjecAgent; the residual 2% is the encoded-and-laundered class, which B3 and L4 are tuned to reduce.' The number, characterized by its residual class, is the engineering deliverable." c2b::b2::analysis "Why is capability minimization (Layer 5) deterministic, and why does B2's gate depend on B5 and B7?" "Layer 5 is deterministic because it does not care HOW CLEVER the injection is — the tool simply does not have the capability the injection requests. An injected agent that cannot call send_email cannot exfiltrate by email. An injected agent whose read_file is allowlisted cannot read the SSH key. This acknowledges the B2.1 architectural truth: if you cannot perfectly prevent the model from being fooled, you can perfectly LIMIT what a fooled model can do. B2's gate DEPENDS ON B5 and B7 because the capability ceiling is only as strong as the credential scoping (B5 — excessive agency means the agent HAS a capability it should not) and the sandbox boundary (B7 — a sandbox escape reaches capabilities outside the ceiling). L5's residual is sandbox escape / over-privilege; B5 closes over-privilege, B7 closes escape." c2b::b2::analysis "How does the B2 layered defense map to Course 1's 6-layer model and 9-layer stack?" "B2's five layers are the INJECTION-RESISTANCE SPECIALIZATION of Course 1 Module 11's stack, not a replacement. Mapping: Course 1's 'input validation' layer = B2's L1 (untrusted tagging). Course 1's 'instruction hierarchy' layer = B2's L2 (instruction isolation). Course 1's 'policy enforcement' layer = B2's L3 (taint tracking + gating) and the deterministic core. Course 1's 'output governance' layer = B2's L4 (secondary-model detection, CrabTrap-generalized) and L5 (output sandboxing, IronCurtain-style). The 9-layer stack's credential quarantine (IronCurtain) underpins L5's capability minimization; its sandbox underpins L5's execution ceiling. B2 is the injection-specific slice of the full defense-in-depth architecture Course 1 introduced." c2b::b2::analysis "What is the role of the separate system role (e.g., OpenAI/Anthropic system role) in Layer 2, and why is it insufficient alone?" "The system role places the developer's instructions in a layer the model attends to with STRUCTURALLY DIFFERENT WEIGHT than user/tool content — it is the provider API's mechanism for instruction isolation. Using it is correct and necessary (L2). INSUFFICIENT ALONE because: (a) it is still tokens in the same model — a sufficiently strong or obfuscated injection can dominate attention despite the privileged position (the architecture has no hard wall); (b) it does not prevent the model from EMITTING a tool call with tainted arguments (that is L3's job); (c) context flooding can degrade the system prompt's effective attention weight. L2 raises the bar and makes L3's job easier (the model is TOLD not to use tainted content for high-impact calls), but when the model does it anyway (because fooled), L3 catches it deterministically. L2 is probabilistic; L3 is the deterministic backstop." c2b::b2::analysis "Why must the Layer 4 detector be a SEPARATE model call, and what should it be prompted to do?" "SEPARATE because: an injection that fooled the PRIMARY model should not automatically fool the detector — different context, different prompt, different objective. If the detector ran in the same context as the agent, the same injection that hijacked the agent would hijack the detector. A separate call with a narrow objective ('does this content contain instructions attempting to override the agent's objectives or extract its system prompt?') gives an independent judgment. PROMPTED TO DECODE AND ANALYZE — explicitly instructed to base64-decode, read comments, consider multi-step assembly — so it catches the ENCODED/OBFUSCATED class (L4's specific job) that pattern-matchers and the taint gate (which checks derivation, not content) miss. It is the CrabTrap pattern applied as a DETECTOR (inbound) rather than an egress filter (outbound)." c2b::b2::analysis "Given an undefended agent with fetch_url injecting into context, trace why InjecAgent measures ~50%, and which B2 layer most directly drops that number." "UNDEFENDED: fetch_url returns attacker content; harness inserts it into context UNTAGGED; model has no signal it is data (no L1 tag); system prompt is not isolated from it (no L2); no check on whether tool calls derive from it (no L3 gate); the model complies with the injected instruction and emits a tool call with the agent's real credentials → impact. ~50% because the model's own judgment (safety RLHF) is the ONLY defense, and it is probabilistic — it fails half the time on indirect injection, which is designed to not look like a refusal request. THE LAYER THAT MOST DIRECTLY DROPS IT: L3 (taint gate) — even when the model is fooled and emits the hijacked tool call, the gate checks whether the call's arguments derive from the tainted fetch_url output and BLOCKS high-impact calls deterministically. L1 (tagging) helps the model resist; L3 (gate) catches it when the model doesn't. L3 is where 50%→near-zero." c2b::b2::analysis "How does context-window flooding (attack class 5) defeat Layer 2, and what is the structural fix?" "Flooding fills the context with so much content that the system prompt's effective ATTENTION WEIGHT degrades — the model's recency bias and finite attention favor the recent, voluminous injected content over the privileged-but-distant system prompt. This defeats L2 (instruction isolation) because L2 relies on the system prompt maintaining priority in attention, and flooding erodes that priority. It is the heap-spray analogue: saturate the space rather than craft a precise exploit. STRUCTURAL FIX: (a) place the system prompt in a layer with GUARANTEED attention weight where the API supports it (separate system role, not concatenated content); (b) CONTEXT-BUDGET LIMITS — cap how much untrusted content can enter the window at all, so flooding is structurally impossible. The fix is not better prompting; it is bounding the input volume the harness admits." c2b::b2::analysis "Why is the argument 'the model's safety training will refuse the injection' called the 1.6% trap, and what is the cure?" "It is the 1.6% trap because the model's refusal training is the 1.6% (the model layer); the harness is the 98.4%. An indirect injection is SPECIFICALLY DESIGNED to bypass refusal — it never appears as a refusal request; it appears as an instruction the model has no reason to question (because the model cannot tell it is data, not intent). Relying on refusal means relying on the one layer an injection is engineered to defeat. CURE: the 98.4% — Layers 1-5 in the harness. L1 tags the content so the model has a signal; L2 isolates the system prompt; L3 gates the tool call deterministically (the model's compliance is irrelevant — the gate blocks); L4 detects encoded overrides; L5 minimizes capabilities. A harness that relies on refusal will comply under indirect injection (the ~50% default)." c2b::b2::analysis "What is the difference between Layer 3 (taint gate) being deterministic and Layer 4 (detector) being advisory, and why must high-impact gating be deterministic?" "L3 (taint gate): DETERMINISTIC — 'are the call's arguments tainted AND is the tool high-impact?' → BLOCK. No model in the loop, no bypass rate, no latency budget. The question is STRUCTURAL (tainted? high-impact?) and enumerable. L4 (detector): ADVISORY — a model returns a score 0..1; if extreme AND not low-impact → require APPROVAL (not auto-block). It raises suspicion but does not gate alone, because it is probabilistic (a clever injection can fool it). HIGH-IMPACT GATING MUST BE DETERMINISTIC because: high-impact actions (send_email, transfer_funds) have irreversible real-world consequences; a probabilistic gate has a nonzero bypass rate that automated attackers (RedAgent, ~5 queries) will find; and the structural question (tainted derivation) CAN be answered deterministically, so using probability there inherits a bypass rate you did not need. Determinism where possible (taint); probability where necessary (content). That is B2.3." c2b::b2::analysis "How do you measure whether B2's defenses actually worked, and what characterizes a good residual-risk report?" "Measure with an INJECAGENT-STYLE HARNESS: construct agentic tasks with indirect injections planted in tool outputs; run the DEFENDED agent; count injection success rate. Compare to the UNDEFENDED baseline (~50%). A good residual-risk report states: (1) the before/after success rate (e.g., 50%→2%); (2) WHICH LAYERS were deployed; (3) the CHARACTERIZED RESIDUAL — what class of attack still succeeds (e.g., 'the residual 2% is the encoded-and-laundered class — injections that obfuscate past L4 AND launder taint through a store L3 doesn't track'); (4) which downstream module closes the residual (B3 for laundering, B7 for sandbox). It does NOT say 'fixed' or 'secure.' It says: 'under harness X, rate dropped from A% to B%; residual is class C, closed by module D.' The number + the characterization + the closure plan is the deliverable." c2b::b2::analysis "Map each B2 layer's residual to the downstream module that closes it." "L1 (tagging) residual = content the model obeys despite the tag → closed by L2 (isolation raises the bar). L2 (isolation) residual = attention domination (model fooled) → closed by L3 (gate doesn't care if model was fooled). L3 (taint gate) residual = taint laundering through memory/files → closed by B3 (harness-managed writes, retrieval-time tagging so laundered values re-enter tainted). L4 (detector) residual = obfuscation that fools the detector → closed by L5 (capability ceiling — a fooled detector still cannot grant a capability the agent lacks). L5 (capability) residual = sandbox escape / excessive agency → closed by B7 (sandbox hardening) and B5 (least-privilege credentials). EVERY residual has a destination module. B2 builds the injection-defense core; B3/B5/B7 close the residuals B2's layers cannot." c2b::b2::analysis "Explain why 'tagging without gating' is an anti-pattern and what breaks." "Tagging (L1) wraps content in and tells the model 'this is data.' But the tag does NOT prevent the model from obeying the data as instruction (the model may ignore the tag — it is just tokens), and it does NOT prevent a downstream tool call whose arguments derive from the tagged content. The tag is an INPUT to the gate (L3), not the defense itself. WITHOUT the gate, you have told the model 'this is data' but provided no enforcement when the model treats it as instruction anyway — so you inherit the model's probabilistic compliance (~50% on indirect injection) with a tag the model may or may not respect. WHAT BREAKS: the indirect-injection path proceeds to the tool call unchecked; the ~50% rate barely moves. CURE: L1 (tagging) MUST be paired with L3 (taint gating) — the tag marks the taint, the gate enforces it at the tool call. The tag without the gate is decoration; the gate without the tag has no taint to check." c2b::b2::analysis