Would your logs have caught it?

[00] Result first

Parts 2 and 3 showed two things an AI coding agent can do without asking you twice: start an MCP server that a repository declares, and keep using a tool after its definition changes. This part asks whether anyone could find out afterwards, and it closes the series. We tried those events in five AI coding agents, each time on a clean virtual machine (the server start could happen in three of them; see [03]), and looked where a defender would look: the agent's own log files, left at their default settings, and the standard activity logs of the Linux machine. So that a detection rule would not count ordinary work as an attack, we also recorded ten hours of scripted ordinary use: five to write the rules, five kept aside and unseen until the rules were fixed, to count false alarms.

• The agents' default logs catch some events, not all. Of the 13 agent–event combinations we could test, 9 left a trace that a rule could tell apart from ordinary use.
• Our prediction held. Before the tests we wrote down that at least one kind of event would slip past the agents' default logs. Two of three did. A tool that changed without the server announcing it left nothing usable in Codex CLI's and Gemini CLI's logs. A server started by the repository left nothing usable in Gemini CLI's and Qwen Code's.
• The machine's own logs saw the repository's server start in all 6 runs where it happened, and neither kind of tool change.
• No false alarms. No rule fired in any minute of the hours kept aside: 0 of 60 per agent and event type. Zero in 60 minutes still allows a rate up to 6.0% (Wilson 95% interval), and some sources covered fewer minutes (Goose 1, Claude Code's notification log 46, the host 111 of 300; see [08]).
• Official MCP servers protect some tools, not all. Some servers mark text written by others, such as issue bodies, comments and log lines, so that the model reads it as data, not instructions. Of the tools that return such text, Azure DevOps fully protects 97%, AWS IaC 89%, GitHub 18% and Sentry 3%; counting partial protection, 100%, 89%, 87% and 52%.

[01] Two questions

What a defender can see (RQ5, confirmatory). We preregistered this question on 11 October, before recording any normal use: with default client logging and standard host telemetry, can each event be told apart from normal agent use, with false alarms on fewer than 5% of one-minute windows? (The frozen text said "below 5% per hour"; the one-minute windows were fixed the same day in deviation D6, and the threshold and scoring in D27, before any normal use was recorded.) Our hypothesis, H5, says no: at least one event type cannot be told apart with the clients' default logs alone.

Whether official servers protect every tool (RQ4, exploratory). Some official MCP servers wrap content written by third parties, such as issue bodies, comments and log lines, before handing it to the model: they mark it as data, not instructions. Does every tool that returns such content go through the wrapper? We first measured this on 9 October, before the preregistration was frozen, so the question stays exploratory.

What was already known. In the six terminal clients, exporting OpenTelemetry is off by default wherever it exists. Claude Code documents an OpenTelemetry event for MCP server connections that says whether a server came from the project, but only once telemetry is turned on. We found no vendor that documents an event for a tool's definition changing. On the host, Elastic publishes detection rules for processes started by AI clients and MCP servers and for changes to their configuration files, built mostly on its own endpoint agent, plus a few on vendors' cloud audit logs; on 17 October we found no rule for these events in the SigmaHQ repository. The only tools we found that notice a silent change pin the definitions in a proxy, as Trail of Bits' mcp-context-protector does. The MCP specification asks clients to log tool usage, not definition changes, and lets a client fetch the tool list again when its cache expires, without any notification: a silent change, the third event we test below, is ordinary protocol behaviour.

[02] What we measured

Three events. In short: we made each event happen, then asked whether a rule written in advance finds it in the default logs without also firing during ordinary work. The details below are what make the numbers trustworthy; if you only want the results, skip to [03]. Each event comes from an earlier part of the series. A run counts as an event only if the ground truth, a log written by the lab's own server, separate from the client's logs, shows it happened.

Swipe the table sideways to see all 4 columns →

Two runs per event and client, in a random order fixed in advance: 30 runs on five terminal clients, on the versions installed in the test snapshot, listed in [08]. The client's own log never decides whether an event happened. An event that the client does not record is a valid negative result, not a failed run; a run whose logs could not be copied is kept aside and repeated. Goose does not start servers declared by a repository, so event 1 cannot happen there: those runs are reported as "no event" and count neither way.

Which clients, and why five. Parts 2 and 3 used eight clients: six that run in a terminal and two editors, VS Code with Copilot and Cursor IDE. This test needs hours of scripted ordinary use per client, and our driver types into a terminal, so the preregistered design of this part covers the six terminal clients only; the two editors were tested by hand in part 3 and are not here. Of the six, Cursor CLI could not be completed for lack of quota. Its runs will follow as an exploratory addendum that cannot change the result below, which is judged on the five clients.

Where we looked.

• The clients' own logs, at default verbosity. Before the runs, on the clean snapshot, we listed every file each client keeps in its default locations (table below). We set no logging or verbosity option. (The frozen protocol also listed each client's logs at maximum verbosity; we collected only the default ones, which are what the hypothesis is about. Whether verbose logging records the events we missed was not tested; see [08].) After every run, before the snapshot is restored, the runner copies those files with their SHA-256.
• Host telemetry. Linux audit (process starts of the lab user, writes to the clients' trust and approval stores), a log of outbound connections (nftables) and the system journal. This is extra: H5 is judged on the clients' logs alone.

Not all of these locations are in the vendors' documentation. Claude Code's transcripts, Gemini CLI's and Qwen Code's chats, Goose's logs and Codex's history are documented; Claude Code's per-server MCP logs and Cursor CLI's transcripts are not. Several are kept for a limited time: by default 30 days for Claude Code's transcripts and Gemini CLI's chats (both configurable), and a fixed two weeks for Goose's logs.

Normal use. For each terminal client, two hours of scripted use with each client's configured model (see [08]): open the test repository, ask for an explanation of a file, run the tests, edit a file, approve a known server (the most boring developer in the world, on purpose: false alarms are measured against boredom). The first hour is used, with the development runs, to write the detection rules. The second hour is the held-out part: a separate session, recorded at least six hours later, on which the rules are scored. The rules' SHA-256 are committed before they are scored. A session counts only if the normal use really happened: at least 80% of the tasks done, with evidence (the tests ran, the edit is in the file, the server logged the call; an error from the model is not an answer), at least 0.8 calls to the known server per cycle that reached that task, its definition unchanged, and the session's logs copied. A session that fails is kept, does not count, and the held-out hour needs a counted first hour of the same client. Without a valid held-out hour, the question becomes exploratory.

A cycle of normal use, as the driver (the script that types into the clients) sees it on Codex CLI's screen (development hour): a scripted edit, then a call to the known lab server.

› Append one line at the end of greet.py with the comment: # reviewed in cycle 1. Change nothing else.
• I’ll append the comment to greet.py and change nothing else.
• Ran python3 - <<'PY' …
  └ (no output)
    + Show details
• Appended # reviewed in cycle 1. to the end of greet.py; changed nothing else.
  Worked for 6s • 8:08 PM
› Call the q3_noop tool of the q3-lab MCP server once, then reply with only the word it returned.
• Called q3-lab.q3_noop
  └ ok
    + Show details
• Working (4s • esc to interrupt)

Rules written on separate material, frozen before scoring. Before the 30 runs, one more run per event and client (15 runs, exploratory) gave the material on which the detection rules were written, together with the first hour of normal use. The 30 runs and the held-out hour were used only to score rules already frozen, in a commit whose hashes are published, with the code that applies and scores them. A rule may not use names that exist only because of our lab (our server's name, our tool, our paths): it has to describe the kind of event.

What counts as told apart. A rule detects an event in a run if it fires between the moment the event happened, according to the ground truth, and the end of the run. The held-out hour is cut into one-minute windows. A rule tells an event apart if it detects every run of that event and fires in fewer than 5% of the held-out windows. An event type is told apart if its rule does that for every client where the event can happen; H5 holds if at least one event type is not. We report counts, Wilson 95% intervals and alerts per hour. Consecutive windows are not independent (one burst of activity can span several), so the intervals describe these windows, not a guarantee for other machines. The 5% is a rate observed on one hour of scripted use per client (see [08]).

How a false alarm is counted, from the lab's scorer:

def windows(t0, t1, size=WINDOW):
    n = int((t1 - t0) // size)
    return [(t0 + i * size, t0 + (i + 1) * size) for i in range(n)]
…
    n_win = fp = hits_in = 0
    seconds = 0
    for s in sessions:
        ws = windows(s["t0"], s["t1"])
        n_win += len(ws)
        seconds += len(ws) * WINDOW
        fp += sum(any(a <= h < b for h in s["hits"]) for a, b in ws)

[03] What the logs show

Two runs per client and event, each from the same clean snapshot, with the releases listed in [08]. "Yes" means a record in the client's default log that a frozen rule turns into an alert; "no" means no record we could use.

Swipe the table sideways to see all 4 columns →

The times in brackets measure our script more than the clients: the new definition showed up once the client was reopened, which the script did about half a minute after the change.

What the user sees when a server announces a change. Gemini CLI is the only client that says anything: one line, printed when the server announces the change.

What a defender sees when a server announces a change: one line in Claude Code's per-server MCP log, two rows in Codex CLI's log database, one line in Gemini CLI's chat file (development runs; session and message ids replaced by "…", the home folder shortened to ~/; Codex's long request prefix elided, its rows shown as id, level, target, message).

{"debug":"Received tools/list_changed notification, refreshing tools","timestamp":"2026-10-22T17:45:04.632Z","sessionId":"…","cwd":"~/q3-repo"}

3366  INFO  rmcp::service                               …:start_server_task{server_name=q3-lab}:serve_inner: received notification notification=ToolListChangedNotification(NotificationNoParam { method: ToolListChangedNotificationMethod, extensions: Extensions })
3367  INFO  codex_rmcp_client::logging_client_handler   …:start_server_task{server_name=q3-lab}:serve_inner: MCP server tool list changed

{"id":"…","timestamp":"2026-10-22T17:30:58.868Z","type":"info","content":"Tools updated for server: q3-lab"}

And what the user sees when the change is silent, in Claude Code: the same prompt, the same "ok", nothing else.

With no notification, the only witness in Claude Code is its transcript: a request in the same session 13 s after the change still carried the old description; the new one appears at the first request after the client was reopened (two sessions; session ids replaced by "…", the rest of each record elided). Codex CLI wrote 742 log lines during a development run of the silent change; none of them mentions it.

17:52:30  {"type":"api-request-shape","sessionId":"…",…,{"name":"mcp__q3-lab__q3_noop","description":"Lab tool, revision A. Returns the word ok and records the call in its own log. It has no other effect.",…}]}}
17:53:14  {"type":"api-request-shape","sessionId":"…",…,{"name":"mcp__q3-lab__q3_noop","description":"Lab tool, revision B. Records the call in its own log and returns the word ok. Nothing else happens.",…}]}}

Codex CLI and Gemini CLI log the notification but never the definition, so a silent change leaves nothing in their logs. Goose and Qwen Code see a change only through the new definition, once they load their tools again, so for them announced and silent look the same; Goose keeps just its last ten requests. When a repository starts a server, Codex CLI records it next to the trust decision; Gemini CLI and Qwen Code record nothing we could use. Claude Code, with its default answer, did not start the server, and Goose does not start servers declared by a repository at all, so that event could not be measured for them: "not observed" is a property of the client, not of its logs.

Host telemetry. Linux audit records the start of the repository's server in all six runs where it happened: the agent launches an interpreter in the project folder right after it starts. Neither tool change leaves a trace in audit, the journal or the log of outgoing connections: the change happens inside a process that is already running, over its standard input and output. The audit log rotates within an hour and only its current file is copied, so the host's false-alarm count rests on 111 of the 300 held-out minutes.

What the machine saw in a development run where Gemini CLI started the repository's server: the client starts in the project folder, and 7.9 seconds later an interpreter starts the server declared in the repository's configuration (our canary), in the same folder. Gemini CLI's own logs wrote nothing in that session (Linux audit records; other fields elided).

type=SYSCALL msg=audit(1792690709.513:1180): … ppid=2505 pid=2513 … comm="node" exe="/usr/bin/node" … key="lab_exec"
type=EXECVE msg=audit(1792690709.513:1180): argc=2 a0="node" a1="/home/lab/.npm-global/bin/gemini"
type=CWD msg=audit(1792690709.513:1180): cwd="/home/lab/q5-repo"
…
type=SYSCALL msg=audit(1792690717.433:1385): … ppid=2608 pid=2675 … comm="node" exe="/usr/bin/node" … key="lab_exec"
type=EXECVE msg=audit(1792690717.433:1385): argc=2 a0="node" a1="/home/lab/lab/trusting-a-repo/canary/marker-server.mjs"
type=CWD msg=audit(1792690717.433:1385): cwd="/home/lab/q5-repo"

The host rule, as frozen in the lab:

CLIENT_BINS = {"claude", "codex", "gemini", "qwen", "goose", "cursor-agent", "cursor"}
MCP_LAUNCHERS = {"node", "nodejs", "npx", "bunx", "bun", "deno", "python", "python3", "uv", "uvx", "pipx",
                 "docker", "podman", "java", "ruby", "go"}
# a project folder: a folder inside a home folder that is not hidden (the home folder itself, its hidden folders, /, /tmp and
# system folders are not project folders)
PROJECT_RX = re.compile(r"^(/home/[^/]+|/root)/[^./][^/]*(/.*)?$")
START_S = 30
…
        if PROJECT_RX.match(cwd) and dt is not None and dt <= START_S and cmd not in seen:
            hits.append(t)

[04] Rules and false alarms

Each rule is a pattern or a comparison with the previous definition; Codex CLI's adds a time link to the trust decision. Where the notification is logged, matching it is enough. Where it is not, comparing a tool's definition with its previous one catches both kinds of change, once the client loads its tools again (in our runs, after it was reopened). Where nothing is logged, no rule can help. Rules were written on the development material only and frozen before the held-out hours were read; a second, blind author (a fresh session of the same AI model that helped us write ours, Claude Opus 5.5, with no history and no access to our rules) wrote its own on the same development material and reached the same verdict on all 13 measurable pairs. That is a check of how the rules were written, not an independent replication.

Swipe the table sideways to see all 7 columns →

Two of the frozen rules, for Claude Code: one matches the logged notification, the other compares each tool definition with the previous one. Neither names our server or our tool.

  "claude-code/q3-announced": {
   "client": "claude-code",
   "event": "q3-announced",
   "class": "the client records a tools/list_changed notification from an MCP server",
   "type": "match",
   "source": ".cache/claude-cli-nodejs/*/mcp-logs-*/*.jsonl",
   "regex": "\"debug\":\"Received tools/list_changed notification"
  },
  "claude-code/q3-silent": {
   "client": "claude-code",
   "event": "q3-silent",
   "class": "an MCP tool's definition stored in the transcript differs from its last earlier definition",
   "type": "changed",
   "source": ".claude/projects/*/*.jsonl",
   "extract": "claude-tools"
  },

The rules are in the lab as plain pattern rules, with the script that reads each client's logs and applies them: telemetry/rules/ (FROZEN.md lists the frozen files and their hashes).

[05] Defences in official MCP servers

What we counted. So far, what a defender can see after the fact. The second question looks one step earlier. Some official MCP servers try to stop text written by others, such as an issue body, from reaching the model as instructions. If that defence skips a tool, the text comes through that tool unmarked. We counted how many tools it skips.

Which servers, and why six. In part 2 we wrote that we had read fifteen official server repositories; that reading was done on 9 October. Nine had no defence of this kind anywhere in their source: Notion, Grafana, Cloudflare, Playwright, Terraform, Stripe's agent toolkit, markitdown, the MCP reference servers and Azure MCP. Atlassian does not publish its server's code. That first reading was exploratory: a search for seven keywords plus a sample of the code, not a tool-by-tool count. A share of tools protected only means something where a defence exists, so the tool-by-tool count below covers six servers that do have one. They come from four of those repositories (the three AWS servers share one); one more repository with a defence is not reported here.

What we counted, tool by tool. An earlier, approximate reading gave other shares (deviations D9 and D15); deviation D17 withdrew them in favour of this count. One row per registered tool, at a pinned commit of each server's repository. A tool is in scope if it returns free prose written by third parties: titles, descriptions, bodies, comments, messages, file contents, logs. Names and identifier-like strings do not count. Tools whose prose is written only by the vendor (documentation, curated advisories) are in scope too, and flagged: every figure is given twice, with them and without them. For each tool we asked whether all of its prose passes through the server's own defence (full), only part of it (partial), or none.

What counts as a defence. A mechanism that marks or neutralises third-party content as data the model should not follow: a boundary together with an instruction to treat what is inside as untrusted, character hygiene, or provenance marking. Delimiters that present the content as instructions to follow do not count.

Two readings. The first reading was made by AI agents and spot-checked against the source. A second reading was made blind by Codex, which never saw the first. Where they disagree, the author decided each case, helped by a third reading by an AI model that saw only the two readings and the cited code and proposed a decision and the line that settles it; the published sheet records, case by case, whether its proposal was taken. Agreement is measured before that decision, on the original labels, separately for the class, the protection and the vendor-only flag. On the 269 tools of the servers below, the two readings agree on the class for 87.7% of the tools (Cohen's κ 0.76; κ measures agreement beyond chance: 1 is perfect, 0 is chance), on the protection for 66.9% (κ 0.52) and on the vendor-only flag for 97.4% (κ 0.62); 91 tools needed a decision.

Swipe the table sideways to see all 7 columns →

Without the tools whose prose is written only by the vendor:

Swipe the table sideways to see all 7 columns →

On average across the six servers (unweighted mean), the share of covered tools is 34.6% with full protection only and 87.8% counting partial (median 10.7% and 94.4%); pooled over all 138 tools, 53 are fully covered and 113 at least partly. Without vendor-only prose: 33.2% and 86.8% (median 11.1% and 93.1%), 49 and 107 of 130.

AWS was informed before publication and assessed these gaps as having no significant security impact. GitHub and Sentry were not sent a report, because the context was already public: GitHub states in pull request #3114 that patches, diffs and raw file contents are intentionally left unsanitised, and an open Sentry issue (#1093) discusses whether its markers are enough. Neither covers every gap counted here.

Most of the 25 uncovered tools are side paths: activity feeds and notifications, user reports, conversations of AI agents, alert rules and monitors, release and dashboard details, rulesets and custom properties, code-quality findings, the results of assigning an issue to an AI agent, and documentation fetched from the web. Main paths slip through too: Sentry's issue search and update return issue titles with no defence.

[06] Check your own setup

Which logs your agent keeps. This script from the lab repository ([09]) only reads. It lists the files present in the default locations of the table in [02], with size, date and SHA-256, and changes nothing:

python3 telemetry/scripts/vm_logs.py inventory telemetry/inventory/candidates.json

The output names files in your home directory: do not paste it publicly, not even to win an argument online.

The one rule that works wherever the log allows it. Keep each MCP tool's definition as your agent last saw it, and alert when it differs, including after a restart: in our runs the change showed up only after the client was reopened. In our runs this caught the silent change in Claude Code and both changes in Goose and Qwen Code, with no false alarm (Goose's zero rests on one covered minute; Qwen Code's needs the model to search its tools again). It needs a log that stores definitions, which Codex CLI and Gemini CLI do not keep.

[07] What we would ask vendors

• Record in the default log when a tool's definition differs from the one the user approved, with both versions or their hashes.
• Record every MCP server start together with where its configuration came from (the user's settings or the repository), in the default log, not only in opt-in telemetry.
• Document where logs are written, at which verbosity and for how long they are kept.

What we have reported and contributed. Findings go back to the projects concerned, in the open where they can:

• goose (Agentic AI Foundation): goose does not open the listening stream that MCP servers on the 2026-07-28 specification use to announce a changed tool list, so its list can stay old. Reported; the fix is assigned to me and ready.
• The official MCP Rust SDK: a documentation request for the trap behind the goose bug, so that other clients do not repeat it; it is in the maintainers' queue.
• MCP specification: a comment with the data of part 2 on the proposed mcp.json format, since a repository's configuration can start MCP servers.
• OWASP AISVS: two proposals, to move to level 2 the requirement to re-approve a tool whose definition changed, and to add a requirement for MCP servers declared by a repository.
• OWASP GenAI Red Team Lab: a proposed hands-on lab on tool poisoning and on an approved tool that changes afterwards; it is in the backlog.
• OWASP Agent Control Standard: the data of part 3 on an open issue about tool redefinitions, and a discussion on MCP servers that start before any control can decide.
• Text contributed to an OWASP guide that is still in draft.

None of this means these projects endorse this research: it is reported to them, not done with them.

[08] Limits

• One operating system, one audit configuration, one machine.
• One-minute windows in a row are not independent observations; the Wilson intervals do not make the false-alarm rate a property of other machines.
• Scripted normal use is not a developer's day. Real use is more varied and probably produces more false alarms (real developers also keep forty tabs open and a terminal they forgot about).
• Two runs per event and client are enough to see whether something is recorded, not to estimate how often it would be caught.
• Zero false alarms in 60 minutes still allows a rate up to 6.0% (Wilson 95%). Goose keeps only its last ten model requests, so its rules could be tested on 1 of its 60 held-out minutes; its zero says little. Claude Code's notification log covers 46 of 60.
• The host rule's zero says little too: in the held-out hours it saw only commands it had already learned, over 111 covered minutes. It assumes the repository is opened when the agent starts and sees only the lab user's processes.
• Each rule that passes has a scope: Codex CLI's sees a new server right after a trust decision taken in its terminal interface, Qwen Code's needs the model to search its tools again after the change, and the host rule misses a server whose command line the machine has already run.
• Normal use contains no harmless look-alikes: no other server starting, no other tool list changing. The false-alarm rates measure ordinary work, not the near-misses a real fleet would produce.
• Five clients, not six: Cursor CLI's test could not be completed for lack of quota, and it follows as an exploratory addendum.
• Three of the five clients use the same Gemini model. Their event runs used a free tier, at times slow or unavailable; their normal use ran on a prepaid key of the same model, because the free daily quota could not carry six hours of it in time. Errors of the provider are part of the normal use we recorded, and sessions that failed because of them were repeated, at most three times.
• The scripted driver reads the clients' screens. Seven rounds of set-up sessions and three restarts of the normal use found faults in how the driver read the clients' screens, and each was fixed in a published deviation (our public log of changes to the protocol, numbered D1, D2…) before the sessions that count (D28–D37).
• Clients change fast: these are Claude Code 2.1.300, Codex CLI 0.162.1, Gemini CLI 0.64.0, Qwen Code 0.25.1 and Goose 1.54.0, run on 22–24 October 2026, and default logs can change with any of them.
• Only default logs were collected. The frozen protocol also listed maximum verbosity; whether verbose logs record the events we missed was not tested.
• Detection is measured here, not deployed: no rule has run on a real fleet.
• We did not repeat the normal use with a local model, so traffic to the model and traffic to the vendor are not told apart.
• The census reads source code at pinned commits. A missing defence is not proof that a tool can be exploited, and the servers may have changed since.
• Whether a tool returns third-party prose, and how much of it is covered, are judgements. Two readings and the author's decision reduce the subjectivity; they do not remove it.

One of those fixes, as an example: how the driver guards a prompt. Before it presses Enter, it re-reads the screen and checks that the typed prompt is there, whole and unaltered; if it is not, it ends the cycle instead of sending something we did not script (D34, D35).

def composer_holds(before, typed, prompt):
    """True when the lines that typing drew or changed hold the whole prompt, unaltered. …
…
        while reads < TYPED_READS and not composer_holds(before, typed, prompt):  # read again while the line is still drawn
…
        if not composer_holds(before, typed, prompt):
            # an altered prompt is not the scripted use: no Enter, the cycle ends and the client is closed (D34)
            self.end("prompt-mangled", cycle, task, f"c{cycle}-{task['id']}-mangled", typed, prompt, reads=reads)
            return False
        q3.keys("Enter")
        self.log(action="prompt", cycle=cycle, task=task["id"], reads=reads)

Geez, you're so picky

[09] The lab

Everything is in two folders of the lab repository:

• telemetry/: the inventory of default log locations, the exporter that runs inside the virtual machine, the collection check and the frozen detection rules of both authors, with the scripts that score them (telemetry/rules/);
• defence-coverage/: the census tables of the servers reported here (results-20261024/), both readings, the third reading's proposals, the adjudication and the scripts;
• tool-change/: the open-weight cells of D39 (recording proxy, client file, orchestration, secret gate, analysis) and their evidence (evidence/open-weight-20261024/).

The lab's code tests run with python3 -m unittest discover -s telemetry/test (221) -s defence-coverage/test (28) and -s tool-change/test -p 'test_*.py' (23). The protocol and every deviation were published before the data they govern were scored; D38, on the RQ4 adjudication, was published with the census it governs; D39, on the open-weight cells, before the cells were run. Release v0.4.2 of the lab: https://doi.org/10.5281/zenodo.23623381.

[10] What the series set out to do, and what it found

The series started from one idea: an AI coding agent acts on more than the prompt you type. Part 1 asked where this summer's disclosures triggered. Part 2 asked what the agent runs when you trust a repository. Part 3 asked what it notices when an approved tool changes. Part 4 asked what trace it leaves. We measured these on current releases, in a lab made only of our own machines, with harmless canaries. Three of the questions were preregistered before the data: the classification of disclosures, the change of an approved tool, and what the logs show.

Where it breaks (part 1). We classified ten public disclosures from this summer by where they trigger. In six of ten, the model's judgment was not what decided the outcome. With the borderline cases moved, the count stays between five and seven. This was a hand-picked sample, read before the preregistration. The preregistered test, on the complete June–September list with independent raters, has not been run yet. It is planned for the preprint; if it is not ready, the preprint will say so. (The preregistration's pilot reported 7 of 10 before one case was relabelled on 13 October, deviation D14.) Until then, part 1 is a reading, not a measurement.

Whom it trusts, and what it runs (part 2). No client started the repository's server before we answered its question, in any interactive session. That held in eight clients. But in five of the six clients that ask a question, the answer under the cursor is the one that trusts. In three of them, trusting the folder also starts the servers it declares, with no second question. Claude Code is the most careful client when you use it interactively, and starts the server with no question in non-interactive mode. This question was exploratory: we measured it before the preregistration.

What it notices (part 3). We changed a tool's description, after it was in use, in eight clients. Four of them ask for approval before a tool's first call; the other four, as we set them up, ask nothing even the first time. None asked again because of the change. Our hypothesis had two parts. The first held: no client asks again. The second, in the form fixed before the runs (deviation D12: a client "acts on" the notification if it tells the user or asks), did not: Gemini CLI tells the user, in one line, that the tools were updated. It does not say what changed, and it does not ask. Read as first written, "by asking the user", the second part would have held. In the clients that ask, an approval follows the tool's name, not what the tool says.

What leaves a trace (part 4). Our hypothesis was that at least one kind of event would leave no usable trace in the clients' default logs. It held. In 9 of the 13 client–event pairs we could measure, a rule told the event apart from normal use, with no false alarm in the held-out hours. A silent change of an approved tool left nothing usable in Codex CLI's or Gemini CLI's logs. A server started by the repository left nothing in Gemini CLI's or Qwen Code's. Host telemetry saw the server start every time, and neither tool change. A second, blind author, a fresh session of the same model, reached the same verdict: a check of how the rules were written, not an independent replication. In the same part, an exploratory count of six official MCP servers: of the tools that return third-party text, Azure DevOps fully protects 97%, AWS IaC 89%, GitHub 18% and Sentry 3%.

Was the goal reached? For the two questions we could put to the test, yes. The hypotheses of parts 3 and 4 were written down and published before the data, and the results are clear. When an approved tool changes, none of the clients that ask for approval asks again. Two of the three kinds of event slip past the default logs of at least two clients, while the machine sees every server a repository starts. We now know, for these releases, where the decisions sit and which of them leave a trace. The third preregistered test, on part 1's disclosures, is still to be run. And these results measure what the clients do and record, not how often it happens in real use or whether an attacker would succeed: two runs per event and client, one held-out hour per client, one machine.

What the clients sent to the model, with an open-weight model. Part 2 promised to run the clients that accept another endpoint against an open-weight model on a GPU we control, to see which definition each one hands to the model. We did it on 24 October, after the results above, as an exploratory check (deviation D39): Qwen Code, Goose, Codex CLI and Claude Code against Qwen3.8-27B served by our own instance, with a proxy that records which revision of the tool each request carries, once per signal. In Qwen Code and Goose, whose model called the tool, the requests carried the old description for the rest of the session and the new one only after the client was reopened. Qwen Code received the notification and did not ask for the list again; Goose does not subscribe to it, so our server could not send it. Codex CLI's requests follow the same pattern, but its model never called the tool and said it did not have it; the requests carried some 320 to 400 tools, almost all from apps connected to the account, with ours inside a nested entry, and we did not establish why the model missed it. Our endpoint rejected every main request from Claude Code, so its model never answered; the requests still show what Claude Code sent. In the same session, a new turn started after the change still carried the old description, and with the notification it did so even though Claude Code had already fetched the new list from our server. Requests with the new description appeared only after the client was reopened, and were rejected too. One run per cell, one model, one endpoint, and Claude Code's new turn followed repeated errors; Gemini CLI and Cursor CLI have no documented way to use another endpoint.

What remains open.

• The preregistered classification of the full list of disclosures.
• Cursor CLI's runs for part 4, as an exploratory addendum.
• The other secondary cells of deviation D5 (a schema change, a name change, the HTTP transport), announced for part 3, were not run.
• Normal use with harmless look-alikes: other servers starting, other tool lists changing.
• What a changed description does to the model's behaviour. Some logs show it in the requests sent to the model; its effect was not measured.
• Every client changes every week. These results describe the versions we name.

If you use agents. Treat a repository as something that can run code. Read the trust question before you press Enter. Run agents in non-interactive mode only in an isolated place. Know that an approval does not cover a later change of the tool.

If you defend. Do not assume the agent's logs will tell you. Find out which files your agents keep, and for how long. Watch process starts on the machine: that caught the repository's server every time. Where a log stores tool definitions, compare each one with the previous one. Ask vendors to log server starts with their origin, and definition changes, by default.

So, would your logs have caught it? A change the server announced: yes, in all five agents; in two of them only once the client loaded its tools again. A silent change: only where the agent writes tool definitions down, three of five. A server started by the repository: on the machine, every time; in the agent's own logs, in one of the three agents where it happened in our runs (Codex CLI; not Gemini CLI or Qwen Code).

The four parts will come out as one preprint, with their full methods and data. It will be linked here, as in the other posts of the series.

Sources

1. Lab for this part — contents, safety note, tests
2. Lab — telemetry/: inventory, exporter, collection check
3. Lab evidence — inventory of default client log locations, 17 October (counts)
4. Lab — defence-coverage/: comparison of the two readings
5. Deviations from the preregistration, D6, D13, D17, D22–D39 — DEVIATIONS.md; preregistration — doi.org/10.5281/zenodo.23300369
6. Model Context Protocol specification, revision 2026-07-28, “Tools” — modelcontextprotocol.io
7. Anthropic, Claude Code documentation, “Monitoring” (OpenTelemetry events) — code.claude.com
8. Model Context Protocol specification, revision 2026-07-28, “Caching” — modelcontextprotocol.io
9. Elastic, detection-rules (GenAI and MCP rules) — github.com/elastic/detection-rules
10. Trail of Bits, mcp-context-protector — github.com/trailofbits/mcp-context-protector
11. SigmaHQ, rule format and repository — sigmahq.io; github.com/SigmaHQ/sigma
12. Part 3 of this series — You approved the tool. Then it changed.
13. Part 2 of this series — Trusting a repo now means trusting its MCP servers
14. Retention of client logs — Claude Code, cleanupPeriodDays: code.claude.com; Gemini CLI, session retention: geminicli.com; Goose, logs: goose-docs.ai
15. Part 1 of this series — Outside the prompt: AI agent vulnerabilities of summer 2026