There's a module for that

On 9 October I gave a talk at the Drupal AI Learners Club about two projects I have been playing around with for about two months: agent-module-documentation, agent-first documentation for every Drupal 11 module, and the Agent Module Knowledge MCP, a DDEV add-on that lets your coding agent search it. Thanks to Amber and Angie for having me. This is the written version. The recording is below, and the slides are online if you want to click through them yourself.

Ask a coding agent for a Facebook login on your Drupal site and it happily builds one. No question asked, no module searched for. You get a brand new OAuth flow written from scratch, enabled on your site and maintained by nobody. I have tried this with Claude, and it just starts creating a custom module, even though there are contrib modules for this that are covered by the Security Team.

That is the first of six problems agents have with Drupal right now.

Problem #2 is not only an agent problem. Search on drupal.org works when you already use the maintainer's words. Shy One-Time fixes the "one-time login link has already been used" error that Outlook's link scanner causes, but search for "password reset link already used outlook" and it isn't in the top 12 (checked on 8 October).

Problem #4 costs money. In the evals further down, an agent that had to read the code spent on average 47,600 more tokens, 24 more seconds and 10 more cents per task than one that had the docs. If 10,000 Drupal developers did that once a day, that is about 476 million tokens a day, or $376,000 a year, to learn again what one documentation run already wrote down. That is a back-of-the-envelope number, not a measurement.

And problem #5 is where Drupal should win. The Security Team, security advisories for covered contrib and Update Status put a whole security process behind a covered module. A custom OAuth module written by an agent has none of that, and a non-developer can't spot a missing state check on the callback.

I worked a lot with Express.js before, and bootstrapping an Express.js app with an agent just works better. npm has a public search API and a CLI, the README ships in the package, and the model has seen a massive amount of training data about it. Drupal's long tail of modules is barely in the training data, and if you don't ask for a framework, the agent is likely to suggest Python or Node.js.

When agents pick the stack, "it just worked" wins, and Drupal gets left behind.

The first attempt was the Drupal Module Finder, which still exists. It is a semantic search index over every Drupal 11 contrib module, plus a skill so agents can call it:

It works. But it is another server to host, scale and pay for. I gave about 50 people access to it, and that is fine for me to host. If every Drupal developer used it, it would start costing money. The other option is asking drupal.org's infrastructure to run something we don't even know gives better results than traditional search.

So the second attempt asks a different question: what if every developer ran it locally? DDEV is already the default local stack for Drupal developers, so the search became an MCP server shipped as a DDEV add-on. One command, a restart, and your agent knows the Drupal ecosystem:

The agent gets five tools: search_modules, get_module, list_docs, read_doc and list_categories. It uses plain HTTP on port 9130 on purpose: DDEV's HTTPS uses a local mkcert certificate, and Node-based clients like Claude Code don't trust it. The traffic never leaves your machine.

In the talk I showed it live, with two Drupal sites set up with the One Line Installer, which runs the agents in containers and can add this MCP server through its agent-ready Drupal build kit. Both agents got the same prompt about Outlook breaking one-time login links, asking them to fix it by writing some code. The one with the MCP server searched the local index and told me to install Shy One-Time. The vanilla one wrote its own code instead.

The MCP server is the small part. The part that is worth something is the dataset behind it. In popularity order, an agent installed every Drupal 11 compatible project on a real Drupal 11 site in DDEV, read its code and configuration, and wrote compact, agent-first documentation for it.

It took about 1,000 commits over 51 days, and it stays current: scanners watch the drupal.org release feeds and document new stable minor versions again. You can browse the catalog without an agent. Every module version gets three kinds of documentation, one for each of the problems above.

For finding modules

usage.md has a short summary, a long summary and then 15 to 30 use cases written in the user's words, like "Fix 'link already used' complaints", and data.json adds categories and keywords. These are the sentences people actually type, so the search from problem #2 now has something to match. It works very well on small and medium-sized modules. Where it misses is modules that can do everything: it suggests ECA less often than it should, because you can't write down every use case ECA covers.

For judging modules

data.json holds the core version requirement, security advisory coverage, maintenance status, contributor count, dependencies, and whether the module ships Drush commands, tool plugins or agent skills. The long summary is an honest assessment with trade-offs and caveats, not marketing.

For using modules

agent/start.md is a cheap index with the key facts, always under 500 tokens. A bigger module links to solution docs for configuration, APIs, plugins, Drush commands and hooks. The rule: every agent doc must be shorter than reading the source. For Social Auth Facebook, start.md says that its settings config doesn't exist until the form is saved and which OAuth callback to register in the Meta app: exactly the things an agent would otherwise dig out of the source, one file at a time.

And a security read

If we say "Drupal is more secure", we have to check it. The agent reading the source also looks for insecure defaults, SSRF, injection, access bypass, CSRF and credentials stored in plain text. It is not a formal audit. AI is very good at finding security issues, and just as good at finding false positives, so people from the AI initiative do a first human review before anything goes to the Security Team or a maintainer. We don't want to spam them. The notes stay out of the public repository, and real issues go through coordinated disclosure. The Security Team are the true heroes here; we try to make their work easier, not add to it.

Why write docs for agents and not for humans? Because the agent docs can be the source. They are dense, structured and grounded in the real code, so you can render them into whatever a person needs. From the 109 lines of one module's agent docs I generated a narrated video, a PDF guide, a game and a podcast. The podcast is the one I use myself: I go for long walks, and when I want to understand a concept, I have three hosts talk about it. Write the knowledge once, and render it for every audience.

Whenever you write a skill, you have to write evals for it. It is very easy to create a skill and think it works so well, but vanilla Claude Code is also very good, and you might just be feeding it context it doesn't need. We measure three things: is it more correct, and if it is just as correct, is it faster or does it use fewer tokens? If none of that is true, you don't need it.

The evals have three arms: a vanilla agent, an agent with a skill that tells it where the docs are, and an agent with the right docs already in its context. The tasks come in three levels. Easy is answering a question, medium is inspecting a live site, and hard is building something, checked against the live site's state.

Precision goes up on long-tail modules, most of all for smaller models. Time drops by about a third with the skill and by almost three quarters with the docs in context. To be honest about the tokens: the skill arm uses more of them than vanilla, because the agent spends them learning where the docs are. Handing over the right doc directly is what cuts tokens to a third, and that is the MCP server's job. The evals did not test the MCP server itself; it sits somewhere between those two arms. And the extra tokens are a first-question cost: when you ask a second question about the same module, the token use goes down again.

Then the plot twist. With a frontier model and the top 100 modules (Pathauto, Token, Webform, Metatag and friends), the model already knows. Claude Opus 4.8 got 97% right without the docs and 98% with the skill, which added overhead: 94.6k instead of 82.6k tokens and $0.169 instead of $0.159 per task. The only gain was some speed, 36 seconds down to 31. Smaller models still benefit even there: Claude Haiku on hard top-100 builds went from 33% to 78%, but that is three cases, so treat it as a hint.

That frontier result is the good news. The fact that the docs help at all means the model doesn't know the module. If every module were as well known as Pathauto, you could ask about any of them and get the right answer, and that is what we want for the rest of Drupal.

Why does Opus know Pathauto? Because the internet does. The top 100 modules have years of blog posts, Stack Exchange answers, documentation and issues, and all of it is in the training data. The long tail has a README, and maybe a project page.

I didn't think of this side effect myself. Scott Falconer pointed it out after seeing the project: the 24,642 agent docs are published openly on GitHub. Today they are context: agents read them through the MCP server. Tomorrow they could be knowledge, because public, structured docs grounded in real code are exactly what a training run can pick up. They are synthetic, yes, but generated from installing and reading the actual module, which is still better Drupal training data than nothing. Whether any lab will train on them, I can't promise. It is a hope with a good reason behind it.

I started this in my free time, and it has now moved into the Drupal AI initiative. It is MIT licensed (the docs for each module keep that module's license, almost always GPL-2.0-or-later), it will keep being updated, and you can use it for whatever you want. If you need a SKILL.md for your own module, you can start from what is there.

The idea is that GitHub takes the hit, and drupal.org gets no load at all. A GitHub Action in the add-on's repository pulls the documentation, builds one SQLite file, docs.db, and a container image, and publishes them as a GitHub Release (about 200 MB gzipped) and on GHCR. Your DDEV project downloads them, and search runs on your laptop. There is no new infrastructure to approve or scale.

Search is hybrid, because each kind misses something:

• Lexical (BM25 in SQLite FTS5) matches the words you typed. Great for exact names and config keys like graph_version or pathauto, but it misses "mail scanner eats my reset link".
• Semantic (KNN with sqlite-vec over embeddings) matches meaning, so "reset link already used in Outlook" lands near "Stop a mail scanner consuming a reset link". It is fuzzy on exact identifiers.
• Fused with Reciprocal Rank Fusion: each result scores the sum of 1 / (60 + its rank) in both lists. I won't pretend I know the theory behind it; it is how the search people say you should merge the two, and the results are very good for such a small database.

The embeddings come from Model2Vec, static embeddings where every token has a precomputed vector, so embedding a text is mostly a lookup. The model, minishlab/potion-retrieval-32M, is baked into the image and runs fully offline. It indexes the roughly 10,000 documents in about 20 seconds, with no PyTorch and no GPU.

It is also locked down: a non-root container, a read-only database, doc bodies served from the database with no filesystem access, and the endpoint is only exposed through the DDEV router.

Ship agent skills inside your module. The AI module 1.5.x has eight of them in .agents/skills/, from creating an AI provider to a guardrail plugin. Ask an agent to "add an AI provider for X" and it follows the maintainers' own recipe, and gets it right almost every time. A skill in the module is versioned with the code and written by the people who know the module best. When I gave the talk, only 11 of the 8,794 projects shipped skills. Add .agents/skills/<name>/SKILL.md to yours; the knowledge base already detects shipped skills and lists them per module. Skills make your module easy to extend and easy to set up, and they teach future models how to use it.

The dataset is free. It is open on GitHub under the MIT license: use it, fork it, index it your own way. That is the part that took time, about 1,000 commits over 51 days, and it is the part worth something.

The MCP server is just one way to put it in front of an agent, and it doesn't have to be this one. It was vibe coded in about four hours, mostly by searching and putting common Python libraries together, and it is about 600 lines of Python. Replace it freely. It could be hosted search, a drupal.org integration, Project Browser, skills with scripts, or your own agent.

And the DDEV idea might be a bad one. It is one container, model and database per project, so memory and disk for every project you run, and about 200 MB to download for every install. It uses about 180 MB of memory even when it does nothing. There are newer Gemma-based embedding models from Google that perform better and run in about 2 GB of RAM, so the search could improve too. A shared global service or a hosted endpoint may be the better long-term shape.

The Q&A ran past the hour, and most of it was about documentation. These are the parts I want to keep.

How do the evals work?

Each module with evals has typical questions with the outcome you want. Some can be checked with code: for the Address module, a correct answer mentions getCountryCode(). Others need an AI to judge how well the agent did. A wrapper runs Claude Code, Codex or OpenCode headless, and their JSONL logs show how they got to the answer. For evals of Drupal modules and skills in general, look at AI Evals.

Why GitHub and not drupal.org?

Amber asked the big one. Documentation is often the last thing a maintainer does, and sometimes it never happens: some modules don't even have a README, or worse, a project page that doesn't say what the module does. Well-thought-out human documentation is a lot better than what AI writes, but AI documentation is a lot better than nothing. We also thought about opening a merge request with skills on every module, but not everyone wants AI in their module, and not everyone has time to review it. This dataset is the middle ground, until the knowledge lives inside the modules.

Angie suggested the front door: drupal.org is the source agents already trust, so ideally the information comes from there, and an agent could read the maintainer's own skills first and fall back to this dataset when there are none. We will continue that conversation.

Why an MCP server and not just the files?

I tried giving the agent the documentation as skills with scripts, and it worked about as well. But Vercel's skills tool flags skills as malicious when they tell the agent to download things; our video tutorial skill gets flagged because it installs FFmpeg. And in the Drupal community, more people know DDEV.

Does it work offline?

Yes. It recommends a module and knows how to develop for it without internet access. Only the Composer commands it gives you need the internet.

Can the docs help humans too?

Around 1,000 modules in the repository also have human documentation, written for MkDocs, the documentation system in the Drupal Association's GitLab template. Doing it for every module would take three or four months, but for a small or medium-sized module it takes about ten minutes. So the skill is what we should give maintainers: if you don't have time to write documentation, run this, because it is better than none. It could also be an opt-in, where a maintainer asks for documentation and gets a merge request.

Christopher Weber said that MkDocs documentation on drupal.org has been discussed with the Drupal Association before and stalled on money. One argument for it: the industry is already paying for the missing information in token costs.

• ddev-drupal-agent-module-knowledge-mcp: the DDEV add-on with the MCP server.
• agent-module-documentation: the dataset, the method and the evals.
• The catalog: browse the docs in a browser.
• Drupal Module Finder: the hosted semantic search from attempt #1.
• The slides from the talk.

I also used the MCP server to build this site; How I vibe-configed this website in under 10 hours has the details.