Dynamic tool selection for LangChain agents

Your LangChain agent has 40 tools, and create_agent sends all 40 to the model on every call. Prompts get long, the model confuses look-alike tools, and every new tool makes it worse. ToolcueMiddleware fixes that with one line: each user turn, the model gets only the tools that turn needs, plus the lookups they depend on.

Verified on 2026-11-02 with langchain 1.4.8 and langgraph 1.2.19, with 10 end-to-end tests. Toolcue is in early access: join the waitlist for an endpoint and keys.

Install#

bash

pip install "toolcue[langchain]"           # toolcue + langchain>=1 + langgraph>=1
export TOOLCUE_URL=...  TOOLCUE_TOKEN=...  TOOLCUE_ADMIN_TOKEN=...

Add the middleware#

python

from dataclasses import dataclass
from langchain.agents import create_agent
from toolcue.langchain import ToolcueMiddleware, publish_tools

@dataclass
class Session:
    roles: list[str]

# once, at deploy time (and whenever your tools change)
publish_tools("acme-billing", TOOLS, extras={
    "find_customer": {"produces": ["customer_id"]},     # returns an id other tools need
    "ask_user": {"core": True},                         # always attached
})

agent = create_agent(
    "anthropic:claude-opus-5-5",
    tools=TOOLS,                                        # register every tool, as before
    middleware=[ToolcueMiddleware("acme-billing", user=lambda s: {"roles": s.roles})],
    context_schema=Session,
)

agent.invoke({"messages": [{"role": "user", "content": "Refund the last invoice for Globex"}]},
             context=Session(roles=["agent"]))

For "Refund the last invoice for Globex", the model gets refund_invoice, get_customer_invoices and find_customer (the refund needs an invoice id, the invoice list needs a customer id), two close matches, and your always-on tools. For "thanks!", it gets only the always-on tools.

How it works#

Hook	When	What it does
`before_agent`	Once per user message	Sends the message and the last 10 messages (with tool calls and results) to Toolcue; keeps the tool names in the agent state
`wrap_model_call`	Every model call of the turn	Passes the model only those tools. The list stays the same for the whole tool loop
`find_tools` tool	When the model asks	Searches your whole catalog and adds what it finds for the rest of the turn
`wrap_tool_call`	Every tool call	Refuses a catalog tool that wasn't offered this turn, so a guessed admin-only tool can't run

Every tool stays registered with the agent, and only tools in your Toolcue catalog are filtered, so tools added by other middleware (a to-do list, a file system) are never hidden. If Toolcue fails or takes more than 2 seconds, the turn gets every tool: your agent never depends on Toolcue to answer. The middleware works with invoke and ainvoke, and with checkpointers for multi-turn threads.

Options: user maps your runtime context to the user Toolcue sees (set roles from your authenticated session, so admin-only tools stay hidden); history_limit (default 10); find_tools=False to leave out the recovery tool; always_include=[...] for tools every turn should get.

Compared with LLMToolSelectorMiddleware#

LangChain ships LLMToolSelectorMiddleware, which asks a second LLM to choose tools before every model call.

	LLMToolSelectorMiddleware	ToolcueMiddleware
How it picks	An extra LLM call	A dedicated service, no LLM
How often	Before every model call in the loop	Once per user message
Added latency	A full model round trip per call	~35 ms per turn
Lookups a tool depends on	Only if the selector thinks of them	Added from your tools' ids (`produces`, required parameters)
Conversation	The last user message	The recent conversation, including tool results
A missed tool	Stays missing for the turn	`find_tools` adds it mid-turn

Try it with a real model#

The examples include agent.py: three turns with Claude or an OpenAI model, printing the tools the model got on every call. The middleware itself is covered by 10 end-to-end tests in the toolcue package, with a scripted model that records exactly which tools it receives.

Building the graph yourself? See Toolcue with LangGraph.