Bind only the tools each turn needs in LangGraph

In a hand-built LangGraph graph, the model node usually calls model.bind_tools(ALL_TOOLS) on every step. With Toolcue you add one node: it asks which tools this user message needs, and the model node binds only those. The ToolNode keeps every tool, so whatever the model calls can still run.

Using create_agent instead of your own graph? The LangChain middleware does all of this for you. Toolcue is in early access: join the waitlist.

The graph#

START -> select -> agent -> tools -> agent -> ... -> END

from typing_extensions import NotRequired          # typing.NotRequired on Python 3.11+
from langgraph.graph import END, START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode, tools_condition
from toolcue.langchain import filter_tools, select_tools

class State(MessagesState):
    toolcue_tools: NotRequired[list[str] | None]     # this turn's tools; None means every tool

def select(state: State):                            # once per user message
    return {"toolcue_tools": select_tools("acme-billing", state["messages"], user={"roles": ["agent"]})}

def agent(state: State):                             # every model step binds only this turn's tools
    tools = filter_tools(TOOLS, state.get("toolcue_tools"))
    return {"messages": [model.bind_tools(tools).invoke(state["messages"])]}

graph = StateGraph(State)
graph.add_node("select", select)
graph.add_node("agent", agent)
graph.add_node("tools", ToolNode(TOOLS))             # every tool, so any call can run
graph.add_edge(START, "select")
graph.add_edge("select", "agent")
graph.add_conditional_edges("agent", tools_condition, {"tools": "tools", END: END})
graph.add_edge("tools", "agent")
app = graph.compile()

ToolNode runs any registered tool the model names, even one that wasn't bound this turn. If some tools are role-restricted, check the name against state["toolcue_tools"] before running it (or use the LangChain middleware, which does).

select_tools sends the newest user message and the conversation before it (with tool calls and their results) to Toolcue. If Toolcue fails, it returns None and filter_tools passes every tool, so the graph keeps working.

Install with pip install "toolcue[langchain]"; select_tools and filter_tools come from toolcue.langchain. The complete graph is in the examples, and the package's end-to-end tests cover it.

Why select once per turn#

• One tool list for the whole tool loop. The model plans its steps with the tools it sees, and tool definitions sit at the start of the prompt, so a list that stays put keeps the plan stable and your provider's prompt cache warm. Select on the user's message, keep the list until the next one.
• The lookups come with it. A refund needs an invoice id; the invoice list needs a customer id. Toolcue adds both lookups, so a multi-step task doesn't stall on the first step.
• Tens of milliseconds. No LLM call: the select node adds about 35 ms to a turn.

Recovering a missed tool#

To let the model pull in a tool that was left out, give it a find_tools tool that calls ToolcueClient().find_tools(...) and writes the found names into toolcue_tools (return a Command with a state update). The LangChain middleware includes a ready-made one you can copy.