MCP Persistent Connections#2346
Open
ngafar wants to merge 3 commits into
Open
Conversation
…g logic. Modified `list_server_tools` to accept `server_id` for session reuse, added `MCPConnectionManager` for managing persistent sessions, and updated relevant handlers to support these changes.
…troduced methods to get, set, and invalidate tool cache based on server_id, enhancing the tool listing logic in `list_server_tools` to utilize cached results when available.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 151baa3. Configure here.
…eter. Modified MCPConnectionManager to invalidate tool cache based on session config, ensuring accurate tool listings when necessary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Description
Every time the AI needed to know what MCP tools were available — or actually call one — the code would:
This happened on every single AI message that involved MCP tools.
The old approach meant each agent loop was paying subprocess startup cost plus a tools/list round-trip on top of that. With many tools across several servers, this adds up fast and makes the agent feel sluggish.
What we've done is the same basic idea as a database connection pool: you pay the setup cost once, then reuse the connection for the lifetime of the session.
Testing
In theory, multiple calls to the same MCP server should be faster, this is hard to test without proper benchmarks. However, the logic behind the persistent connections is sound.
Documentation
N/A - BTS changes
Note
Medium Risk
Changes how long-lived MCP child processes are managed; stale sessions, leaked subprocesses, or incorrect invalidation could affect agent tool calls until reconnect, though errors trigger eviction and delete clears the pool entry.
Overview
Replaces per-call MCP subprocess spawn with a persistent connection pool keyed by
server_id, so listing tools and invoking tools reuse one stdioClientSessioninstead of starting the server on every agent message.MCPConnectionManagerholds sessions, configs, per-server locks, and a cached tool list; failures and timeouts evict the session so the next call reconnects. Short-lived connections remain only whenserver_idis omitted (e.g. validating a new server on POST).Call sites now pass
server_idfrom the JupyterLab tool executor, MCP settings GET, andget_available_mcp_tools. Deleting an MCP server is async and invalidates the pooled connection for that id.Reviewed by Cursor Bugbot for commit 151baa3. Configure here.