Provide some filters for fetch_trace as CC has a 25k token limit on tool responses

[shreyas-shinde]

Checklist

• I have searched the existing issues for similar feature requests.
• I added a descriptive title and summary to this issue.

Summary

exceeds maximum allowed tokens (25000). Please use pagination, filtering, or limit parameters to reduce
the response size.

Why?

No response

How?

No response

Additional Context

No response

[PeriniM]

Hey @shreyas-shinde ! That's a good point, we should definitely pre-process the trace before sending it to the MCP client. Any thoughts?
Are you mainly interested in traces stats, llm calls, retrieved documents, errors, etc?

[shreyas-shinde]

@PeriniM thanks for reply. I am setting up eval loop with claude code and this mcp such that it can run some tests after making some change and verify. So it will be good to be able to specify some tags in this mcp's tool's request and then it could be any step of the entire trace and i could get only that specific step (could be llm call, tool call etc..) response thus it wont go over the 25k token limit. Probably we should also be able to just get the skeleton of the entire trace like each step's name and the inputs with which it was called (no response as that causes too many tokens issue). Then we could use the name or metadata of each step to get its response. Also would love to be able to get the attachments associated if any with a step. I want to confrim the contents of csv generated in the trace.
Currently im able to do all this with the playwright MCP by navigating to langsmith's UI but its bit flaky like any UI test so will be good if langsmith mcp could support such filters on its tools.

What do you think?

[nick-youngblut]

There is no fetch_trace tool according to the README.

The tool seems to be commented out.

This tool would be very useful for helping to summarize long traces, especially if the tool included:

• provide token efficient per-step summaries by default
• allow the MCP client to obtain details about specific steps in the trace, when needed

[PeriniM]

Hey @nick-youngblut
we replaced that tool with fetch_runs to make it more flexible by adding filters.
I still think summarizing long content is a good idea, are you willing to pass your own model api key for summarization?
We could still have a tool to fetch the overview of the trace by showing only the dictionary keys given the ability to choose the search depth eventually

[nick-youngblut]

I still think summarizing long content is a good idea, are you willing to pass your own model api key for summarization?

Why would this be necessary? Why not use a progressive disclosure approach, where the default is high-level, and the MCP client can request more detail when needed (e.g., detailed information for steps 5-8)?