Large tool results lead to looping

[roblourens]

Thanks for looking into it- Is the MCP tool output really large? This could still happen in that case, and if you saw it in 1.99, that's likely.

You could also check the Copilot Chat output channel, and see whether there are any errors or "400"

I hard-limited the MCP tool's output instead of letting copilot choose the limit, and that fixed the issue on the insiders version.

If copilot had simply caught the issue and told me that the context was too long to work with and that I needed to alter my approach instead of retrying over and over again, I never would have filed this bug report. Maybe that would be a good solution?

Originally posted by @oenic in #9507

[tarbaiev-smg]

For me it happens after running tests in a iOS project with e.g. "Run tests and fix errors" prompt. xcodebuild produces quite large logs so summarization is a good idea. I guess it's the quality of the summarization which causes the issue. Either it scrambles the logs to some meaningless soup or just deletes everything. In the chat it prints nothing.
Is there a way to check the raw summarization results?

[connor4312]

MCP does not allow passing context window hints in tool calls, and we could also never depend on an MCP server repecting that. Also not just exclusive to MCP tool calls, but maybe most common.

One thing we could do is that, if we see a tool call leads to a massive amount of data beyond our context window, we specifically give the conversation and tool result to the agent and ask it to summarize the result in isolation before returning it to the main agent loop.

[tarbaiev-smg]

As a workaround I instructed it to trim the logs using sed or awk, e.g. xcodebuild <...> | awk 'p; /Test Suite|Testing failed/ { p = 1 }'. If only it could figure this out by itself, e.g. summarize the first call results and craft a trimming pipeline for the command for future calls. This would drastically decrease the amount of tokens used.