Conversation history summarization makes everything worse

[oenic]

Type: Bug

Last week, I was able to tell copilot to "check our test env logs for anything that would help troubleshoot " or something along those lines, and it would be able to, via an MCP tool, query our logs and surface helpful things to me. If it didn't find anything, it would try a couple different ways, maybe try searching the codebase for exact logging statements to search for, etc. It was very very good and helpful. If it couldn't find anything at all, then it would typically admit that and ask for clarification or more information.

However, now that the conversation history gets summarized on every single tool call, (see attached screenshot), it just tries the same thing over and over and over ad infinitum. Please allow me to opt out of it.

Extension version: 0.27.0
VS Code version: Code 1.100.0 (19e0f9e681ecb8e5c09d8784acaa601316ca4571, 2025-05-07T12:48:53.763Z)
OS version: Windows_NT x64 10.0.22631
Modes:

System Info

Item	Value
CPUs	13th Gen Intel(R) Core(TM) i7-13850HX (28 x 2304)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	undefined
Memory (System)	31.69GB (4.54GB free)
Process Argv	--crash-reporter-id d0271f0f-9ace-4549-be77-9efaf08306e9
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
binariesv615:30325510
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
h48ei257:31000450
pythontbext0:30879054
cppperfnew:31000557
dwnewjupytercf:31046870
pythonrstrctxt:31112756
nativeloc2:31192216
5fd0e150:31155592
dwcopilot:31170013
471b6256:31263136
6074i472:31201624
dwoutputs:31242946
customenabled:31248079
hdaa2157:31222309
copilot_t_ci:31222730
e5gg6876:31282496
pythoneinst12:31285622
bgtreat:31268568
4gafe986:31271826
31787653:31262186
3e8i5726:31271747
996jf627:31283433
pythonrdcb7cf:31303019
usemplatestapi:31297334
7bj51361:31289155
747dc170:31275177
aj953862:31281341
generatesymbolt:31295002
convertfstringf:31295003
gendocf:31295004
2d52d103:31303029

[oenic]

Got another example. It is comically bad lol

[isidorn]

Thanks for opening this.
I also noticed folks on reddit hitting this https://www.reddit.com/r/GithubCopilot/comments/1kl0dwt/partial_file_read_sometimes_causes_issues/

@roblourens let me know if you need more details or anything to help with the investigation.

[utapyngo]

I also noticed folks on reddit hitting this https://www.reddit.com/r/GithubCopilot/comments/1kl0dwt/partial_file_read_sometimes_causes_issues/

This is another issue related to partial file read.

[Cubox]

are you able to get the output of the MCP tool call and check the size? If it's hitting around the context window, I'm guessing the answer is too long for Copilot

[oenic]

It wasn't too long last week. I also downgraded and tried the same thing and it worked great.

[Cubox]

What model are you using?

[oenic]

I tried with GPT 4.1, Claude Sonnet 3.7, and Gemini 2.5.

Gemini and Claude were the ones that produced this behavior. I had other issues with GPT 4.1 (it just hung forever and didn't output anything), so I stopped trying with it.

I have primarily been using Claude Sonnet 3.7 though, and it never did this before.

[oenic]

Just checked the MCP tool output. Unfortunately I can't paste it here because it contains potentially confidential information, but it's around 75,000 characters, which admittedly is quite large.

[Cubox]

iirc they introduced the "summarized conversation history" feature recently, so it might need more work.