LLM chat prompts are duplicated when logged to the database if a template is used

[simonw]

Split from:

• llm chat -t template doesn't correctly load tools #1239

I chat with just hi in it resulted in this log:

# 2025-08-12T03:49:36    conversation: 01k2e6bmgtvxbgr4ggqf7ea548 id: 01k2e6bn4swxq0dj9hpsz49mtb

Model: **gpt-5-mini**

## Prompt

hi
hi

## Response

Hi — how can I help you today?

And in the DB:

[simonw]

I ran this:

LLM_OPENAI_SHOW_RESPONSES=1 llm chat

And said "hi" and saw this:

Request: POST https://api.openai.com/v1/chat/completions
  Headers:
    host: api.openai.com
    connection: keep-alive
    accept: application/json
    content-type: application/json
    user-agent: OpenAI/Python 1.92.2
    x-stainless-lang: python
    x-stainless-package-version: 1.92.2
    x-stainless-os: MacOS
    x-stainless-arch: arm64
    x-stainless-runtime: CPython
    x-stainless-runtime-version: 3.10.10
    authorization: [...]
    x-stainless-async: false
    x-stainless-retry-count: 0
    x-stainless-read-timeout: 600
    content-length: 120
  Body:
    {
      "messages": [
        {
          "role": "user",
          "content": "hi"
        }
      ],
      "model": "gpt-5-mini",
      "stream": true,
      "stream_options": {
        "include_usage": true
      }
    }

This suggests the bug is in the way logging happens (on in the way a chat reply happens perhaps) since here the "content": "hi" only included one hi.

[simonw]

That used 1.92.2 and it looks like OpenAI are up to https://github.com/openai/openai-python/releases/tag/v1.99.9 as of an hour ago.

[simonw]

This happens when ANY template is used, it's not a tools thing.

$ llm chat -t wild-french
Chatting with gpt-5-mini
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
Type '!edit' to open your default editor and modify the prompt
Type '!fragment <my_fragment> [<another_fragment> ...]' to insert one or more fragments
> hi 
Bonjour ! Comment puis-je vous aider aujourd'hui ?
> Aborted!

Then:

llm logs -c

Outputs:

# 2025-08-12T04:26:58    conversation: 01k2e8g21222vp88gg9smrkacz id: 01k2e8g2s3s48g91ttjby29a1c

Model: **gpt-5-mini**

## Prompt

hi 
hi 

## System

Speak in French

## Response

Bonjour ! Comment puis-je vous aider aujourd'hui ?

That wild-french template is just:

name: wild-french
options:
    temperature: 1.8
system: Speak in French

[simonw]

llm -f issue:https://github.com/simonw/llm/issues/1240 -f llm/cli.py -m gpt-5

Output: https://gist.github.com/simonw/c6971109b80edeede7deaf58200fff5e#response

Relevant bit:

• In chat(), when a template is present we always do:
  • evaluate(prompt, params)
  • then, if template_prompt is truthy, we prepend it to the user’s prompt: prompt = template_prompt + "\n" + prompt
• Many templates that only set system (like wild-french) return the user input as template_prompt (i.e. evaluate echoes the input back).
• So we end up with input + "\n" + input → “hi\nhi” in the logs.

[simonw]

Got Codex to fix the bug: https://gist.github.com/simonw/7fc5e10d01c5dfeb099ea3efc2dc915b

[simonw]

It spotted and fixed another bug too:

• Separate potential issue: --system-fragment values are reused across turns in chat() (argument fragments persist while system is reset).

[simonw]

That's this code:

llm/llm/cli.py

Lines 1213 to 1226 in e6ac18f

	response = conversation.chain(
	prompt,
	fragments=[str(fragment) for fragment in fragments],
	system_fragments=[
	str(system_fragment) for system_fragment in argument_system_fragments
	],
	attachments=attachments,
	system=system,
	**kwargs,
	)
	# System prompt only sent for the first message:
	system = None
	system_fragments = []

Note that it should be using argument_system_fragments but it resets system_fragments = [].