Feature/add support for OpenAI text to speech models #2829

vladimirrotariu · 2025-07-25T08:19:42Z

Details

Introduces end-to-end support for OpenAI Text-to-Speech (TTS):

Python SDK
- openai_speech_decorator.py + speech_stream_buffer.py + speech_stream_events_aggregator.py – trace audio.speech requests (sync & streaming) and aggregate stream chunks.
- opik_tracker.py & stream_patchers.py patched to hook the new decorator and process chunks.
- openai_speech_usage.py model records per-character usage for cost analytics.
Java backend
- ModelPrice.java, SpanCostCalculator.java, CostService.java extended with inputCharacterPrice and new speechTtsCost calculator to bill TTS spans by character.
Front-end
- TracesSpansTab.tsx adds a Chars column, surfacing character_count and tying it into existing cost display.
Documentation
- openai.mdx updated with TTS tracing instructions, environment setup, and pricing explanation.

Issues

Resolves #2202
/claim #2202

Testing

Python integration tests: test_openai_speech.py verify sync & streaming paths.
Java unit tests: CostServiceTest.java check character-based cost computation.
All existing suites run green.

Documentation

Primary guide updated at apps/opik-documentation/documentation/fern/docs/tracing/integrations/openai.mdx, including examples and cost tables for TTS.

… opik sdk

…ch events

…ricing, although with hardcoded open-ai tts values

…ver hit it

apps/opik-backend/src/main/java/com/comet/opik/domain/cost/CostService.java

apps/opik-backend/src/main/java/com/comet/opik/domain/cost/SpanCostCalculator.java

apps/opik-backend/src/test/java/com/comet/opik/domain/cost/CostServiceTest.java

apps/opik-backend/src/main/java/com/comet/opik/domain/cost/CostService.java

Copilot

Pull Request Overview

This PR introduces comprehensive support for OpenAI Text-to-Speech (TTS) models by implementing end-to-end tracking, cost calculation, and UI integration. The changes enable Opik to trace audio.speech requests, calculate costs based on character usage, and display TTS-specific data in the frontend.

Implements TTS request tracing for both sync and streaming speech generation with audio attachment support
Adds character-based cost calculation for TTS models (tts-1, tts-1-hd) in the backend pricing system
Extends the frontend UI with a "Chars" column to display character count usage across traces, spans, threads, and projects

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`test_openai_speech.py`	Comprehensive integration tests for sync and streaming TTS tracing
`openai_speech_decorator.py`	Main decorator implementing TTS request tracking with audio buffer management
`speech_stream_*.py`	Supporting classes for streaming TTS: buffer management and event aggregation
`opik_usage*.py`	Usage tracking extensions to support character-based metrics for TTS models
`opik_tracker.py`	Integration of speech decorator into the main OpenAI tracking system
`stream_patchers.py`	Enhanced stream patching to support TTS chunk processing
Frontend components	Addition of "Chars" column across TracesSpansTab, ThreadsTab, and ProjectsPage
Backend cost system	Extended ModelPrice and cost calculation to support character-based pricing
`openai.mdx`	Updated documentation with TTS usage examples and cost information

Comments suppressed due to low confidence (3)

apps/opik-backend/src/main/java/com/comet/opik/domain/cost/SpanCostCalculator.java:96

[nitpick] The method name 'speechTtsCost' is redundant since TTS already stands for Text-to-Speech. Consider renaming to 'speechCost' or 'textToSpeechCost'.

    public static BigDecimal speechTtsCost(ModelPrice modelPrice, Map<String, Integer> usage) {

apps/opik-backend/src/main/java/com/comet/opik/domain/cost/CostService.java:107

Missing braces around the else clause. According to Java coding standards, all control flow statements should use braces for clarity and maintainability.

                } else

apps/opik-backend/src/main/java/com/comet/opik/domain/cost/CostService.java:112

Missing braces around the else clause. All control flow statements should use braces for consistency and maintainability.

                            } else

Copilot · 2025-07-25T14:21:21Z

sdks/python/tests/library_integration/openai/test_openai_speech.py

+    def __init__(self, audio: bytes):
+        self._audio = audio
+
+    def model_dump(self, mode: str = "json") -> dict:  # noqa: D401 – follow OpenAI style


The comment uses an en dash (–) instead of a hyphen (-). Use a standard hyphen for consistency.

Suggested change

def model_dump(self, mode: str = "json") -> dict: # noqa: D401 – follow OpenAI style

def model_dump(self, mode: str = "json") -> dict: # noqa: D401 - follow OpenAI style

Copilot · 2025-07-25T14:21:22Z

sdks/python/src/opik/integrations/openai/openai_speech_decorator.py

+                if error_info is None and buffer.should_attach():
+                    path: Optional[Path] = buffer.flush_to_tempfile(".mp3")
+                    if path is not None:
+                        client = opik_client_module.get_client_cached()
+                        attach_obj = attachment_module.Attachment(data=str(path))
+                        client.update_span(
+                            id=generators_span_to_end.id,
+                            trace_id=generators_span_to_end.trace_id,
+                            parent_span_id=generators_span_to_end.parent_span_id,
+                            project_name=generators_span_to_end.project_name or "",
+                            attachments=[attach_obj],
+                        )
+                        try:
+                            path.unlink()
+                        except Exception:
+                            pass


The attachment logic is duplicated between sync and async handlers. Consider extracting this into a separate method to reduce code duplication.

Suggested change

if error_info is None and buffer.should_attach():

path: Optional[Path] = buffer.flush_to_tempfile(".mp3")

if path is not None:

client = opik_client_module.get_client_cached()

attach_obj = attachment_module.Attachment(data=str(path))

client.update_span(

id=generators_span_to_end.id,

trace_id=generators_span_to_end.trace_id,

parent_span_id=generators_span_to_end.parent_span_id,

project_name=generators_span_to_end.project_name or "",

attachments=[attach_obj],

)

try:

path.unlink()

except Exception:

pass

if error_info is None:

self._handle_attachment(buffer, generators_span_to_end)

Copilot · 2025-07-25T14:21:22Z

sdks/python/src/opik/integrations/openai/openai_speech_decorator.py

+                if error_info is None and buffer.should_attach():
+                    path: Optional[Path] = buffer.flush_to_tempfile(".mp3")
+                    if path is not None:
+                        client = opik_client_module.get_client_cached()
+                        attach_obj = attachment_module.Attachment(data=str(path))
+                        client.update_span(
+                            id=generators_span_to_end.id,
+                            trace_id=generators_span_to_end.trace_id,
+                            parent_span_id=generators_span_to_end.parent_span_id,
+                            project_name=generators_span_to_end.project_name or "",
+                            attachments=[attach_obj],
+                        )
+                        try:
+                            path.unlink()
+                        except Exception:
+                            pass


This attachment logic is identical to the sync version. The duplication could be eliminated by extracting the attachment functionality into a helper method.

Suggested change

if error_info is None and buffer.should_attach():

path: Optional[Path] = buffer.flush_to_tempfile(".mp3")

if path is not None:

client = opik_client_module.get_client_cached()

attach_obj = attachment_module.Attachment(data=str(path))

client.update_span(

id=generators_span_to_end.id,

trace_id=generators_span_to_end.trace_id,

parent_span_id=generators_span_to_end.parent_span_id,

project_name=generators_span_to_end.project_name or "",

attachments=[attach_obj],

)

try:

path.unlink()

except Exception:

pass

if error_info is None:

self._attach_buffer_to_span(buffer, generators_span_to_end)

aadereiko

Hey @vladimirrotariu,
regarding changes in apps/opik-frontend/, could you please avoid reformatting large parts of files (like changing the order of imports) unless you’ve actually modified that code? It makes reviewing PRs a bit more complicated

This reverts commit 4338898.

…h-models

vladimirrotariu · 2025-08-06T18:30:49Z

Hi @andrescrz, is there anything I should do right now?

vincentkoc · 2025-08-07T11:01:37Z

@vladimirrotariu see comments raised by team as well as open issues and merge conflicts. Although we appreciate the contribution given the volume of changes and review required the team may close this PR as there are other submissions and/or close the bounty entirely.

vladimirrotariu · 2025-08-07T15:46:43Z

@vincentkoc but I reverted the changes to the non-formatted ones more than a week ago, now the changes strictly adheres to the implementation logic. Please explain further.

andrescrz · 2025-08-12T12:45:27Z

Hi @vladimirrotariu

Thanks for your contributions! As per the bounty guidelines, please note that we process one bounty PR at a time, so we'll focus first on PR #2850 since it appears closer to being ready.

In the meantime, for this PR, could you please:

Rebase on the latest main branch and resolve any conflicts.
Address the remaining open comments from both Opik engineers and the co-pilot review.

Once the first PR is merged, we'll review your other PRs one by one in accordance with the bounty process. Thanks for your patience and for helping improve Opik!

vladimirrotariu added 15 commits July 24, 2025 23:00

add middleware processing usage data between open-ai tts endpoint and…

3598893

… opik sdk

add decorator for openai tts models, and aggregator for streamed sppe…

cfcd49a

…ch events

add patching for openai tts models in the opik tracker

af87dc0

remove repetition in decorator for openai tts

5cdb3f0

add tests

c3a1ca4

remove env vars stubs

7d2ba0f

extend calculate cost method of the cost service with per character p…

89c7a63

…ricing, although with hardcoded open-ai tts values

extend calculate cost method of the cost service with per character p…

4d01ff7

…ricing, although with hardcoded open-ai tts values

read pricing of models from file

19cf56d

add separate test for character-based cost as the other code paths ne…

49fb6af

…ver hit it

apply spotless

4338898

fix the test by making it independent of changes in prices files

223413c

add audio player conditional on audio attachment with fitting MIME type

be38182

add audio player conditional on audio attachment with fitting MIME type

49b13cb

document the newly supported models

7033486

vladimirrotariu requested review from a team as code owners July 25, 2025 08:19

algora-pbc bot added the 🙋 Bounty claim label Jul 25, 2025

algora-pbc bot mentioned this pull request Jul 25, 2025

[FR]: Support Openai TTS models tracking #2202

Open

BorisTkachenko reviewed Jul 25, 2025

View reviewed changes

aadereiko requested a review from Copilot July 25, 2025 14:20

Copilot AI reviewed Jul 25, 2025

View reviewed changes

aadereiko requested changes Jul 25, 2025

View reviewed changes

vladimirrotariu added 6 commits July 28, 2025 04:15

revert to passing usage without null guarding

98af193

Revert "apply spotless"

45234ab

This reverts commit 4338898.

aborb the dynamic tracing tests in the SpansResourceTest class

b74627b

revert import order

2be80a7

extract duplicate logic in helper private method

1c881c9

reconcile with main

c0dcc80

vladimirrotariu requested a review from BorisTkachenko July 28, 2025 13:06

vladimirrotariu requested a review from aadereiko July 28, 2025 13:06

vladimirrotariu added 6 commits July 29, 2025 22:02

Merge branch 'main' into feature/add-support-for-openai-text-to-speec…

0728008

…h-models

revert import reordering on autosave

0e77d70

Merge branch 'main' into feature/add-support-for-openai-text-to-speec…

d2c49ac

…h-models

Merge branch 'main' into feature/add-support-for-openai-text-to-speec…

94f6fd7

…h-models

Merge branch 'main' into feature/add-support-for-openai-text-to-speec…

a6e6f96

…h-models

Merge branch 'main' into feature/add-support-for-openai-text-to-speec…

6e87c5e

…h-models

andrescrz added the Pending user response label Aug 6, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feature/add support for OpenAI text to speech models #2829

Feature/add support for OpenAI text to speech models #2829

Uh oh!

vladimirrotariu commented Jul 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jul 25, 2025

Uh oh!

Copilot AI Jul 25, 2025

Uh oh!

Copilot AI Jul 25, 2025

Uh oh!

aadereiko left a comment

Uh oh!

vladimirrotariu commented Aug 6, 2025

Uh oh!

vincentkoc commented Aug 7, 2025

Uh oh!

vladimirrotariu commented Aug 7, 2025

Uh oh!

andrescrz commented Aug 12, 2025

Uh oh!

Uh oh!

	def model_dump(self, mode: str = "json") -> dict: # noqa: D401 – follow OpenAI style
	def model_dump(self, mode: str = "json") -> dict: # noqa: D401 - follow OpenAI style

Feature/add support for OpenAI text to speech models #2829

Are you sure you want to change the base?

Feature/add support for OpenAI text to speech models #2829

Uh oh!

Conversation

vladimirrotariu commented Jul 25, 2025

Details

Issues

Testing

Documentation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

aadereiko left a comment

Choose a reason for hiding this comment

Uh oh!

vladimirrotariu commented Aug 6, 2025

Uh oh!

vincentkoc commented Aug 7, 2025

Uh oh!

vladimirrotariu commented Aug 7, 2025

Uh oh!

andrescrz commented Aug 12, 2025

Uh oh!

Uh oh!