22
22
@component
23
23
class OpenAIChatGenerator :
24
24
"""
25
- A Chat Generator component that uses the OpenAI API to generate text .
25
+ Completes chats using OpenAI's large language models (LLMs) .
26
26
27
- Enables text generation using OpenAI's large language models (LLMs). It supports `gpt-4` and `gpt-3.5-turbo`
28
- family of models accessed through the chat completions API endpoint.
27
+ It works with the gpt-4 and gpt-3.5-turbo models and supports streaming responses
28
+ from OpenAI API. It uses [ChatMessage](https://docs.haystack.deepset.ai/docs/data-classes#chatmessage)
29
+ format in input and output.
29
30
30
- Users can pass any text generation parameters valid for the `openai.ChatCompletion.create` method
31
- directly to this component via the `generation_kwargs` parameter in `__init__` or the `generation_kwargs`
32
- parameter in `run` method.
31
+ You can customize how the text is generated by passing parameters to the
32
+ OpenAI API. Use the `**generation_kwargs` argument when you initialize
33
+ the component or when you run it. Any parameter that works with
34
+ `openai.ChatCompletion.create` will work here too.
33
35
34
- For more details on the parameters supported by the OpenAI API, refer to the OpenAI
35
- [documentation](https://platform.openai.com/docs/api-reference/chat).
36
+ For details on OpenAI API parameters, see
37
+ [OpenAI documentation](https://platform.openai.com/docs/api-reference/chat).
38
+
39
+ ### Usage example
36
40
37
41
```python
38
42
from haystack.components.generators.chat import OpenAIChatGenerator
@@ -56,17 +60,6 @@ class OpenAIChatGenerator:
56
60
]
57
61
}
58
62
```
59
-
60
- Key Features and Compatibility:
61
- - Primary Compatibility: designed to work seamlessly with the OpenAI API Chat Completion endpoint and `gpt-4`
62
- and `gpt-3.5-turbo` family of models.
63
- - Streaming Support: supports streaming responses from the OpenAI API Chat Completion endpoint.
64
- - Customizability: supports all parameters supported by the OpenAI API Chat Completion endpoint.
65
-
66
- Input and Output Format:
67
- - ChatMessage Format: this component uses the ChatMessage format for structuring both input and output,
68
- ensuring coherent and contextually relevant responses in chat-based text generation scenarios. Details on the
69
- ChatMessage format can be found at [here](https://docs.haystack.deepset.ai/v2.0/docs/data-classes#chatmessage).
70
63
"""
71
64
72
65
def __init__ (
@@ -81,30 +74,31 @@ def __init__(
81
74
max_retries : Optional [int ] = None ,
82
75
):
83
76
"""
84
- Initializes the OpenAIChatGenerator component.
85
-
86
- Creates an instance of OpenAIChatGenerator. Unless specified otherwise in the `model`, this is for OpenAI's
87
- GPT-3.5 model.
77
+ Creates an instance of OpenAIChatGenerator. Unless specified otherwise in `model`, uses OpenAI's GPT-3.5.
88
78
89
- By setting the 'OPENAI_TIMEOUT' and 'OPENAI_MAX_RETRIES' you can change the timeout and max_retries parameters
79
+ Before initializing the component, you can set the 'OPENAI_TIMEOUT' and 'OPENAI_MAX_RETRIES'
80
+ environment variables to override the `timeout` and `max_retries` parameters respectively
90
81
in the OpenAI client.
91
82
92
83
:param api_key: The OpenAI API key.
84
+ You can set it with an environment variable `OPENAI_API_KEY`, or pass with this parameter
85
+ during initialization.
93
86
:param model: The name of the model to use.
94
87
:param streaming_callback: A callback function that is called when a new token is received from the stream.
95
- The callback function accepts StreamingChunk as an argument.
88
+ The callback function accepts [StreamingChunk](https://docs.haystack.deepset.ai/docs/data-classes#streamingchunk)
89
+ as an argument.
96
90
:param api_base_url: An optional base URL.
97
- :param organization: The Organization ID, defaults to `None`. See
91
+ :param organization: Your organization ID, defaults to `None`. See
98
92
[production best practices](https://platform.openai.com/docs/guides/production-best-practices/setting-up-your-organization).
99
- :param generation_kwargs: Other parameters to use for the model. These parameters are all sent directly to
93
+ :param generation_kwargs: Other parameters to use for the model. These parameters are sent directly to
100
94
the OpenAI endpoint. See OpenAI [documentation](https://platform.openai.com/docs/api-reference/chat) for
101
95
more details.
102
96
Some of the supported parameters:
103
97
- `max_tokens`: The maximum number of tokens the output text can have.
104
98
- `temperature`: What sampling temperature to use. Higher values mean the model will take more risks.
105
99
Try 0.9 for more creative applications and 0 (argmax sampling) for ones with a well-defined answer.
106
100
- `top_p`: An alternative to sampling with temperature, called nucleus sampling, where the model
107
- considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens
101
+ considers the results of the tokens with top_p probability mass. For example, 0.1 means only the tokens
108
102
comprising the top 10% probability mass are considered.
109
103
- `n`: How many completions to generate for each prompt. For example, if the LLM gets 3 prompts and n is 2,
110
104
it will generate two completions for each of the three prompts, ending up with 6 completions in total.
@@ -116,11 +110,11 @@ def __init__(
116
110
- `logit_bias`: Add a logit bias to specific tokens. The keys of the dictionary are tokens, and the
117
111
values are the bias to add to that token.
118
112
:param timeout:
119
- Timeout for OpenAI Client calls, if not set it is inferred from the `OPENAI_TIMEOUT` environment variable
120
- or set to 30 .
113
+ Timeout for OpenAI client calls. If not set, it defaults to either the
114
+ `OPENAI_TIMEOUT` environment variable, or 30 seconds .
121
115
:param max_retries:
122
- Maximum retries to stablish contact with OpenAI if it returns an internal error, if not set it is inferred
123
- from the `OPENAI_MAX_RETRIES` environment variable or set to 5.
116
+ Maximum number of retries to contact OpenAI after an internal error.
117
+ If not set, it defaults to either the `OPENAI_MAX_RETRIES` environment variable, or set to 5.
124
118
"""
125
119
self .api_key = api_key
126
120
self .model = model
@@ -190,14 +184,14 @@ def run(
190
184
generation_kwargs : Optional [Dict [str , Any ]] = None ,
191
185
):
192
186
"""
193
- Invoke the text generation inference based on the provided messages and generation parameters.
187
+ Invokes chat completion based on the provided messages and generation parameters.
194
188
195
189
:param messages: A list of ChatMessage instances representing the input messages.
196
190
:param streaming_callback: A callback function that is called when a new token is received from the stream.
197
191
:param generation_kwargs: Additional keyword arguments for text generation. These parameters will
198
- potentially override the parameters passed in the `__init__` method .
199
- For more details on the parameters supported by the OpenAI API, refer to the
200
- OpenAI [ documentation](https://platform.openai.com/docs/api-reference/chat/create).
192
+ override the parameters passed during component initialization .
193
+ For details on OpenAI API parameters, see
194
+ [ OpenAI documentation](https://platform.openai.com/docs/api-reference/chat/create).
201
195
202
196
:returns:
203
197
A list containing the generated responses as ChatMessage instances.
0 commit comments