Skip to content

Commit 3776f05

Browse files
jongallowayCopilotMackinnonBuck
authored andcommitted
Add note on extending timeout for Ollama client (#6258)
* Add note on extending timeout for Ollama client Standard HttpClient timeout of 10 seconds applied by Service Defaults under .NET Aspire is likely too short for Ollama calls. Add note with code to extend this timeout. * Update src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData/README.Aspire.md Co-authored-by: Copilot <[email protected]> * Update src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData/README.Aspire.md Co-authored-by: Mackinnon Buck <[email protected]> --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Mackinnon Buck <[email protected]>
1 parent 4489695 commit 3776f05

File tree

1 file changed

+17
-0
lines changed
  • src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData

1 file changed

+17
-0
lines changed

src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData/README.Aspire.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,23 @@ This project is configured to run Ollama in a Docker container. Docker Desktop m
8181

8282
Download, install, and run Docker Desktop from the [official website](https://www.docker.com/). Follow the installation instructions specific to your operating system.
8383

84+
The response time from Ollama depends on your GPU. If you find that default resilience timeout of 10 seconds used by .NET Aspire is timing out for the HTTP call from the web app to the Ollama service, you can update the web app's `Program.cs` to extend the timeout by adding the following below the line which reads `builder.AddServiceDefaults();`:
85+
```csharp
86+
// Extend the HTTP Client timeout for Ollama
87+
builder.Services.ConfigureHttpClientDefaults(http =>
88+
{
89+
#pragma warning disable EXTEXP0001
90+
http.RemoveAllResilienceHandlers();
91+
http.AddStandardResilienceHandler(config =>
92+
{
93+
config.AttemptTimeout.Timeout = TimeSpan.FromMinutes(3);
94+
config.CircuitBreaker.SamplingDuration = TimeSpan.FromMinutes(10); // must be at least double the AttemptTimeout to pass options validation
95+
config.TotalRequestTimeout.Timeout = TimeSpan.FromMinutes(10);
96+
});
97+
#pragma warning restore EXTEXP0001
98+
});
99+
```
100+
84101
Note: Ollama and Docker are excellent open source products, but are not maintained by Microsoft.
85102
#### ---#endif
86103
#### ---#if (IsAzureOpenAI)

0 commit comments

Comments
 (0)