Skip to content

Add security comments for GetResponseAsync/GetStreamingResponseAsync #6386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
May 13, 2025
Merged

Add security comments for GetResponseAsync/GetStreamingResponseAsync #6386

merged 6 commits into from
May 13, 2025

Conversation

jeffhandley
Copy link
Member

@jeffhandley jeffhandley commented May 7, 2025 •
edited by dotnet-policy-service bot
Loading

Fixes #6245

An outcome from the threat model review. The docs for GetResponseAsync/GetStreamingResponseAsync should be clear that:

  1. We are just passing through any messages and returning whatever results come back.
  2. MEAI doesn't do any defence against prompt injection; it's up to application code to do that
  3. MEAI doesn't impose any limits on data sizes, either outbound or returned. Apps that need to limit this must do it themselves.
  4. MEAI doesn't impose any limits on the number of calls (per user or in total, over any period of time) since it's stateless. Apps that need to impose limits (e.g., using rate limiting) must do it themselves
Microsoft Reviewers: Open in CodeFlow

@jeffhandley jeffhandley self-assigned this May 7, 2025
@jeffhandley jeffhandley requested a review from a team as a code owner May 7, 2025 05:05
@github-actions github-actions bot added the area-ai Microsoft.Extensions.AI libraries label May 7, 2025
@jeffhandley
Copy link
Member Author

@gewarren @stephentoub I used GitHub Copilot in VS to coach me through how to address the problem of needing to set expectations on the interface API, while then providing clear statements on the implementations. I'm happy with the results; what do you think?

Also, @gewarren, is it OK to use <inheritdoc /> and then add additional <remarks>?

stephentoub
stephentoub reviewed May 10, 2025
View reviewed changes
@gewarren
Copy link
Contributor

Also, @gewarren, is it OK to use <inheritdoc /> and then add additional <remarks>?

Yes, that just means that no remarks will be inherited.

@jeffhandley jeffhandley merged commit b2298a6 into dotnet:main May 13, 2025
6 checks passed
jeffhandley added a commit that referenced this pull request May 15, 2025
@jeffhandley
Add security comments for chat clients (#6386)
d1e1319
jeffhandley added a commit that referenced this pull request May 15, 2025
@jeffhandley @stephentoub
@eiriktsarpalis @jozkee @gewarren @SteveSandersonMS @MackinnonBuck @jongalloway @peterwald
[9.5] Merge from main into release/9.5 for MEAI stable package releas…
988e9ee 
…es (#6451)

* Translate OpenAI refusals to ErrorContent (#6393)

Refusals in OpenAI are errors reported when the service can't generate an output that matches the requested schema. Translate refusals to ErrorContent now that we have it.

* Add JSON schema transformation functionality to `AIJsonUtilities` (#6383)

* Add initial schema transformation functionality and incorporate into the OpenAI leaf client.

* Update all leaf client implementions, improve naming, add testing.

* Remove redundant suppressions

* Address feedback.

* Add ChatOptions.RawRepresentationFactory (#6319)

* Look for OpenAI.ChatCompletionOptions in top-level additional properties and stop looking for individually specific additional properties

* Add RawRepresentation to ChatOptions and use it in OpenAI and AzureAIInference

* Remove now unused locals

* Add [JsonIgnore] and update roundtrip tests

* Overwirte properties only if the underlying model don't specify it already

* Clone RawRepresentation

* Reflection workaround for ToolChoice not being cloned

* Style changes

* AI.Inference: Bring back propagation of additional properties

* Don't use 0.1f, it doesn't roundtrip properly in .NET Framework

* Add RawRepresentationFactory instead of object? property

* Augment remarks to discourage returning shared instances

* Documentation feedback

* AI.Inference: keep passing TopK as AdditionalProperty if not already there

* Fix streaming chat response example (#6408)

* Move AIFunctionFactory down to M.E.AI.Abstractions (#6412)

* Remove AIFunctionFactory dependency on M.E.DI

This means reverting the recent changes to it that:
- Special-cased KeyedServices
- Special-cased IServiceProviderIsService
- Used ActivatorUtilities.CreateInstance

* Move AIFunctionFactory down to M.E.AI.Abstractions

* Add CreateInstance delegate to AIFunctionFactoryOptions

To enable use of ActivatorUtilities.CreateInstance or alternative.

* Add some comments

* Fix handling of tool calls with some OpenAI endpoints (#6405)

* Fix handling of tool calls with some endpoints

Most assistant messages containing tool calls don't contain text as well (though some can). In such a case, we were still creating the assistant with empty text. While OpenAI's service permits that, some other endpoints are more finicky about it. This avoids doing so.

* Reduce to single iteration through assistant content

* Delete Microsoft.Extensions.AI.Abstractions APIs marked [Obsolete] during preview (#6414)

* Add WriteAsync overrides to stream helper in AIFunctionFactory (#6419)

We use JsonSerializer.SerializeAsync but were missing the async overrides. As with MemoryStream, these don't need to queue.

* Replace Type targetType AIFunctionFactory.Create parameter with a func (#6424)

* Add an AIJsonSchemaTransformOptions property inside AIJsonSchemaCreateOptions and mark redundant properties in the latter as obsolete. (#6427)

* Add an AIJsonSchemaTransformOptions property inside AIJsonSchemaCreateOptions and mark redundant properties in the latter as obsolete.

* s/inferred/created

* Eliminate ingestion cache from AI Chat Web template (#6428)

* Begin updating to latest MEVD

* Reimplement JsonVectorStore to match updated MEVD APIs

* Remove ingestion cache and track ingestion status inside the vector DB

* Track the document metadata in a separate collection so we don't have to fetch literally everything from the vector DB in order to update ingestion

* Fix equality comparison issue with Qdrant connector

* Tidying

* More tidying

* Update MEAI.Templates test snapshots

---------

Co-authored-by: Jeff Handley <[email protected]>

* Update the template test README with snapshot update instructions (#6431)

* AI Chat Web template fixes for Azure AI Search (#6429)

* AI Chat Web template fixes for Azure AI Search

* Update snapshots

* Add security comments for chat clients (#6386)

* Remove unused select param (#6341)

CreateRecordsForDocumentAsync includes `Select((pair, index) =>` but index is never used

Co-authored-by: Jeff Handley <[email protected]>

* Add RawRepresentationFactory to other options types (#6433)

* Add RawRepresentationFactory to other options types

* Undo changes in Azure.AI.Inference

* Address documentation feedback

* Ensure the type keyword is included when generating schemas for nullable enums. (#6440)

* Remove obsolete members from AIJsonSchemaCreateOptions (#6432)

* Use RawRepresentationFactory on AzureAIInference embedding generators (#6445)

* Mark Microsoft.Extensions.AI and Microsoft.Extensions.AI.Abstractions as stable (#6446)

* Bump ICSharpCode.Decompiler for record struct support in ApiChief tool

Needed the fix for icsharpcode/ILSpy#3159 to fix "record struct" formatting (it was "recordstruct" before the fix).

* Generate ApiChief baselines for MEAI libraries

Ran .\scripts\MakeApiBaselines.ps1 and discarded other libraries' updates.

* Hand-edit MEAI ApiChief baseline to fix params ReadOnlySpan

Params collections are not yet supported in ICSharpCode.Decompiler:
icsharpcode/ILSpy#829

The result is an emitted 'scoped' keyword instead of 'params'. This was edited by hand in the baseline MEAI file.

* Mark Microsoft.Extensions.AI and Microsoft.Extensions.AI.Abstractions as stable

* Update MEAI and MEAI.Abstractions NuGet package documentation

* Update NuGet package documentation for MEAI implementation packages

* Update MEAI.Templates package references, including SemanticKernel for a coherent build.

* Lower OllamaSharp for integration tests to use version available on feed

* Empty the ApiChief baselines for Ollama, AzureAIInference, and OpenAI adapters since they are not shipping stable

* Apply code review feedback to the MEAI package READMEs

* Update MEAI.Templates test snapshots for version bumps

* Apply documentation review feedback to the MEAI package READMEs

* Add comments to the MEAI API baseline file for the hand-editing required

* Restore documentation blurb into Microsoft.Extensions.AI.AzureAIInference per other feedback

* Use stable version for MEAI in templates

* Mark Microsoft.Extensions.AI.Evaluation.* Libraries as stable (#6450)

* Mark Microsoft.Extensions.AI packages stable

All packages except Microsoft.Extensions.AI.Evaluation.Safety are being marked stable.

* Remove primary constructors from API json files.

* Remove more primary constructors from API Chief json

---------

Co-authored-by: Jeff Handley <[email protected]>

---------

Co-authored-by: Stephen Toub <[email protected]>
Co-authored-by: Eirik Tsarpalis <[email protected]>
Co-authored-by: David Cantú <[email protected]>
Co-authored-by: Genevieve Warren <[email protected]>
Co-authored-by: Steve Sanderson <[email protected]>
Co-authored-by: Mackinnon Buck <[email protected]>
Co-authored-by: Jon Galloway <[email protected]>
Co-authored-by: Peter Waldschmidt <[email protected]>
@github-actions github-actions bot locked and limited conversation to collaborators Jun 12, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@stephentoub stephentoub stephentoub left review comments

@SteveSandersonMS SteveSandersonMS SteveSandersonMS approved these changes

@gewarren gewarren gewarren approved these changes

Assignees

@jeffhandley jeffhandley

Labels
area-ai Microsoft.Extensions.AI libraries
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

MEAI docs: GetResponseAsync/GetStreamingResponseAsync
4 participants
@jeffhandley @gewarren @SteveSandersonMS @stephentoub