Test templates #6086
Merged
Test templates #6086
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jeffhandley
reviewed
Mar 12, 2025
...rojectTemplates/Microsoft.Extensions.AI.Templates.InegrationTests/AichatwebTemplatesTests.cs
Outdated
Show resolved
Hide resolved
RussKie
force-pushed
the
test_templates
branch
2 times, most recently
from
a71ffcc to
00ef90e
Compare
|
I'm surprised it worked, but it did. |
RussKie
added
area-ai-templates
evgenyfedorov2
approved these changes
Mar 13, 2025
MackinnonBuck
approved these changes
Mar 13, 2025
test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/TestBase.cs
Outdated
Show resolved
Hide resolved
Comment on lines
+10
to
+36
| <ItemGroup> | ||
| <!--#if (IsOllama) | ||
| <PackageReference Include="OllamaSharp" Version="${OllamaSharpVersion}" /> | ||
| #elif (IsGHModels) | ||
| <PackageReference Include="OpenAI" Version="${OpenAIVersion}" /> | ||
| <PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="${MicrosoftExtensionsAIVersion}" /> | ||
| #elif (IsAzureAiFoundry) | ||
| <PackageReference Include="Azure.AI.Projects" Version="${AzureAIProjectsVersion}" /> | ||
| <PackageReference Include="Azure.AI.OpenAI" Version="${AzureAIOpenAIVersion}" /> | ||
| <PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="${MicrosoftExtensionsAIVersion}" /> | ||
| #else --> | ||
| <PackageReference Include="Azure.AI.OpenAI" Version="${AzureAIOpenAIVersion}" /> | ||
| <PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="${MicrosoftExtensionsAIVersion}" /> | ||
| <!--#endif --> | ||
| <!--#if (UseManagedIdentity) --> | ||
| <PackageReference Include="Azure.Identity" Version="${AzureIdentityVersion}" /> | ||
| <!--#endif --> | ||
| <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="9.0.2" /> | ||
| <PackageReference Include="Microsoft.Extensions.AI" Version="${MicrosoftExtensionsAIVersion}" /> | ||
| <PackageReference Include="Microsoft.SemanticKernel.Core" Version="${MicrosoftSemanticKernelCoreVersion}" /> | ||
| <PackageReference Include="PdfPig" Version="${PdfPigVersion}" /> | ||
| <PackageReference Include="System.Linq.Async" Version="${SystemLinqAsyncVersion}" /> | ||
| <!--#if (UseAzureAISearch) | ||
| <PackageReference Include="Azure.Search.Documents" Version="${AzureSearchDocumentsVersion}" /> | ||
| <PackageReference Include="Microsoft.SemanticKernel.Connectors.AzureAISearch" Version="${MicrosoftSemanticKernelConnectorsAzureAISearchVersion}" /> | ||
| #endif --> | ||
| </ItemGroup> |
There was a problem hiding this comment.
Why is the .in file included in the snapshot? Does this mean that the .in file gets included in the generated project?
There was a problem hiding this comment.
That's a good question which I can't answer :)
I guess it does...
There was a problem hiding this comment.
@MackinnonBuck - yes, the *.in file is included in the generated content.
I followed the steps the test performs:
PS D:\Development> dotnet new install D:\Development\dotnet-extensions\src\ProjectTemplates\Microsoft.Extensions.AI.Templates\src\ChatWithCustomData --debug:custom-hive C:\Users\user\AppData\Local\Temp\hkgl3nh4.eq4\home
The following template packages will be installed:
D:\Development\dotnet-extensions\src\ProjectTemplates\Microsoft.Extensions.AI.Templates\src\ChatWithCustomData
Success: D:\Development\dotnet-extensions\src\ProjectTemplates\Microsoft.Extensions.AI.Templates\src\ChatWithCustomData installed the following templates:
Template Name Short Name Language Tags
--------------- ---------- -------- -------------
AI Chat Web App aichatweb [C#] Common/AI/Web
PS D:\Development> pushd C:\Users\user\AppData\Local\Temp\TemplateEngine.Tests\test
PS C:\Users\user\AppData\Local\Temp\TemplateEngine.Tests\test> dotnet new aichatweb -n aichatweb -o aichatweb --debug:custom-hive C:\Users\user\AppData\Local\Temp\hkgl3nh4.eq4\home --no-update-check
The template "AI Chat Web App" was created successfully.
PS C:\Users\user\AppData\Local\Temp\TemplateEngine.Tests\test> ls .\aichatweb\
Directory: C:\Users\user\AppData\Local\Temp\TemplateEngine.Tests\test\aichatweb
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 14/03/2025 11:17 AM Components
d----- 14/03/2025 11:17 AM Properties
d----- 14/03/2025 11:17 AM Services
d----- 14/03/2025 11:17 AM wwwroot
-a---- 14/03/2025 11:17 AM 889 aichatweb.csproj
-a---- 14/03/2025 11:17 AM 2053 aichatweb.csproj.in
-a---- 14/03/2025 11:17 AM 178 appsettings.Development.json
-a---- 14/03/2025 11:17 AM 202 appsettings.json
-a---- 14/03/2025 11:17 AM 2678 Program.cs
-a---- 14/03/2025 11:17 AM 1194 README.mdThere was a problem hiding this comment.
Oh, it's probably because the test is looking at files directly inside the repo and treating them as template content. This ignores any preprocessing of such content that happens as part of building the Microsoft.Extensions.AI.Templates project (for example, here).
There was a problem hiding this comment.
We don't have to block the PR on it, but it might be good to figure out how to fix that for the future. Maybe the Microsfot.Extensions.AI.Templates project copies its content to an output folder at build time, and then that's what gets used by the tests.
There was a problem hiding this comment.
Oops, your comment didn't show up for me until I refreshed the page. If you produce a NuGet package from Microsfot.Extensions.AI.Templates, I'm guessing there's no .in file there, right? That's what I've seen from my local testing.
There was a problem hiding this comment.
Correct - no *.in files in the nupkg
peterwald
pushed a commit
to peterwald/extensions
that referenced
this pull request
Mar 14, 2025
Leveraging [template testing infra](https://github.com/dotnet/templating/wiki/Templates-Testing-Tooling) for a basic test of the AI template. More tests can be added later. During a test the template is getting installed locally and then a new app is created, which is then gets snapshot-verified using Verify tooling. For the local devex, it is very much worth installing [Verify DiffEngineTray](https://github.com/VerifyTests/DiffEngine/blob/main/docs/tray.md) to approve all incoming snapshots (for the initial iteration). It is also possible that git block staging the snapshot files due to line ending issues, and "git config core.safecrlf false" could be used as a way to work around this.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Leveraging template testing infra for a basic test of the AI template. More tests can be added later.
During a test the template is getting installed locally and then a new app is created, which is then gets snapshot-verified using Verify tooling.
For the local devex, it is very much worth installing Verify DiffEngineTray to approve all incoming snapshots (for the initial iteration).
It is also possible that git block staging the snapshot files due to line ending issues, and
git config core.safecrlf falsecould be used as a jailbreak card.Microsoft Reviewers: Open in CodeFlow