Move tests and test debuggees to repo root 'tests' directory #5554

max-charlamb · 2025-08-28T01:00:58Z

This pull request reorganizes and improves the project and build configuration, with a particular focus on test infrastructure and project structure. The changes streamline how test projects are referenced and built, clarify property settings for packaging and shipping, and update solution/project references to reflect a new directory structure for test assets.

Project and Build Configuration Improvements:

Added a new Build.proj traversal project that includes all main diagnostic tool projects and conditionally includes test projects, improving build orchestration and clarity.
Updated Directory.Build.props to set IsShipping and IsPackable to false by default, and suppress warnings about packing non-packable projects, ensuring that internal/test projects are not accidentally shipped or packed.
Added architecture-specific properties for test .NET install roots and registry roots, improving test environment configuration for different target architectures.

Test Infrastructure and Project Structure:

Added add_subdirectory(tests) to CMakeLists.txt to ensure that test projects are included in native builds.
Updated the Visual Studio solution file debuggees.sln to reference test debuggee projects from the new tests/Debuggees directory, and added several new test debuggee projects to the solution.
Corrected a path in THIRD-PARTY-NOTICES.TXT to point to the new location of the Xunit.Extensions code under tests/Microsoft.Diagnostics.TestHelpers, reflecting the directory reorganization.edited version of Move tests and test debuggees to repo root 'tests' directory #5469

…es with build in them. Flags as duplicate params

… (NuGet.config, Version.props, and a debuggee Directory.Build.props+targets)

eng/Build.props

eng/build.ps1

noahfalk

What is the rationale for moving stuff in \src\tests -> \tests? For example dotnet\runtime or dotnet\aspnetcore both put their tests under the src directory. It seems like this creates a lot of churn and makes our repo different than others.

I'm not necessarily opposed but I do want to understand before we churn 100s of files in the repo versioning history. I just marked request changes to make sure it doesn't get accidentally checked in before we talked about it.

steveisok

I think we're good now.

noahfalk · 2025-09-05T10:45:45Z

src/tests/Directory.Build.props

+    <TestDotNetRootDir>$(DotNetInstallRoot)</TestDotNetRootDir>
+    <TestPackageArtifactsRootDir>$(NuGetPackageRoot)</TestPackageArtifactsRootDir>
+    <TestAuxMSBuildRootDir>$(AuxMSBuildRootDir)</TestAuxMSBuildRootDir>
+    <TestVersionConfigDir>$(ArtifactsBinDir)</TestVersionConfigDir>


What causes Debugger.Tests.Versions.txt to be located in the artifacts bin dir? I thought this is a file we create when we install the runtimes to test.

Since there will be two modes of running tests, it made sense to decouple generating this file from when we install the runtimes. When we build tests for helix, installing runtimes at that point is useless.

A couple thoughts:

Could you or Max make a little doc in our repo that describes how we expect the Helix testing to work? I'm hoping this wouldn't need to be more than a few paragraphs describing what assets are going to be put in what places and how are the different pieces going to find what they need. Right now its challenging to reconstruct the high level design intent from the fragments of scripts and paths spread around the repo.

When we set up a Helix run I assume we have some control over where the runtime gets copied to and where the Debugger.Tests.Versions.txt file gets copied to. Certainly we don't have to put those two things in the same place but is it problematic to do so? The fewer different directories and variables we need the simpler it is to keep track of.

ArtifactsBinDir initially seems like a surprising place to have the versions file but maybe its just because I don't understand the bigger picture of how the Helix testing is intended to work. I'm trusting once I see how the pieces fit together it will make more sense.

noahfalk · 2025-09-05T10:55:36Z

src/tests/DbgShim.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt

-  <DotNetRoot Condition="'$(TargetArchitecture)' == 'x86'">$(RepoRootDir)\.dotnet-test\x86</DotNetRoot>
-  <Import ConfigFile="$(DotNetRoot)\Debugger.Tests.Versions.txt" />
+  <DotNetRoot Condition="'$(TestDotNetRoot)' != ''">$(TestDotNetRoot)</DotNetRoot>
+  <Import ConfigFile="$(TestVersionConfigDir)\Debugger.Tests.Versions.txt" />


For the SOS tests we assume this file is in DotNetRoot directory. Why have different tests looking in different directories?

(I notice this pattern of parameterizing the DbgShim tests differently than the SOS tests occurs in other places too but I'm guessing there is no benefit to commenting each spot)

We took an earlier commit from #5469 as it started to get too much from one PR. This will be the way we want to have the debugger type tests configured moving forward. DbgShim was the first one and SOS just hadn't been done yet.

It would be easier to leave this change in my opinion as we're going to follow up with similar behavior in the other tests.

Got it, the differences are because its only partially complete. 👍

noahfalk · 2025-09-05T11:01:35Z

src/tests/DbgShim.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt

-  <DotNetRoot Condition="'$(TargetArchitecture)' != 'x86'">$(RepoRootDir)\.dotnet-test</DotNetRoot>
-  <DotNetRoot Condition="'$(TargetArchitecture)' == 'x86'">$(RepoRootDir)\.dotnet-test\x86</DotNetRoot>
-  <Import ConfigFile="$(DotNetRoot)\Debugger.Tests.Versions.txt" />
+  <DotNetRoot Condition="'$(TestDotNetRoot)' != ''">$(TestDotNetRoot)</DotNetRoot>


The SOS tests have hard-coded $(RepoRootDir)/.dotnet-test as DotNetRoot. Any reason DbgShim tests are doing all this parameterization and the SOS tests are essentially hard-coding it? It seems like we'd want all our tests looking for DotNetRoot in the same place.

Similar comment as the one above. The rationale for creating $(TestDotNetRoot) is local = '.dotnet-test' and helix = 'read from DOTNET_ROOT env variable'. All debugger tests will opt into this behavior.

noahfalk · 2025-09-05T23:25:56Z

src/tests/DbgShim.UnitTests/ConfigFiles/Unix/Debugger.Tests.Config.txt

@@ -1,14 +1,14 @@
 <Configuration>
  <Import ConfigFile="Debugger.Tests.Common.txt" />
-  <DotNetRoot>$(RepoRootDir)/.dotnet-test</DotNetRoot>
-  <Import ConfigFile="$(DotNetRoot)/Debugger.Tests.Versions.txt" />
+  <DotNetRoot Condition="'$(TestDotNetRoot)' != ''">$(TestDotNetRoot)</DotNetRoot>


I assume the design intent is that TestDotNetRoot won't be defined when running Helix? Its not yet clear to me why that will be the case, but I'm hoping things will make sense once I've got a better understanding how the Helix run is executed.

Yeah, that's the only thing that can't be relative pathed. When the sdk is sent along, the DOTNET_ROOT env variable is also set. We can just read from the env variable in that case.

Mike McLaughlin and others added 14 commits May 29, 2025 09:45

Move tests and test debuggees to repo root 'tests' directory

8186c59

Remove debuggee from DbgShim.UnitTest project

ad7cdcd

Everything under the tests directory is not packable

1f9a870

Fix packing in official build

93a5efa

Fix single-file build step

a57bf97

Merge remote-tracking branch 'upstream/main' into organize_tests

b8315c8

Merge remote-tracking branch 'upstream/main' into organize_tests

cd2e7d8

Split out build tools and tests from running them

aebac9c

Use -withtests instead because powershell doesn't seem to like prefix…

7c06f2d

…es with build in them. Flags as duplicate params

WIP - DbgShim.UnitTests can run outside of the repo with a few tweaks…

b118af2

… (NuGet.config, Version.props, and a debuggee Directory.Build.props+targets)

Merge remote-tracking branch 'upstream/main' into organize_tests

2a8905b

fix paths

6eecb79

Merge branch 'main' into organize_tests2

37097ef

modify build.proj

878a01a

jkoritzinsky reviewed Aug 28, 2025

View reviewed changes

eng/Build.props Outdated Show resolved Hide resolved

max-charlamb added 4 commits August 28, 2025 13:56

fix source-build

56fd8e8

support helix in build.ps1

2ed0c53

remove references to helix

1c87d1c

remove helix references

418570a

max-charlamb marked this pull request as ready for review August 28, 2025 21:46

max-charlamb requested a review from a team as a code owner August 28, 2025 21:46

max-charlamb changed the title ~~Prepare tests to run on helix~~ Move tests and test debuggees to repo root 'tests' directory Aug 28, 2025

This was referenced Aug 29, 2025

Support running test out of proc #5560

Draft

[cDAC] Run runtime-diagnostics pipeline on more platforms dotnet/runtime#119212

Open

steveisok previously approved these changes Aug 29, 2025

View reviewed changes

modify cli arguments

5dc3574

max-charlamb dismissed steveisok’s stale review via 5dc3574 August 29, 2025 18:48

max-charlamb commented Aug 29, 2025

View reviewed changes

eng/build.ps1 Show resolved Hide resolved

max-charlamb added 2 commits August 29, 2025 14:59

Update eng/build.ps1

12da84a

fix pipeline

4d0b29f

undo change

7a4f23e

steveisok self-requested a review August 29, 2025 22:28

steveisok previously approved these changes Aug 29, 2025

View reviewed changes

noahfalk requested changes Aug 29, 2025

View reviewed changes

Move tests into src/tests and adjust any pathing weirdness

73ff521

steveisok dismissed their stale review via 73ff521 September 4, 2025 18:53

steveisok added 3 commits September 4, 2025 15:03

Add binaries and other ignored files that were in tree back

a6e0aaf

Merge remote-tracking branch 'upstream/main' into organize_tests2

f615832

Correct lldbplugin.tests location

8b10fe4

steveisok self-requested a review September 4, 2025 23:42

steveisok approved these changes Sep 4, 2025

View reviewed changes

noahfalk reviewed Sep 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Move tests and test debuggees to repo root 'tests' directory #5554

Move tests and test debuggees to repo root 'tests' directory #5554

max-charlamb commented Aug 28, 2025 •

edited by steveisok

Loading

Uh oh!

Uh oh!

Uh oh!

noahfalk left a comment

Uh oh!

steveisok left a comment

Uh oh!

noahfalk Sep 5, 2025

Uh oh!

steveisok Sep 5, 2025

Uh oh!

noahfalk Sep 5, 2025 •

edited

Loading

Uh oh!

noahfalk Sep 5, 2025

Uh oh!

steveisok Sep 5, 2025

Uh oh!

noahfalk Sep 5, 2025

Uh oh!

noahfalk Sep 5, 2025

Uh oh!

steveisok Sep 5, 2025

Uh oh!

noahfalk Sep 5, 2025

Uh oh!

steveisok Sep 5, 2025

Uh oh!

Uh oh!

Move tests and test debuggees to repo root 'tests' directory #5554

Are you sure you want to change the base?

Move tests and test debuggees to repo root 'tests' directory #5554

Conversation

max-charlamb commented Aug 28, 2025 • edited by steveisok Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

noahfalk left a comment

Choose a reason for hiding this comment

Uh oh!

steveisok left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

noahfalk Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

max-charlamb commented Aug 28, 2025 •

edited by steveisok

Loading

noahfalk Sep 5, 2025 •

edited

Loading