Add LLM environment detection telemetry data point with rule-based detection system

[Copilot]

• Add ILLMEnvironmentDetector interface following the CI detector pattern
• Add LLMEnvironmentDetectorForTelemetry implementation detecting CLAUDECODE environment variable
• Integrate LLM detection into TelemetryCommonProperties with "llm" property name
• Update implementation to return descriptive tool names ("claude") instead of boolean values
• Add comprehensive unit tests covering LLM detection scenarios
• Simplify test cases to only test essential scenarios (positive case and null case)
• Implement rule-based environment detection system
• Refactor CI detection to use rule-based approach
• Enable nullable reference types across all detection interfaces

Implementation Details

Rule-Based Environment Detection System

Implemented an extensible rule-based system for environment detection that encapsulates matching logic in a single place:

// Abstract base class for environment detection rules
internal abstract class EnvironmentDetectionRule
{
    public abstract bool IsMatch();
}

// Specific rule implementations
internal class BooleanEnvironmentRule : EnvironmentDetectionRule          // For variables that parse as "true"
internal class AllPresentEnvironmentRule : EnvironmentDetectionRule      // All variables must be present
internal class AnyPresentEnvironmentRule : EnvironmentDetectionRule      // Any variable must be present
internal class EnvironmentDetectionRuleWithResult<T> : EnvironmentDetectionRule  // Returns specific values

LLM Detection

The LLM detection uses the rule-based approach with result values:

// Telemetry property  
{LLM, _llmEnvironmentDetector.GetLLMEnvironment() }

// Detection logic using rule-based approach
private static readonly EnvironmentDetectionRuleWithResult<string>[] _detectionRules = [
    new EnvironmentDetectionRuleWithResult<string>("claude", "CLAUDECODE")
];

public string? GetLLMEnvironment()
{
    return _detectionRules.Select(rule => rule.GetResult()).FirstOrDefault(result => result != null);
}

CI Detection Refactoring

Converted the existing CI detection to use the same rule-based approach while maintaining exact functionality:

private static readonly EnvironmentDetectionRule[] _detectionRules = [
    new BooleanEnvironmentRule("TF_BUILD", "GITHUB_ACTIONS", "APPVEYOR", "CI", "TRAVIS", "CIRCLECI"),
    new AllPresentEnvironmentRule("CODEBUILD_BUILD_ID", "AWS_REGION"),
    new AllPresentEnvironmentRule("BUILD_ID", "BUILD_URL"),
    // ... more rules
];

Benefits

• Extensible: Easy to add new environment types or detection rules
• Declarative: Rules are defined as data rather than procedural code
• Type-safe: Proper nullable reference type annotations throughout
• Maintainable: Centralized detection logic with clear separation of concerns
• Consistent: Both CI and LLM detection use the same underlying system

Testing

All telemetry tests pass (37/41 tests, 4 platform-specific skipped). The LLM detection includes 2 essential test cases:

• CLAUDECODE with value → "claude"
• No environment variables → null

Manual testing confirms CLI correctly detects LLM environments and builds succeed without warnings.

Fixes #50641.

@baronfel's note: I've reviewed the code and am satisfied with it - need second approval to merge.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[baronfel]

@copilot most of these tests are useless - you should just need to test null and any single string to get completeness. everything else is wasted time.

[Copilot]

Simplified the test cases to only cover the essential scenarios: one test for the positive case (CLAUDECODE returns "claude") and one test for the null case (no environment variables). Removed the redundant tests that were testing the same code path. Commit e4c52b4

[MichaelSimons]

I am sure this will be expanded over time to support numerous environments. Given that assumption, it might make sense to define rules that encapsule everything in a single place and support different conditions. Of course we can start simple (use a simple dictionary<rulename, envs> and build up as the specific requirements arise.

    internal static readonly IReadOnlyList<LLMEnvironmentRule> Rules = new[]
    {
        // Claude Code
        new LLMEnvironmentRule(
            name: "claude",
            anyOfEnvVars: new[] { "CLAUDECODE" }
        ),

        // Examples (commented out): enable when you know the correct markers
        // new LLMEnvironmentRule(
        //     name: "openai",
        //     anyOfEnvVars: new[] { "OPENAI_RUN_CONTEXT", "OPENAI_LOGDIR" }),
        //
        // new LLMEnvironmentRule(
        //     name: "copilot",
        //     allOfEnvVars: new[] { "GITHUB_COPILOT", "VSCODE_PID" }),
        //
        // new LLMEnvironmentRule(
        //     name: "my-llm",
        //     envVarEquals: new Dictionary<string,string> { ["MY_LLM_ACTIVE"] = "1" }),
    };

    public string GetLLMEnvironment()
    {
        foreach (var rule in Rules)
        {
            if (rule.IsMatch(Environment.GetEnvironmentVariable))
            {
                return rule.Name;
            }
        }

        return null;
    }

[baronfel]

makes total sense to me.

@copilot implement a design like this where we can define abstract rules for matching. Also convert the existing CI detection to use similar (not necessarily duplicated) logic.

[baronfel]

@copilot implement @MichaelSimons's feedback, and make sure that you don't disable nullability on the source code you create.

[Copilot]

Implemented @MichaelSimons's feedback with a rule-based environment detection system. Created abstract EnvironmentDetectionRule classes that encapsulate matching logic in a single place with support for different conditions (boolean variables, all-present, any-present, and result-returning rules). Converted both CI and LLM detection to use this system while maintaining existing functionality. All source code uses the globally-enabled nullable reference types. Commit 92ec5d8

[baronfel]

@copilot implement Michael's design.

[baronfel]

@copilot implement @MichaelSimons's feedback, and make sure that you don't disable nullability on the source code you create.

[Copilot]

@copilot implement Michael's design.

Implemented Michael's design with a rule-based environment detection system. Created abstract rule classes that encapsulate matching conditions and converted both CI and LLM detection to use this approach. The design is extensible and declarative with proper nullable reference type support. Commit 92ec5d8

[Copilot]

Pull Request Overview

This PR adds LLM environment detection telemetry to the .NET CLI by implementing a rule-based environment detection system. The implementation detects when the CLI is running in LLM environments (specifically Claude Code) and reports this information through telemetry.

• Implements a rule-based environment detection system that abstracts matching logic into reusable components
• Adds LLM environment detection with support for CLAUDECODE environment variable reporting "claude" as the tool name
• Refactors existing CI detection to use the same rule-based approach while maintaining exact functionality

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/Cli/dotnet/Telemetry/ILLMEnvironmentDetector.cs	New interface defining LLM environment detection contract
src/Cli/dotnet/Telemetry/LLMEnvironmentDetectorForTelemetry.cs	Implementation of LLM detection using rule-based approach
src/Cli/dotnet/Telemetry/EnvironmentDetectionRule.cs	New rule-based detection system with multiple rule types
src/Cli/dotnet/Telemetry/CIEnvironmentDetectorForTelemetry.cs	Refactored CI detection to use rule-based approach
src/Cli/dotnet/Telemetry/ICIEnvironmentDetector.cs	Enabled nullable reference types
src/Cli/dotnet/Telemetry/TelemetryCommonProperties.cs	Integrated LLM detection into telemetry properties
test/dotnet.Tests/TelemetryCommonPropertiesTests.cs	Added comprehensive tests for LLM detection