Skip to content

[android] Add Test History visualizations for new Android scenarios #4832

@matouskozak

Description

@matouskozak

With the recent additions to Android scenario testing (#4808, #4770, ...), we need to have convenient access to the measured data.

Proposal

Add new section to Test History Index with Mobile Scenarios name. Note, we already have Mobile Tests section, but it is mostly non-functional at the moment and until we unify the flow for both iOS and Android scenarios it would be better to keep them separate to avoid confusion.

  1. To the Mobile Scenarios section add the following links:

    • dotnet/runtime Android Sample App
    • .NET Android Default Template
    • MAUI Android Default Template
    • MAUI Blazor Android Default Template
    • (... extend with more scenarios as we need ...)
  2. The individual subpages should contain the following links:

    • dotnet/runtime Android Sample App
      • dotnet/runtime Android Sample App - Mono JIT
      • dotnet/runtime Android sample app - Mono AOT
      • dotnet/runtime Android sample app - CoreCLR JIT
      • dotnet/runtime Android sample app - CoreCLR R2R
    • .NET Android Default Template
      • .NET Android Default Template - Mono ProfiledAOT
      • .NET Android Default Template - CoreCLR JIT
    • MAUI Android Default Template
      • MAUI Android Default Template - Mono ProfiledAOT
      • MAUI Android Default Template - CoreCLR JIT
    • MAUI Blazor Android Default Template
      • MAUI Blazor Android Default Template - Mono ProfiledAOT
      • MAUI Blazor Android Default Template - CoreCLR JIT
  3. The individual scenario - configuration links should lead to a page with measurement results (links)

    • Startup
    • Startup NoAnimation (when available)
    • SOD - APK size
    • SOD - Extracted size
    • Memory Consumption (when available)
  4. Each measurement result page should has the following UX

    • View of the full history trend
    Image
    • Upon clicking on the history trend, it should be possible to select a time range, representing a commit history of the underlying repositories (currently, only dotnet/runtime and dotnet/performance are shown).
    Image Image
    • After selecting the end point of the range, generated links to GitHub compare (e.g., compare) should be provided for ALL underlying repositories (currently, only link to dotnet/runtime is provided). The underlying repositories for the Android scenarios are:
      • dotnet/runtime Android Sample App: dotnet/runtime and dotnet/performance
      • .NET Android Default Template: dotnet/runtime, dotnet/android, and dotnet/performance
      • MAUI Android Default Template: dotnet/runtime, dotnet/android, dotnet/maui, and dotnet/performance
      • MAUI Blazor Android Default Template: dotnet/runtime, dotnet/android, dotnet/maui, and dotnet/performance

    The commit and version history data is stored in BuildAdditionalData column

    Image

Things to consider

  • Instead of step (3) where we list links to individual measurements results, to avoid extensive nesting, we could directly redirect to a "Full History Trend" page (4) where instead of having a single measurement, we could list all relevant measurements (startup, SOD, etc.). There seems to be a capability of doing this as we already show multiple measurements for auto-filling issues:
Image
  • Currently, Measurements kusto table doesn't provide reliably datetime information for .NET Android, MAUI, and MAUI-Blazor measurements. To provide the information, cross-cluster join with TimelineBuilds table is required:
| join kind=inner (
    cluster(<eng_cluster>).database(<eng_data>).TimelineBuilds
    | where Definition contains "dotnet-performance"
    | where SourceBranch == "main"
    | project BuildNumber, QueueTime
) on $left.BuildName == $right.BuildNumber

This is not ideal, as it requires access to the <eng_cluster> + VPN.
After #4838, the BuildAdditionalInformation['SDK_INSTALL_DATETIME'] contains the datetime information.

Additional work

  • remove the old Android links from Mobile Tests section

Future work

  • when iOS measurements are migrated to the new flow, update their respective links as well

Metadata

Metadata

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions