Skip to content

Commit 3692a01

Browse files
Copilotkarpikpl
andauthored
Update README with latest screenshots, comprehensive feature documentation, and improved teams comparison with E2E test fixes (#252)
* Initial plan * Update README with latest screenshots and feature documentation Co-authored-by: karpikpl <[email protected]> * Update teams-comparison screenshot with team selection and add E2E test Co-authored-by: karpikpl <[email protected]> * Fix teams-comparison Playwright test selectors and logic Co-authored-by: karpikpl <[email protected]> * Fix Playwright test selectors and add UI validation for teams comparison Co-authored-by: karpikpl <[email protected]> * fix tests * lint * update pw * fixing test * no screenshot * 4 workers? * 2 workers --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: karpikpl <[email protected]> Co-authored-by: Piotr Karpala <[email protected]>
1 parent a42b8d2 commit 3692a01

20 files changed

+1501
-2047
lines changed

Dockerfile

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ USER node
5353
ENTRYPOINT [ "/entrypoint.sh" ]
5454

5555
#----------------------------------- PW layer - not used in production
56-
FROM mcr.microsoft.com/playwright:v1.49.1 AS base-playwright
56+
FROM mcr.microsoft.com/playwright:v1.54.2 AS base-playwright
5757

5858
WORKDIR /pw
5959

@@ -66,9 +66,9 @@ COPY --chown=1000:1000 --from=base-prod /app /app
6666
COPY --chown=1000:1000 e2e-tests ./e2e-tests
6767
COPY --chown=1000:1000 playwright.config.ts playwright.docker.config.ts tsconfig.json package*.json ./
6868

69-
RUN npm install --only=dev
69+
RUN NODE_ENV=development npm install
7070

71-
ENTRYPOINT [ "npx", "playwright", "test", "-c", "playwright.docker.config.ts" ]
71+
ENTRYPOINT [ "npx", "playwright", "test", "-c", "playwright.docker.config.ts", "--workers", "2"]
7272

7373
#-----------------------------------
7474
FROM base-${mode} AS final

README.md

Lines changed: 75 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,47 @@ _NOTE: For information on support and assistance, click [here](https://github.co
77

88
This application displays a set of charts with various metrics related to GitHub Copilot for your <i>GitHub Organization</i> or <i>Enterprise Account</i>. These visualizations are designed to provide clear representations of the data, making it easy to understand and analyze the impact and adoption of GitHub Copilot. This app utilizes the [GitHub Copilot Metrics API](https://docs.github.com/en/enterprise-cloud@latest/rest/copilot/copilot-usage?apiVersion=2022-11-28).
99

10-
## Video
10+
## Application Overview
1111

12-
https://github.com/github-copilot-resources/copilot-metrics-viewer/assets/3329307/bc7e2a16-cc73-43c4-887a-b50809c08533
12+
The GitHub Copilot Metrics Viewer provides comprehensive analytics through an intuitive dashboard interface:
13+
14+
<p align="center">
15+
<img width="800" alt="Main Dashboard Overview" src="./images/main-metrics-dashboard.png">
16+
</p>
17+
18+
## New Features
19+
20+
### Date Range Filtering (up to 100 days)
21+
Users can now filter metrics for custom date ranges up to 100 days, with an intuitive calendar picker interface. The system also supports excluding weekends and holidays from calculations.
22+
23+
<p align="center">
24+
<img width="800" alt="Date Range Filter" src="./images/date-range-filter.png">
25+
</p>
26+
27+
### Teams Comparison
28+
Compare Copilot metrics across multiple teams within your organization to understand adoption patterns and identify high-performing teams.
29+
30+
<p align="center">
31+
<img width="800" alt="Teams Comparison" src="./images/teams-comparison.png">
32+
</p>
33+
34+
### GitHub.com Integration & Model Analytics
35+
View comprehensive statistics for GitHub.com features including Chat, PR Summaries, and detailed model usage analytics. Each section provides expandable details showing model types, editors, and usage patterns.
36+
37+
<p align="center">
38+
<img width="800" alt="GitHub.com Tab" src="./images/gh.seave.top-tab.png">
39+
</p>
40+
41+
<p align="center">
42+
<img width="800" alt="Model Usage Details" src="./images/gh.seave.top-models-expanded.png">
43+
</p>
44+
45+
### CSV Export Functionality
46+
Export your metrics data in multiple formats for further analysis or reporting. Options include summary reports, full detailed exports, and direct clipboard copying.
47+
48+
<p align="center">
49+
<img width="800" alt="CSV Export Options" src="./images/csv-export-functionality.png">
50+
</p>
1351

1452
## Charts
1553

@@ -19,7 +57,7 @@ https://github.com/github-copilot-resources/copilot-metrics-viewer/assets/332930
1957
2058
Here are the key metrics visualized in these charts:
2159
<p align="center">
22-
<img width="800" alt="image" src="./images/KeyMetrics.png">
60+
<img width="800" alt="Key Metrics Overview" src="./images/main-metrics-dashboard.png">
2361
</p>
2462

2563
1. **Acceptance Rate:** This metric represents the ratio of accepted lines and suggestions to the total suggested by GitHub Copilot. This rate is an indicator of the relevance and usefulness of Copilot's suggestions. However, as with any metric, it should be used with caution as developers use Copilot in many different ways (research, confirm, verify, etc., not always "inject").
@@ -50,31 +88,28 @@ Here are the key metrics visualized in these charts:
5088

5189
Pie charts with the top 5 languages by accepted prompts and acceptance rate (by count/by lines) are displayed at the top.
5290
<p align="center">
53-
<img width="800" alt="image" src="./images/Language_breakdown.png">
91+
<img width="800" alt="Updated Language breakdown with charts and data table" src="./images/languages-breakdown.png">
5492
</p>
5593

56-
The language breakdown analysis tab also displays a table showing the Accepted Prompts, Accepted Lines of Code, and Acceptance Rate (%) for each language over the past 28 days. The entries are sorted by the number of _accepted lines of code descending_.
57-
<p align="center">
58-
<img width="800" alt="image" src="./images/Language_breakdown_list.png">
59-
</p>
94+
The language breakdown analysis tab also displays a table showing the Accepted Prompts, Accepted Lines of Code, and Acceptance Rate (%) for each language over the selected time period. The entries are sorted by the number of _accepted lines of code descending_.
6095

6196
## Copilot Chat Metrics
6297

6398
<p align="center">
64-
<img width="800" alt="image" src="https://github.com/github-copilot-resources/copilot-metrics-viewer/assets/3329307/79867d5f-8933-4509-a58a-8c6deeb47536">
99+
<img width="800" alt="Copilot Chat Metrics Dashboard" src="./images/copilot-chat-metrics.png">
65100
</p>
66101

67-
1. **Cumulative Number of Turns:** This metric represents the total number of turns (interactions) with the Copilot over the past 28 days. A 'turn' includes both user inputs and Copilot's responses.
102+
1. **Cumulative Number of Turns:** This metric represents the total number of turns (interactions) with the Copilot over the selected time period. A 'turn' includes both user inputs and Copilot's responses.
68103

69-
2. **Cumulative Number of Acceptances:** This metric shows the total number of lines of code suggested by Copilot that have been accepted by users over the past 28 days.
104+
2. **Cumulative Number of Acceptances:** This metric shows the total number of lines of code suggested by Copilot that have been accepted by users over the selected time period.
70105

71106
3. **Total Turns | Total Acceptances Count:** This is a chart that displays the total number of turns and acceptances.
72107

73-
4. **Total Active Copilot Chat Users:** A bar chart that illustrates the total number of users who have actively interacted with Copilot over the past 28 days.
108+
4. **Total Active Copilot Chat Users:** A bar chart that illustrates the total number of users who have actively interacted with Copilot over the selected time period.
74109

75110
## Seat Analysis
76111
<p align="center">
77-
<img width="800" alt="image" src="https://github.com/github-copilot-resources/copilot-metrics-viewer/assets/54096296/51747194-df30-4bfb-8849-54a0510fffcb">
112+
<img width="800" alt="Seat Analysis Dashboard" src="./images/seat-analysis.png">
78113
</p>
79114

80115
1. **Total Assigned:** This metric represents the total number of Copilot seats assigned within the current organization/enterprise.
@@ -85,6 +120,33 @@ The language breakdown analysis tab also displays a table showing the Accepted P
85120

86121
4. **No Activity in the Last 7 Days (including never used seats):** A table to display seats that have had no activity in the past 7 days, ordered by the date of last activity. Seats that were used earlier are displayed at the top.
87122

123+
## Advanced Features
124+
125+
### Flexible Date Range Selection
126+
The application supports flexible date range selection allowing users to analyze metrics for any period up to 100 days. The date picker provides an intuitive calendar interface with options to exclude weekends and holidays from the analysis.
127+
128+
### Data Export Capabilities
129+
Multiple export options are available in the API Response tab:
130+
- **Download CSV (Summary)**: Exports key metrics in a condensed format
131+
- **Download CSV (Full)**: Exports comprehensive detailed data
132+
- **Copy Metrics to Clipboard**: Quick copy functionality for immediate use
133+
- **Check Metric Data Quality**: Validates data integrity and completeness
134+
135+
### Team Analytics
136+
Organizations can compare metrics across different teams to:
137+
- Identify high-performing teams
138+
- Understand adoption patterns
139+
- Share best practices across teams
140+
- Monitor team-specific engagement levels
141+
142+
### Model Usage Analytics
143+
Detailed insights into AI model usage including:
144+
- IDE Code Completions by editor and model type
145+
- IDE Chat interactions and model preferences
146+
- GitHub.com Chat usage patterns
147+
- PR Summary generation statistics
148+
- Custom vs. default model adoption rates
149+
88150
## Setup Instructions
89151

90152
In the `.env` file, you can configure several environment variables that control the behavior of the application.

app/components/TeamsComponent.vue

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@
1414
<v-card-text>
1515
<v-row>
1616
<v-col cols="12" md="8">
17-
<v-select v-model="selectedTeams" :items="availableTeams" item-value="slug"
17+
<v-select
18+
v-model="selectedTeams" :items="availableTeams" item-value="slug"
1819
label="Select teams to compare" multiple chips clearable variant="outlined" :menu-props="{
1920
contentClass: 'teams-select-menu',
2021
maxHeight: 360,
@@ -31,7 +32,8 @@
3132
</v-select>
3233
</v-col>
3334
<v-col cols="12" md="4" class="d-flex align-center">
34-
<v-btn v-if="selectedTeams.length > 0" color="primary" variant="outlined" size="small"
35+
<v-btn
36+
v-if="selectedTeams.length > 0" color="primary" variant="outlined" size="small"
3537
@click="clearSelection">
3638
Clear All
3739
</v-btn>
@@ -51,7 +53,8 @@
5153
<v-card-title class="text-h6">Selected Teams</v-card-title>
5254
<v-card-text>
5355
<v-chip-group>
54-
<v-chip v-for="team in selectedTeamObjects" :key="team.slug" :href="getTeamDetailUrl(team.slug)"
56+
<v-chip
57+
v-for="team in selectedTeamObjects" :key="team.slug" :href="getTeamDetailUrl(team.slug)"
5558
class="selected-team-chip" target="_blank" link>
5659
{{ team.name }} - View Details
5760
<v-icon end>mdi-open-in-new</v-icon>
@@ -67,7 +70,8 @@
6770
<div v-if="selectedTeams.length > 0">
6871
<!-- Summary Cards -->
6972
<div class="tiles-container">
70-
<v-card elevation="4" color="white" variant="elevated" class="mx-auto my-3"
73+
<v-card
74+
elevation="4" color="white" variant="elevated" class="mx-auto my-3"
7175
style="width: 300px; height: 175px;">
7276
<v-card-item>
7377
<div class="tiles-text">
@@ -87,7 +91,8 @@
8791
</v-card-item>
8892
</v-card>
8993

90-
<v-card elevation="4" color="white" variant="elevated" class="mx-auto my-3"
94+
<v-card
95+
elevation="4" color="white" variant="elevated" class="mx-auto my-3"
9196
style="width: 300px; height: 175px;">
9297
<v-card-item>
9398
<div class="tiles-text">

e2e-tests/copilot.team.spec.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,5 +24,5 @@ test('has title', tag, async () => {
2424
});
2525

2626
test('team tab', tag, async () => {
27-
await dashboard.expectTeamsTabVisible();
27+
await dashboard.expectTeamTabVisible();
2828
});

e2e-tests/pages/DashboardPage.ts

Lines changed: 45 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
1-
import { expect, type Locator, type Page } from '@playwright/test';
2-
import { LanguagesTab } from './LanguagesTab';
3-
import { EditorsTab } from './EditorsTab';
4-
import { SeatAnalysisTab } from './SeatAnalysisTab';
5-
import { ApiResponseTab } from './ApiResponseTab';
6-
import { CopilotChatTab } from './CopilotChatTab';
7-
import { GitHubTab } from './GitHubTab';
1+
import { expect, type Locator, type Page } from "@playwright/test";
2+
import { LanguagesTab } from "./LanguagesTab";
3+
import { EditorsTab } from "./EditorsTab";
4+
import { SeatAnalysisTab } from "./SeatAnalysisTab";
5+
import { ApiResponseTab } from "./ApiResponseTab";
6+
import { CopilotChatTab } from "./CopilotChatTab";
7+
import { GitHubTab } from "./GitHubTab";
88

99
export class DashboardPage {
1010
readonly page: Page;
@@ -16,6 +16,7 @@ export class DashboardPage {
1616
readonly toolbarTitle: Locator;
1717

1818
readonly teamTabLink: Locator;
19+
readonly teamsTabLink: Locator;
1920
readonly orgTabLink: Locator;
2021
readonly enterpriseTabLink: Locator;
2122

@@ -29,25 +30,39 @@ export class DashboardPage {
2930
constructor(page: Page) {
3031
this.page = page;
3132

32-
this.acceptanceRateByCountLabel = page.getByText('Acceptance Rate (by count)')
33-
this.totalCountOfSuggestionsLabel = page.getByText('Total count of Suggestions (Prompts)')
34-
this.totalLinesSuggestedLabel = page.getByRole('heading', { name: 'Total Lines Suggested | Total' })
35-
this.totalLinesSuggestedValue = page.locator('.v-card-item').filter({ has: page.getByText('Total Lines of code Suggested') }).locator('.text-h4')
36-
this.toolbarTitle = page.locator(".toolbar-title")
37-
38-
this.languagesTabLink = page.getByRole('tab', { name: 'languages' })
39-
this.editorsTabLink = page.getByRole('tab', { name: 'editors' })
40-
this.seatAnalysisTabLink = page.getByRole('tab', { name: 'seat analysis' })
41-
this.apiResponseTabLink = page.getByRole('tab', { name: 'api response' })
42-
this.copilotChatTabLink = page.getByRole('tab', { name: 'copilot chat' })
43-
this.githubTabLink = page.getByRole('tab', { name: 'github.com' })
44-
45-
this.teamTabLink = page.getByRole('tab', { name: 'team' })
46-
this.orgTabLink = page.getByRole('tab', { name: 'organization' })
47-
this.enterpriseTabLink = page.getByRole('tab', { name: 'enterprise' })
33+
this.acceptanceRateByCountLabel = page.getByText(
34+
"Acceptance Rate (by count)"
35+
);
36+
this.totalCountOfSuggestionsLabel = page.getByText(
37+
"Total count of Suggestions (Prompts)"
38+
);
39+
this.totalLinesSuggestedLabel = page.getByRole("heading", {
40+
name: "Total Lines Suggested | Total",
41+
});
42+
this.totalLinesSuggestedValue = page
43+
.locator(".v-card-item")
44+
.filter({ has: page.getByText("Total Lines of code Suggested") })
45+
.locator(".text-h4");
46+
this.toolbarTitle = page.locator(".toolbar-title");
47+
48+
this.languagesTabLink = page.getByRole("tab", { name: "languages" });
49+
this.editorsTabLink = page.getByRole("tab", { name: "editors" });
50+
this.seatAnalysisTabLink = page.getByRole("tab", { name: "seat analysis" });
51+
this.apiResponseTabLink = page.getByRole("tab", { name: "api response" });
52+
this.copilotChatTabLink = page.getByRole("tab", { name: "copilot chat" });
53+
this.githubTabLink = page.getByRole("tab", { name: "github.com" });
54+
55+
this.teamTabLink = page.getByRole("tab", { name: "team" });
56+
this.teamsTabLink = page.getByRole("tab", { name: "teams" });
57+
this.orgTabLink = page.getByRole("tab", { name: "organization" });
58+
this.enterpriseTabLink = page.getByRole("tab", { name: "enterprise" });
4859
}
4960

5061
async expectTeamsTabVisible() {
62+
await expect(this.teamsTabLink).toBeVisible();
63+
}
64+
65+
async expectTeamTabVisible() {
5166
await expect(this.teamTabLink).toBeVisible();
5267
}
5368

@@ -106,7 +121,13 @@ export class DashboardPage {
106121
return new GitHubTab(this.page);
107122
}
108123

124+
async gotoTeamsTab() {
125+
await this.teamsTabLink.click();
126+
// No specific page object for teams tab, just return this for fluent interface
127+
return this;
128+
}
129+
109130
async close() {
110131
await this.page.close();
111132
}
112-
}
133+
}

0 commit comments

Comments
 (0)