Skip to content

Commit 705280d

Browse files
PankajChaudhari-AtosAndreiS-gh
PankajChaudhari-Atos
and
AndreiS-gh
authored
feat(gh-copilot): add AI credit usage billing metrics and fix report-body double-read (#9019)
* feat(gh-copilot): add AI credit usage billing metrics Adds collection and extraction of GitHub Copilot AI credit usage billing metrics at org, user, and enterprise levels, including new models, tasks (ai_credit_collector/extractor), a migration for the new billing tables, and e2e snapshot fixtures. Ports the work from #8980. Co-authored-by: Andrei Savu <54935810+AndreiS-gh@users.noreply.github.com> Signed-off-by: Pankaj Chaudhari <pankaj.chaudhari@atos.net> * fix(gh-copilot): parse report metadata from already-read body The user and enterprise Copilot metrics collectors read the report metadata HTTP response body with io.ReadAll, then called parseReportMetadataResponse(res, ...) which read res.Body a second time. Since the body was already consumed, the second read returned empty, parseReportMetadata logged "Report metadata response was empty, skipping" and returned nil, so the collectors produced zero records. Parse the metadata from the body already read instead. The organization collector was unaffected because it reads the body only once inline. Fixes empty _raw_copilot_user_metrics and _raw_copilot_enterprise_metrics (and downstream _tool_copilot_user_daily_metrics) while org metrics work. Signed-off-by: Pankaj Chaudhari <pankaj.chaudhari@atos.net> * fix(gh-copilot): resolve migration-linter and golangci-lint failures - migrationscripts: inline snapshot structs instead of importing models pkg - user_metrics: fix gofmt struct-tag alignment - ai_credit_collector/extractor: fix ASF license header (capital The) - report_download_helper: remove unused readReportMetadataBody/parseReportMetadataResponse - metrics_collector_test: retarget tests to parseReportMetadata/ignoreNoContent --------- Signed-off-by: Pankaj Chaudhari <pankaj.chaudhari@atos.net> Co-authored-by: Andrei Savu <54935810+AndreiS-gh@users.noreply.github.com>
1 parent 0194a41 commit 705280d

19 files changed

Lines changed: 709 additions & 76 deletions

‎backend/plugins/gh-copilot/e2e/metrics/snapshot_tables/_tool_copilot_enterprise_ai_credit_usage.csv‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
connection_id,scope_id,year,month,day,enterprise,model,organization,user,product,cost_center_id,cost_center_name,gross_quantity,discount_quantity,net_quantity,price_per_unit,gross_amount,discount_amount,net_amount
2+
1,octodemo,2025,12,10,octodemo,gpt-4.1,,,copilot,,,100.5,10.0,90.5,1.0,100.5,10.0,90.5
3+
1,octodemo,2025,12,10,octodemo,gpt-4o,,,copilot,cc-eng,Engineering,50.0,5.0,45.0,1.0,50.0,5.0,45.0
4+
1,octodemo,2025,12,10,octodemo,claude-3,,,copilot,cc-ml,Machine Learning,30.0,2.0,28.0,1.0,30.0,2.0,28.0

‎backend/plugins/gh-copilot/e2e/metrics/snapshot_tables/_tool_copilot_org_ai_credit_usage.csv‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
connection_id,scope_id,year,month,day,organization,model,user,product,gross_quantity,discount_quantity,net_quantity,price_per_unit,gross_amount,discount_amount,net_amount
2+
1,octodemo,2025,12,10,octodemo,gpt-4.1,alice,copilot,50.0,5.0,45.0,1.0,50.0,5.0,45.0
3+
1,octodemo,2025,12,10,octodemo,gpt-4o,bob,copilot,30.0,3.0,27.0,1.0,30.0,3.0,27.0
4+
1,octodemo,2025,12,10,octodemo,claude-3,charlie,copilot,20.0,0.0,20.0,1.0,20.0,0.0,20.0

‎backend/plugins/gh-copilot/e2e/metrics/snapshot_tables/_tool_copilot_user_ai_credit_usage.csv‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
connection_id,scope_id,year,month,day,user,model,product,gross_quantity,discount_quantity,net_quantity,price_per_unit,gross_amount,discount_amount,net_amount
2+
1,octodemo,2025,12,10,alice,gpt-4.1,copilot,50.0,5.0,45.0,1.0,50.0,5.0,45.0
3+
1,octodemo,2025,12,10,alice,gpt-4o,copilot,30.0,3.0,27.0,1.0,30.0,3.0,27.0
4+
1,octodemo,2025,12,10,bob,claude-3,copilot,20.0,2.0,18.0,1.0,20.0,2.0,18.0

‎backend/plugins/gh-copilot/models/enterprise_ai_credit_usage.go‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
/*
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
*/
17+
18+
package models
19+
20+
import (
21+
"github.com/apache/incubator-devlake/core/models/common"
22+
)
23+
24+
// GhCopilotEnterpriseAiCreditUsage tracks AI credit consumption at the enterprise level.
25+
// One row per time period per model per entity (user, org, or cost center).
26+
type GhCopilotEnterpriseAiCreditUsage struct {
27+
ConnectionId uint64 `gorm:"primaryKey" json:"connectionId"`
28+
ScopeId string `gorm:"primaryKey;type:varchar(191)" json:"scopeId"`
29+
Year int `gorm:"primaryKey" json:"year"`
30+
Month int `gorm:"primaryKey" json:"month"`
31+
Day int `gorm:"primaryKey" json:"day"`
32+
33+
Enterprise string `gorm:"primaryKey;type:varchar(191);comment:Enterprise slug" json:"enterprise"`
34+
Model string `gorm:"primaryKey;type:varchar(191);comment:AI model name (e.g., gpt-4.1)" json:"model"`
35+
Organization string `gorm:"index;type:varchar(255);comment:Organization within enterprise, if specified" json:"organization"`
36+
User string `gorm:"index;type:varchar(255);comment:Username, if specified" json:"user"`
37+
38+
Product string `gorm:"type:varchar(32);comment:Product name (e.g., copilot)" json:"product"`
39+
CostCenterId string `gorm:"index;type:varchar(255);comment:Cost center identifier" json:"costCenterId"`
40+
CostCenterName string `gorm:"type:varchar(255);comment:Cost center display name" json:"costCenterName"`
41+
42+
// Credit usage breakdown
43+
GrossQuantity float64 `json:"grossQuantity" gorm:"comment:Raw credits consumed"`
44+
DiscountQuantity float64 `json:"discountQuantity" gorm:"comment:Credits discounted"`
45+
NetQuantity float64 `json:"netQuantity" gorm:"comment:Credits after discount"`
46+
PricePerUnit float64 `json:"pricePerUnit" gorm:"comment:Price per credit unit"`
47+
GrossAmount float64 `json:"grossAmount" gorm:"comment:Gross cost before discount"`
48+
DiscountAmount float64 `json:"discountAmount" gorm:"comment:Discount amount"`
49+
NetAmount float64 `json:"netAmount" gorm:"comment:Net cost after discount"`
50+
51+
common.NoPKModel
52+
}
53+
54+
func (GhCopilotEnterpriseAiCreditUsage) TableName() string {
55+
return "_tool_copilot_enterprise_ai_credit_usage"
56+
}

‎backend/plugins/gh-copilot/models/migrationscripts/20260708_add_ai_credit_usage_metrics.go‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
/*
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
*/
17+
18+
package migrationscripts
19+
20+
import (
21+
"time"
22+
23+
"github.com/apache/incubator-devlake/core/context"
24+
"github.com/apache/incubator-devlake/core/errors"
25+
"github.com/apache/incubator-devlake/core/models/migrationscripts/archived"
26+
"github.com/apache/incubator-devlake/helpers/migrationhelper"
27+
)
28+
29+
type addAiCreditUsageMetrics struct{}
30+
31+
// --- Snapshot structs for migration (avoid importing models package to prevent drift) ---
32+
33+
type creditUsageBreakdown20260708 struct {
34+
GrossQuantity float64
35+
DiscountQuantity float64
36+
NetQuantity float64
37+
PricePerUnit float64
38+
GrossAmount float64
39+
DiscountAmount float64
40+
NetAmount float64
41+
}
42+
43+
type enterpriseAiCreditUsage20260708 struct {
44+
ConnectionId uint64 `gorm:"primaryKey"`
45+
ScopeId string `gorm:"primaryKey;type:varchar(191)"`
46+
Year int `gorm:"primaryKey"`
47+
Month int `gorm:"primaryKey"`
48+
Day int `gorm:"primaryKey"`
49+
50+
Enterprise string `gorm:"primaryKey;type:varchar(191)"`
51+
Model string `gorm:"primaryKey;type:varchar(191)"`
52+
Organization string `gorm:"index;type:varchar(255)"`
53+
User string `gorm:"index;type:varchar(255)"`
54+
55+
Product string `gorm:"type:varchar(32)"`
56+
CostCenterId string `gorm:"index;type:varchar(255)"`
57+
CostCenterName string `gorm:"type:varchar(255)"`
58+
59+
creditUsageBreakdown20260708 `gorm:"embedded"`
60+
archived.NoPKModel
61+
}
62+
63+
func (enterpriseAiCreditUsage20260708) TableName() string {
64+
return "_tool_copilot_enterprise_ai_credit_usage"
65+
}
66+
67+
type orgAiCreditUsage20260708 struct {
68+
ConnectionId uint64 `gorm:"primaryKey"`
69+
ScopeId string `gorm:"primaryKey;type:varchar(191)"`
70+
Year int `gorm:"primaryKey"`
71+
Month int `gorm:"primaryKey"`
72+
Day int `gorm:"primaryKey"`
73+
74+
Organization string `gorm:"primaryKey;type:varchar(191)"`
75+
Model string `gorm:"primaryKey;type:varchar(191)"`
76+
User string `gorm:"index;type:varchar(255)"`
77+
78+
Product string `gorm:"type:varchar(32)"`
79+
80+
creditUsageBreakdown20260708 `gorm:"embedded"`
81+
archived.NoPKModel
82+
}
83+
84+
func (orgAiCreditUsage20260708) TableName() string {
85+
return "_tool_copilot_org_ai_credit_usage"
86+
}
87+
88+
type userAiCreditUsage20260708 struct {
89+
ConnectionId uint64 `gorm:"primaryKey"`
90+
ScopeId string `gorm:"primaryKey;type:varchar(191)"`
91+
Year int `gorm:"primaryKey"`
92+
Month int `gorm:"primaryKey"`
93+
Day int `gorm:"primaryKey"`
94+
95+
User string `gorm:"primaryKey;type:varchar(191)"`
96+
Model string `gorm:"primaryKey;type:varchar(191)"`
97+
98+
Product string `gorm:"type:varchar(32)"`
99+
100+
creditUsageBreakdown20260708 `gorm:"embedded"`
101+
archived.NoPKModel
102+
}
103+
104+
func (userAiCreditUsage20260708) TableName() string {
105+
return "_tool_copilot_user_ai_credit_usage"
106+
}
107+
108+
// userDailyMetrics20260708 adds AI credit, CLI and code-review columns to the existing table.
109+
type userDailyMetrics20260708 struct {
110+
ConnectionId uint64 `gorm:"primaryKey"`
111+
ScopeId string `gorm:"primaryKey;type:varchar(255)"`
112+
Day time.Time `gorm:"primaryKey;type:date"`
113+
UserId int64 `gorm:"primaryKey"`
114+
115+
OrganizationId string `gorm:"type:varchar(100)"`
116+
EnterpriseId string `gorm:"type:varchar(100)"`
117+
UserLogin string `gorm:"type:varchar(255);index"`
118+
UsedAgent bool
119+
UsedChat bool
120+
UsedCli bool `gorm:"comment:Whether user used Copilot CLI"`
121+
UsedCopilotCodeReviewActive bool `gorm:"comment:Whether user actively used code review"`
122+
UsedCopilotCodeReviewPassive bool `gorm:"comment:Whether user passively used code review"`
123+
AiCreditsUsed float64 `gorm:"comment:AI credits consumed on this day"`
124+
125+
UserInitiatedInteractionCount int
126+
CodeGenerationActivityCount int
127+
CodeAcceptanceActivityCount int
128+
LocSuggestedToAddSum int
129+
LocSuggestedToDeleteSum int
130+
LocAddedSum int
131+
LocDeletedSum int
132+
133+
CliSessionCount int
134+
CliRequestCount int
135+
CliPromptCount int
136+
CliOutputTokenSum int
137+
CliPromptTokenSum int
138+
139+
archived.NoPKModel
140+
}
141+
142+
func (userDailyMetrics20260708) TableName() string {
143+
return "_tool_copilot_user_daily_metrics"
144+
}
145+
146+
func (u *addAiCreditUsageMetrics) Up(basicRes context.BasicRes) errors.Error {
147+
return migrationhelper.AutoMigrateTables(
148+
basicRes,
149+
&enterpriseAiCreditUsage20260708{},
150+
&orgAiCreditUsage20260708{},
151+
&userAiCreditUsage20260708{},
152+
&userDailyMetrics20260708{},
153+
)
154+
}
155+
156+
func (u *addAiCreditUsageMetrics) Version() uint64 {
157+
return 20260708000000
158+
}
159+
160+
func (u *addAiCreditUsageMetrics) Name() string {
161+
return "add AI credit usage billing tables"
162+
}

‎backend/plugins/gh-copilot/models/migrationscripts/register.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,6 @@ func All() []plugin.MigrationScript {
3131
new(addPRFieldsToEnterpriseMetrics),
3232
new(addOrganizationIdToUserMetrics),
3333
new(addCopilotMetricsGaps),
34+
new(addAiCreditUsageMetrics),
3435
}
3536
}

‎backend/plugins/gh-copilot/models/models.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,5 +47,9 @@ func GetTablesInfo() []dal.Tabler {
4747
&GhCopilotSeat{},
4848
// User-team mappings
4949
&GhCopilotUserTeam{},
50+
// AI credit usage billing (enterprise, org, user levels)
51+
&GhCopilotEnterpriseAiCreditUsage{},
52+
&GhCopilotOrgAiCreditUsage{},
53+
&GhCopilotUserAiCreditUsage{},
5054
}
5155
}

‎backend/plugins/gh-copilot/models/models_test.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,9 @@ func TestGetTablesInfo(t *testing.T) {
4141
(&GhCopilotUserMetricsByModelFeature{}).TableName(): false,
4242
(&GhCopilotSeat{}).TableName(): false,
4343
(&GhCopilotUserTeam{}).TableName(): false,
44+
(&GhCopilotEnterpriseAiCreditUsage{}).TableName(): false,
45+
(&GhCopilotOrgAiCreditUsage{}).TableName(): false,
46+
(&GhCopilotUserAiCreditUsage{}).TableName(): false,
4447
}
4548

4649
if len(tables) != len(expected) {

‎backend/plugins/gh-copilot/models/org_ai_credit_usage.go‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
/*
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
*/
17+
18+
package models
19+
20+
import (
21+
"github.com/apache/incubator-devlake/core/models/common"
22+
)
23+
24+
// GhCopilotOrgAiCreditUsage tracks AI credit consumption at the organization level.
25+
// One row per time period per model per user (within the organization).
26+
type GhCopilotOrgAiCreditUsage struct {
27+
ConnectionId uint64 `gorm:"primaryKey" json:"connectionId"`
28+
ScopeId string `gorm:"primaryKey;type:varchar(191)" json:"scopeId"`
29+
Year int `gorm:"primaryKey" json:"year"`
30+
Month int `gorm:"primaryKey" json:"month"`
31+
Day int `gorm:"primaryKey" json:"day"`
32+
33+
Organization string `gorm:"primaryKey;type:varchar(191);comment:Organization name" json:"organization"`
34+
Model string `gorm:"primaryKey;type:varchar(191);comment:AI model name (e.g., gpt-4.1)" json:"model"`
35+
User string `gorm:"index;type:varchar(255);comment:Username consuming the credits" json:"user"`
36+
37+
Product string `gorm:"type:varchar(32);comment:Product name (e.g., copilot)" json:"product"`
38+
39+
// Credit usage breakdown
40+
GrossQuantity float64 `json:"grossQuantity" gorm:"comment:Raw credits consumed"`
41+
DiscountQuantity float64 `json:"discountQuantity" gorm:"comment:Credits discounted"`
42+
NetQuantity float64 `json:"netQuantity" gorm:"comment:Credits after discount"`
43+
PricePerUnit float64 `json:"pricePerUnit" gorm:"comment:Price per credit unit"`
44+
GrossAmount float64 `json:"grossAmount" gorm:"comment:Gross cost before discount"`
45+
DiscountAmount float64 `json:"discountAmount" gorm:"comment:Discount amount"`
46+
NetAmount float64 `json:"netAmount" gorm:"comment:Net cost after discount"`
47+
48+
common.NoPKModel
49+
}
50+
51+
func (GhCopilotOrgAiCreditUsage) TableName() string {
52+
return "_tool_copilot_org_ai_credit_usage"
53+
}

‎backend/plugins/gh-copilot/models/user_ai_credit_usage.go‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
/*
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
*/
17+
18+
package models
19+
20+
import (
21+
"github.com/apache/incubator-devlake/core/models/common"
22+
)
23+
24+
// GhCopilotUserAiCreditUsage tracks AI credit consumption at the individual user level.
25+
// One row per time period per model per authenticated user.
26+
type GhCopilotUserAiCreditUsage struct {
27+
ConnectionId uint64 `gorm:"primaryKey" json:"connectionId"`
28+
ScopeId string `gorm:"primaryKey;type:varchar(191)" json:"scopeId"`
29+
Year int `gorm:"primaryKey" json:"year"`
30+
Month int `gorm:"primaryKey" json:"month"`
31+
Day int `gorm:"primaryKey" json:"day"`
32+
33+
User string `gorm:"primaryKey;type:varchar(191);comment:GitHub username" json:"user"`
34+
Model string `gorm:"primaryKey;type:varchar(191);comment:AI model name (e.g., gpt-4.1)" json:"model"`
35+
36+
Product string `gorm:"type:varchar(32);comment:Product name (e.g., copilot)" json:"product"`
37+
38+
// Credit usage breakdown
39+
GrossQuantity float64 `json:"grossQuantity" gorm:"comment:Raw credits consumed"`
40+
DiscountQuantity float64 `json:"discountQuantity" gorm:"comment:Credits discounted"`
41+
NetQuantity float64 `json:"netQuantity" gorm:"comment:Credits after discount"`
42+
PricePerUnit float64 `json:"pricePerUnit" gorm:"comment:Price per credit unit"`
43+
GrossAmount float64 `json:"grossAmount" gorm:"comment:Gross cost before discount"`
44+
DiscountAmount float64 `json:"discountAmount" gorm:"comment:Discount amount"`
45+
NetAmount float64 `json:"netAmount" gorm:"comment:Net cost after discount"`
46+
47+
common.NoPKModel
48+
}
49+
50+
func (GhCopilotUserAiCreditUsage) TableName() string {
51+
return "_tool_copilot_user_ai_credit_usage"
52+
}

0 commit comments

Comments
 (0)