feat: [IANDT-237] add active contributor emitter by jonnyowenpowell · Pull Request #697 · snyk/go-application-framework

jonnyowenpowell · 2026-08-12T13:03:44Z

Description

This PR adds an API client for the public submit active contributors endpoint served by the entitlements-service (internal/apiclients/contributors_ingest), a helper function to collect contributor data (contributors.go) from the Git repository the command is being run in, and an emitter object (emitter.go) which ties the two together, exposing an Emit function to perform the work.

Checklist

Tests added and all succeed (make test)
Regenerated mocks, etc. (make generate)
Linted (make lint)
Test your changes work for the CLI (currently dead code, will be done for later PRs in the stack)
1. Clone / pull the latest CLI main.
2. Run go get github.com/snyk/go-application-framework@YOUR_LATEST_GAF_COMMIT in the cliv2 directory.
  - Tip: for local testing, you can uncomment the line near the bottom of the CLI's go.mod to point to your local GAF code.
3. Run go mod tidy in the cliv2 directory.
4. Run the CLI tests and do any required manual testing.
5. Open a PR in the CLI repo now with the go.mod and go.sum changes.
- Once this PR is merged, repeat these steps, but pointing to the latest GAF commit on main and update your CLI PR.

Performance Investigation for counting contributors

Investigation into what % of CLI command runtime counting contributors would add.

Contributor timing = BenchmarkCollectContributors (-benchtime 5x, avg ns/op) on local clones on my Macbook Pro (M5 Pro).
Command timing = Datadog logs, @analytics-service.runtime.performance.durationMs, 15-day window, service:analytics-service filtered to @analytics-service.interaction.target.id:pkg\:git/github.com/snyk/* and the category set per command.

Count Contributors benchmark

repo	contrib ms
registry	215.56
deepcode	111.30
maverick	42.67
app-ui	26.74
cli	23.91
datadog-infra	21.42
topcoat-dbt	21.86
workspace-service	20.71
api-gateway	18.52
assets-api	18.01
target-service	17.46
entitlements-service	17.15
maverick-ui	16.55
tenants-service	14.44
data-schema	14.10
go-application-framework	13.31
data-layer	11.50
identity-service	11.36
sca-engine	10.10
code-gateway	8.37
managed-idp-resources	7.86
vervet	6.92
janus	6.82
admin-access	6.73
cerberus	6.23
rest-node-libs	6.21
go-common	5.68
projects-usage-producer	5.16
go-code-integrations-common	3.92
snyk-images	3.21

OSS Monitor (30d, snyk/* targets)

repo	n	avg ms	p50 ms	p90 ms	p99 ms	contrib ms	%avg	%p50	%p90
snyk-images	2276	40096	39748	62964	95830	3.21	0.008%	0.008%	0.005%
deepcode	497	9536	9801	12213	14917	111.30	1.167%	1.136%	0.911%
registry	122	14895	13498	20958	25092	215.56	1.447%	1.597%	1.029%
maverick	83	8663	8352	9801	14622	42.67	0.493%	0.511%	0.435%
maverick-ui	73	20707	19738	22704	31898	16.55	0.08%	0.084%	0.073%
datadog-infra	66	9772	10618	12712	15526	21.42	0.219%	0.202%	0.169%
assets-api	66	11196	10407	12968	23162	18.01	0.161%	0.173%	0.139%
data-layer	56	9333	9048	10407	11502	11.50	0.123%	0.127%	0.11%
cli	44	44695	43928	49529	65534	23.91	0.053%	0.054%	0.048%
app-ui	41	9402	9048	11734	15526	26.74	0.284%	0.296%	0.228%
entitlements-service	40	7043	6838	8693	9417	17.15	0.244%	0.251%	0.197%
workspace-service	25	24309	18220	36692	59297	20.71	0.085%	0.114%	0.056%
projects-usage-producer	18	9574	9048	10407	10407	5.16	0.054%	0.057%	0.05%
topcoat-dbt	17	5067	4867	6312	6703	21.86	0.432%	0.449%	0.346%
go-application-framework	15	10266	11502	12712	12968	13.31	0.13%	0.116%	0.105%
vervet	14	38917	38189	43058	43928	6.92	0.018%	0.018%	0.016%
admin-access	12	7712	7261	8693	8868	6.73	0.087%	0.093%	0.077%
janus	11	13079	12712	14917	14917	6.82	0.052%	0.054%	0.046%
tenants-service	9	33850	35253	35965	35965	14.44	0.043%	0.041%	0.04%
data-schema	9	22838	22704	24595	24595	14.10	0.062%	0.062%	0.057%
code-gateway	8	13887	13498	14917	14917	8.37	0.06%	0.062%	0.056%
identity-service	5	14392	16160	16819	16819	11.36	0.079%	0.07%	0.068%
go-common	4	31192	28291	31266	31266	5.68	0.018%	0.02%	0.018%
target-service	4	10334	9417	10618	10618	17.46	0.169%	0.185%	0.164%
go-code-integrations-common	4	14872	14622	14917	14917	3.92	0.026%	0.027%	0.026%
managed-idp-resources	3	7755	7710	7710	7710	7.86	0.101%	0.102%	0.102%
rest-node-libs	3	4658	4404	4404	4404	6.21	0.133%	0.141%	0.141%
cerberus	2	13461	12712	12712	12712	6.23	0.046%	0.049%	0.049%

Code --report (30d, snyk/* targets)

repo	n	avg ms	p50 ms	p90 ms	p99 ms	contrib ms	%avg	%p50	%p90
cli	3644	8733	9048	12712	26116	23.91	0.274%	0.264%	0.188%
registry	122	97098	73889	129358	311879	215.56	0.222%	0.292%	0.167%
maverick	82	34846	27731	39748	140133	42.67	0.122%	0.154%	0.107%
deepcode	82	95710	80044	151804	217588	111.30	0.116%	0.139%	0.073%
maverick-ui	73	65718	61717	78459	93932	16.55	0.025%	0.027%	0.021%
datadog-infra	66	16302	12213	18964	46644	21.42	0.131%	0.175%	0.113%
assets-api	66	133246	23630	52592	1484255	18.01	0.014%	0.076%	0.034%
data-layer	55	19488	13230	30647	58123	11.50	0.059%	0.087%	0.038%
app-ui	41	30816	23162	60495	83310	26.74	0.087%	0.115%	0.044%
entitlements-service	40	72756	36692	81661	240473	17.15	0.024%	0.047%	0.021%
workspace-service	25	28462	16160	30040	83310	20.71	0.073%	0.128%	0.069%
projects-usage-producer	18	15500	11274	18220	23162	5.16	0.033%	0.046%	0.028%
topcoat-dbt	17	13847	11502	17859	21813	21.86	0.158%	0.19%	0.122%
vervet	14	23714	13498	21813	48548	6.92	0.029%	0.051%	0.032%
go-application-framework	13	20299	17506	28291	29445	13.31	0.066%	0.076%	0.047%
admin-access	12	55214	16819	17859	18588	6.73	0.012%	0.04%	0.038%
janus	11	21610	12968	21813	21813	6.82	0.032%	0.053%	0.031%
tenants-service	9	17100	16160	19738	19738	14.44	0.084%	0.089%	0.073%
data-schema	9	30056	18964	42205	42205	14.10	0.047%	0.074%	0.033%
code-gateway	8	120447	20958	84993	84993	8.37	0.007%	0.04%	0.01%
sca-engine	8	18819	14332	21382	21382	10.10	0.054%	0.07%	0.047%
target-service	4	17281	13230	13770	13770	17.46	0.101%	0.132%	0.127%
go-common	4	22454	20543	24108	24108	5.68	0.025%	0.028%	0.024%
go-code-integrations-common	4	11934	10201	13230	13230	3.92	0.033%	0.038%	0.03%
identity-service	4	17625	11274	18588	18588	11.36	0.064%	0.101%	0.061%
managed-idp-resources	3	132463	28862	28862	28862	7.86	0.006%	0.027%	0.027%

IaC --report (30d, snyk/* targets)

repo	n	avg ms	p50 ms	p90 ms	p99 ms	contrib ms	%avg	%p50	%p90
registry	122	7076	6838	8024	10201	215.56	3.046%	3.152%	2.686%
maverick-ui	73	16299	15840	17859	20958	16.55	0.102%	0.105%	0.093%
datadog-infra	66	176937	171160	204916	240473	21.42	0.012%	0.013%	0.01%
assets-api	66	7823	7407	8868	13498	18.01	0.23%	0.243%	0.203%
data-layer	55	15318	14622	18588	22704	11.50	0.075%	0.079%	0.062%
app-ui	41	6578	6440	7407	7866	26.74	0.407%	0.415%	0.361%
entitlements-service	40	12298	11734	14048	17506	17.15	0.139%	0.146%	0.122%
api-gateway	39	684688	666901	722447	955900	18.52	0.003%	0.003%	0.003%
workspace-service	25	5475	5272	5712	7407	20.71	0.378%	0.393%	0.363%
topcoat-dbt	17	6068	5945	6312	6440	21.86	0.36%	0.368%	0.346%
vervet	14	6659	6312	7117	7407	6.92	0.104%	0.11%	0.097%
go-application-framework	13	6193	6187	6570	6570	13.31	0.215%	0.215%	0.203%
admin-access	12	9183	8868	9999	10832	6.73	0.073%	0.076%	0.067%
janus	11	8806	8024	9999	9999	6.82	0.077%	0.085%	0.068%
tenants-service	9	9377	9048	10201	10201	14.44	0.154%	0.16%	0.142%
code-gateway	8	6023	5712	6187	6187	8.37	0.139%	0.147%	0.135%
sca-engine	8	6090	6187	6440	6440	10.10	0.166%	0.163%	0.157%
target-service	4	8874	8693	8868	8868	17.46	0.197%	0.201%	0.197%
go-common	4	5101	5066	5272	5272	5.68	0.111%	0.112%	0.108%
go-code-integrations-common	4	5328	4965	5272	5272	3.92	0.074%	0.079%	0.074%
identity-service	4	7653	7261	8024	8024	11.36	0.148%	0.156%	0.142%
rest-node-libs	3	7065	7117	7117	7117	6.21	0.088%	0.087%	0.087%
managed-idp-resources	3	37735	37433	37433	37433	7.86	0.021%	0.021%	0.021%
cerberus	2	5235	5066	5066	5066	6.23	0.119%	0.123%	0.123%

Secrets --report

Query restricted to snyk/* targets returns almost nothing (3 repos total, ≤238 samples). The global (all orgs, all targets) 15d stats are: n=2196, avg=13087ms, p50=11971ms, p90=15840ms, p99=32542ms. Contributor overhead against that global baseline:

repo	contrib ms	%avg(global)	%p50(global)
registry	215.56	1.647%	1.801%
deepcode	111.30	0.851%	0.93%
maverick	42.67	0.326%	0.356%
app-ui	26.74	0.204%	0.223%
cli	23.91	0.183%	0.2%
topcoat-dbt	21.86	0.167%	0.183%
datadog-infra	21.42	0.164%	0.179%
workspace-service	20.71	0.158%	0.173%
api-gateway	18.52	0.142%	0.155%
assets-api	18.01	0.138%	0.15%
target-service	17.46	0.133%	0.146%
entitlements-service	17.15	0.131%	0.143%
maverick-ui	16.55	0.127%	0.138%
tenants-service	14.44	0.11%	0.121%
data-schema	14.10	0.108%	0.118%
go-application-framework	13.31	0.102%	0.111%
data-layer	11.50	0.088%	0.096%
identity-service	11.36	0.087%	0.095%
sca-engine	10.10	0.077%	0.084%
code-gateway	8.37	0.064%	0.07%
managed-idp-resources	7.86	0.06%	0.066%
vervet	6.92	0.053%	0.058%
janus	6.82	0.052%	0.057%
admin-access	6.73	0.051%	0.056%
cerberus	6.23	0.048%	0.052%
rest-node-libs	6.21	0.047%	0.052%
go-common	5.68	0.043%	0.047%
projects-usage-producer	5.16	0.039%	0.043%
go-code-integrations-common	3.92	0.03%	0.033%
snyk-images	3.21	0.025%	0.027%

Summary

This gives global % averages of: min 0.003%, median 0.088%, mean 0.182%, max 3.046%.

In Registry, the cominbation of IaC --report being fast and registry having a large git history means the contributor collection cost is proportionally largest there.
Next largest is deepcode on OSS Monitor at 1.167%, again via a combination of a relatively fast scan and large git history.

For every other repo/command combination, contributor collection is under 1% of command runtime.

snyk-io · 2026-08-12T13:03:59Z

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues
✅	Code Security	0	0	0	0	0 issues
✅	Secrets	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

snyk-io · 2026-08-12T13:03:59Z

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues
✅	Code Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

cursor · 2026-08-17T10:58:02Z

+
+			if commit.Committer.When.Before(since) {
+				return storer.ErrStop
+			}


Committer stop skips author window

Medium Severity

Contributor collection stops the commit walk when commit.Committer.When is before the window, before evaluating commit.Author.When. Billing uses author dates, so commits with a recent author date and an older committer date are dropped entirely, under-reporting active contributors.

^{Reviewed by Cursor Bugbot for commit 14a9d10. Configure here.}

snyk-pr-review-bot · 2026-08-17T13:06:53Z

PR Reviewer Guide 🔍

🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Tight Operation Timeout 🟠 [major] The `requestTimeout` of 10 seconds (line 25) is used for the entire operation including all retries. With `requestAttempts` set to 3 and an initial `retryInterval` of 1 second, the total duration can easily exceed 10 seconds if the target API experiences high latency (e.g., 3-4 seconds per response). This will cause the context to cancel the final retry attempt prematurely, leading to billing data loss even if the service was recoverable. requestTimeout = 10 * time.Second Silent Data Loss 🟠 [major] In `Emit`, the code returns `nil` (success) when `len(contributors) == 0`. While this handles legitimate cases of inactive repos, it also silences scenarios where `collectContributors` fails to find data due to environment issues (e.g., shallow clones, git index corruption). Because this is billing data, a zero-contributor result should potentially be distinguished from a failed scan to ensure billing integrity. } if len(contributors) == 0 { e.logger.Debug().Str("repo_path", repoPath).Msg("contributor billing: no contributors to report") return nil Resource Exhaustion Risk 🟡 [minor] `openRepositoryFast` initializes filesystem storage with `KeepDescriptors: true`. In the context of a long-running IDE process or a server-side consumer (like an MCP server) that may open many repositories, this can lead to File Descriptor (FD) exhaustion if the returned `closeRepo` function is not called immediately or if the repository contains a very high number of packfiles. st := filesystem.NewStorageWithOptions(fs, cache.NewObjectLRUDefault(), filesystem.Options{KeepDescriptors: true})
📚 Repository Context Analyzed This review considered 51 relevant code sections from 14 files (average relevance: 0.96) internal/contributorbilling/billing.go (lines 1-131) internal/contributorbilling/contributors.go (lines 1-148) pkg/apiclients/contributors_ingest/2024-10-15/contributors_ingest.go (2 segments, lines 1-191) pkg/apiclients/contributors_ingest/2024-10-15/spec.yaml (lines 1-114) pkg/apiclients/contributors_ingest/2024-10-15/spec.config.yaml (lines 1-6) pkg/apiclients/feature_flag_gateway/2024-10-15/gen.go (lines 1-5) pkg/apiclients/fileupload/client.go (lines 1-178) ... and 7 more file(s)
🤖 Repository instructions applied (from `AGENTS.md`)

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 43fe465. Configure here.}

cursor · 2026-08-17T13:07:42Z

+		name := ref.Name()
+		if !name.IsBranch() && !name.IsRemote() {
+			return nil
+		}


Detached HEAD commits never scanned

Medium Severity

Reference iteration only follows local branches and remote-tracking refs. The HEAD ref is skipped, so commits that exist only on a detached HEAD (not on any branch or remote ref) are never walked and their authors are omitted from contributor reporting.

^{Reviewed by Cursor Bugbot for commit 43fe465. Configure here.}

jonnyowenpowell force-pushed the feat/IANDT-237-contributor-billing branch from dfe3785 to 3a994a8 Compare August 12, 2026 13:42

jonnyowenpowell commented Aug 12, 2026

View reviewed changes

Comment thread internal/contributors/collect.go

NikkiL-Snyk reviewed Aug 12, 2026

View reviewed changes

Comment thread pkg/apiclients/contributors_ingest/ingest.go Outdated

jonnyowenpowell force-pushed the feat/IANDT-237-contributor-billing branch 2 times, most recently from 1894254 to 8796cbc Compare August 13, 2026 11:08

jonnyowenpowell commented Aug 13, 2026

View reviewed changes

Comment thread internal/contributorbilling/contributors.go Outdated

jonnyowenpowell changed the title ~~feat: add active contributor emitter~~ feat: [IANDT-237] add active contributor emitter Aug 13, 2026

jonnyowenpowell marked this pull request as ready for review August 13, 2026 11:12

jonnyowenpowell requested review from a team as code owners August 13, 2026 11:12

cursor Bot reviewed Aug 13, 2026

View reviewed changes

Comment thread internal/contributorbilling/contributors.go Outdated

Comment thread internal/contributorbilling/contributors.go Outdated