Skip to content

Add grouped nodeTypeCounts query; use COUNT(*) for totalCount#2296

Merged
shangyian merged 2 commits into
DataJunction:mainfrom
shangyian:perf/landing-node-counts
Jul 5, 2026
Merged

Add grouped nodeTypeCounts query; use COUNT(*) for totalCount#2296
shangyian merged 2 commits into
DataJunction:mainfrom
shangyian:perf/landing-node-counts

Conversation

@shangyian

@shangyian shangyian commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

The namespace landing page shows per-type filter pills ("Metric (342)", "Source (…)" etc). Today the UI populates them by firing one findNodesPaginated(nodeTypes:[X], limit:1){ totalCount } per node type, which results in 5 separate count scans per page load (aliased into a single request, but still N scans server-side).

This PR adds a generic server-side primitive to do it in one grouped scan, plus a small fix to the count query that backs every paginated totalCount.

New generic nodeCounts query

nodeCounts(groupBy: NodeGroupBy!, namespace: String): [NodeCount!]!

type NodeCount { value: String!, count: Int! }
enum NodeGroupBy { TYPE }

This runs the existing filter builder with GROUP BY <column> and returns {value: count} in one query. We deliberately kept this as a separate field, not folded into findNodesPaginated since counting and pagination are different concerns. NodeGroupBy starts at TYPE, but adding STATUS/MODE later is just an enum value + a column mapping.

COUNT(*) instead of COUNT(DISTINCT id) for totalCount

Node.count_by (which backs every paginated totalCount) used COUNT(DISTINCT Node.id). However, COUNT(*) is exactly equivalent and skips the sort/hash DISTINCT forces. This change yields identical results, and it stops degrading with table size.

Test Plan

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

@netlify

netlify Bot commented Jul 5, 2026

Copy link
Copy Markdown

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit de802af
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/6a4a81402a097600098d03f8

@shangyian shangyian force-pushed the perf/landing-node-counts branch from 5634ebd to a6b9369 Compare July 5, 2026 15:32
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@shangyian shangyian marked this pull request as ready for review July 5, 2026 16:00
@shangyian shangyian force-pushed the perf/landing-node-counts branch from a6b9369 to 7483264 Compare July 5, 2026 16:02
@shangyian shangyian merged commit f9d87a4 into DataJunction:main Jul 5, 2026
21 checks passed
@shangyian shangyian deleted the perf/landing-node-counts branch July 5, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant