Skip to content

fix: resolve catalog pagination limits and add GHCR CI/CD#2592

Open
kelvinzer0 wants to merge 6 commits into
evolution-foundation:mainfrom
kelvinzer0:main
Open

fix: resolve catalog pagination limits and add GHCR CI/CD#2592
kelvinzer0 wants to merge 6 commits into
evolution-foundation:mainfrom
kelvinzer0:main

Conversation

@kelvinzer0

@kelvinzer0 kelvinzer0 commented Jun 18, 2026

Copy link
Copy Markdown

Summary

This PR fixes critical pagination limitations in the WhatsApp Business catalog API that prevent fetching catalogs with more than 50 products.

Changes

  1. Increased default catalog limit from 10 to 50 products (whatsapp.baileys.service.ts:4904)
  2. Increased pagination loops from 4 to 20 (whatsapp.baileys.service.ts:4927)
  3. Removed arbitrary 20-item cap on getCollections - now defaults to 100 (whatsapp.baileys.service.ts:4974)
  4. Added cursor parameter to validation schemas for manual pagination control (business.schema.ts)
  5. Added cursor to getCollectionsDto (business.dto.ts)
  6. Added GHCR workflow for publishing to GitHub Container Registry

Problem

The current implementation has these limitations:

  • getCatalog: Default limit=10 with max 4 loops = 50 products max
  • getCollections: Hard-capped at 20 items per collection
  • cursor parameter stripped by validation schema, preventing manual pagination

Solution

  • Default limit increased to 50 for better out-of-box experience
  • Auto-pagination now supports up to 20 loops (50 × 20 = 1000 products)
  • getCollections default increased to 100 items
  • Cursor parameter now exposed for manual pagination when needed

Testing

  • Verified with catalog containing 63+ products across 6 collections
  • Both getCatalog and getCollections now return complete data

Closes #2589

Summary by Sourcery

Adjust WhatsApp Business catalog APIs to support larger, cursor-based pagination and add automated container publishing to GitHub Container Registry.

New Features:

  • Expose an optional cursor parameter in catalog and collections DTOs and validation schemas to support manual pagination control.
  • Introduce a GitHub Actions workflow to build multi-architecture Docker images and publish them to GitHub Container Registry on pushes, releases, and relevant pull requests.

Enhancements:

  • Increase default catalog page size and pagination loop count to handle significantly larger product catalogs automatically.
  • Relax collection fetch limits by raising the default maximum number of items returned per request.

CI:

  • Add a CI workflow that builds and optionally pushes Docker images to GHCR with proper tagging and caching.

- Increase default catalog limit from 10 to 50 products
- Increase pagination loops from 4 to 20 for larger catalogs
- Remove arbitrary 20-item cap on getCollections (now default 100)
- Add cursor parameter to validation schemas for manual pagination
- Add cursor support to getCollectionsDto
- Add GitHub Actions workflow for GHCR publishing

Fixes evolution-foundation#2589
@sourcery-ai

sourcery-ai Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Adjusts WhatsApp Business catalog pagination defaults and exposes cursor-based pagination, while adding a GitHub Actions workflow to build and publish Docker images to GHCR.

File-Level Changes

Change Details Files
Relax and expand catalog pagination to support larger catalogs and cursor-based manual control.
  • Increase default product limit in fetchCatalog from 10 to 50 to improve out-of-the-box catalog size coverage.
  • Allow up to 20 pagination loops instead of 4 when auto-fetching catalog pages, enabling retrieval of up to ~1000 products with default settings.
  • Change fetchCollections limit handling to default to 100 items when limit is not provided, removing the previous hard cap of 20.
  • Propagate an optional cursor parameter through validation schemas and DTOs so clients can pass manual pagination cursors into catalog and collection fetches.
src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts
src/validate/business.schema.ts
src/api/dto/business.dto.ts
Introduce CI/CD workflow to build and publish container images to GitHub Container Registry (GHCR).
  • Add a GitHub Actions workflow that builds multi-architecture Docker images on pushes to main, release branches, semantic version tags, and pull requests targeting main.
  • Configure docker/metadata-action to generate tags from branches, PRs, semantic versions, and commit SHA, and use GitHub Container Registry as the target registry.
  • Set up QEMU and Buildx, authenticate to GHCR for non-PR events, and push built images with caching enabled.
.github/workflows/publish_ghcr.yml

Assessment against linked issues

Issue Objective Addressed Explanation
#2589 Increase getCatalog auto-pagination capacity beyond 50 items and allow manual pagination by accepting a cursor parameter through the DTO and validation schema.
#2589 Remove or significantly raise the 20-item-per-collection cap in getCollections and support cursor-based manual pagination via DTO and validation schema.
#2589 Improve getCatalog API behavior by exposing nextPageCursor in the response and/or addressing the unreliable isBusiness: false fallback behavior. The PR only changes pagination parameters (limit, loops, cursor support) and schemas. It does not modify the getCatalog response shape to expose nextPageCursor, nor does it change the error handling that returns { wuid: jid, name: null, isBusiness: false }.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The new cursor parameter is exposed in the DTO and schema but only used in fetchCatalog; if collections also support cursor-based pagination it would be good to wire cursor through fetchCollections for consistency and functionality.
  • The increased hard-coded limits in fetchCatalog (limit=50, up to 20 loops) and fetchCollections (limit=100) might benefit from being configurable (e.g., via environment or config) rather than inlined magic numbers, so they can be tuned without code changes.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The new `cursor` parameter is exposed in the DTO and schema but only used in `fetchCatalog`; if collections also support cursor-based pagination it would be good to wire `cursor` through `fetchCollections` for consistency and functionality.
- The increased hard-coded limits in `fetchCatalog` (limit=50, up to 20 loops) and `fetchCollections` (limit=100) might benefit from being configurable (e.g., via environment or config) rather than inlined magic numbers, so they can be tuned without code changes.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

- Add swagger-jsdoc for auto-generating OpenAPI spec from JSDoc comments
- Create swagger.config.ts with base schemas and security setup
- Add JSDoc comments to Business router endpoints (getCatalog, getCollections)
- Setup swagger-ui at /docs endpoint
- Add /openapi.json endpoint for raw spec access
- Add npm script 'openapi:generate' to generate static spec
- Generate initial openapi.json with Business endpoints

Access:
- Swagger UI: http://localhost:8080/docs
- OpenAPI JSON: http://localhost:8080/openapi.json
- Fix package.json import in swagger.config.ts (use readFileSync instead of import)
- Fix package.json import in generate-openapi.ts
- Fix prettier formatting in main.ts swagger-ui setup
- Addresses CI failures from PR evolution-foundation#2592
- Add Number() conversion for limit in fetchCatalog and fetchCollections
- Fixes 'limit is not of a type(s) number' error when limit is sent as string
- Change limit schema type from 'number' to ['number', 'string']
- Fixes 400 error when n8n sends limit as string (e.g. "50" instead of 50)
- Service layer already converts with Number() for safety
- fetchBusinessProfile failure no longer blocks catalog/collections fetch
- Both endpoints now continue even if business profile check fails
- Prevents returning isBusiness:false when profile API is temporarily down
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 getCatalog pagination limited to 50 products & getCollections capped at 20 items per collection

1 participant