feat: EntityPagination — lazily loaded paginated view over a select (v1.12.0)

CHANGELOG.md

@@ -1,3 +1,60 @@
+## 1.12.0
+
+- New `EntityPagination<O>`: a lazily loaded, paginated view over a select,
+  for reading a result page by page without knowing its total length upfront.
+
+  ```dart
+  var p = userRepository.paginateByQuery(' state == ? ',
+      parameters: ['NY'], limit: 20);
+
+  await p.loadNextPage();   // page 1
+  p[0];                     // sync, already loaded
+  await p.getAt(45);        // loads page 3 on demand, leaving page 2 a gap
+  await p.loadAll();        // fills the gaps and resolves the total
+  ```
+
+  - Pages are **1-based** (matching the `page` parameter of the `select*`
+    methods); entry indexes are **0-based** (matching a Dart `List`). See
+    `indexOfPage` / `pageOfIndex`.
+  - Pages can be loaded out of order, leaving gaps: `getAt(45)` with a `limit`
+    of 20 loads only page 3.
+  - Synchronous access (`operator []`, `loadedEntities`) **never fetches**;
+    only the `FutureOr` methods (`getAt`, `getPage`, `getRange`,
+    `loadNextPage`, `loadPage`, `loadAll`, `stream`) do. `operator []` returns
+    `null` for a gap, an unloaded page or an out-of-range index alike; use
+    `isPageLoaded` / `isIndexKnownOutOfRange` to tell them apart.
+  - It is deliberately **not** a `List` or an `Iterable`: both require a
+    `length`, which is exactly what a paginated select can't answer until it
+    reaches the end. Use `loadAll` when a complete list is really needed.
+
+  What it knows: `loadedPages`, `loadedPagesLength`, `loadedEntities`,
+  `loadedEntitiesLength`, `maxLoadedPage`, `maxLoadedIndex`, `maxKnownPage`,
+  `isFinalPageResolved`, `finalPage`, `totalLength`, `isKnownEmpty`,
+  and `information()`.
+
+  Since every page except the last holds exactly `limit` entries, identifying
+  the final page yields the total even with gaps:
+  `totalLength == (finalPage - 1) * limit + entries(finalPage)`.
+  The end resolves when a page comes back short, when an empty page has a
+  loaded and full predecessor, or when page 1 comes back empty. An empty page
+  *without* a loaded predecessor does **not** resolve it — jumping to page 50
+  of a 3-page result only proves the end is somewhere before page 50 — but it
+  is still recorded, to avoid re-fetching that page or any page after it.
+
+  Concurrent requests for the same page share a single fetch, and a failed
+  load is evicted so a retry actually retries.
+
+- New `paginateByQuery`, `paginate` and `paginateAll` on `EntitySource`,
+  `EntityRepository` (with `resolutionRules`) and `APIRepository`. They return
+  immediately without loading anything. `orderByID` defaults to `true` there,
+  rather than following the `offset != null` rule of the `select*` methods:
+  a paginated read is only meaningful over a stable order.
+
+- Note: each page is an independent select, without a shared `Transaction`.
+  Entries inserted or deleted between two page loads shift the offsets, so a
+  page loaded later can repeat or skip entries. This is inherent to
+  offset-based pagination; ordering by ID makes it as stable as it can be.
+
 ## 1.11.0
 
 - `selectByQuery` and its siblings gained 4 optional parameters, for pagination

README.md

@@ -459,6 +459,42 @@ The `select*` methods accept `limit`, `offset`, `page`, `orderByID` and
   combined with an `offset`, if there is no positive `limit` to page by, or if
   it is below 1.
 
+### Paginated reads: `EntityPagination`
+
+For reading a result page by page, `paginateByQuery` (and `paginate` /
+`paginateAll`) returns an `EntityPagination`, which keeps the pages it has
+already loaded:
+
+```dart
+var page = accountRepository.paginateByQuery(
+  ' address.state == ? ',
+  parameters: ['NY'],
+  limit: 20,
+);
+
+await page.loadNextPage();   // loads page 1
+page[0];                     // synchronous: already loaded
+page[45];                    // null: not loaded (never fetches)
+
+await page.getAt(45);        // loads page 3 on demand
+page.loadedPages;            // [1, 3] — page 2 is a gap
+page.maxLoadedIndex;         // 59
+page.totalLength;            // null: the end is not known yet
+
+await page.loadAll();        // fills the gaps and resolves the end
+page.totalLength;            // 57
+page.finalPage;              // 3
+```
+
+Pages are 1-based and entry indexes 0-based. Synchronous access
+(`operator []`, `loadedEntities`) never fetches — only the `FutureOr` methods
+(`getAt`, `getPage`, `getRange`, `loadNextPage`, `loadAll`, `stream`) do.
+
+It is deliberately not a `List`: a paginated select does not know its length
+until it reaches the end, so `totalLength` is `null` until the final page is
+identified (`isFinalPageResolved`). Until then you still know
+`maxLoadedIndex`, `maxKnownPage` and which pages are loaded.
+
 The config file used above:
 
 File: `api-local.yaml`

lib/bones_api.dart

@@ -325,6 +325,7 @@ export 'src/bones_api_entity_db_memory.dart';
 export 'src/bones_api_entity_db_object_memory.dart';
 export 'src/bones_api_entity_db_relational.dart';
 export 'src/bones_api_entity_db_sql.dart';
+export 'src/bones_api_entity_pagination.dart';
 export 'src/bones_api_entity_reference.dart';
 export 'src/bones_api_entity_rules.dart';
 export 'src/bones_api_entity_sql.dart';

lib/src/bones_api_base.dart

@@ -48,7 +48,7 @@ typedef APILogger =
 /// Bones API Library class.
 class BonesAPI {
   // ignore: constant_identifier_names
-  static const String VERSION = '1.11.0';
+  static const String VERSION = '1.12.0';
 
   static bool _boot = false;
 

lib/src/bones_api_entity.dart

@@ -19,6 +19,7 @@ import 'package:swiss_knife/swiss_knife.dart' show EventStream, DataURLBase64;
 import 'bones_api_base.dart';
 import 'bones_api_condition.dart';
 import 'bones_api_entity_annotation.dart';
+import 'bones_api_entity_pagination.dart';
 import 'bones_api_entity_reference.dart';
 import 'bones_api_entity_rules.dart';
 import 'bones_api_error_zone.dart';
@@ -3632,6 +3633,88 @@ abstract class EntitySource<O extends Object> extends EntityAccessor<O> {
     EntityResolutionRules? resolutionRules,
   });
 
+  /// {@template bones_api.paginate}
+  /// Returns an [EntityPagination] over this select, loading nothing yet:
+  /// pages are fetched only when asked for. See [EntityPagination].
+  ///
+  /// - [limit]: the page size. Required, and must be `> 0`.
+  /// - [orderByID]: defaults to `true`. A paginated read is only meaningful
+  ///   over a stable order, so unlike the `select*` methods this is on by
+  ///   default instead of being implied by an offset.
+  /// - [orderDirection]: the [OrderDirection] of the ordering.
+  /// {@endtemplate}
+  EntityPagination<O> paginateByQuery(
+    String query, {
+    Object? parameters,
+    List? positionalParameters,
+    Map<String, Object?>? namedParameters,
+    required int limit,
+    bool? orderByID,
+    OrderDirection? orderDirection,
+    Transaction? transaction,
+  }) => EntityPagination<O>(
+    limit: limit,
+    query: query,
+    pageLoader:
+        (page, limit) => selectByQuery(
+          query,
+          parameters: parameters,
+          positionalParameters: positionalParameters,
+          namedParameters: namedParameters,
+          transaction: transaction,
+          limit: limit,
+          page: page,
+          orderByID: orderByID ?? true,
+          orderDirection: orderDirection,
+        ).resolveMapped((os) => os.toList()),
+  );
+
+  /// {@macro bones_api.paginate}
+  EntityPagination<O> paginate(
+    EntityMatcher<O> matcher, {
+    Object? parameters,
+    List? positionalParameters,
+    Map<String, Object?>? namedParameters,
+    required int limit,
+    bool? orderByID,
+    OrderDirection? orderDirection,
+    Transaction? transaction,
+  }) => EntityPagination<O>(
+    limit: limit,
+    query: '$matcher',
+    pageLoader:
+        (page, limit) => select(
+          matcher,
+          parameters: parameters,
+          positionalParameters: positionalParameters,
+          namedParameters: namedParameters,
+          transaction: transaction,
+          limit: limit,
+          page: page,
+          orderByID: orderByID ?? true,
+          orderDirection: orderDirection,
+        ).resolveMapped((os) => os.toList()),
+  );
+
+  /// {@macro bones_api.paginate}
+  EntityPagination<O> paginateAll({
+    required int limit,
+    bool? orderByID,
+    OrderDirection? orderDirection,
+    Transaction? transaction,
+  }) => EntityPagination<O>(
+    limit: limit,
+    query: 'ALL',
+    pageLoader:
+        (page, limit) => selectAll(
+          transaction: transaction,
+          limit: limit,
+          page: page,
+          orderByID: orderByID ?? true,
+          orderDirection: orderDirection,
+        ).resolveMapped((os) => os.toList()),
+  );
+
   FutureOr<Iterable<dynamic>> selectRelationship<E>(
     O? o,
     String field, {
@@ -5725,6 +5808,94 @@ abstract class EntityRepository<O extends Object> extends EntityAccessor<O>
     );
   }
 
+  /// {@macro bones_api.paginate}
+  ///
+  /// - [resolutionRules]: applied to every loaded page.
+  @override
+  EntityPagination<O> paginateByQuery(
+    String query, {
+    Object? parameters,
+    List? positionalParameters,
+    Map<String, Object?>? namedParameters,
+    required int limit,
+    bool? orderByID,
+    OrderDirection? orderDirection,
+    Transaction? transaction,
+    EntityResolutionRules? resolutionRules,
+  }) => EntityPagination<O>(
+    limit: limit,
+    query: query,
+    pageLoader:
+        (page, limit) => selectByQuery(
+          query,
+          parameters: parameters,
+          positionalParameters: positionalParameters,
+          namedParameters: namedParameters,
+          transaction: transaction,
+          limit: limit,
+          page: page,
+          orderByID: orderByID ?? true,
+          orderDirection: orderDirection,
+          resolutionRules: resolutionRules,
+        ).resolveMapped((os) => os.toList()),
+  );
+
+  /// {@macro bones_api.paginate}
+  ///
+  /// - [resolutionRules]: applied to every loaded page.
+  @override
+  EntityPagination<O> paginate(
+    EntityMatcher<O> matcher, {
+    Object? parameters,
+    List? positionalParameters,
+    Map<String, Object?>? namedParameters,
+    required int limit,
+    bool? orderByID,
+    OrderDirection? orderDirection,
+    Transaction? transaction,
+    EntityResolutionRules? resolutionRules,
+  }) => EntityPagination<O>(
+    limit: limit,
+    query: '$matcher',
+    pageLoader:
+        (page, limit) => select(
+          matcher,
+          parameters: parameters,
+          positionalParameters: positionalParameters,
+          namedParameters: namedParameters,
+          transaction: transaction,
+          limit: limit,
+          page: page,
+          orderByID: orderByID ?? true,
+          orderDirection: orderDirection,
+          resolutionRules: resolutionRules,
+        ).resolveMapped((os) => os.toList()),
+  );
+
+  /// {@macro bones_api.paginate}
+  ///
+  /// - [resolutionRules]: applied to every loaded page.
+  @override
+  EntityPagination<O> paginateAll({
+    required int limit,
+    bool? orderByID,
+    OrderDirection? orderDirection,
+    Transaction? transaction,
+    EntityResolutionRules? resolutionRules,
+  }) => EntityPagination<O>(
+    limit: limit,
+    query: 'ALL',
+    pageLoader:
+        (page, limit) => selectAll(
+          transaction: transaction,
+          limit: limit,
+          page: page,
+          orderByID: orderByID ?? true,
+          orderDirection: orderDirection,
+          resolutionRules: resolutionRules,
+        ).resolveMapped((os) => os.toList()),
+  );
+
   @override
   FutureOr<O?> deleteEntity(O o, {Transaction? transaction}) =>
       deleteByID(getEntityID(o), transaction: transaction);