diff --git a/README.md b/README.md index eeca7e2..82e4e6e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # fastapi-listing -Advanced items listing library that gives you freedom to design really complex listing APIs using component based architecture. +A listing API library for FastAPI, built around small, composable components rather than one large +endpoint function. [![.github/workflows/deploy.yml](https://github.com/danielhasan1/fastapi-listing/actions/workflows/deploy.yml/badge.svg)](https://github.com/danielhasan1/fastapi-listing/actions/workflows/deploy.yml) [![.github/workflows/tests.yml](https://github.com/danielhasan1/fastapi-listing/actions/workflows/tests.yml/badge.svg)](https://github.com/danielhasan1/fastapi-listing/actions/workflows/tests.yml) ![PyPI - Programming Language](https://img.shields.io/pypi/pyversions/fastapi-listing.svg?color=%2334D058) @@ -11,38 +12,34 @@ Advanced items listing library that gives you freedom to design really complex l > unaffected. See [CHANGELOG.md](CHANGELOG.md) for the migration table. Comes with: -- pre defined filters -- pre defined paginator -- pre defined sorter +- a predefined set of filters +- a predefined paginator +- a predefined sorter - SQLAlchemy support out of the box, and a backend-agnostic core so you're not locked into one ORM -## Advantage -- simplify the intricate process of designing and developing complex listing APIs -- Design components(USP) and plug them from anywhere -- Components can be **reusable** -- Best for fast changing needs +## Why + +- Simplifies designing and maintaining complex listing APIs +- Components are independent, reusable, and can be swapped in from anywhere +- Well suited to fast-changing requirements - Not an ORM captive: filters/sorter/paginator are written against a small `QueryContext` contract, not a raw SQLAlchemy `Query` - swap in a different backend without rewriting your filters ## Installing Using [pip](https://pip.pypa.io/): -```python +```bash pip install fastapi-listing ``` -## Quick Example - -Attaching example of it running against the [mysql employee db](https://dev.mysql.com/doc/employee/en/) +## Quick example -There are two ways to implement a listing API using fastapi listing +The example below runs against the [MySQL employee sample DB](https://dev.mysql.com/doc/employee/en/). -- inline implementation -- class based implementation +There are two ways to implement a listing API with this library: **inline** or **class-based**. Both +need a DAO (data access object) class. -for both we will be needing a dao(data access object) class - -### First let's look at inline implementation. +### Inline implementation ```python # main.py @@ -76,9 +73,9 @@ class Employee(Base): # Dao class class EmployeeDao(GenericDao): - """write your data layer access logic here. keep it raw!""" + """Data access logic lives here - keep it raw.""" name = "employee" - model = Employee # sqlalchemy model class. Not on SQLAlchemy? See "Backend support" below. + model = Employee # SQLAlchemy model class. Not on SQLAlchemy? See "Backend support" below. class EmployeeListDetails(BaseModel): @@ -92,43 +89,49 @@ class EmployeeListDetails(BaseModel): class Config: orm_mode = True allow_population_by_field_name = True - + +@app.get("/employees", response_model=ListingPage[EmployeeListDetails]) +def get_employees(db: Session): + dao = EmployeeDao(read_db=db) + # passing a pydantic serializer is optional - it generates a select query from the + # serializer's fields for simple cases (columns all on the same table); otherwise + # provide the select query yourself in the DAO layer + return FastapiListing(dao=dao, pydantic_serializer=EmployeeListDetails + ).get_response(MetaInfo(default_srt_on="emp_no")) # sorts descending by default +``` + +If your Pydantic model has computed fields (fields with no matching column), pass `custom_fields=True` to +avoid an "unknown attribute" error: + +```python @app.get("/employees", response_model=ListingPage[EmployeeListDetails]) def get_employees(db: Session): dao = EmployeeDao(read_db=db) - # passing pydantic serializer is optional, automatically generates a - # select query based on pydantic class fields for easy cases like columns of same table - # if not passed then provide a select query in dao layer - return FastapiListing(dao=dao, pydantic_serializer=EmployeeListDetails - ).get_response(MetaInfo(default_srt_on="emp_no")) # by default sort in desc order - # let's say pydantic class contains compute fields then pass custom_fields=True (by default False) return FastapiListing(dao=dao, pydantic_serializer=EmployeeListDetails, - custom_fields=True # here setting custom field True to avoid unknown attributes error + custom_fields=True ).get_response(MetaInfo(default_srt_on="emp_no")) ``` -Voila 🎉 your very first listing response +That's your first listing response. ![](https://drive.google.com/uc?export=view&id=1amgrAdGP7WvXfiNlCYJZPC9fz4_1CidE) - -Auto generated query doesn't fulfil your use case❓️ +If the auto-generated query doesn't fit your use case, override `get_default_read` in the DAO instead: ```python # Overwriting default read method in dao class class EmployeeDao(GenericDao): - """write your data layer access logic here. keep it raw!""" + """Data access logic lives here - keep it raw.""" name = "employee" model = Employee - + def get_default_read(self, fields_to_read: Optional[list]): """ Extend and return your query from here. - Use it when use cases are comparatively easier than complex. - Alternatively fastapi-listing provides a robust way to write performance packed queries - for complex APIs which we will look at later. + Use this when the use case is simpler than a full custom query strategy; + for more complex cases, see the query-customisation docs. """ query = self._read_db.query(Employee) return query @@ -137,77 +140,75 @@ class EmployeeDao(GenericDao): @app.get("/employees", response_model=ListingPage[EmployeeListDetails]) def get_employees(db: Session): dao = EmployeeDao(read_db=db) - # note we removed all optional named params here + # note the optional named params are gone here return FastapiListing(dao=dao).get_response(MetaInfo(default_srt_on="emp_no")) ``` -# Adding client site features +## Adding client-site features -Django admin users gonna love filter feature. But before that lets do a little setup which no once can avoid to support a broad spectrum of clients unless you use native query param format which I doubt. +Before adding filters, sorters, or pagination, most existing services need one bit of setup: an adapter +that reads your client's actual request-parameter format, unless it already matches FastAPI Listing's +native format exactly. -## Add your custom adaptor class for reading filter/sorter/paginator client request params +### Add a custom adapter for reading filter/sorter/paginator parameters -Below is the default implementation. You will be writing your own adaptor definition +Below is the default implementation - you'll typically extend it with your own parameter format. ```python from typing import Literal from fastapi_listing.service.adapters import CoreListingParamsAdapter from fastapi_listing import utils -class YourAdapterClass(CoreListingParamsAdapter): # Extend to add your behaviour - """Utilise this adapter class to make your remote client site: - - filter, - - sorter, - - paginator. - query params adapt to fastapi listing library. - With this you can utilise same listing api to multiple remote client - even if it's a front end server or other backend server. +class YourAdapterClass(CoreListingParamsAdapter): # extend to add your own behavior + """Adapts your client's filter/sorter/paginator query params to what FastAPI + Listing expects natively. This lets the same listing API serve multiple + clients - a frontend, another backend service, whatever - each using their + own parameter format. - fastapi listing is always going to request one of the following fundamental key if you want to use it + FastAPI Listing looks for up to three keys: - sort - filter - pagination - supported formats for + Supported formats: + filter: - simple filter - [{"field":"", "value":{"search":""}}, ...] - if you are using a range filter - - [{"field":"", "value":{"start":"", "end": ""}}, ...] - if you are using a list filter i.e. search on given items - [{"field":"", "value":{"list":[""]}}, ...] + single value - [{"field": "", "value": {"search": ""}}, ...] + range - [{"field": "", "value": {"start": "", "end": ""}}, ...] + list - [{"field": "", "value": {"list": [""]}}, ...] sort: - [{"field":<"key used in sort mapper>", "type":"asc or "dsc"}, ...] - by default single sort allowed you can change it by extending sort interceptor + [{"field": "", "type": "asc" | "dsc"}, ...] + single-field sort by default; extend the sort interceptor for multi-field sort. pagination: - {"pageSize": , "page": } + {"pageSize": , "page": } """ - + def get(self, key: Literal["sort", "filter", "pagination"]): """ @param key: Literal["sort", "filter", "pagination"] - @return: List[Optional[dict]] for filter/sort and dict for paginator + @return: List[Optional[dict]] for filter/sort, dict for pagination """ return utils.dictify_query_params(self.dependency.get(key)) ``` -### Once your adaptor class is set - -## Adding filter feature - -➡️ lets add filters on Employee for: -1. gender - return only **Employees** belonging to 'X' gender where X could be anything. -2. DOB - return **Employees** belonging to a specific range of DOB. -3. First Name - return **Employees** only starting with specific first names. + +### Adding filters + +Add filters on `Employee` for: +1. **gender** - only employees matching a given gender +2. **date of birth** - employees within a date range +3. **first name** - employees whose first name starts with a given value + ```python from fastapi import Request from sqlalchemy.orm import Session from fastapi_listing.paginator import ListingPage -from fastapi_listing.filters import generic_filters # collection of inbuilt filters -from fastapi_listing.factory import filter_factory # register filter against a listing +from fastapi_listing.filters import generic_filters # collection of inbuilt filters +from fastapi_listing.factory import filter_factory # register a filter mapper for use from fastapi_listing import MetaInfo, FastapiListing @@ -226,12 +227,16 @@ def get_employees(request: Request, db: Session): MetaInfo(default_srt_on="emp_no", filter_mapper=emp_filter_mapper, feature_params_adapter=YourAdapterClass)) - - # or you dont wanna pass request? - # extract required data from reqeust and pass it directly +``` + +If you'd rather not pass the request object through, extract what you need and pass it directly instead: + +```python +@app.get("/employees", response_model=ListingPage[EmployeeListDetails]) +def get_employees(request: Request, db: Session): params = request.query_params filter_, sort_, pagination = params.get("filter"), params.get("sort"), params.get("paginator") - + dao = EmployeeDao(read_db=db) return FastapiListing(dao=dao).get_response( MetaInfo(default_srt_on="emp_no", @@ -240,22 +245,21 @@ def get_employees(request: Request, db: Session): filter=filter_, sort=sort_, paginator=pagination)) - ``` -### Let's break it down +### Breaking it down -**Filter mapper** - a collection of allowed filters on your listing API. Any request outside of this mapper scope -will not be executed for filtering safeguarding you from creepy API users. +**Filter mapper** - the set of filters allowed on this listing API. A request for anything outside this +mapper is simply not executed, which keeps clients from probing for fields you didn't intend to expose. -`generic_filters` a collection of inbuilt filters supported by sqlalchemy orm -A dictionary is defined with structure: +`generic_filters` is a collection of inbuilt filters supported by the SQLAlchemy ORM. The mapper's +structure: `{"alias": tuple("sqlalchemy_model.field", filter_implementation)}` -`alias` - A string used by client in case if you wanna avoid actual column names to client site. +`alias` - what the client sends, so the real column name never has to be exposed. -`tuple` - will contain two items field name and filter implementation +`tuple` - the field name and the filter implementation. ```python from fastapi_listing.filters import generic_filters @@ -268,27 +272,27 @@ emp_filter_mapper = { } ``` -Register the above mapper with filter factory. +Register the mapper with the filter factory, at module level: ```python from fastapi_listing.factory import filter_factory -filter_factory.register_filter_mapper(emp_filter_mapper) # Register in global space or module level. +filter_factory.register_filter_mapper(emp_filter_mapper) ``` -A client could request you like `v1/employees?filter=[{"gdr":"M"}]` - -parse the above query_param in your adapter class like `[{"field":"gdr", "value":{"search":"M"}}]` if passed externally as kwarg then access it via `self.extra_context` in your adapter class or if passed request then -access `self.request` directly there. +A client could then request `v1/employees?filter=[{"gdr":"M"}]`, which your adapter parses into +`[{"field":"gdr", "value":{"search":"M"}}]` - if the adapter is given kwargs directly rather than the +request, access them via `self.extra_context`; if it's given the request, access `self.request` +directly. -Assuming everything goes right above will produce a response with items filtered on gender field matching rows with 'M' +That produces a response filtered to rows where `gender` is `M`. -**Sort Mapper** - a collection of allowed sort on listing any request outside of this mapper scope will -not be permitted for sort. +**Sort mapper** - the set of fields allowed for sorting; a request for anything outside this mapper is +not permitted. -Simply define a dictionary with structure `{"alias": "field"}` if sorting on same column them omit model name & -if sorting on a joined table column then add sqlalchemy class name like we did for filter `{"alias":"sqlalchemy_model.field"}` +Structure: `{"alias": "field"}` - omit the model name when sorting on the primary model's own column, or +qualify it (`{"alias": "sqlalchemy_model.field"}`) for a joined table's column, same as with filters. ```python listing_sort_mapper = { @@ -303,7 +307,7 @@ return FastapiListing(dao=dao).get_response( sort=sort_, paginator=pagination)) -# OR if passing request obj +# or, passing the request object instead return FastapiListing(request=request, dao=dao).get_response( MetaInfo(default_srt_on="emp_no", filter_mapper=emp_filter_mapper, @@ -311,32 +315,33 @@ return FastapiListing(request=request, dao=dao).get_response( feature_params_adapter=YourAdapterClass)) ``` -A client could request you like `v1/employees?sort={"code":}` or followed by filter `v1/employees?filter=[{"gdr":"M"}]&sort={"code":, "type":"asc"}` -and the response should contain list items sorted by employee code column in ascending order. - -**Note** we didn't registered sort mapper like we did for filter mapper. +A client could then request `v1/employees?sort={"code":}`, or combine it with a filter - +`v1/employees?filter=[{"gdr":"M"}]&sort={"code":, "type":"asc"}` - and the response would +be sorted by employee code, ascending. -Similarly, for paginator `v1/employees?pagination={"page":1, "pageSize":10}` or followed by filter and sort `v1/employees?filter=[{"gdr":"M"}]&sort={"code":, "type":"asc"}&pagination={"page":1, "pageSize":10}` +**Note**: unlike the filter mapper, the sort mapper doesn't need to be registered with a factory. -Above will produce listing page of items 10 or dynamically client could change page size. +Pagination works the same way: `v1/employees?pagination={"page":1, "pageSize":10}`, or combined with +filter and sort - `v1/employees?filter=[{"gdr":"M"}]&sort={"code":, "type":"asc"}&pagination={"page":1, "pageSize":10}`. -One thing to **Note** here is fastapi listing by default limits the client to reuqest maximum of 50 items at a time to safeguard your database -if you want to increase/decrease this default limit then simply pass the limit in `MetaInfo` +That returns a page of 10 items, or however many the client requests via `pageSize`. -**You can also change the default page size from 10 to anything you would want** +By default, FastAPI Listing caps a single request at 50 items to protect the database. Change that (and +the default page size) via `MetaInfo`: ```python return FastapiListing(request=request, dao=dao).get_response( MetaInfo(default_srt_on="emp_no", filter_mapper=emp_filter_mapper, sort_mapper=listing_sort_mapper, - max_page_size=25, # here change max page size - default_page_size=10, # here change default page size + max_page_size=25, # cap on requested page size + default_page_size=10, # page size when the client doesn't specify one feature_params_adapter=YourAdapterClass)) ``` -### Class Based implementation -Quick Example to convey the context +### Class-based implementation + +The same listing API, structured as a class: ```python from fastapi import FastAPI @@ -362,25 +367,25 @@ class Title(Base): employee = relationship('Employee') - + class EmployeeDao(GenericDao): name = "employee" model = Employee - + class TitleDao(GenericDao): name = "title" model = Title - + @loader.register() class EmployeeListingService(ListingService): - """Class based listing API implementation""" + """Class-based listing API implementation""" filter_mapper = { "gdr": ("Employee.gender", generic_filters.EqualityFilter), "bdt": ("Employee.birth_date", generic_filters.MySqlNativeDateFormateRangeFilter), "fnm": ("Employee.first_name", generic_filters.StringStartsWithFilter), "lnm": ("Employee.last_name", generic_filters.StringEndsWithFilter), - # below feature will require customisation to work at query level - "desg": ("Employee.Title.title", generic_filters.StringLikeFilter, lambda x: getattr(Title, x)) # registering filter with joined table field + # a joined-table field needs a resolver, same as in the query-customisation docs + "desg": ("Employee.Title.title", generic_filters.StringLikeFilter, lambda x: getattr(Title, x)) } sort_mapper = { @@ -390,35 +395,33 @@ class EmployeeListingService(ListingService): default_dao = EmployeeDao def get_listing(self): - # similar to above inline but instead of passing meta info uncompressed we pass self - # rest is handled implicityly like filter register - # one advantage here is every expect is validated so you get error when running server + # same idea as the inline version, but MetaInfo is populated from self + # rather than passed explicitly - filter/sort registration is handled implicitly, + # and @loader.register() validates the whole definition at startup resp = FastapiListing(self.request, self.dao, pydantic_serializer=EmployeeListDetails).get_response(self.MetaInfo(self)) return resp - + @app.get("/employees", response_model=ListingPage[EmployeeListDetails]) def get_employees(db: Session): return EmployeeListingService(read_db=db).get_listing() ``` -Check out [docs](https://fastapi-listing.readthedocs.io/en/latest/tutorials.html#adding-filters-to-your-listing-api) for supported list of filters. -Additionally, you can create **custom filters** as well. +See the [docs](https://fastapi-listing.readthedocs.io/en/latest/tutorials.html#adding-filters-to-your-listing-api) +for the full list of supported filters. You can also write your own custom filters. -## Provided features are not meeting your requirements??? +## Need something the built-ins don't cover? -The Applications are endless with customisations +You can write a custom: -➡️ You can write custom: - -* Query +* Query strategy * Filter * Sorter * Paginator -You can check out customisation section in docs after going through basics and tutorials. +See the customisation section of the docs, after basics and tutorials. -Check out my other [repo](https://github.com/danielhasan1/test-fastapi-listing/blob/master/app/router/router.py) to see some examples +A second, example-focused repo is available [here](https://github.com/danielhasan1/test-fastapi-listing/blob/master/app/router/router.py). ## Backend support @@ -434,7 +437,7 @@ parameterized SQL via `clickhouse-driver`, no ORM at all. The same `generic_filt against it completely unmodified, because they only ever talk to the `QueryContext`, never to SQLAlchemy directly. -```python +```bash pip install fastapi-listing[clickhouse] ``` @@ -459,25 +462,18 @@ Want a different ORM or database driver (Tortoise, Django ORM, raw psycopg2, pym `QueryContext` + DAO pair the same way `ClickHouseQueryContext`/`ClickHouseDao` do it - see `docs/query.rst`. Neither SQLAlchemy nor clickhouse-driver are required to install the package; both are opt-in extras. -## Features and Readability hand in hand 🤝 - - - Well defined interface for filter, sorter, paginator - - Support Dependency Injection for easy testing - - Room to adapt the existing remote client query param semantics - - Write standardise listing APIs that will be understood by generations of upcoming developers - - Write listing features which is easy on human mind to extend or understand - - Break down the most complex listing data APIs into digestible piece of code - -Why readability and code quality matters in one picture... +## Design goals - +- A well-defined interface for filter, sorter, and paginator +- Dependency injection, for easy testing +- Adapters, so an existing client's query-param format doesn't have to change +- Listing APIs that stay legible as they grow, rather than accumulating branches over time -# Documentation -View full documentation at: https://fastapi-listing.readthedocs.io (A work in progress) +## Documentation +Full documentation: https://fastapi-listing.readthedocs.io (a work in progress) -# Feedback, Questions? +## Feedback and questions -Any form of feedback and questions are welcome! Please create an issue 💭 -[here](https://github.com/danielhasan1/fastapi-listing/issues/new). +Feedback and questions are welcome - please [open an issue](https://github.com/danielhasan1/fastapi-listing/issues/new). diff --git a/docs/advanced_user_guide.rst b/docs/advanced_user_guide.rst index 69528bb..4148649 100644 --- a/docs/advanced_user_guide.rst +++ b/docs/advanced_user_guide.rst @@ -1,7 +1,7 @@ Customisation ============= -Learn how to customise your listing service without losing any performance with FastAPI Listing ✨ +How to customise your listing service without giving up any of the performance of a hand-written query. .. toctree:: :maxdepth: 3 diff --git a/docs/basics.rst b/docs/basics.rst index faaf3f8..5fcf564 100644 --- a/docs/basics.rst +++ b/docs/basics.rst @@ -10,88 +10,97 @@ To install FastAPI Listing, run: pip install fastapi-listing +For the ClickHouse reference backend, install the extra as well: + +.. code-block:: bash + + pip install fastapi-listing[clickhouse] .. _dao overview: -The Dao (Data Access Object) layer ----------------------------------- +The DAO (Data Access Object) layer +----------------------------------- -FastAPI Listing uses a `dao `_ -layer. +FastAPI Listing uses a `DAO `_ +layer as the one place responsible for talking to the database. -Benefits +Benefits: * A dedicated place for writing queries -* Better Separation -* Ability to change queries independently -* Provides common code usage for more than one place -* imports look cleaner +* Clear separation between data access and business logic +* Queries can change independently of the rest of the service +* Shared query logic has one obvious home instead of being copied across endpoints +* Cleaner imports at the call site -Metaphorically "a dedicated place where you cultivate your ingredients for cooking purpose" (stolen from sqlalchemy docs) - -FastAPI Listing uses single table Dao. Each dao class will be bound with single orm model class 📝. +FastAPI Listing uses a single-table DAO: each DAO class is bound to one model. Dao objects ^^^^^^^^^^^ .. py:class:: GenericDao -when creating dao ``class`` extend ``GenericDao`` which comes with necessary setup code. -Each Dao object support two protected (limiting their scope to dao layer only) session attributes. +When creating a DAO class, extend ``GenericDao``, which comes with the necessary setup code. Every DAO +object exposes two session attributes, scoped to the DAO layer only: ``_read_db`` and ``_write_db`` -You can use these attributes to communicate with the database. Provides early preparation of when you might need to implement master slave architecture. -Non master slave arch users can point both of these attributes to same db as well. It's simple. +Use these to communicate with the database. Keeping them separate is preparation for a read +replica/primary split, should you need one later - if you don't have one, point both at the same +session; there's no cost to doing so. + +``GenericDao`` is the SQLAlchemy-backed default. For a non-ORM backend, extend ``DaoAbstract`` +directly instead - see :doc:`query` for how the reference ``ClickHouseDao`` does this. Dao class attributes -^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^ .. py:attribute:: GenericDao.model - The sqlalchemy model class. Attribute type - **Required** + The SQLAlchemy model class. **Required.** .. py:attribute:: GenericDao.name - User defined name of dao class, should be unique. Attribute type - **Required** + A user-defined, unique name for the DAO class. **Required.** The Strategy layer ------------------- -Encapsulates process of: -* writing data fetch logics (Query Strategy) -* applying sorting(if any) after fetching data (Sorting Strategy) -* paginating data at the end (Paginating Strattegy) -Inspiration of using strategy pattern for this: -Depending upon logged in user/applied query filter/performance requirement/legacy based database schema(poorly managed)/data visual limiting due to maybe role of user. -You will write multiple ways to prepare queries to fetch data, or different technique to handle sorting, or a lazy paginator etc. -In any case this is a really good way to handle multiple logic implementations and their compositions. +Encapsulates: + +* fetching data (Query Strategy) +* applying sorting, if any, after fetching data (Sorting Strategy) +* paginating the fetched data (Paginating Strategy) + +The strategy pattern fits well here because these concerns tend to vary independently: which query to +run can depend on the logged-in user's role, which data layer they're allowed to see, performance +constraints, or a legacy schema you can't change. You'll often end up with multiple ways to build a +query, sort, or paginate - the strategy pattern gives each variant its own object rather than a growing +pile of conditionals in one place. Query Strategy ^^^^^^^^^^^^^^ -Logical layer to decide on a listing query in a context. By default comes with a ``default_query`` strategy which generates a -``select a,b,c,d from some_table`` query using sqlalchemy where a,b,c,d are columns given by the user. +Decides what listing query to run, in context. The default ``default_query`` strategy generates a +``select a, b, c, d from some_table`` query using SQLAlchemy, where ``a, b, c, d`` are the columns you +provide. .. _querybasics: -For simple use cases this gets the work done. +That covers most simple cases. .. py:class:: QueryStrategy -You can easily create your custom Query Strategy by extending base class. - -➡️ Taking a real world example where using strategy pattern can be helpful: +Create your own query strategy by extending the base class. -You have an employee table and hierarchy Director*->Assistant Director*->Division Managers*->Managers*->Leads*->teams. +A concrete example of where the strategy pattern helps: you have an employee table and an +organisational hierarchy - Director, Assistant Director, Division Manager, Manager, Lead, then +individual contributors. You need an API that only shows employees under the logged-in user. -You need to design an API to show list of employees associated to logged-in user only. For the sake of this example lets focus on query part only. +There are two reasonable ways to structure that with strategies. -With strategy you have two ways of achieving this. - -➡️ Creating context related query strategies: +**One strategy class per context:** ``class DirectorQueryForEmp(QueryStrategy)`` @@ -103,118 +112,111 @@ With strategy you have two ways of achieving this. ``class LeadsQueryForEmp(QueryStrategy)`` -You can abstract and encapsulate relevant logic to make a decision on logged in user basis. You can choose which one to call at runtime. - -Or +Encapsulate the logic for deciding which one applies, and choose at runtime. -➡️ Encapsulate the whole thing into one: +**Or one strategy class handling every context:** ``class EmployeeQuery(QueryStrategy)`` -And implement context based logics in one place. Choosing to write in any of above style is a personal decision based on project requirements. +Branch on context inside a single class. Which style fits is a judgment call based on how the branches +are likely to grow. +Benefits of separating by context: -Benefit of above approach: - -- Context is clear by just a look -- light weight containers of logical instructions -- Decoupled and easy to extend -- Much Easier to incorporate new relevant features like adding for new role or super user. +* the intent of each class is clear at a glance +* each is a small, focused unit +* easy to extend with a new role or a superuser case without touching the others Sorting Strategy ^^^^^^^^^^^^^^^^ -Responsible for applying sorting scheme(sql native sorting) on your query. Simple as it sound, nothing fancy here. +Applies a sort order to your query. Nothing more than that. .. py:class:: SortingOrderStrategy -**SortingOrderStrategy** ``class`` knows two *client* site keywords ``asc`` or ``dsc`` and applies sorting scheme on basis of this 📝. +``SortingOrderStrategy`` understands two client-facing keywords, ``asc`` and ``dsc``, and sorts +accordingly. -🤯You are using different keywords to make sorting decision? No worries 😉 :ref:`Make FastAPI Listing adapt to it`. +Using different keywords on the client side? See :ref:`the adapter layer `. Paginator Strategy ^^^^^^^^^^^^^^^^^^^ -Simple Paginator to paginate your database queries and return paginated response to your clients. +Paginates query results and returns a paginated response to the client. .. py:class:: PaginationStrategy -* Easily define pagination params. -* Support dynamic page resizing. -* You can configure ``default_page_size`` to return default number of items if client made a request without pagination params -* You can configure ``max_page_size``, to avoid memory choke on absurd page size demands from clients. -* Easily implement your own custom paginator to add more features like lazy loading or range based slicing. +* Configure pagination parameters directly. +* Supports dynamic page sizing. +* Set ``default_page_size`` for requests that don't specify a page size. +* Set ``max_page_size`` to cap how large a page a client can request. +* Write your own paginator for lazy loading, range-based slicing, or other strategies. -🤯You have an existing set of pagination params. can you still use it? Yes! 😉 :ref:`Make FastAPI Listing adapt to it`. +Have an existing set of pagination parameters? See :ref:`the adapter layer `. The Filters layer -^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^ -The most used feature of any listing service easily, and maintaining filters is an art in itself ❤️. +Usually the most-used part of a listing service, and one where things get messy fast without a bit of +discipline. -Abstracts away the complex procedure of applying filters, No more branching (if else) in your listing API even if you have more than a dozen filters, -with this you can write performance packed robust filters. +Filtering is abstracted so you never write a chain of ``if``/``else`` branches in a listing endpoint, +even with a dozen filters applied. -Inspired by **django-admin** design of writing and maintaining filters. Create filter anywhere easy to import ❤️ like any independent -facade API. You will see how inbuilt ``generic_filters`` will make it easy and super fast to integrate filters in your listing APIs. +Inspired by Django admin's approach to filters: define a filter once, import it anywhere, and reuse it +across listing services. ``generic_filters`` ships a set of these ready to use. -🤯 Can it support your existing clients filter parameters? Ofcourse! 😉 :ref:`Make FastAPI Listing adapt to it`. +Need this to work with an existing client's filter parameters? See :ref:`the adapter layer `. .. _intereptorbasics: The Interceptor layer -^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^ -Allows users to write custom execution plan for filters/Sorters. +Lets you write a custom execution plan for filters or sorters. -* Default filter execution plan follows iterative approach when one or more filters are applied by clients. -* Default sorter execution plan allows sort on one param at a time. +* The default filter execution plan applies filters one at a time, iteratively. +* The default sort execution plan sorts on one field at a time. -Reason of existence❓️ - In my personal experience there are special situations when applying two or many filters directly could cause -multitude of problems if applied in one by one iterative fashion. Maybe you wanna skip one or combine two filter into one -and form a more optimised and robust query for your db to avoid performance hiccups. +Why this exists: applying several filters independently, one after another, doesn't always give the +same result as applying them together. You may also want to combine two filters into a single, more +efficient query rather than running them in sequence. -Or +Similarly for sorting - multi-field sort is supported, but on large tables it tends to hurt performance +more than it helps. Filtering the data down first, then sorting, is usually the better trade-off. -Allow sorting on more than one field at a time (I personally don't like the idea as for larger tables it degrades the performance) The best way in my humble opinion -is to shorten your data via filters and then sort on your will. - -So now you know you can intercept the way filters and sorters are applied and add your custom behaviours to it. +An interceptor is where you take control of *how* filters and sorters get applied, beyond the default +one-at-a-time behavior. .. _adapterbenefit: Params Adapter layer -^^^^^^^^^^^^^^^^^^^^ - -Everyone implements filter/sorter/paginator layers at their client site differently. For example stackoverflow🧐: - -.. image:: https://drive.google.com/uc?export=view&id=1X1DiX7zRhnmJfw-t71Vgk4jnKVIExJzP - :width: 500 - :alt: Stockoverflow client site params study +^^^^^^^^^^^^^^^^^^^^^ -You might have a different approach, which is perfectly fine. This is where you can use FastAPI Listing to adjust to the -parameters of your client's site by utilizing ``CoreListingParamsAdapter`` 🤓. With this, you can access your HTTP request -object and parse the query parameters in a way that FastAPI Listing can comprehend. +Every client encodes filter/sort/pagination parameters a little differently. See, for example, this +`Stack Overflow discussion `_ +of how differently teams approach it. -FastAPI Listing uses ``sort``, ``filter`` and ``pagination`` as keys for the adapter. The adapter should then return the -translated parameters signaled at the native level. +Whatever convention your client already uses, ``CoreListingParamsAdapter`` lets FastAPI Listing adapt to +it: read the raw HTTP request, and translate its query parameters into the shape FastAPI Listing expects +natively. -Now, you may wonder how FastAPI Listing natively understands the mentioned parameters: +FastAPI Listing looks for three keys - ``sort``, ``filter``, and ``pagination`` - and the adapter is +responsible for returning them translated into the native shape: -- Filter: ``[{"field": "", "value": {"search": ""}}]`` - This represents a list of filters applied by clients, where multiple filters can be applied simultaneously. -- Sort: ``[{"field": "", "type": ""}]`` - This indicates a list of sorting instructions. While the default supports single sorting (as explained above), customization is possible. -- Pagination: ``{"pageSize": "", "page": ""}`` - These are pagination parameters that support dynamic resizing of the page. +- **Filter**: ``[{"field": "", "value": {"search": ""}}]`` - a list of filters, any number of which may be applied together. +- **Sort**: ``[{"field": "", "type": ""}]`` - a list of sort instructions (single-field sort by default; customisable). +- **Pagination**: ``{"pageSize": "", "page": ""}`` - supports dynamic page sizing. -This feature proves immensely beneficial for user with existing operational services seeking an enhanced solution to manage -their current codebase. By leveraging this library, user can potentially integrate it without necessitating modificatin to their remote client -site code. Consequently, FastAPI Listing Service can seamlessly adapt to their requirements. +This is particularly useful if you're adding FastAPI Listing to an existing service: you can adopt it +without changing anything on the client side. -Moreover, Filters also provide varying semantics for parameters based on ranges and list. +Filters also support range- and list-based semantics beyond a single value - see :doc:`filters`. Conclusion ---------- -That's it folks that's all for the theory. If you were able to come this far I believe you have a basic understanding of all the components. -In the next section we will start with Tutorials. +That covers the theory. With a basic understanding of each component, you're ready for the tutorial, +which walks through building a listing API end to end. diff --git a/docs/filters.rst b/docs/filters.rst index 63aa2c1..dd5a71d 100644 --- a/docs/filters.rst +++ b/docs/filters.rst @@ -3,11 +3,10 @@ Adding Filters to your listing API ---------------------------------- -The most interesting part of a listing that becomes the most hated part of any listing super easily. +Filtering is the most-used part of a listing API, and also the part most likely to turn messy without +some discipline. -Starting with an easy request. - -Adding a filter that will filter your employee listing on basis of ``gender``. +Start with a simple request: filter the employee listing by ``gender``. .. code-block:: python :emphasize-lines: 1, 7 @@ -22,82 +21,79 @@ Adding a filter that will filter your employee listing on basis of ``gender``. "gdr": ("Employee.gender", generic_filters.EqualityFilter), } - # rest of the definition is going to be same no change required. - -In above example we have imported a module ``generic_filters`` which holds some of the very commonly used query filters supported by FastAPI Listing. -These are highly reusable and support a cross model in place hook when you may wanna provide secondary model field. -There are a bunch of filters out of the box to speed up your regular listing API development.😉 + # rest of the definition is unchanged +``generic_filters`` holds the commonly used filters that ship with FastAPI Listing - reusable, and each +supports referencing a secondary model's field in place. There's a filter here for most common cases: .. list-table:: :widths: auto * - ``EqualityFilter`` - - equality filter ``a == b`` + - equality filter, ``a == b`` * - ``InEqualityFilter`` - - inequality filter ``a != b`` + - inequality filter, ``a != b`` * - ``InDataFilter`` - - ``in`` filter ``a in (b)`` + - ``in`` filter, ``a in (b)`` * - ``BetweenUnixMilliSecDateFilter`` - - best way to avoid conflict between date formate awareness. deal in unix timestamp. range filter ``between(start,end)`` + - range filter, ``between(start, end)``, over Unix timestamps - avoids ambiguity between date formats * - ``StringStartsWithFilter`` - - like filter ``a like b%`` + - like filter, ``a like b%`` * - ``StringEndsWithFilter`` - - like filter ``a like %b`` + - like filter, ``a like %b`` * - ``StringContainsFilter`` - - contains substring filter ``a like %b%``. recommended use on only small tables + - substring filter, ``a like %b%`` - recommended only on small tables * - ``StringLikeFilter`` - - string equality filter ``a like b`` + - string equality filter, ``a like b`` * - ``DataGreaterThanFilter`` - - greater than filter ``a > b`` + - greater-than filter, ``a > b`` * - ``DataGreaterThanEqualToFilter`` - - greater than equal to filter ``a >= b`` + - greater-than-or-equal filter, ``a >= b`` * - ``DataLessThanFilter`` - - less than filter a < b + - less-than filter, ``a < b`` * - ``DataLessThanEqualToFilter`` - - less than equal to filter a <= b + - less-than-or-equal filter, ``a <= b`` * - ``DataGropByElementFilter`` - - aggregation filter ``a group by b`` + - aggregation filter, ``a group by b`` * - ``DataDistinctByElementFilter`` - - distinct data filter ``distinct a`` + - distinct filter, ``distinct a`` * - ``HasFieldValue`` - - has field filter ``a is null`` or ``a is not null`` + - null-check filter, ``a is null`` or ``a is not null`` * - ``MySqlNativeDateFormateRangeFilter`` - - native date formate range filter between(a,b) + - range filter, ``between(a, b)``, over MySQL's native date format -I hope you still remember :ref:`filter_mapper ` +Recall :ref:`filter_mapper ` from the tutorial - each entry has three parts: -Each item of this mapping dict has 3 key components. +1. the key sent by the remote client +2. the tuple: -1. the key itself which will be sent in remote client request. -2. The tuple - * first item is ``model.field`` -> Field associated to primary table. The filter will be applied on it. - * second item is your filter class definition. + * first item: ``model.field`` - the field on the primary table the filter applies to + * second item: the filter class -And that's it you have successfully implemented your first filter. +That's a complete, working filter. +Aliasing your fields (the dict key) over their real names has a few concrete benefits: -Several benefits of having an alias over your actual fields as shown in the above dict key. -1. You will never expose your actual field name to the remote client which help to secure your service. -2. You will have a more cleaner looking request urls which will only make sense to software developers. -3. It will trim out the extra information exposing from urls. +1. the actual column name is never exposed to the client +2. request URLs stay short and meaningful to other developers, not database internals +3. less information leaks through the URL than would otherwise -How FastAPI Listing reads filter params: +How FastAPI Listing reads filter parameters: -* when you have a single value filter - ``[{"field": "alias<(filter mapper dict key)>", "value":{"search":}}]`` 📝 -* when you have multi value filter - ``[{"field": "alias<(filter mapper dict key)>", "value":{"list":}}]`` 📝 -* when you have a range value filter - ``[{"field": "alias<(fileter mapper dict key)>", "value":{"start":, "end":}}]`` 📝 +* single-value filter - ``[{"field": "", "value": {"search": ""}}]`` +* multi-value filter - ``[{"field": "", "value": {"list": []}}]`` +* range filter - ``[{"field": "", "value": {"start": "", "end": ""}}]`` -**If you have an existing running service that means you already have running remote client setup that will be sending different named query params for filter, then -use the :ref:`adapter` to make your existing listing service adapt to your existing code.** +Adapting an existing client's filter parameter names? See :ref:`the adapter layer `. Customising your filters ^^^^^^^^^^^^^^^^^^^^^^^^ -Using secondary model field. Lets say you wanna use a field from ``DeptEmp`` model. If you give the write your filter like this +Say you want to filter on a field from the ``DeptEmp`` model rather than the listing's primary model. A +filter written like this: .. code-block:: python @@ -105,10 +101,10 @@ Using secondary model field. Lets say you wanna use a field from ``DeptEmp`` mod "gdr": ("Employee.dept_no", generic_filters.EqualityFilter), } -it will raise an attribute error which is expected as your primary model doesnt have this field. -We have a rule to only allow a primary model plugged to our listing service. +raises an ``AttributeError``, as expected - the primary model has no such field, and only a primary +model may be attached to a listing service directly. -To allow passing secondary model field +To filter on a secondary model's field, add a resolver as a third tuple item: .. code-block:: python :emphasize-lines: 2 @@ -117,27 +113,23 @@ To allow passing secondary model field "dpt": ("Employee.DeptEmp.dept_no", generic_filters.EqualityFilter, lambda x: getattr(DeptEmp, x)) } -Lets see what extra we have in our tuple above. - -We have an extra lambda definition which tells what model field to use when this filter gets applied. -As to why I chained two model names ``Employee.DeptEmp.dept_no``? +The lambda tells the filter which model's field to use when applying it. -There is a filter factory which centrally encapsulates all application logic. It works on unique field names(So you can't provide duplicate names). -the **alias(filter mapper dict key)** could be same for multiple listing services and multiple database schema could contain same field names -but any database asks you to provide unique schema(table) name similarly we register the filter under `schema.field` name to reduce for users to always coming -up with random unique names. -Chaining the name like this shows a clear relation that from ``Employee`` to ``DeptEmp`` where field is ``dept_no``. -Though you can argue with it and still choose a different way of adding your filter field. Just make sure it is understandable. +Why the chained name, ``Employee.DeptEmp.dept_no``? Filters register centrally in a factory keyed by +field path, which must be unique - two filters can't register under the same path. The alias +(``filter_mapper`` key) can repeat across listing services, and different schemas can share column +names, but a chained name like ``Employee.DeptEmp.dept_no`` makes the relationship explicit (``Employee`` +to ``DeptEmp``, field ``dept_no``) while staying unique. You're free to use a different naming +convention, as long as it stays unique and legible. -Note that if we use filter with this query strategy :ref:`dept emp query strategy ` then only this would work. becuase our base query is aware of -``DeptEmp``. +Note that a filter like this only works if the listing's query strategy already joins in ``DeptEmp`` - +see :ref:`the dept-emp query strategy `. Writing a custom filter -^^^^^^^^^^^^^^^^^^^^^^^ - -You wanna write your own filter because FastAPI Listing default filters were unable to fulfill your use case 🥹. +^^^^^^^^^^^^^^^^^^^^^^^^ -Its easy to do as well. You wanna write a filter which does a full name scan combining first_name and last_name columns. +Sometimes the built-in filters don't cover a use case - here, a filter that scans across both +``first_name`` and ``last_name`` for a full-name match: .. code-block:: python :emphasize-lines: 2, 4, 6 @@ -149,7 +141,7 @@ Its easy to do as well. You wanna write a filter which does a full name scan com class FullNameFilter(generic_filters.CanonicalFilter): def filter(self, *, field: str = None, value: dict = None, context: QueryContext = None) -> QueryContext: - # field is not necessary here as this is a custom filter and user have full control over its implementation + # field isn't needed here - this filter has full control over its own implementation if value: emp_dao: EmployeeDao = dao_factory.create("employee", replica=True) emp_ids: list[int] = emp_dao.get_emp_ids_contain_full_name(value.get("search")) @@ -157,64 +149,58 @@ Its easy to do as well. You wanna write a filter which does a full name scan com context = context.with_native(native) return context -As you can see in above filter class we are inheriting from ``CanonicalFilter``, part of our ``generic_filters`` -module (``CommonFilterImpl`` is kept as a deprecated alias for one release if you're upgrading existing code). -In our filter class we have a single filter method with fixed signature - note the last argument is now -``context`` (a backend-agnostic ``QueryContext``) rather than a raw SQLAlchemy ``query``. When you need SQLAlchemy-specific -behaviour like a fluent ``.filter()`` chain, use ``context.native`` to reach the underlying ``Query`` and -``context.with_native(...)`` to hand the mutated query back. you will receive your filter value as a dict. -We have also used **dao factory** which allows us to use anywhere dao policy. -You basically filter your query and return it. -And just like that voila your custom filter is ready. No need to think how you will call it, this will be handled implicitly by filter mechanics(interceptor). +This inherits from ``CanonicalFilter`` (``generic_filters``); ``CommonFilterImpl`` remains as a +deprecated alias for one release if you're upgrading existing code. A custom filter implements a single +``filter`` method with a fixed signature - note the last argument is ``context``, a backend-agnostic +``QueryContext``, rather than a raw SQLAlchemy ``query``. For SQLAlchemy-specific behavior like a fluent +``.filter()`` chain, reach the underlying ``Query`` via ``context.native``, mutate it, and hand it back +via ``context.with_native(...)``. The filter's value arrives as a ``dict``. + +This example also uses the DAO factory, which lets any registered DAO be used from anywhere, not just +its own listing service. Filter, then return the (possibly rewrapped) context - the filter interceptor +calls this implicitly; there's nothing else to wire up. -Most built-in filters don't need any of this: they simply declare a canonical ``op`` (see ``fastapi_listing.ops.Op``) -and hand off to the context - ``EqualityFilter``, ``InDataFilter`` and the rest of ``generic_filters`` work -unmodified whether your DAO is backed by SQLAlchemy or a non-ORM backend like the reference ``ClickHouseDao``. -Write a custom filter with ``context.native`` only when the canonical ``Op`` vocabulary genuinely can't express -what you need. +Most built-in filters need none of this: they declare a canonical ``op`` (see ``fastapi_listing.ops.Op``) +and hand off to the context - ``EqualityFilter``, ``InDataFilter``, and the rest of ``generic_filters`` +work unmodified whether the DAO is backed by SQLAlchemy or a non-ORM backend like the reference +``ClickHouseDao``. Reach for ``context.native`` only when the canonical ``Op`` vocabulary genuinely can't +express what you need. -Why do we need an interceptor? Just bear with this example to have an idea of when you may wanna use or write your own interceptor. +Why an interceptor, and when to write one +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Lets say you have a listing of products and a mapping table where products are mapped to some groups and each group belongs to a bigger group. +An example of where a custom interceptor earns its keep: -Your mapping table looks like this +Say you have a product listing with a mapping table linking products to groups, and groups to a parent +group: .. code-block:: sql id | product_id | group_id | sub_group_id +You've added filters for group, sub-group, and product, each resolving IDs from the mapping table before +applying an ``in`` filter. So when a client applies: -You added filters for group sub group and product on your listing. You wrote your custom filters to either apply **lazy join** or resolve mapping data -and then apply the filter. So when: +* a group filter - your group filter runs +* a group and sub-group filter together - both run, the sub-group filter still seeing the group filter, since the client hasn't removed it +* group, sub-group, and product together - all three run -* A user applies Group filter - Your custom Group Filter gets called. -* A user applies Sub Group filter - Your custom SubGroup Filter gets called with above Group Filter because user hasn't removed above filter. -* A user applies Product filter with above two filters Your Product filter gets called with maybe with existing ``generic_filters.EqualityFilter`` Filter. - -Group -> Sub Group -> Product - -As the default interceptor runs in an iterative fashion which applies filter one by one you may end up getting different results. Why? lets see: - -You may try to find id of products mapped to Group A and applies filter on these ids. Perfect ✅ +The default interceptor applies filters one at a time, iteratively, which can give the wrong result here. +Consider filtering by group ``A`` and sub-group ``A_a`` together: ``select product_id from mapping where group_id = 'A';`` -and then feed these product_id into your filter via ``in`` query. - -On application of second filter you will repeat above process to find product ids and apply the filter again but wait will you receive sane results? I doubt it. ❌ - -``select product_id from mapping where sub_group_id = "A_a";`` - -First your Group Filter is called. It returned product_ids. Then your Sub Group Filter is called and it may return different product_ids -again you will feed these product_ids into your filter via ``in`` query. To avoid this you could create an advanced filter which is combination of both. -Create a custom filter where you could find product_ids with below query +feeds those product IDs into an ``in`` filter. Applying the sub-group filter next repeats the process +independently: -``select product from mapping where group_id = 'A' and sub_group_id = 'A_a';`` ✅ +``select product_id from mapping where sub_group_id = 'A_a';`` -This will give you accurate product_ids. Once you have a custom filter you could detect if these two filters are applied together -and modify their application by combining these two into one. +Each filter resolves its own product IDs and applies them separately, rather than the two constraints +being applied together - the two ``in`` filters don't compose into "products in group A AND sub-group +A_a." What's actually needed is: -Hope this gives you a more clear picture of situations where filter interceptor could play a significance role in reducing code complexity and -providing a more cleaner approach towards writing your code. +``select product_id from mapping where group_id = 'A' and sub_group_id = 'A_a';`` -I've faced situations like this in some system and to resolve such situation interceptor could be a big help. \ No newline at end of file +A custom interceptor can detect that both filters are applied together and combine them into a single +query like this one, rather than resolving each independently. This is exactly the kind of case where an +interceptor earns its complexity: reducing several dependent filters into one correct, efficient query. diff --git a/docs/index.rst b/docs/index.rst index 79462c1..e815ae2 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -6,32 +6,31 @@ Welcome to fastapi-listing documentation! ========================================= -FastAPI Listing is an advanced data listing library that works on top of `fastAPI `_ -to reduce the efforts in writing and maintaining your listing APIs by providing -a highly extensible, decoupled and reusable interface. +FastAPI Listing is a data listing library that sits on top of `FastAPI `_ +to reduce the effort of writing and maintaining listing APIs, through a small set of composable, +decoupled components rather than one large endpoint function. -**Component** based Plug & Play architecture allows you to write easy to use and more **quickly** readable block of code. -Inject dependencies or swap components as you write more and more complex logics. +Filtering, sorting, pagination, and query construction are each their own **component** with a well +defined contract. Compose the defaults for the common case, or swap any one of them out for a custom +implementation without touching the rest. -It ships with SQLAlchemy support out of the box, but the Filter/Sorter/Paginator/QueryStrategy contracts are -backend-agnostic (see the *Customising your listing query* guide) - a non-ORM ClickHouse backend ships as a -reference implementation proving the same abstraction works for raw parameterized SQL too, and the same -approach extends to other ORMs/database toolkits. +It ships with SQLAlchemy support out of the box, but the Filter/Sorter/Paginator/QueryStrategy contracts +are backend-agnostic (see :doc:`query`) - a non-ORM ClickHouse backend ships as a reference implementation +proving the same abstraction works for raw parameterized SQL too, and the same approach extends to other +ORMs or database toolkits. Features -------- -* **Component Based Architecture**: Small collection of independent instructions. Easy to create and attach. -* **Maintenance**: Fast to code and maintain, Light weight components are easy to create in case of multiple development iteration/customisations. -* **Fewer Bugs**: Reduce the amount of bugs by always having single responsibility modules, Focus on one sub problem at a time to solve the bigger one. -* **Easy**: Designed to be easy to use and never having the need to extend core modules. -* **Short**: Minimize code duplication. -* **Filters**: A predefined set of filters. Create new one or extend existing ones. An approach Inspired by **django admin**. Allows you to write powerful robust and reusable filters. -* **Backport Compatibility**: Level up your existing listing APIs by using FastAPI Listing without changing any client site dependency utilizing adapters. -* **Anywhere Dao objects**: Dao object powered by sqlalchemy sessions are just an import away. Use them anywhere to interact with database. - -Having some knowledge of design patterns such as strategy pattern, adapter pattern and solid principles could be a plus going forward in this documentation 📚️. +* **Component-based architecture** - independent, single-responsibility pieces that are easy to create, test, and attach. +* **Fewer bugs by construction** - each component does one thing, so a change in one rarely ripples into the others. +* **No core modules to extend** - customisation happens by writing new components, not by subclassing internals. +* **A predefined set of filters** - inspired by Django admin's approach to writing and maintaining filters; create your own alongside the built-ins. +* **Backward compatibility via adapters** - adapt FastAPI Listing to an existing client's query-param format without changing the client. +* **DAO objects usable anywhere** - import a registered DAO directly wherever you need database access, not just inside a listing endpoint. +Some familiarity with the strategy and adapter patterns, and with SOLID principles generally, will make +this documentation easier to follow, though it isn't required. The manual ---------- diff --git a/docs/paginator.rst b/docs/paginator.rst index 0f83972..c459004 100644 --- a/docs/paginator.rst +++ b/docs/paginator.rst @@ -1,15 +1,10 @@ - - Customising Paginator Strategy ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -We have a default pagination class. Which handles slicing of our data into pages with variable size. The provided pagination ``class`` -is simple and gets the work done. If you wanna write your own efficient paginating strategy for huge tables or any other use case -you could write one by extending existing base or abstract paginating strategy ``class``. - -For example you may wanna implement a paginating strategy which works on range ids for huge tables or only `previous` `next` pagination strategy and avoid -any count query. - +The default pagination strategy slices data into variable-sized pages and covers most cases. For a huge +table, or a use case the default doesn't fit, extend the base paginating strategy to write your own - +for example, a keyset/range-based strategy, or a "previous"/"next" style paginator that avoids a count +query entirely. .. code-block:: python :emphasize-lines: 3, 4 @@ -17,12 +12,11 @@ any count query. @loader.register() class EmployeeListingService(ListingService): paginate_strategy: str = "default_paginator" - default_page_size: int = 10 # default page size modify this to change default page size. - + default_page_size: int = 10 # change to alter the default page size Post-fetch business logic -------------------------- +-------------------------- Not everything belongs in the query. Filling in zero-value rows for missing time buckets, a tie-break re-sort that can't be expressed in SQL, reshaping rows differently for CSV export than for the JSON @@ -47,10 +41,10 @@ are assembled into the response. Do post-fetch business logic here, not by overr .. _alias overview: -Why use alias -------------- +Why use an alias +----------------- -* Avoid giving away original column names at client level. A steps towards securing and maintaining abstraction at api level. -* Shorter alias names are light weight. payload looks more friendly. -* Saves a little bit of bandwidth by saving communicating some extra characters. -* save coding time with shorter keys. \ No newline at end of file +* Avoids exposing real column names to the client - a small step toward keeping the API's abstraction boundary intact. +* Shorter aliases keep response payloads lighter. +* Saves a little bandwidth by not sending longer key names. +* Saves coding time with shorter keys. diff --git a/docs/query.rst b/docs/query.rst index 3f73892..001ee0f 100644 --- a/docs/query.rst +++ b/docs/query.rst @@ -1,55 +1,47 @@ -Customising your listing query +Customising your listing query ------------------------------- -By default FastAPI Listing prepares simple queries which may look like: +By default, FastAPI Listing prepares a simple query, roughly: -``select a,b,c,d from table`` +``select a, b, c, d from table`` -where ``a,b,c,d`` are columns that you provide either via pydantic serializer or as a list of strings. +where ``a, b, c, d`` are the columns you provide, either via a Pydantic serializer or as a list of +strings. -Remember this? +Recall from the tutorial: ``FastapiListing(self.request, self.dao, pydantic_serializer=EmployeeListindDetail).get_response(self.MetaInfo(self))`` ``FastapiListing(self.request, self.dao, fields_to_fetch=['a', 'b', 'c', 'd']).get_response(self.MetaInfo(self))`` -core ``class`` invokes ``get_default_read`` to prepare above mentioned vanilla query. You can easily overwrite this method -in your dao class to write your custom query. +Internally, this calls ``get_default_read`` on your DAO to build that query. Override it on your own DAO +class to write a custom query instead - ``pydantic_serializer``/``fields_to_fetch`` become optional once +you're building the query yourself. -You can either pass ``pydantic_serializer``/``fields_to_fetch`` or not as you will be writing custom ``query``. +Advanced guide for generating listing queries +----------------------------------------------- -Advanced guide for generating listing query -------------------------------------------- +Most non-trivial listing APIs need more than one query, chosen based on context - and getting this wrong +tends to be where listing API performance actually breaks down. -Most of the time you will be writing your own custom optimised queries for retrieving listing data and it isn't unusual to write -multiple queries that gets fired on different context. +A representative example: users belong to different roles, and each role should only see a subset of +the data. Every listing request needs to answer two questions: -A brief example could be: +1. what role does the logged-in user have? +2. which data layer does that role's data live in? -You have a system where users are grouped together in different roles. Each group of user are separated on -different layer of data levels so you need to check two thing in every listing API call +Different roles may need meaningfully different queries - some simple, some more involved, some backed +by a cache. As covered in the basics, :ref:`query strategies ` are how you encapsulate this, +keeping query construction separate from the rest of the service. -1. What role logged in user have, +First example: context-based switching at the service level +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -2. On which data layer the user lies and show only relevant or allowed data, - -To tackle this situation you may wanna write different query for each group of users. -Some queries may look simple some may look advanced some may even corporate caching layer. -This part could easily kill your listing API performance if not handled well or a small change could induce huge errors. - -Going back to the topic. - -As mentioned in the basics section you can create :ref:`strategies` encapsulating query generation logics and abstracting query preparation from rest of the code. - -First Example -^^^^^^^^^^^^^ - -Lets say you have a dept manager table +Say you have a department-manager table: .. code-block:: python - class DeptManager(Base): __tablename__ = 'dept_manager' @@ -62,14 +54,11 @@ Lets say you have a dept manager table department = relationship('Department') employee = relationship('Employee') - -Whenever department managers logs into the app they should only see employees who are associated to them (engineering department manager should only see engineering staff) - -Writing your own query strategy +A department manager should only see employees in their own department (an engineering manager sees +engineering staff, and nothing else). Here's a query strategy for that: .. code-block:: python - from fastapi_listing.strategies import QueryStrategy from fastapi_listing.factory import strategy_factory @@ -78,22 +67,20 @@ Writing your own query strategy def get_query(self, *, request: FastapiRequest = None, dao: EmployeeDao = None, extra_context: dict = None) -> QueryContext: - # as request and dao args are self explanatory - # extra_context is a chained variable that can carry contextual data from one place - # to another place. extremely helpful when passing args from router or client. - dept_no: str = dept_no # assuming we found dept no of logged in user - return dao.get_employees_by_dept(dept_no) # method defined in dao class + # extra_context threads contextual data from one stage of the pipeline to + # another - handy for passing values in from the router or the client + dept_no: str = dept_no # assume we've already resolved the logged-in user's dept_no + return dao.get_employees_by_dept(dept_no) # defined on the DAO below - # it is important to register your strategy with factory for use. - strategy_factory.register("", DepartmentWiseEmployeesQuery) + # strategies must be registered with the factory before use + strategy_factory.register_strategy("", DepartmentWiseEmployeesQuery) .. _dept_emp_q_stg: -Add your new listing query to employee dao +Add the corresponding method to the employee DAO: .. code-block:: python - from fastapi_listing.context.sqlalchemy import SqlAlchemyQueryContext class EmployeeDao(ClassicDao): @@ -101,12 +88,13 @@ Add your new listing query to employee dao model = Employee def get_employees_by_dept(self, dept_no: str) -> SqlAlchemyQueryContext: - # assuming we have one to one mapping and we are passing manager department here + # assumes a one-to-one mapping; dept_no here is the manager's own department query = self._read_db.query(self.model ).join(DeptEmp, Employee.emp_no == DeptEmp.emp_no ).filter(DeptEmp.dept_no == dept_no) return SqlAlchemyQueryContext(query) +Then switch to it at the service level, based on context: .. code-block:: python :emphasize-lines: 9 @@ -116,55 +104,50 @@ Add your new listing query to employee dao default_srt_on = "Employee.emp_no" default_dao = EmployeeDao - query_strategy = "default_query" # strategy chosen in case runtime switch condition not satisfied + query_strategy = "default_query" # used unless the switch below fires def get_listing(self): - if user == manager: # imaginary conditions - self.switch("query_strategy","") # switch strategy on the fly on object/request level + if user == manager: # illustrative condition + self.switch("query_strategy", "") # switch strategy for this request resp = FastapiListing(self.request, self.dao).get_response(self.MetaInfo(self)) return resp -In above example I have decided to make a switch for query strategy at runtime. So whenever a department manager logs in ``query_strategy`` will be -switched to fetch relative data and whenever other user logs in they will see global data because you have a default ``query_strategy`` placed as well. Lets call it context based switching. - -Second Example -^^^^^^^^^^^^^^ +Here, the switch happens at the service level: a department manager gets the department-scoped query, +every other user gets the default. Call this context-based switching. -1. **Different Ways to Handle Queries:** +Second example: encapsulating the switch inside the strategy +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - If you want to deal with context based switching separately, you can encapsulate logic in a single strategy class. Add instructions to generate context based queries. Inject this class into your listing service ``default_strategy = ``. +If you'd rather keep context-based branching out of the service entirely, put it inside a single +strategy class instead, and inject that strategy as ``query_strategy``: .. code-block:: python from fastapi_listing.strategies import QueryStrategy from fastapi_listing.factory import strategy_factory - from sqlalchemy.orm import Query + from fastapi_listing.context import QueryContext class EmployeesQuery(QueryStrategy): def get_query(self, *, request: FastapiRequest = None, dao: EmployeeDao = None, - extra_context: dict = None) -> Query: - # assuming in this scope we know about logged in user - user = logged_in_user + extra_context: dict = None) -> QueryContext: + user = logged_in_user # assume this scope has access to the logged-in user match user.role: - case "manager" : - query = self.get_manager_query(user) - ... # you define other contexts like manager - ... - ... - case _" : #encountering any unknown context return empty query - query = dao.get_empty_query() # defined in classic dao + case "manager": + query = self.get_manager_query(user, dao) + # ... other roles handled the same way + case _: + query = dao.get_empty_query() # any unrecognised role gets nothing back return query - def get_manager_query(self, user, dao) -> Query: - # assuming we have a way to get dept_no + def get_manager_query(self, user, dao) -> QueryContext: dept_no = dao.get_dept_no_via_user(user) return dao.get_employees_by_dept(dept_no) - # it is important to register your strategy with factory for use. - strategy_factory.register("", EmployeesQuery) + # strategies must be registered with the factory before use + strategy_factory.register_strategy("", EmployeesQuery) .. code-block:: python @@ -172,27 +155,23 @@ Second Example default_srt_on = "Employee.emp_no" default_dao = EmployeeDao - query_strategy = "" + query_strategy = "" def get_listing(self): - # if user == manager: # imaginary conditions - # self.switch("query_strategy","") # switch strategy on the fly on object/request level - - # we made our query strategy class to exhibit different behaviour no need of above code + # the strategy itself now handles context - no switch call needed here resp = FastapiListing(self.request, self.dao).get_response(self.MetaInfo(self)) return resp -2. **Two Approaches for Query Handling:** - - Some people might want to decide which query method to use right where the service is like we did in first example. They like to keep the way queries work separate and simple. They can use ``switch`` to easily switch between different methods. +Which approach to use +^^^^^^^^^^^^^^^^^^^^^^ -3. **Choosing the Right Approach:** +Both are valid; it comes down to where you'd rather keep the branching logic: - It's completely a users choice to make their objects behave in a certain way. FastAPI Listing is capable of adhering to users need 😍 whether you wanna keep your context based switching at service level - or at strategy level (query strategy class) inject it in your listing service as mention in first point and make your query strategy ``object`` capable of behaving context wise. +* keep it at the service level (first example) if you like seeing the switch happen right where the service is defined +* keep it inside the strategy (second example) if you'd rather the service stay simple and let the strategy object handle context on its own -Personally I mixes both of these when I know strategies are going to be simple I tend to make strategy objects capable of handlind different contexts but -when I know or see my single strategy class is becoming hard to maintain I tend to breakdown them to handle specefic context at a time as a result having -single responsibility objects. +In practice, a mix often works best: when a strategy is simple, let it handle its own context; once a +single strategy class becomes hard to follow, split it into one class per context so each stays focused +on a single responsibility. Backend-agnostic query objects (SQLAlchemy is no longer the only option) -------------------------------------------------------------------------- @@ -287,4 +266,4 @@ has an escape hatch for each level of "not simple enough": The rule of thumb across all four: reach for the narrowest escape hatch that solves your problem. Need a different clause or expression for one filter/one sort? Use ``having``/``order_by_raw``/ ``add_raw_condition``. Need a fundamentally different query shape (CTEs, joins, a table function)? -``from_raw_sql`` is the one that hands you full control. \ No newline at end of file +``from_raw_sql`` is the one that hands you full control. diff --git a/docs/sorter.rst b/docs/sorter.rst index f0538e1..003b1de 100644 --- a/docs/sorter.rst +++ b/docs/sorter.rst @@ -1,28 +1,26 @@ - - Adding Sorters to your listing API ---------------------------------- -This part is simple. As we leave it in the hand of db to sort our data in its own cluster FastAPI listing provides a strategy class -to apply sort on our listing query. +Sorting is left to the database, so this is simple - FastAPI Listing provides a strategy class that +applies a sort order to your listing query. .. code-block:: python :emphasize-lines: 3, 4, 5 @loader.register() class EmployeeListingService(ListingService): - default_srt_ord: str = "dsc" # change the value to asc if you want ascending order. default value is dsc for latest data. - default_srt_on = "Employee.emp_no" # default sorting field used when no loading listing with no sorting parameter. + default_srt_ord: str = "dsc" # "asc" for ascending; "dsc" is the default (latest data first) + default_srt_on = "Employee.emp_no" # field used when the request specifies no sort parameter sort_mapper = { "empid": "emp_no", } -``sort_mapper`` is similar to ``filter_mapper`` where ``empid`` is what remote client sends and ``emp_no`` is what gets used to sort our dataset. -it is a collection of allowed sorting parameters. +``sort_mapper`` works like ``filter_mapper``: ``empid`` is what the remote client sends, ``emp_no`` is +the field actually used to sort. It's the set of sort fields a client is allowed to request. -If using primary model you could use it just like shown above. +Sorting on the primary model looks like the example above. -Or if sorting is implemented on joined table field and like filter mapper +To sort on a joined table's field, add a resolver just like you would for a filter: .. code-block:: python :emphasize-lines: 2 @@ -31,15 +29,58 @@ Or if sorting is implemented on joined table field and like filter mapper "deptno": ("dept_no", lambda x: getattr(DeptEmp, x)) } -like filter mapper there is no central sorter factory. As we leave the heavy lifting to DB. so there is no need to provide unique field names for registration purpose. -Although its better to use ``model.field`` convention like we used in filter mapper to keep the similarity. +Unlike filters, there's no central sorter factory requiring unique names - since sorting is delegated +entirely to the database, there's no registration step to worry about. Using the ``model.field`` +convention is still recommended for consistency with ``filter_mapper``. + +Just like the filter interceptor, a sorter interceptor lets you override the default one-field-at-a-time +sort behavior and apply your own multi-field sorting logic. + +Sorting on a computed or aggregated field (e.g. a CTE) +-------------------------------------------------------- + +``sort_mapper``'s callable form (above) resolves a field once, at class-definition time - that works for +a joined table's column, but not for a column that only exists on a query built per-request, such as a +CTE aggregating a metric. For that, write a custom sorting strategy that reads the column back from +``extra_context`` instead of the static model - stash it there from your ``QueryStrategy`` when you build +the CTE, since both ``get_query()`` and ``sort()`` receive the same ``extra_context`` dict for a given +request: + +.. code-block:: python + + from fastapi_listing.abstracts import AbsQueryStrategy, AbsSortingStrategy + from fastapi_listing.context import QueryContext + from fastapi_listing.factory import strategy_factory + + class EmployeeMetricsQueryStrategy(AbsQueryStrategy): + def get_query(self, *, request=None, dao=None, extra_context=None) -> QueryContext: + context = dao.get_default_read(...) # builds/joins your CTE + extra_context["metrics_cte"] = ... # keep a handle to the CTE for the sort stage + return context + + class ComputedFieldSortingStrategy(AbsSortingStrategy): + def sort(self, *, context: QueryContext = None, value=None, extra_context=None) -> QueryContext: + cte = extra_context["metrics_cte"] + column = getattr(cte.c, value["field"]) + return context.order_by(field=column, direction=value["type"]) + + strategy_factory.register_strategy("employee_metrics_query", EmployeeMetricsQueryStrategy) + strategy_factory.register_strategy("computed_field_sorter", ComputedFieldSortingStrategy) + + @loader.register() + class EmployeeListingService(ListingService): + query_strategy = "employee_metrics_query" + sorting_strategy = "computed_field_sorter" + sort_mapper = { + "indexedpages": "total_indexed_pages", + } -Just like filter interceptor you also have an option of sorter interceptor where you could interrupt the default behaviour of applying sort on your query -and customise how you may wanna apply multi field sorting on your query. +``context.order_by()`` works with any SQLAlchemy column-like object - a CTE's labeled column included, +not just a mapped model attribute - so no library change is needed to sort on one; only a sorting +strategy that knows where to find it. -How FastAPI Listing reads sorter params: +How FastAPI Listing reads sort parameters: -``[{"field":"alias", "type":"asc"}]`` or ``[{"field":"alias", "type":"dsc"}]`` 📝 +``[{"field": "alias", "type": "asc"}]`` or ``[{"field": "alias", "type": "dsc"}]`` -**If you have an existing running service that means you already have running remote client setup that will be sending different named query params for filter, then -use the** :ref:`adapter ` **to make your existing listing service adapt to your existing code.** \ No newline at end of file +Adapting an existing client's sort parameter names? See :ref:`the adapter layer `. diff --git a/docs/tutorials.rst b/docs/tutorials.rst index 6df4ca8..4104b9e 100644 --- a/docs/tutorials.rst +++ b/docs/tutorials.rst @@ -4,18 +4,14 @@ Tutorials Preparations ------------ -A simple example showing how easy it is to get started. Lets look at a little bit of context for better understanding. - -**Note** First lets setup the app to use the library - -Your project structure may differ, but all FastAPI related flow is similar in context. - -I'll be using the following structure for this tutorial: +A walkthrough of building a listing API end to end. Your project layout may differ, but the FastAPI +wiring is the same regardless. +This tutorial uses the following structure: .. parsed-literal:: - employess + employees \|-- app | \|-- __init__.py | \|-- :ref:`dao` @@ -51,17 +47,16 @@ I'll be using the following structure for this tutorial: \|-- main.py \`-- requirements.txt -Lets call this app **employees** +Call this app **employees**. models ------ -model classes +Model classes: .. code-block:: python - class Employee(Base): __tablename__ = 'employees' @@ -96,39 +91,36 @@ model classes Dao --- -Here we have a local dao package where we will be adding all our :ref:`Dao` classes. -I've additionally added a package to keep generic methods for common use, e.g. ``dao_generics.py`` file which looks something -like this + +A local ``dao`` package holds every :ref:`DAO ` class. It's also common to keep a small +package of generic, reusable DAO helpers - here, ``dao_generics.py``: .. code-block:: python + import sqlalchemy from fastapi_listing.dao import GenericDao + from fastapi_listing.context.sqlalchemy import SqlAlchemyQueryContext class ClassicDao(GenericDao): # noqa """ - Not to be used directly as this class is missing required attributes 'model' and 'name' to be given by users. - model class is given when we are linking a new dao class with a new model/table - name is dao name that a user will use to invoke dao objects. + Not meant to be used directly - it's missing the required 'model' and 'name' + attributes, which a concrete subclass provides when binding to a model/table. """ def check_pk_exist(self, id: int | str) -> bool: - # check if id exists in linked dao table - return self._read_db.query(self._read_db.query(self.model - ).filter(self.model.id == id).exists()).scalar() - + return self._read_db.query( + self._read_db.query(self.model).filter(self.model.id == id).exists() + ).scalar() - def get_empty_query(self): - # return empty query - return self._read_db.query(self.model).filter(sqlalchemy.sql.false()) + def get_empty_query(self) -> SqlAlchemyQueryContext: + return SqlAlchemyQueryContext(self._read_db.query(self.model).filter(sqlalchemy.sql.false())) -Dao classes +Concrete DAO classes, one per model, each in their own module: .. code-block:: python - - # each dao will be placed in their own module/files from fastapi_listing.dao import dao_factory from app.dao import ClassicDao @@ -137,29 +129,28 @@ Dao classes name = "title" model = Title - dao_factory.register_dao(TitleDao.name, TitleDao) # registering dao with app to use anywhere + dao_factory.register_dao(TitleDao.name, TitleDao) # makes the DAO usable anywhere via dao_factory class EmployeeDao(ClassicDao): name = "employee" model = Employee - dao_factory.register_dao(EmployeeDao.name, EmployeeDao) # registering dao with app to use anywhere + dao_factory.register_dao(EmployeeDao.name, EmployeeDao) class DeptEmpDao(ClassicDao): name = "deptemp" model = DeptEmp - dao_factory.register_dao(DeptEmpDao.name, DeptEmpDao) # registering dao with app to use anywhere + dao_factory.register_dao(DeptEmpDao.name, DeptEmpDao) schema ------ -Response Schema (Support for pydantic 2 is added.) +Response schema (Pydantic v2 is supported): .. code-block:: python - class GenderEnum(enum.Enum): MALE = "M" FEMALE = "F" @@ -179,17 +170,16 @@ Response Schema (Support for pydantic 2 is added.) main ---- -Add middleware at main file + +Add the session-binding middleware in your main file: .. code-block:: python :emphasize-lines: 17 def get_db() -> Session: """ - replicating sessionmaker for any fastapi app. - anyone could be using a different way or opensource packages like fastapi-sqlalchemy - it all comes down to a single result that is yielding a session. - for the sake of simplicity and testing purpose I'm replicating this behaviour in this naive way. + Stand-in for a sessionmaker. Use whatever gives you a Session - + fastapi-sqlalchemy or your own factory both work the same way here. :return: Session """ engine = create_engine("mysql://root:123456@127.0.0.1:3307/employees", pool_pre_ping=1) @@ -198,23 +188,23 @@ Add middleware at main file app = FastAPI() - # fastapi-listing middleware offering anywhere dao usage policy. Just like anywhere door use sessions and dao - # anywhere in your code via single import. + # DaoSessionBinderMiddleware makes a registered dao usable anywhere via a + # single import, without threading a session through every function call. - # if you have a master slave architecture + # if you have a primary/replica architecture: app.add_middleware(DaoSessionBinderMiddleware, master=get_db, replica=get_db) - # if you have only a master database + # if you have a single database: app.add_middleware(DaoSessionBinderMiddleware, master=get_db) - # if you want fastapi listing to close session when returning a response + # if you want FastAPI Listing to close the session before returning the response: app.add_middleware(DaoSessionBinderMiddleware, master=get_db, session_close_implicit=True) router ------ -Write abstract listing api routers with FastAPI Listing. -calling listing endpoint from routers +Write listing endpoint routers with FastAPI Listing - calling the listing endpoint from a router looks +like this: .. code-block:: python :emphasize-lines: 1, 5, 8 @@ -228,14 +218,14 @@ calling listing endpoint from routers resp = EmployeeListingService(request).get_listing() return resp -service definition is given in below. +The service definition follows below. .. _service: -Writing your very first listing API using fastapi-listing ---------------------------------------------------------- +Writing your first listing API with FastAPI Listing +----------------------------------------------------- .. code-block:: python :emphasize-lines: 1, 6, 10, 13, 14 @@ -243,10 +233,10 @@ Writing your very first listing API using fastapi-listing from fastapi_listing import ListingService, FastapiListing, loader from app.dao import EmployeeDao - from app.schema.response.employee_responses import EmployeeListDetails # optional + from app.schema.response.employee_responses import EmployeeListDetails # optional - @loader.register() # run system checks to validate your listing service + @loader.register() # validates the listing service's semantics at startup class EmployeeListingService(ListingService): default_srt_on = "Employee.emp_no" @@ -257,30 +247,28 @@ Writing your very first listing API using fastapi-listing ).get_response(self.MetaInfo(self)) return resp - # that's it your very first listing api is ready to be serverd. - # + # that's it - the first listing API is ready to serve. -You actually began writing your listing API here. Before this everything was vanilla FastAPI code excluding doa setup 🤠 +Everything before this point was plain FastAPI/DAO setup; this is where the listing API itself begins. -* **loader**: A utility decorator used on startup when classes gets loaded into the memory validates the semantics also helps to identify any abnormality within - your defined listing class. -* **ListingService**: High level base class. All Listing Service classes will extend this. -* **Attributes**: :ref:`attributes overview` -* **EmployeeListDetails**: Optional pydantic class containing required fields to render. These field will get added automatically in vanilla query. - if you are not using pydantic then you could leave it or use list of fields. -* **get_listing**: High level function, entrypoint for listing service. -* **FastapiListing**: Low level class that you will only use as an expression which returns a result. Extending this is forbidden. +* **loader**: a startup-time decorator that validates a listing service's semantics and flags mistakes early, rather than at request time. +* **ListingService**: the base class every listing service extends. +* **Attributes**: see :ref:`attributes overview`. +* **EmployeeListDetails**: an optional Pydantic class listing the fields to render; these are added to the query automatically. Without Pydantic, pass a plain list of field names instead. +* **get_listing**: the entry point for the listing service. +* **FastapiListing**: a low-level class used as an expression that returns a result - not meant to be subclassed. -Once you runserver, hit the endpoint ``localhost:8000/v1/employees`` and you will receive a json response with page size 10 (default page size). +Start the server and hit ``localhost:8000/v1/employees`` to get a JSON response with 10 items (the +default page size). .. _attributes overview: ``ListingService`` high level attributes ----------------------------------------- +------------------------------------------ -This library is divided down to fundamental level blocks of any listing API, You can create these blocks independent from each other -inject them into your listing service and their composition will communicate implicitly so you can focus more on writing solutions and leave their communication on the core service. +Each of these blocks - filter, sort, pagination, query - is independent and composes implicitly through +the core service, so you can focus on the logic of each rather than how they're wired together. .. py:currentmodule:: fastapi_listing.service.listing_main @@ -288,80 +276,77 @@ inject them into your listing service and their composition will communicate imp .. py:attribute:: ListingService.filter_mapper - A ``dict`` containing allowed filters on the listing. ``{alias: value}`` where key should be an alias of field and value is - a tuple. You can use actual field names in place of alias its a matter of personal preferrence 🤓 + A ``dict`` of allowed filters: ``{alias: value}``, where the key is an alias for the field (or the + field name itself, if you'd rather not alias it) and the value is a tuple. - for example: ``{"fnm": ("Employees.first_name", filter_class)}`` + Example: ``{"fnm": ("Employees.first_name", filter_class)}`` - value ``"Employees.first_name"`` shows relation. ``first_name`` from primary model ``Employees``. - This should always be unique. You could go sane defining your values - like this which will help you when debugging. + ``"Employees.first_name"`` shows the relation - ``first_name`` on the primary model, ``Employees``. + This value should always be unique; keeping it descriptive like this also helps when debugging. - alias/filter field will be sent in request by clients. for those who directly jumped here🤯 checkout :ref:`basics adapter layer` first - to see how FastAPI Listing is capable of adapting to your existing clients without any modification. + The alias is what the client sends. If you're adapting an existing client's parameters rather than + starting fresh, see :ref:`the adapter layer ` first. -For customising the behaviour you can check out customisation section ✏️. - -:ref:`alias overview`? +See :ref:`alias overview` for why aliasing is worth doing in the first place. .. py:attribute:: ListingService.sort_mapper - A ``dict`` containing allowed sorting on the listing. + A ``dict`` of allowed sort fields. - for example: ``{"empno": "Employees.emp_no"}`` + Example: ``{"empno": "Employees.emp_no"}`` - sorter alias/fields will be sent in request by clients and you know FastAPI Listing can :ref:`adapt` to them. + As with filters, sort aliases can be adapted to an existing client via :ref:`the adapter layer `. .. py:attribute:: ListingService.default_srt_on - attribute provides field name used to sort listing item by default + The field to sort by when the request specifies no sort parameter. .. py:attribute:: ListingService.default_srt_ord - attributes provides sorting order, allowed ``asc`` and ``dsc`` 📝. + The default sort order: ``asc`` or ``dsc``. .. py:attribute:: ListingService.paginate_strategy - attribute provides pagination strategy name used by listing service to apply pagination on query. - Default strategy - ``default_paginator`` + The pagination strategy name. + Default: ``default_paginator``. .. py:attribute:: ListingService.query_strategy - attribute provides query strategy name, used to get base query for your listing service. - Default strategy - ``default_query`` + The query strategy name, used to build the base query. + Default: ``default_query``. .. py:attribute:: ListingService.sorting_strategy - attribute provides sorting strategy name, used to apply sorting on your base query. - Default strategy - ``default_sorter`` + The sort strategy name, used to apply sorting to the base query. + Default: ``default_sorter``. .. py:attribute:: ListingService.sort_mecha - attribute provides interceptor name. :ref:`interceptors` ❓️ - Default interceptor - ``indi_sorter_interceptor`` + The sort interceptor name - see :ref:`interceptors `. + Default: ``indi_sorter_interceptor``. .. py:attribute:: ListingService.filter_mecha - attribute provides interceptor name. :ref:`interceptors` ❓️ - Default interceptor - ``iterative_filter_interceptor`` + The filter interceptor name - see :ref:`interceptors `. + Default: ``iterative_filter_interceptor``. .. py:attribute:: ListingService.default_dao - provides listing service :ref:`dao` class. - every listing service should contain one primary doa only. You can use multiple dao/sqlalchemy models/tables in defintion via dao_factory. + The listing service's :ref:`DAO ` class. Each listing service has exactly one primary + DAO, though a DAO can reference other models/tables via ``dao_factory`` when needed. .. py:attribute:: ListingService.default_page_size - default number of items in a single page. + The default number of items per page. .. _adapter_attr: .. py:attribute:: ListingService.feature_params_adapter - default adapter to resolve issue between incompatible objects. Users are advices to design their - own adapters to support their existing remote client site filter/sorter/page params. :ref:`adapters` ❓️ - + The adapter used to reconcile an existing client's filter/sort/pagination parameter shape with + FastAPI Listing's own. Write your own to support your client's existing format - see + :ref:`the adapter layer `.