diff --git a/docs/README.md b/docs/README.md index a1ceef71..8b0edf2e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -16,18 +16,29 @@ - [Failover Plugin](using-the-python-wrapper/using-plugins/UsingTheFailoverPlugin.md) - [Failover v2 Plugin](using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md) - [Failover Configuration Guide](using-the-python-wrapper/FailoverConfigurationGuide.md) + - [Global Database Failover Plugin](using-the-python-wrapper/using-plugins/UsingTheGdbFailoverPlugin.md) - [Host Monitoring Plugin](using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md) - [Aurora Connection Tracker Plugin](using-the-python-wrapper/using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) - [Aurora Initial Connection Strategy Plugin](using-the-python-wrapper/using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) - [Host Availability Strategy](using-the-python-wrapper/HostAvailabilityStrategy.md) + - [Custom Endpoint Plugin](using-the-python-wrapper/using-plugins/UsingTheCustomEndpointPlugin.md) - [IAM Authentication Plugin](using-the-python-wrapper/using-plugins/UsingTheIamAuthenticationPlugin.md) - [AWS Secrets Manager Plugin](using-the-python-wrapper/using-plugins/UsingTheAwsSecretsManagerPlugin.md) - [Federated Authentication Plugin](using-the-python-wrapper/using-plugins/UsingTheFederatedAuthenticationPlugin.md) + - [Okta Authentication Plugin](using-the-python-wrapper/using-plugins/UsingTheOktaAuthenticationPlugin.md) - [Read Write Splitting Plugin](using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md) - [Reader Selection Strategies](using-the-python-wrapper/ReaderSelectionStrategies.md) - [Simple Read Write Splitting Plugin](using-the-python-wrapper/using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) + - [Global Database Read/Write Splitting Plugin](using-the-python-wrapper/using-plugins/UsingTheGdbReadWriteSplittingPlugin.md) - [Fastest Response Strategy Plugin](using-the-python-wrapper/using-plugins/UsingTheFastestResponseStrategyPlugin.md) + - [Blue/Green Deployment Plugin](using-the-python-wrapper/using-plugins/UsingTheBlueGreenPlugin.md) + - [Limitless Connection Plugin](using-the-python-wrapper/using-plugins/UsingTheLimitlessPlugin.md) + - [Developer Plugin](using-the-python-wrapper/using-plugins/UsingTheDeveloperPlugin.md) - [Host Availability Strategy](using-the-python-wrapper/HostAvailabilityStrategy.md) + - [Plugins Compatibility](using-the-python-wrapper/compatibility/Compatibility.md) + - [Database Type Compatibility](using-the-python-wrapper/compatibility/CompatibilityDatabaseTypes.md) + - [Database URL Type Compatibility](using-the-python-wrapper/compatibility/CompatibilityEndpoints.md) + - [Cross Plugin Compatibility](using-the-python-wrapper/compatibility/PluginChainCompatibility.md) - [Development Guide](./development-guide/DevelopmentGuide.md) - [Setup](./development-guide/DevelopmentGuide.md#setup) - [Testing Overview](./development-guide/DevelopmentGuide.md#testing-overview) diff --git a/docs/using-the-python-wrapper/PluginChainCompatibility.md b/docs/using-the-python-wrapper/PluginChainCompatibility.md deleted file mode 100644 index f41694c9..00000000 --- a/docs/using-the-python-wrapper/PluginChainCompatibility.md +++ /dev/null @@ -1,49 +0,0 @@ -# Plugin Chain Compatibility - -Some plugins depend on or conflict with others. This page captures -non-obvious pairings that aren't enforced by code but will silently -degrade behavior at runtime if violated. - -If a plugin's combination is **incompatible**, the wrapper does not raise -at construction time — the chain is accepted but behavior at runtime is -incorrect (silent fallback to a writer-only connection, monitor threads -that can't abort, etc.). Check the relevant row below before composing -a chain you haven't used before. - -## Required pairings - -| Plugin | Requires | Why | -|---------------------|-----------------------------------|-----| -| `read_write_splitting` | `failover_v2` (not `failover`) | `failover_v2` starts the cluster topology monitor on the **initial** connect. With plain `failover`, the host list stays at `{writer-only}` until a later failover signal fires, and `conn.read_only = True` flips silently fall back to the writer rather than splitting to a reader. | - -## Driver-specific incompatibilities - -| Plugin | Incompatible with | Why | -|------------------------------|------------------------------------------------|-----| -| `host_monitoring` (EFM v1) | sync MySQL (`mysql-connector-python`) | EFM requires a thread-based connection abort that `mysql-connector-python` doesn't expose. Symptoms: monitor threads hang on shutdown, "Python hangs on exit" when a host is unreachable. Use the chain **without EFM**, or switch to the async driver (`aiomysql`). | -| `host_monitoring_v2` (EFM v2) | sync MySQL (`mysql-connector-python`) | Same root cause as v1 — EFM v2 still depends on thread-based abort. | -| `iam_authentication` | MySQL `use_pure=True` (pure-Python connector) | The pure-Python connector truncates passwords at 255 chars; IAM tokens are typically longer. Expect `int1store requires 0 <= i <= 255` or `struct.error: ubyte format requires 0 <= number <= 255`. See README "Known Limitations". | - -## Recommended canonical chains - -| Use case | Sync chain | Async chain | -|----------------------------------------------------------------|---------------------------------------------------------|----------------------------------------------------------| -| Aurora PG — R/W splitting + failover + EFM | `read_write_splitting,failover_v2,host_monitoring_v2` | `read_write_splitting,failover_v2,host_monitoring_v2` | -| Aurora MySQL (sync, `mysql-connector-python`) — R/W splitting + failover | `read_write_splitting,failover_v2` *(no EFM)* | — | -| Aurora MySQL (async, `aiomysql`) — R/W splitting + failover + EFM | — | `read_write_splitting,failover_v2,host_monitoring_v2` | -| Aurora PG — failover only | `failover_v2,host_monitoring_v2` | `failover_v2,host_monitoring_v2` | -| Aurora MySQL (sync) — failover only | `failover_v2` | — | - -> **Note (async federated/Okta auth):** the async `federated_auth` and `okta` plugins perform their -> IdP HTTP round-trips with [`aiohttp`](https://docs.aiohttp.org/), which is not a runtime dependency -> of this package — install it alongside your async driver (`pip install aiohttp`) when using either -> plugin in an asyncio application. The sync plugins use `requests` and are unaffected. - -## Related docs - -- [UsingTheReadWriteSplittingPlugin.md](./using-plugins/UsingTheReadWriteSplittingPlugin.md) -- [UsingTheFailover2Plugin.md](./using-plugins/UsingTheFailover2Plugin.md) -- [UsingTheFailoverPlugin.md](./using-plugins/UsingTheFailoverPlugin.md) (v1, not recommended for new code per the table above) -- [UsingTheHostMonitoringPlugin.md](./using-plugins/UsingTheHostMonitoringPlugin.md) -- [UsingTheIamAuthenticationPlugin.md](./using-plugins/UsingTheIamAuthenticationPlugin.md) -- [FailoverConfigurationGuide.md](./FailoverConfigurationGuide.md) — retry-budget knobs at the SQLAlchemy / Django boundary diff --git a/docs/using-the-python-wrapper/compatibility/Compatibility.md b/docs/using-the-python-wrapper/compatibility/Compatibility.md new file mode 100644 index 00000000..31971f51 --- /dev/null +++ b/docs/using-the-python-wrapper/compatibility/Compatibility.md @@ -0,0 +1,23 @@ +# Plugins compatibility + +The AWS Advanced Python Wrapper uses plugins to execute database method calls. You can think of a plugin as an extensible code module that adds additional logic around driver method calls. Plugins are designed with the intention of being compatible with each other; however, there are logical constraints related to database type or database features that can make plugins inefficient in certain configurations. + +For example, RDS Single-AZ Instance deployments do not support failover, so the `failover` and `failover_v2` plugins are marked as incompatible with that deployment. If either of these plugins is included in the driver configuration, there will be no added value. However, these unnecessary plugins will function without errors and will simply consume additional resources. + +The following matrices help verify plugin compatibility with other plugins and with various database types. Some plugins are sensitive to the database URL provided in the connection string, and this is also presented below. + +We encourage users to verify their configurations and ensure that their configuration contains no incompatible components. + +- [Database type compatibility](./CompatibilityDatabaseTypes.md) +- [Database URL type compatibility](./CompatibilityEndpoints.md) +- [Cross plugin compatibility](./PluginChainCompatibility.md) — the plugin-vs-plugin matrix plus driver/runtime constraints, plugin ordering, and canonical chains + +## Universally Compatible Plugins + +The following plugins operate independently of connection management and are compatible with all plugins, database types, and endpoint types: + +| Plugin | Description | +|----------------|----------------------------------------------------| +| [dev](../using-plugins/UsingTheDeveloperPlugin.md) | Developer utility plugin for debugging and diagnostics. | +| `connect_time` | Logs the time taken to establish a connection. | +| `execute_time` | Logs the time taken to execute any driver method. | diff --git a/docs/using-the-python-wrapper/compatibility/CompatibilityDatabaseTypes.md b/docs/using-the-python-wrapper/compatibility/CompatibilityDatabaseTypes.md new file mode 100644 index 00000000..ee0297ce --- /dev/null +++ b/docs/using-the-python-wrapper/compatibility/CompatibilityDatabaseTypes.md @@ -0,0 +1,59 @@ +# Database type compatibility + +This document is part of the [Compatibility Guide](./Compatibility.md) and explains plugin compatibility with various database types and deployments. Some plugins require specific metadata from particular database types to function properly. + +For example, the `limitless` plugin is incompatible with [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) because it's built on different architectural principles than [Limitless Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/limitless-architecture.html). Aurora Global Database doesn't use transaction routers and doesn't provide the transaction routers' metadata. This lack of required metadata makes it incompatible with the `limitless` plugin. + +For Aurora Global Database configuration details, see [Aurora Global Databases](../GlobalDatabases.md). + +Legend: ✅ compatible  |  ❌ incompatible / no added value + +| Plugin code / Database type | Aurora Global Database
(MySQL and PG) | Aurora Cluster
(MySQL and PG) | RDS Multi-AZ DB Cluster (3 instances)
(MySQL and PG) | +|---|:---:|:---:|:---:| +| [custom_endpoint](../using-plugins/UsingTheCustomEndpointPlugin.md) | ✅ | ✅ | ✅ | +| [host_monitoring](../using-plugins/UsingTheHostMonitoringPlugin.md) (EFM v1) | ✅ | ✅ | ✅ | +| [host_monitoring_v2](../using-plugins/UsingTheHostMonitoringPlugin.md) (EFM v2) | ✅ | ✅ | ✅ | +| [failover](../using-plugins/UsingTheFailoverPlugin.md) | ✅ | ✅ | ✅ | +| [failover_v2](../using-plugins/UsingTheFailover2Plugin.md) | ✅ | ✅ | ✅ | +| [gdb_failover](../using-plugins/UsingTheGdbFailoverPlugin.md) | ✅ | ✅ | ✅ | +| [iam](../using-plugins/UsingTheIamAuthenticationPlugin.md) | ✅ | ✅ | ✅ | +| [aws_secrets_manager](../using-plugins/UsingTheAwsSecretsManagerPlugin.md) | ✅ | ✅ | ✅ | +| [federated_auth](../using-plugins/UsingTheFederatedAuthenticationPlugin.md) | ✅ | ✅ | ✅ | +| [okta](../using-plugins/UsingTheOktaAuthenticationPlugin.md) | ✅ | ✅ | ✅ | +| stale_dns | ✅ | ✅ | ✅ | +| [read_write_splitting](../using-plugins/UsingTheReadWriteSplittingPlugin.md) | ✅ | ✅ | ✅ | +| [srw](../using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) | ✅ | ✅ | ✅ | +| [gdb_rw](../using-plugins/UsingTheGdbReadWriteSplittingPlugin.md) | ✅ | ✅ | ✅ | +| [aurora_connection_tracker](../using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) | ✅ | ✅ | ✅ | +| connect_time | ✅ | ✅ | ✅ | +| [fastest_response_strategy](../using-plugins/UsingTheFastestResponseStrategyPlugin.md) | ✅ | ✅ | ✅ | +| [initial_connection](../using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) | ✅ | ✅ | ✅ | +| [limitless](../using-plugins/UsingTheLimitlessPlugin.md) | ❌ | ✅ (PostgreSQL only) | ✅ | +| [bg](../using-plugins/UsingTheBlueGreenPlugin.md) | ❌ | ✅ | ❌ | + +
+ +| Plugin code / Database type | RDS Multi-AZ DB Instance (2 instances)
(MySQL and PG) | RDS Single-AZ Instance (1 instance)
(MySQL and PG) | Community Database
(MySQL and PG) | +|---|:---:|:---:|:---:| +| [custom_endpoint](../using-plugins/UsingTheCustomEndpointPlugin.md) | ❌ | ❌ | ❌ | +| [host_monitoring](../using-plugins/UsingTheHostMonitoringPlugin.md) (EFM v1) | ✅ | ✅ | ✅ | +| [host_monitoring_v2](../using-plugins/UsingTheHostMonitoringPlugin.md) (EFM v2) | ✅ | ✅ | ✅ | +| [failover](../using-plugins/UsingTheFailoverPlugin.md) | ❌ | ❌ | ❌ | +| [failover_v2](../using-plugins/UsingTheFailover2Plugin.md) | ❌ | ❌ | ❌ | +| [gdb_failover](../using-plugins/UsingTheGdbFailoverPlugin.md) | ❌ | ❌ | ❌ | +| [iam](../using-plugins/UsingTheIamAuthenticationPlugin.md) | ✅ | ✅ | ❌ | +| [aws_secrets_manager](../using-plugins/UsingTheAwsSecretsManagerPlugin.md) | ✅ | ✅ | ❌ | +| [federated_auth](../using-plugins/UsingTheFederatedAuthenticationPlugin.md) | ✅ | ✅ | ❌ | +| [okta](../using-plugins/UsingTheOktaAuthenticationPlugin.md) | ✅ | ✅ | ❌ | +| stale_dns | ❌ | ❌ | ❌ | +| [read_write_splitting](../using-plugins/UsingTheReadWriteSplittingPlugin.md) | ❌ | ❌ | ❌ | +| [srw](../using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) | ✅ | ❌ | ✅ | +| [gdb_rw](../using-plugins/UsingTheGdbReadWriteSplittingPlugin.md) | ❌ | ❌ | ❌ | +| [aurora_connection_tracker](../using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) | ❌ | ❌ | ❌ | +| connect_time | ✅ | ✅ | ✅ | +| [fastest_response_strategy](../using-plugins/UsingTheFastestResponseStrategyPlugin.md) | ❌ | ❌ | ❌ | +| [initial_connection](../using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) | ❌ | ❌ | ❌ | +| [limitless](../using-plugins/UsingTheLimitlessPlugin.md) | ❌ | ❌ | ❌ | +| [bg](../using-plugins/UsingTheBlueGreenPlugin.md) | ✅ | ✅ | ❌ | + +> The `connect_time`, `execute_time`, and [`dev`](../using-plugins/UsingTheDeveloperPlugin.md) plugins are compatible with every database type (see [Universally Compatible Plugins](./Compatibility.md#universally-compatible-plugins)). diff --git a/docs/using-the-python-wrapper/compatibility/CompatibilityEndpoints.md b/docs/using-the-python-wrapper/compatibility/CompatibilityEndpoints.md new file mode 100644 index 00000000..c9e8c316 --- /dev/null +++ b/docs/using-the-python-wrapper/compatibility/CompatibilityEndpoints.md @@ -0,0 +1,124 @@ +# Database URL types compatibility + +This document is part of the [Compatibility Guide](./Compatibility.md) and explains plugin compatibility with various database endpoints. + +There are many different URL types (endpoints) that can be used with the AWS Advanced Python Wrapper, but certain URL types are not compatible with certain plugins. This page outlines the various URL types and which plugins are compatible with each type. + +- [Aurora Global Database Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database-connecting.html) - `.global-.global.rds.amazonaws.com` +- [Aurora Cluster Writer Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Endpoints.Cluster.html) - `.cluster-..rds.amazonaws.com` +- [Aurora Cluster Reader Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Endpoints.Reader.html) - `.cluster-ro-..rds.amazonaws.com` +- [Aurora Cluster Custom Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Endpoints.Custom.html) - `.cluster-custom-..rds.amazonaws.com` +- [Aurora Instance Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Endpoints.Instance.html) - `...rds.amazonaws.com` +- [RDS Multi-AZ DB Cluster Writer Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts-connection-management.html) - `.cluster-..rds.amazonaws.com` +- [RDS Multi-AZ DB Cluster Reader Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts-connection-management.html) - `.cluster-ro-..rds.amazonaws.com` +- [RDS Instance Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts-connection-management.html) - `...rds.amazonaws.com` +- [RDS Proxy Endpoint](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy-endpoints.html) - `.proxy-..rds.amazonaws.com` +- [DB Shard Group Endpoint (Aurora Limitless)](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/limitless-shard.html) - `.shardgrp-..rds.amazonaws.com` +- [IP address](https://en.wikipedia.org/wiki/IP_address) - IPv4 or IPv6 addresses, for example: `8.8.0.0` +- User custom domain (CNAME alias) - any other domain names, for example: `my-database.my-domain.com` + +Legend: ✅ compatible  |  ❌ incompatible + +
+ +| Plugin code / Database URL type | Aurora Global Database Endpoint | +|---|:---:| +| [custom_endpoint](../using-plugins/UsingTheCustomEndpointPlugin.md) | ❌ | +| [host_monitoring](../using-plugins/UsingTheHostMonitoringPlugin.md) | ✅ (requires `initial_connection` plugin) | +| [host_monitoring_v2](../using-plugins/UsingTheHostMonitoringPlugin.md) | ✅ (requires `initial_connection` plugin) | +| [failover](../using-plugins/UsingTheFailoverPlugin.md) | ✅ | +| [failover_v2](../using-plugins/UsingTheFailover2Plugin.md) | ✅ | +| [gdb_failover](../using-plugins/UsingTheGdbFailoverPlugin.md) | ✅ | +| [iam](../using-plugins/UsingTheIamAuthenticationPlugin.md) | ✅ (requires `initial_connection` plugin) | +| [aws_secrets_manager](../using-plugins/UsingTheAwsSecretsManagerPlugin.md) | ✅ | +| [federated_auth](../using-plugins/UsingTheFederatedAuthenticationPlugin.md) | ✅ | +| [okta](../using-plugins/UsingTheOktaAuthenticationPlugin.md) | ✅ | +| stale_dns | ✅ | +| [read_write_splitting](../using-plugins/UsingTheReadWriteSplittingPlugin.md) | ✅ | +| [srw](../using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) | ✅ | +| [gdb_rw](../using-plugins/UsingTheGdbReadWriteSplittingPlugin.md) | ✅ | +| [aurora_connection_tracker](../using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) | ✅ | +| connect_time | ✅ | +| [fastest_response_strategy](../using-plugins/UsingTheFastestResponseStrategyPlugin.md) | ✅ | +| [initial_connection](../using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) | ✅ | +| [limitless](../using-plugins/UsingTheLimitlessPlugin.md) | ❌ | +| [bg](../using-plugins/UsingTheBlueGreenPlugin.md) | ❌ | + +
+ +| Plugin code / Database URL type | Aurora Cluster Writer Endpoint | Aurora Cluster Reader Endpoint | Aurora Cluster Custom Endpoint | Aurora / RDS Instance Endpoint | +|---|:---:|:---:|:---:|:---:| +| [custom_endpoint](../using-plugins/UsingTheCustomEndpointPlugin.md) | ❌ | ❌ | ✅ | ❌ | +| [host_monitoring](../using-plugins/UsingTheHostMonitoringPlugin.md) | ✅ (requires `initial_connection`) | ✅ (requires `initial_connection`) | ✅ | ✅ | +| [host_monitoring_v2](../using-plugins/UsingTheHostMonitoringPlugin.md) | ✅ (requires `initial_connection`) | ✅ (requires `initial_connection`) | ✅ | ✅ | +| [failover](../using-plugins/UsingTheFailoverPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [failover_v2](../using-plugins/UsingTheFailover2Plugin.md) | ✅ | ✅ | ✅ | ✅ | +| [gdb_failover](../using-plugins/UsingTheGdbFailoverPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [iam](../using-plugins/UsingTheIamAuthenticationPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [aws_secrets_manager](../using-plugins/UsingTheAwsSecretsManagerPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [federated_auth](../using-plugins/UsingTheFederatedAuthenticationPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [okta](../using-plugins/UsingTheOktaAuthenticationPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| stale_dns | ✅ | ❌ | ❌ | ❌ | +| [read_write_splitting](../using-plugins/UsingTheReadWriteSplittingPlugin.md) | ✅ | ✅ | ✅[^1] | ✅[^1] | +| [srw](../using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [gdb_rw](../using-plugins/UsingTheGdbReadWriteSplittingPlugin.md) | ✅ | ✅ | ✅[^1] | ✅[^1] | +| [aurora_connection_tracker](../using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| connect_time | ✅ | ✅ | ✅ | ✅ | +| [fastest_response_strategy](../using-plugins/UsingTheFastestResponseStrategyPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [initial_connection](../using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) | ✅ | ✅ | ❌ | ❌ | +| [limitless](../using-plugins/UsingTheLimitlessPlugin.md) | ✅ | ✅ | ✅ | ❌ | +| [bg](../using-plugins/UsingTheBlueGreenPlugin.md) | ✅ | ✅ | ✅ | ✅ | + +
+ +| Plugin code / Database URL type | RDS Multi-AZ Cluster Writer Endpoint | RDS Multi-AZ Cluster Reader Endpoint | RDS Proxy Endpoint | DB Shard Group Endpoint (Limitless) | +|---|:---:|:---:|:---:|:---:| +| [custom_endpoint](../using-plugins/UsingTheCustomEndpointPlugin.md) | ❌ | ❌ | ❌ | ❌ | +| [host_monitoring](../using-plugins/UsingTheHostMonitoringPlugin.md) | ✅ (requires `initial_connection`) | ✅ (requires `initial_connection`) | ❌ | ❌ | +| [host_monitoring_v2](../using-plugins/UsingTheHostMonitoringPlugin.md) | ✅ (requires `initial_connection`) | ✅ (requires `initial_connection`) | ❌ | ❌ | +| [failover](../using-plugins/UsingTheFailoverPlugin.md) | ✅ | ✅ | ✅ | ❌ | +| [failover_v2](../using-plugins/UsingTheFailover2Plugin.md) | ✅ | ✅ | ✅ | ❌ | +| [gdb_failover](../using-plugins/UsingTheGdbFailoverPlugin.md) | ✅ | ✅ | ✅ | ❌ | +| [iam](../using-plugins/UsingTheIamAuthenticationPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [aws_secrets_manager](../using-plugins/UsingTheAwsSecretsManagerPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [federated_auth](../using-plugins/UsingTheFederatedAuthenticationPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [okta](../using-plugins/UsingTheOktaAuthenticationPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| stale_dns | ✅ | ❌ | ❌ | ❌ | +| [read_write_splitting](../using-plugins/UsingTheReadWriteSplittingPlugin.md) | ✅ | ✅ | ❌ | ❌ | +| [srw](../using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) | ✅ | ✅ | ✅ | ✅ | +| [gdb_rw](../using-plugins/UsingTheGdbReadWriteSplittingPlugin.md) | ✅ | ✅ | ❌ | ❌ | +| [aurora_connection_tracker](../using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) | ✅ | ✅ | ✅ | ❌ | +| connect_time | ✅ | ✅ | ✅ | ✅ | +| [fastest_response_strategy](../using-plugins/UsingTheFastestResponseStrategyPlugin.md) | ✅ | ✅ | ✅ | ❌ | +| [initial_connection](../using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) | ✅ | ✅ | ❌ | ❌ | +| [limitless](../using-plugins/UsingTheLimitlessPlugin.md) | ❌ | ❌ | ❌ | ✅ | +| [bg](../using-plugins/UsingTheBlueGreenPlugin.md) | ❌ | ❌ | ✅ | ❌ | + +
+ +| Plugin code / Database URL type | IP address | User custom domain (CNAME alias) | +|---|:---:|:---:| +| [custom_endpoint](../using-plugins/UsingTheCustomEndpointPlugin.md) | ❌ | ❌ | +| [host_monitoring](../using-plugins/UsingTheHostMonitoringPlugin.md) | ❌ | ❌ | +| [host_monitoring_v2](../using-plugins/UsingTheHostMonitoringPlugin.md) | ❌ | ❌ | +| [failover](../using-plugins/UsingTheFailoverPlugin.md) | ✅ (requires special configuration) | ✅ (requires special configuration) | +| [failover_v2](../using-plugins/UsingTheFailover2Plugin.md) | ✅ (requires special configuration) | ✅ (requires special configuration) | +| [gdb_failover](../using-plugins/UsingTheGdbFailoverPlugin.md) | ✅ (requires special configuration) | ✅ (requires special configuration) | +| [iam](../using-plugins/UsingTheIamAuthenticationPlugin.md) | ✅ (requires special configuration) | ✅ (requires special configuration) | +| [aws_secrets_manager](../using-plugins/UsingTheAwsSecretsManagerPlugin.md) | ✅ (requires special configuration) | ✅ (requires special configuration) | +| [federated_auth](../using-plugins/UsingTheFederatedAuthenticationPlugin.md) | ✅ (requires special configuration) | ✅ (requires special configuration) | +| [okta](../using-plugins/UsingTheOktaAuthenticationPlugin.md) | ✅ (requires special configuration) | ✅ (requires special configuration) | +| stale_dns | ❌ | ❌ | +| [read_write_splitting](../using-plugins/UsingTheReadWriteSplittingPlugin.md) | ✅[^1] | ✅[^1] | +| [srw](../using-plugins/UsingTheSimpleReadWriteSplittingPlugin.md) | ✅ | ✅ | +| [gdb_rw](../using-plugins/UsingTheGdbReadWriteSplittingPlugin.md) | ✅[^1] | ✅[^1] | +| [aurora_connection_tracker](../using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) | ✅ | ✅ | +| connect_time | ✅ | ✅ | +| [fastest_response_strategy](../using-plugins/UsingTheFastestResponseStrategyPlugin.md) | ✅ | ✅ | +| [initial_connection](../using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) | ❌ | ❌ | +| [limitless](../using-plugins/UsingTheLimitlessPlugin.md) | ✅ | ✅ | +| [bg](../using-plugins/UsingTheBlueGreenPlugin.md) | ✅ (requires special configuration) | ❌ | + +
+ +[^1]: For custom-endpoint and instance endpoints, keep connection-role verification enabled (`verify_opened_connection_role`, default on). The actual host role may differ from the assumed role, and disabling verification can cause incorrect read/write splitting behavior. See [UsingTheAuroraInitialConnectionStrategyPlugin.md](../using-plugins/UsingTheAuroraInitialConnectionStrategyPlugin.md) for details. diff --git a/docs/using-the-python-wrapper/compatibility/PluginChainCompatibility.md b/docs/using-the-python-wrapper/compatibility/PluginChainCompatibility.md new file mode 100644 index 00000000..df78c684 --- /dev/null +++ b/docs/using-the-python-wrapper/compatibility/PluginChainCompatibility.md @@ -0,0 +1,205 @@ +# Cross plugin compatibility + +This document is part of the [Compatibility Guide](./Compatibility.md) and explains compatibility between various plugins — both the *logical* plugin-vs-plugin matrix and the **Python-specific** constraints (async/sync availability, driver/runtime limitations, plugin ordering, and recommended canonical chains) that the matrix can't capture. + +- [Database type compatibility](./CompatibilityDatabaseTypes.md) +- [Database URL type compatibility](./CompatibilityEndpoints.md) + +While combining plugins in a single driver configuration is common, some plugins may not work properly together. Such incompatibilities can arise from either plugin design constraints or logical conflicts. + +For example, the `failover` plugin is incompatible with `failover_v2`. Both plugins support database cluster failover but implement this functionality differently. Combining them in a single configuration causes interference between their operations, leading to instability. Similarly, the `limitless` plugin and `custom_endpoint` plugin are incompatible because Limitless Database does not support custom endpoints. + +> **Runtime note (Python-specific):** The AWS Advanced Python Wrapper does **not** raise at construction time for an incompatible chain — the chain is accepted but behaves incorrectly at runtime (silent fallback to a writer-only connection, monitor threads that can't abort, etc.). Check the relevant section below before composing a chain you haven't used before. + +## Mutually incompatible groups + +Only **one** plugin from each of the following groups may be used at a time: + +- **Failover:** `failover`, `failover_v2`, `gdb_failover` +- **Host monitoring (EFM):** `host_monitoring`, `host_monitoring_v2` +- **Read/write splitting:** `read_write_splitting`, `srw`, `gdb_rw` +- **Authentication:** `iam`, `aws_secrets_manager`, `federated_auth`, `okta` + +## Other incompatibilities + +- `initial_connection` is incompatible with `stale_dns` and with `srw`. +- `stale_dns` is incompatible with `srw` and `gdb_rw`. +- `limitless` is incompatible with all failover plugins, all read/write-splitting plugins, `custom_endpoint`, `aurora_connection_tracker`, `initial_connection`, `stale_dns`, `fastest_response_strategy`, and `bg`. + +## Plugin-vs-plugin matrix + +Legend: ✅ compatible  |  ❌ incompatible + +
+ +| Plugin code / Plugin code | host_monitoring | host_monitoring_v2 | failover | failover_v2 | gdb_failover | +|---|:---:|:---:|:---:|:---:|:---:| +| host_monitoring_v2 | ❌ | | | | | +| failover | ✅ | ✅ | | | | +| failover_v2 | ✅ | ✅ | ❌ | | | +| gdb_failover | ✅ | ✅ | ❌ | ❌ | | +| iam | ✅ | ✅ | ✅ | ✅ | ✅ | +| aws_secrets_manager | ✅ | ✅ | ✅ | ✅ | ✅ | +| federated_auth | ✅ | ✅ | ✅ | ✅ | ✅ | +| okta | ✅ | ✅ | ✅ | ✅ | ✅ | +| read_write_splitting | ✅ | ✅ | ✅ | ✅ | ✅ | +| srw | ✅ | ✅ | ✅ | ✅ | ✅ | +| gdb_rw | ✅ | ✅ | ✅ | ✅ | ✅ | +| custom_endpoint | ✅ | ✅ | ✅ | ✅ | ✅ | +| aurora_connection_tracker | ✅ | ✅ | ✅ | ✅ | ✅ | +| initial_connection | ✅ | ✅ | ✅ | ✅ | ✅ | +| stale_dns | ✅ | ✅ | ✅ | ✅ | ✅ | +| connect_time | ✅ | ✅ | ✅ | ✅ | ✅ | +| fastest_response_strategy | ✅ | ✅ | ✅ | ✅ | ✅ | +| limitless | ✅ | ✅ | ❌ | ❌ | ❌ | +| bg | ✅ | ✅ | ✅ | ✅ | ✅ | + +
+ +| Plugin code / Plugin code | iam | aws_secrets_manager | federated_auth | okta | +|---|:---:|:---:|:---:|:---:| +| aws_secrets_manager | ❌ | | | | +| federated_auth | ❌ | ❌ | | | +| okta | ❌ | ❌ | ❌ | | +| read_write_splitting | ✅ | ✅ | ✅ | ✅ | +| srw | ✅ | ✅ | ✅ | ✅ | +| gdb_rw | ✅ | ✅ | ✅ | ✅ | +| custom_endpoint | ✅ | ✅ | ✅ | ✅ | +| aurora_connection_tracker | ✅ | ✅ | ✅ | ✅ | +| initial_connection | ✅ | ✅ | ✅ | ✅ | +| stale_dns | ✅ | ✅ | ✅ | ✅ | +| connect_time | ✅ | ✅ | ✅ | ✅ | +| fastest_response_strategy | ✅ | ✅ | ✅ | ✅ | +| limitless | ✅ | ✅ | ✅ | ✅ | +| bg | ✅ | ✅ | ✅ | ✅ | + +
+ +| Plugin code / Plugin code | read_write_splitting | srw | gdb_rw | +|---|:---:|:---:|:---:| +| srw | ❌ | | | +| gdb_rw | ❌ | ❌ | | +| custom_endpoint | ✅ | ✅ | ✅ | +| aurora_connection_tracker | ✅ | ✅ | ✅ | +| initial_connection | ✅ | ❌ | ✅ | +| stale_dns | ✅ | ❌ | ❌ | +| connect_time | ✅ | ✅ | ✅ | +| fastest_response_strategy | ✅ | ✅ | ✅ | +| limitless | ❌ | ❌ | ❌ | +| bg | ✅ | ✅ | ✅ | + +
+ +| Plugin code / Plugin code | custom_endpoint | aurora_connection_tracker | initial_connection | stale_dns | +|---|:---:|:---:|:---:|:---:| +| aurora_connection_tracker | ✅ | | | | +| initial_connection | ✅ | ✅ | | | +| stale_dns | ✅ | ✅ | ❌ | | +| connect_time | ✅ | ✅ | ✅ | ✅ | +| fastest_response_strategy | ✅ | ✅ | ✅ | ✅ | +| limitless | ❌ | ❌ | ❌ | ❌ | +| bg | ✅ | ✅ | ✅ | ✅ | + +
+ +| Plugin code / Plugin code | connect_time | fastest_response_strategy | limitless | bg | +|---|:---:|:---:|:---:|:---:| +| fastest_response_strategy | ✅ | | | | +| limitless | ✅ | ❌ | | | +| bg | ✅ | ✅ | ❌ | | + +## Plugin order + +Plugins are initialized and executed in the order they are listed in the `plugins` property. By default, the wrapper re-sorts them into a safe order. + +- **`plugins`** — comma-separated list of plugin codes. Default: + `initial_connection,aurora_connection_tracker,failover_v2,host_monitoring_v2` + (for `mysql-connector-python`, the default omits `host_monitoring_v2`: + `initial_connection,aurora_connection_tracker,failover_v2`). +- **`auto_sort_wrapper_plugin_order`** — `Boolean`, default `True`. Lets the wrapper sort + connection plugins by weight to prevent misconfiguration. Set to `False` to preserve the + exact order you specify. + +When auto-sort is enabled, plugins are ordered by ascending weight (lowest first). The +built-in weights are: + +| Plugin code | Weight | +|---|---| +| `custom_endpoint` | 40 | +| `initial_connection` | 50 | +| `aurora_connection_tracker` | 100 | +| `stale_dns` | 200 | +| `read_write_splitting` | 300 | +| `srw` | 310 | +| `gdb_rw` | 320 | +| `failover` | 400 | +| `failover_v2` | 410 | +| `gdb_failover` | 420 | +| `host_monitoring` | 500 | +| `host_monitoring_v2` | 510 | +| `bg` | 550 | +| `fastest_response_strategy` | 600 | +| `iam` | 700 | +| `aws_secrets_manager` | 800 | +| `federated_auth` | 900 | +| `limitless` | 950 | +| `okta` | 1000 | +| `connect_time`, `execute_time`, `dev` | relative to prior plugin (keep listed position) | + +## Required pairings + +| Plugin | Requires | Why | +|---|---|---| +| `read_write_splitting` | `failover_v2` (not `failover`) | `failover_v2` starts the cluster topology monitor on the **initial** connect. With plain `failover`, the host list stays at `{writer-only}` until a later failover signal fires, and `conn.read_only = True` flips silently fall back to the writer rather than splitting to a reader. | + +## Async / sync availability + +The Python wrapper ships two execution modes — a synchronous wrapper and an asynchronous (`asyncio`) wrapper. **Every plugin code is registered in both modes**, so the plugin-vs-plugin, database-type, and endpoint matrices above apply equally to sync and async. The differences are in the *target driver* and a few behavioral gaps, not in which plugins you can name. + +| Concern | Sync | Async | +|---|---|---| +| Target drivers | `psycopg` (PG), `mysql-connector-python` (MySQL) | `psycopg` async (PG), `aiomysql` (MySQL) | +| Plugin codes available | all 22 | all 22 (identical set) | +| `host_monitoring` / `host_monitoring_v2` on MySQL | ❌ `mysql-connector-python` (no thread-based abort — see below) | ✅ `aiomysql` | +| `federated_auth` / `okta` IdP HTTP client | `requests` (bundled) | `aiohttp` (install separately — see note below) | + +**Behavioral gaps to be aware of on the async side:** + +- The async `gdb_failover` and `gdb_rw` plugins implement **home-region** logic but do **not** yet apply `gdb_accessible_regions` filtering, and the async topology monitor has no monitoring-connection-priority support. These are sync-only today. If you rely on accessible-regions or monitoring-connection-priority, use the sync wrapper. +- Async plugins are otherwise functionally equivalent to their sync counterparts; differences are documented in the individual plugin pages where they exist. + +## Driver-specific incompatibilities + +These constraints are specific to the Python target drivers and are **not** reflected in the logical compatibility matrices. + +| Plugin | Incompatible with | Why | +|---|---|---| +| `host_monitoring` (EFM v1) | sync MySQL (`mysql-connector-python`) | EFM requires a thread-based connection abort that `mysql-connector-python` doesn't expose. Symptoms: monitor threads hang on shutdown, "Python hangs on exit" when a host is unreachable. Use the chain **without EFM**, or switch to the async driver (`aiomysql`). | +| `host_monitoring_v2` (EFM v2) | sync MySQL (`mysql-connector-python`) | Same root cause as v1 — EFM v2 still depends on thread-based abort. | +| `iam` | MySQL `use_pure=True` (pure-Python connector) | The pure-Python connector truncates passwords at 255 chars; IAM tokens are typically longer. Expect `int1store requires 0 <= i <= 255` or `struct.error: ubyte format requires 0 <= number <= 255`. See README "Known Limitations". | + +## Recommended canonical chains + +| Use case | Sync chain | Async chain | +|---|---|---| +| Aurora PG — R/W splitting + failover + EFM | `read_write_splitting,failover_v2,host_monitoring_v2` | `read_write_splitting,failover_v2,host_monitoring_v2` | +| Aurora MySQL (sync, `mysql-connector-python`) — R/W splitting + failover | `read_write_splitting,failover_v2` *(no EFM)* | — | +| Aurora MySQL (async, `aiomysql`) — R/W splitting + failover + EFM | — | `read_write_splitting,failover_v2,host_monitoring_v2` | +| Aurora PG — failover only | `failover_v2,host_monitoring_v2` | `failover_v2,host_monitoring_v2` | +| Aurora MySQL (sync) — failover only | `failover_v2` | — | +| Aurora Global Database — GDB failover + EFM | `gdb_failover,host_monitoring_v2` | `gdb_failover,host_monitoring_v2` | +| Aurora Global Database — GDB R/W splitting + GDB failover | `gdb_rw,gdb_failover` | `gdb_rw,gdb_failover` | + +> **Note (async federated/Okta auth):** the async `federated_auth` and `okta` plugins perform their +> IdP HTTP round-trips with [`aiohttp`](https://docs.aiohttp.org/), which is not a runtime dependency +> of this package — install it alongside your async driver (`pip install aiohttp`) when using either +> plugin in an asyncio application. The sync plugins use `requests` and are unaffected. + +## Related docs + +- [UsingTheReadWriteSplittingPlugin.md](../using-plugins/UsingTheReadWriteSplittingPlugin.md) +- [UsingTheFailover2Plugin.md](../using-plugins/UsingTheFailover2Plugin.md) +- [UsingTheFailoverPlugin.md](../using-plugins/UsingTheFailoverPlugin.md) (v1, not recommended for new code per the table above) +- [UsingTheHostMonitoringPlugin.md](../using-plugins/UsingTheHostMonitoringPlugin.md) +- [UsingTheIamAuthenticationPlugin.md](../using-plugins/UsingTheIamAuthenticationPlugin.md) +- [FailoverConfigurationGuide.md](../FailoverConfigurationGuide.md) — retry-budget knobs at the SQLAlchemy / Django boundary diff --git a/docs/using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md index 9551ad3d..14577ed8 100644 --- a/docs/using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheFailover2Plugin.md @@ -1,7 +1,7 @@ # Failover Plugin v2 The AWS Advanced Python Wrapper uses the Failover Plugin v2 to provide minimal downtime in the event of a DB instance failure. The plugin is the next version (v2) of the [Failover Plugin](./UsingTheFailoverPlugin.md) and unless explicitly stated otherwise, most of the information and suggestions for the Failover Plugin are applicable to the Failover Plugin v2. -> **Note**: pair this plugin with `read_write_splitting` when you need R/W splitting — the v1 `failover` plugin doesn't start the cluster topology monitor on initial connect, which makes `read_only = True` silently fall back to the writer. See [PluginChainCompatibility.md](../PluginChainCompatibility.md) for the full compatibility matrix and [FailoverConfigurationGuide.md](../FailoverConfigurationGuide.md#retry-behavior-at-the-sqlalchemy--django-pool-boundary) for the retry-budget knobs at the SA / Django pool boundary. +> **Note**: pair this plugin with `read_write_splitting` when you need R/W splitting — the v1 `failover` plugin doesn't start the cluster topology monitor on initial connect, which makes `read_only = True` silently fall back to the writer. See [PluginChainCompatibility.md](../compatibility/PluginChainCompatibility.md) for the full compatibility matrix and [FailoverConfigurationGuide.md](../FailoverConfigurationGuide.md#retry-behavior-at-the-sqlalchemy--django-pool-boundary) for the retry-budget knobs at the SA / Django pool boundary. ## Differences between the Failover Plugin and the Failover Plugin v2 diff --git a/docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md index 8f6d9442..bc16d26c 100644 --- a/docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheHostMonitoringPlugin.md @@ -1,6 +1,6 @@ # Host Monitoring Plugin -> **Warning**: EFM (both v1 and v2) is **incompatible with sync MySQL** (`mysql-connector-python`) — that driver doesn't expose the thread-based connection abort EFM requires. Symptoms include monitor threads hanging on shutdown and "Python hangs on exit" when a host is unreachable. Use the chain without EFM for sync MySQL, or switch to the async driver (`aiomysql`) which supports EFM v2. See [PluginChainCompatibility.md](../PluginChainCompatibility.md) for the full compatibility matrix. +> **Warning**: EFM (both v1 and v2) is **incompatible with sync MySQL** (`mysql-connector-python`) — that driver doesn't expose the thread-based connection abort EFM requires. Symptoms include monitor threads hanging on shutdown and "Python hangs on exit" when a host is unreachable. Use the chain without EFM for sync MySQL, or switch to the async driver (`aiomysql`) which supports EFM v2. See [PluginChainCompatibility.md](../compatibility/PluginChainCompatibility.md) for the full compatibility matrix. ## Enhanced Failure Monitoring diff --git a/docs/using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md b/docs/using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md index 3776c949..6d01d9dc 100644 --- a/docs/using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md +++ b/docs/using-the-python-wrapper/using-plugins/UsingTheReadWriteSplittingPlugin.md @@ -2,7 +2,7 @@ The Read/Write Splitting Plugin adds functionality to switch between writer and reader instances by setting the `read_only` attribute of an `AwsWrapperConnection`. When setting `read_only` to `True`, the plugin will establish a connection to a reader instance and direct subsequent queries to this instance. Future `read_only` settings will switch the underlying connection between the established writer and reader according to the `read_only` setting. -> **Note**: this plugin must be paired with `failover_v2` (not the v1 `failover` plugin). Plain `failover` doesn't start the cluster topology monitor on the initial connect, leaving the host list at `{writer-only}` until a later failover signal fires; `read_only = True` then silently falls back to the writer. See [PluginChainCompatibility.md](../PluginChainCompatibility.md) for the full compatibility matrix. +> **Note**: this plugin must be paired with `failover_v2` (not the v1 `failover` plugin). Plain `failover` doesn't start the cluster topology monitor on the initial connect, leaving the host list at `{writer-only}` until a later failover signal fires; `read_only = True` then silently falls back to the writer. See [PluginChainCompatibility.md](../compatibility/PluginChainCompatibility.md) for the full compatibility matrix. ### Loading the Read/Write Splitting Plugin