Support growing connection pools dynamically

[simolus3]

Originally posted by @davidmartos96 in simolus3/sqlite3.dart#391:

Hello! We are in the process of trying to squeeze the performance of our app start. SQLite it's a hard dependency when starting so we are currently waiting for the sqlite_async (Powersync) factory to initialize before showing the interactive UI to the users. At the moment we are comfortable with 5 readers, but that means that on app start we need to wait for the initialization of all the readers in the pool (about 1.3 s on my emulator). If we use 1 reader, time is reduced to 500 ms. What we were thinking is if there could be a way to open with 1 writer / 1 writer + 1 reader, and then have some way of adding more readers at runtime. We would open the app as thin as possible and then asynchronously spawn the other readers when the user can interact with the UI.

Thanks!

Starting from version 0.2.8 of sqlite3_connection_pool, it's possible to add additional readers to a pool (and to start without any dedicated readers initially), so we could add an option to start with a single SQLite connection on native platforms that would then grow later to shorten the critical path on app starts.

If I remember correctly, this package used to work exactly like that: Additional readers were spawned on-demand when all existing ones are blocked, up until the maximum amount of configured readers is reached. Upgrading to sqlite3_connection_pool is a regression in that regard. Growing the pool like that is not really an option anymore, it's hard to detect what's blocking a read since the state is no longer managed in Dart (and is potentially shared across isolates, where before we would only share the write connection).

A potential improvement might be to:

1. Spawn the pool with a single connection initially, and only wait for that before completing the initialize() future.
2. In parallel (using a background isolate), spawn additional connections and add them to the pool. This would be transparent to the rest of the database.

This adds new failure modes that could be hard to debug: What do we do if opening the initial connection works but then there's a failure opening new ones? We can't complete initialize() with an error retroactively, so this would effectively be an unhandled error. Also, performance might be less predictable while we wait for additional connections.
For this reason, maybe this should be a configurable option that would be disabled by default. That would also allow adding an onError callback, or an option to spawn a subset of readers as part of the original initialization for fine-tuning.