Skip to content

Fix EventStream async streaming under hackney 4 - #81

Open
rvasyl wants to merge 2 commits into
devstopfix:mainfrom
holsee:fix/hackney4-async-streaming
Open

Fix EventStream async streaming under hackney 4#81
rvasyl wants to merge 2 commits into
devstopfix:mainfrom
holsee:fix/hackney4-async-streaming

Conversation

@rvasyl

@rvasyl rvasyl commented Jul 15, 2026

Copy link
Copy Markdown

Supersedes #80.

Problem

With hackney 4.x, ExAws.Bedrock.stream!/2 hangs indefinitely: the connection opens but no chunks are ever delivered. Two incompatibilities in EventStream:

  1. hackney 4 negotiates HTTP/2 via ALPN by default, and the Bedrock endpoint accepts h2 — but hackney's h2 path delivers no async messages, so the very first receive (waiting on the status line) blocks forever. Bedrock's InvokeModelWithResponseStream is an HTTP/1.1 API (botocore has no HTTP/2 support either), and this module's Transfer-Encoding: chunked verification only exists on HTTP/1.1.
  2. hackney 4 identifies async responses by the connection pid, not an Erlang reference, so the ref when is_reference(ref) clause in the body loop raises FunctionClauseError on the first read.

Separately (affects both hackney majors): the streaming path discarded caller :http_opts, so every stream ran with hackney's built-in 5s recv_timeout with no way to override it — slow model responses died mid-stream as {:closed, :timeout}. (The non-streaming path already forwards these via ExAws.Request.)

Changes

  • Force HTTP/1.1 with protocols: [:http1] in the streaming hackney options.
  • Match async refs with is_reference(ref) or is_pid(ref) so both hackney majors work.
  • Honor caller :http_opts (recv_timeout, connect_timeout, pool) by merging them under the streaming defaults via a testable hackney_options/1 — streaming-critical keys (async: :once, protocols) can't be overridden.
  • Raise ExAws.Error on {:error, reason} async messages and on unexpected message shapes instead of hanging the receive or crashing the decoder; guard the body-data clause with is_binary/1.

Testing

  • mix test and mix credo --strict pass.
  • Tested live against Bedrock on both the streaming and non-streaming paths, with hackney 4.0 and hackney 1.9.

With hackney 4.x the streaming path hung indefinitely: the stream opened
but no chunks were ever delivered. Two incompatibilities, plus two
hardening fixes:

- Force HTTP/1.1 (protocols: [:http1]). hackney 4 negotiates HTTP/2 via
  ALPN by default and Bedrock accepts h2, but hackney's h2 path never
  delivers async body messages, so the first receive (status) blocked
  forever. HTTP/1.1 is the documented protocol for
  InvokeModelWithResponseStream and what boto3 uses; it also guarantees
  the Transfer-Encoding: chunked header this module verifies.

- Accept pid async refs. hackney 4 correlates async messages by the
  connection pid (async_ref :: pid()), not an Erlang reference, so the
  `ref when is_reference(ref)` clause raised FunctionClauseError on the
  first body read. Match on reference or pid to support both majors.

- Honor caller :http_opts (recv_timeout/connect_timeout/pool) via
  hackney_options/1, streaming defaults winning on conflict. Previously
  the stream always used hackney's built-in recv_timeout (5s) and
  dropped slow responses regardless of the caller's configured timeout.

- Raise on {:error, reason} async messages. hackney 4 emits e.g.
  {:error, :closed} on mid-stream disconnects; previously any shape
  other than {:closed, :timeout} either blocked the receive forever or
  fell into the body-data clause and crashed decode with a badarg.
  Guard the data clause with is_binary/1 and raise ExAws.Error with the
  real reason instead.

Verified live against eu.anthropic.claude-sonnet-4-5-20250929-v1:0
(eu-west-1) with hackney 4.5.2: full stream decoded, zero bad chunks.
@rvasyl
rvasyl marked this pull request as draft July 15, 2026 16:43
@rvasyl
rvasyl marked this pull request as ready for review July 16, 2026 13:06
@holsee
holsee requested a review from devstopfix July 16, 2026 13:07
@devstopfix devstopfix self-assigned this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants