Skip to content

Feature handlers#114

Open
alowrydi wants to merge 7 commits into
mainfrom
feature-handlers
Open

Feature handlers#114
alowrydi wants to merge 7 commits into
mainfrom
feature-handlers

Conversation

@alowrydi

Copy link
Copy Markdown
Contributor

Central .z.* handler registry, di.handlers

Extracts code/handlers/dotz.q's handler capture-and-install mechanism from TorQ and packages it as a standalone kdb-x module - the single sanctioned place a process assigns .z.* connection-lifecycle callbacks, replacing the per-file, load-order-dependent closure wrapping TorQ uses today, where whichever handler loads last silently overwrites whoever set the event before it.

The other nine files in code/handlers/ - writeaccess.q, controlaccess.q, permissions.q, trackclients.q, trackservers.q, logusage.q, zpsignore.q, ldap.q, apidetails.q - were read as research evidence during design, not as source material this module extracts or consolidates. Their actual content belongs to other modules per the modularisation plan: controlaccess.q/permissions.q/writeaccess.q/ldap.q to di.permissions, trackclients.q/trackservers.q to di.clienttracking, logusage.q to di.querylog, apidetails.q to di.api (zpsignore.q remains unassigned in the plan). Reading all nine directly - not a subset - is what surfaced the real distinction this module is built on: some of TorQ's .z.* handlers fan out safely across multiple files, others are hard-coded to a single owner with real mutual-exclusion guards between them, and no single file shows both patterns at once.

Trello ticket - https://trello.com/c/lMDyVar0/119-kdb-x-handlers-module


Files created

File Description
di/handlers/init.q Loads handlers.q, defines export of 7 functions
di/handlers/handlers.q Full implementation - module state, internal helpers, decider-observer machinery, public API, init
di/handlers/handlers.md Full module documentation - see below
di/handlers/test.csv 61 k4unit unit test assertions (114 total rows including setup/comment rows)
di/handlers/test_integration.csv 3 k4unit integration tests against a real spawned q process, OS-assigned port, self-cleaning on any exit path

How to test

Unit tests:

k4unit:use`di.k4unit
k4unit.moduletest`di.handlers
2026.07.17T10:43:06.123 start
2026.07.17T10:43:06.123 :.../di/handlers/test.csv 114 test(s)
2026.07.17T10:43:06.128 end
...
All tests passed

39 true + 22 fail assertions, 0 failures.

Integration tests (spawns a real second q process on an OS-assigned ephemeral port; skips cleanly if no usable q binary is found, no configuration required). moduletest only ever loads test.csv, so the integration suite is loaded and run directly:

k4unit:use`di.k4unit
.m.di.0k4unit.KUltf .Q.dd[hsym`$.Q.m.mp`di.handlers;`test_integration.csv]
.m.di.0k4unit.KUrt[]
k4unit.getresults[]
action ms bytes lang code                             ... ok
true   0  0     q    .it.roundtrip                     ...  1
true   0  0     q    `closed in exec who from .hh.iev  ...  1
true   0  0     q    .it.gone                          ...  1

3/3 passing. (The sh: kill: No such process line sometimes seen during this run is expected, harmless output from the belt-and-braces forceful cleanup step finding the child already self-exited via its own .z.pc backstop - not an error.)

Unit test coverage includes: observer fan-out order and isolation on throw, the captured original always running last, duplicate-name replace-in-place, decider single-owner claim/reclaim/reject, decider remove restoring the kdb-x built-in default via \x, .z.ph rejected outright with a self-contained error, dependency validation (all four failure modes), .z.pw password redaction to observers (with the owner still receiving the real password), owner-throw stopping the chain before any observer runs, register reclaim after observe leaving the dispatcher installed, full state reset when an observed owner is removed, and init idempotency preserving live registrations across a second init call.

Integration test coverage confirms a real synchronous IPC round-trip works while the module is loaded, that a registered .z.pc observer fires on a genuine connection close (not just a synthetic call), and that the spawned child process is fully cleaned up afterward - proven against both the normal path and a deliberately forced mid-suite failure, since the child is instructed to self-exit whenever its connection to the test process drops, for any reason, rather than relying on the test process shutting down gracefully.


Design decisions

1. Observer vs decider split, not a uniform chaining rule - Not every .z.* event behaves the same way when more than one thing wants to hook it. For .z.pc, .z.po, .z.exit, .z.wo, .z.wc, kdb-x discards the return value, so any number of registrants can safely coexist - register fans out to all of them, each individually isolated (a throw is caught, logged at warn, and doesn't stop the rest). For .z.pg, .z.ps, .z.pi, .z.pp, .z.pw, .z.ws, kdb-x uses the return value as the actual outcome, so only one thing can coherently own it - register enforces single ownership, with same-name reclaim (needed for idempotent re-init) and different-name rejection naming the current owner. This split was derived from reading all of TorQ's code/handlers/ files directly, not assumed - real TorQ resolves decider conflicts three incompatible ways depending on which files are involved (permissions.q's hard mutual-exclusion exit against controlaccess.q, ldap.q's AND-composition, zpsignore.q's conditional bypass), so di.handlers doesn't invent a generic composition rule for deciders - it enforces exclusivity instead. Independently corroborated by KX's own shipped Delta Platform product (.ch.addPC/.ch.addPO/.ch.addExit), which formalises multi-handler chaining for exactly the observer-style events and stops there.

2. .z.ph explicitly out of scope, not silently accepted - On any current kdb-x version, permissions.q doesn't gate HTTP GET via .z.ph at all - it sets .h.val:req instead, a completely different hook outside .z.* entirely. Registering against .z.ph would silently do nothing on a real deployment, so register/observe both reject it outright with a self-contained error rather than accepting it as a normal decider.

3. observe/unobserve - side-effect-only watchers on a decider's owner - Single ownership on decider events is correct and unchanged, but real TorQ proves a decider event also needs pure watchers alongside its owner - logusage.q wraps every decider event with log-before/after logging that passes the result through untouched; trackclients.q wraps pg/ps/ws with byte-counting the same way. observe[event;name;func] attaches a watcher to a decider event that already has an owner; it runs after a successful owner call, receives func[result;args], and can never alter what the caller actually receives - the dispatcher's own return statement is the owner's result, computed before any observer runs, so nothing an observer does or returns can reach the client. For .z.pw specifically, the owner receives the real password (it has to, to authenticate) but observers only ever receive (user;"***") - modelled directly on logusage.q's own real redaction of the same field.

4. OS-assigned ephemeral port for the integration test, no configuration - The integration suite needed a real second process on a real port. Rather than a fixed range or an environment variable, the test asks the OS directly (\p 0W, confirmed against kdb-x's own documented ephemeral-port mechanism) and reads back whatever port the kernel actually assigned - guaranteed free at the moment of assignment, no configuration step, no bespoke variable. Cleanup is robust to a genuine mid-suite crash, not just the happy path: the spawned child is told to self-exit the instant its connection to the test process drops, for any reason, which was verified empirically to survive an uncaught throw partway through the suite, not just assumed to work.

5. Export list reviewed against the public API criterion - All seven exports (init, register, remove, list, version, observe, unobserve) are genuine consumer-facing functions. The dispatch machinery, the per-event state, and the decider-observer internals are all private to the module.


Checklist

  • 61/61 unit test assertions passing (114 total test.csv rows)
  • 3/3 integration tests passing
  • Follows consistency.md and style.md
  • Follows dependency injection guidelines
  • handlers.md documents all exported functions, config, usage examples and notes
  • No hard dependencies on other di.* modules - standalone
  • Verified against no TorQ namespaces, env vars, or hard use imports anywhere in the module

Documentation

See handlers.md for full reference including the managed-events table, dependency table, exported function documentation with examples, usage example, and notes.

Comment thread di/handlers/handlers.q
Comment thread di/handlers/handlers.q
Comment thread di/handlers/handlers.q
Comment thread di/handlers/handlers.q
Comment thread di/handlers/test_integration.csv
Comment thread di/handlers/handlers.q
@DI-Software-Engineering

Copy link
Copy Markdown

DIReview Summary

2 critical | 4 warning(s) | 0 suggestion(s)

⚠️ Spec check skipped — tracker lookup failed (NO_REF_FOUND). Standards axis only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants