Skip to content

fix: give HTTP drivers a MAC-derived device identity - #724

Open
HuggeK wants to merge 2 commits into
srcfl:masterfrom
HuggeK:fix/http-driver-mac-identity
Open

fix: give HTTP drivers a MAC-derived device identity#724
HuggeK wants to merge 2 commits into
srcfl:masterfrom
HuggeK:fix/http-driver-mac-identity

Conversation

@HuggeK

@HuggeK HuggeK commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

What

An HTTP driver never had its MAC resolved. Combined with the fact that the HTTP
branch also sets no endpoint, a LAN device that had not yet reported a serial
had no stable device_id at all — so its persistent state (battery model,
RLS twin, calibration history) was orphaned whenever its address changed.

Registry.Add now ARP-resolves the driver's own configured address, exactly as
it already did for MQTT and Modbus. This is the code enforcing the invariant in
AGENTS.md: "Persistent device state is keyed by stable hardware identity, not
a YAML name."

How the gap went unnoticed

arp.Lookup is only reachable through Registry.ARPLookup, which was called in
the MQTT and Modbus branches but not the HTTP one — while the arp package doc
claimed it served "Modbus TCP / HTTP / on-LAN MQTT devices". The doc
described the intent; the wiring never matched it.

Only the driver's own address is probed

The MAC is resolved from the driver's own config.host / config.url, via a new
httpDeviceHost helper — deliberately not from the merged HTTP allowlist,
which also names vendor cloud endpoints that must never become identity
candidates.

A narrowed ARP gate

arp.Lookup now skips addresses that provably cannot appear in an ARP table
(loopback, CGNAT/tunnel, multicast, broadcast), saving ~150 ms of futile dialing
per affected driver.

Why this is not an RFC1918 check (I backed that out)

My first version gated on RFC1918 private space, which reads as the "obvious"
definition of on-LAN. It is wrong here: an ISP can route a small public block
straight onto the local segment, and those devices resolve via ARP today. Gating
on RFC1918 would have silently re-keyed them off their MAC and orphaned exactly
the state this PR is trying to protect — a regression that would surface only on
a subset of real deployments, long after merge.

The gate now excludes only what is provably impossible, so no address that
resolves today stops resolving.

Tests

ResolveDeviceID, RegisterDevice and the whole arp package had zero test
coverage
before this PR, despite arp.go carrying a readFile seam whose
comment says it exists "so tests can stub /proc/net/arp". That seam is now
actually used.

  • ResolveDeviceID priority order (make:serialmac:ep:) and its
    normalization rules — a compatibility surface, since changing it orphans state.
  • RegisterDevice upsert semantics: COALESCE must not let an empty field erase
    a known MAC, and first_seen_ms is immutable.
  • Both ARP parsers. The darwin one is extracted as parseDarwinARP so it is
    testable without shelling out to /usr/sbin/arp.
  • The new HTTP wiring, including a test asserting cloud endpoints in the
    allowlist are never passed to ARP.
Two incidental parser fixes that came with the extraction

parseDarwinARP now splits on whitespace rather than a literal space, so
tab-separated arp output no longer yields a corrupted MAC like
aa:bb:cc:dd:ee:ff\ton; and it rejects an all-zero MAC, matching what the Linux
path already did for incomplete entries.

One test documents a pre-existing wart rather than asserting it is correct

TestRegisterDeviceRekeysWhenSerialArrivesAfterMAC pins current behavior:
registerAllDevices resolves the id afresh on every pass, so a device first seen
by MAC is re-keyed the moment its driver reports a serial — leaving the mac:
row behind and orphaning anything still keyed on it. Not fixed here (it needs a
migration decision, and it predates this PR), but it is now visible and covered.

Not in scope

The HTTP branch still sets no endpoint. Adding one would give cloud drivers an
ep:-keyed row where they currently have none, and those rows would be orphaned
the moment a serial arrived — the wart above, amplified. That needs its own
decision, so it is left alone.

Verification

  • go vet ./... and go build ./... clean.
  • Full go test ./... run on this branch and on clean master: identical
    set of 7 pre-existing Windows-only failures (api, backup, config,
    driverrepo, modbus, mpc, state — SQLite URI and unix-socket issues on
    win32). The only delta is internal/arp now reporting ok instead of
    "no test files". No regressions introduced.
Origin of this PR

Found while answering "does FTW work with IPv6?". arp.Lookup is IPv4-only, so a
device reached over IPv6 gets no MAC and falls back to ep: identity — which,
given SLAAC privacy addresses rotate by design, is actively unstable. That IPv6
story is broader (bare IPv6 literals also break MQTT/HA-bridge/CTEK/Nova via
%s:%d string concatenation instead of net.JoinHostPort, and scan/mDNS are
IPv4-only) and is not addressed here.

claude and others added 2 commits July 30, 2026 15:53
An HTTP driver never had its MAC resolved, and the HTTP branch sets no
endpoint either, so a LAN device that had not yet reported a serial had
no stable device_id at all — its persistent state (battery model, RLS
twin, calibration history) was orphaned whenever its address changed.

Registry.Add now ARP-resolves the driver's own configured address, as it
already did for MQTT and Modbus. Only the driver's own host/url is
probed, never the merged HTTP allowlist, so vendor cloud endpoints are
never treated as identity candidates.

arp.Lookup now skips addresses that provably cannot appear in an ARP
table (loopback, CGNAT/tunnel, multicast, broadcast), saving 150 ms of
futile dialing. Public addresses stay eligible on purpose: an ISP can
route a block straight onto the LAN, and excluding them would silently
re-key those devices off their MAC.

Adds the missing coverage for this path: ResolveDeviceID priority and
RegisterDevice upsert semantics had no tests at all, and the arp package
had none despite carrying a readFile seam built for them. The darwin
parser is extracted so it is testable without shelling out, and now
tolerates tab-separated output and rejects all-zero MACs.

Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Branch was cut from a stale fork master. Two conflicts in registry.go,
both additive:

- Registry struct: keep upstream's new SerialFactory field alongside the
  corrected ARPLookup doc comment.
- HTTP capability branch: keep upstream's AllowWrite wiring (srcfl#716) and
  append the ARP lookup after it.

Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
@HuggeK

HuggeK commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Merged master in — the branch was cut from a stale fork master and had drifted behind by ~20 commits. Two conflicts in registry.go, both additive and resolved by keeping both sides:

CI is green. Note that core (Go) is the meaningful signal here: the Lua-dependent driver tests can't run locally since #712 moved driver source out of this repo, and scripts/sync-bundled-drivers.sh currently fails against the pin in drivers/BUNDLED_SOURCE.json with the pin names drivers that do not exist upstream — unrelated to this PR, but worth a look on its own.

Filed #725 for the wider IPv6 findings this PR came out of, including the part it deliberately leaves alone (a v6-addressed device gets no MAC and falls back to a rotating ep: identity).

@HuggeK
HuggeK marked this pull request as ready for review July 30, 2026 14:14
@HuggeK
HuggeK requested a review from frahlg as a code owner July 30, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants