fix: give HTTP drivers a MAC-derived device identity - #724
Open
HuggeK wants to merge 2 commits into
Open
Conversation
An HTTP driver never had its MAC resolved, and the HTTP branch sets no endpoint either, so a LAN device that had not yet reported a serial had no stable device_id at all — its persistent state (battery model, RLS twin, calibration history) was orphaned whenever its address changed. Registry.Add now ARP-resolves the driver's own configured address, as it already did for MQTT and Modbus. Only the driver's own host/url is probed, never the merged HTTP allowlist, so vendor cloud endpoints are never treated as identity candidates. arp.Lookup now skips addresses that provably cannot appear in an ARP table (loopback, CGNAT/tunnel, multicast, broadcast), saving 150 ms of futile dialing. Public addresses stay eligible on purpose: an ISP can route a block straight onto the LAN, and excluding them would silently re-key those devices off their MAC. Adds the missing coverage for this path: ResolveDeviceID priority and RegisterDevice upsert semantics had no tests at all, and the arp package had none despite carrying a readFile seam built for them. The darwin parser is extracted so it is testable without shelling out, and now tolerates tab-separated output and rejects all-zero MACs. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Branch was cut from a stale fork master. Two conflicts in registry.go, both additive: - Registry struct: keep upstream's new SerialFactory field alongside the corrected ARPLookup doc comment. - HTTP capability branch: keep upstream's AllowWrite wiring (srcfl#716) and append the ARP lookup after it. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Contributor
Author
|
Merged
CI is green. Note that Filed #725 for the wider IPv6 findings this PR came out of, including the part it deliberately leaves alone (a v6-addressed device gets no MAC and falls back to a rotating |
HuggeK
marked this pull request as ready for review
July 30, 2026 14:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
An HTTP driver never had its MAC resolved. Combined with the fact that the HTTP
branch also sets no endpoint, a LAN device that had not yet reported a serial
had no stable
device_idat all — so its persistent state (battery model,RLS twin, calibration history) was orphaned whenever its address changed.
Registry.Addnow ARP-resolves the driver's own configured address, exactly asit already did for MQTT and Modbus. This is the code enforcing the invariant in
AGENTS.md: "Persistent device state is keyed by stable hardware identity, nota YAML name."
How the gap went unnoticed
arp.Lookupis only reachable throughRegistry.ARPLookup, which was called inthe MQTT and Modbus branches but not the HTTP one — while the
arppackage docclaimed it served "Modbus TCP / HTTP / on-LAN MQTT devices". The doc
described the intent; the wiring never matched it.
Only the driver's own address is probed
The MAC is resolved from the driver's own
config.host/config.url, via a newhttpDeviceHosthelper — deliberately not from the merged HTTP allowlist,which also names vendor cloud endpoints that must never become identity
candidates.
A narrowed ARP gate
arp.Lookupnow skips addresses that provably cannot appear in an ARP table(loopback, CGNAT/tunnel, multicast, broadcast), saving ~150 ms of futile dialing
per affected driver.
Why this is not an RFC1918 check (I backed that out)
My first version gated on RFC1918 private space, which reads as the "obvious"
definition of on-LAN. It is wrong here: an ISP can route a small public block
straight onto the local segment, and those devices resolve via ARP today. Gating
on RFC1918 would have silently re-keyed them off their MAC and orphaned exactly
the state this PR is trying to protect — a regression that would surface only on
a subset of real deployments, long after merge.
The gate now excludes only what is provably impossible, so no address that
resolves today stops resolving.
Tests
ResolveDeviceID,RegisterDeviceand the wholearppackage had zero testcoverage before this PR, despite
arp.gocarrying areadFileseam whosecomment says it exists "so tests can stub /proc/net/arp". That seam is now
actually used.
ResolveDeviceIDpriority order (make:serial→mac:→ep:) and itsnormalization rules — a compatibility surface, since changing it orphans state.
RegisterDeviceupsert semantics:COALESCEmust not let an empty field erasea known MAC, and
first_seen_msis immutable.parseDarwinARPso it istestable without shelling out to
/usr/sbin/arp.allowlist are never passed to ARP.
Two incidental parser fixes that came with the extraction
parseDarwinARPnow splits on whitespace rather than a literal space, sotab-separated
arpoutput no longer yields a corrupted MAC likeaa:bb:cc:dd:ee:ff\ton; and it rejects an all-zero MAC, matching what the Linuxpath already did for incomplete entries.
One test documents a pre-existing wart rather than asserting it is correct
TestRegisterDeviceRekeysWhenSerialArrivesAfterMACpins current behavior:registerAllDevicesresolves the id afresh on every pass, so a device first seenby MAC is re-keyed the moment its driver reports a serial — leaving the
mac:row behind and orphaning anything still keyed on it. Not fixed here (it needs a
migration decision, and it predates this PR), but it is now visible and covered.
Not in scope
The HTTP branch still sets no endpoint. Adding one would give cloud drivers an
ep:-keyed row where they currently have none, and those rows would be orphanedthe moment a serial arrived — the wart above, amplified. That needs its own
decision, so it is left alone.
Verification
go vet ./...andgo build ./...clean.go test ./...run on this branch and on cleanmaster: identicalset of 7 pre-existing Windows-only failures (
api,backup,config,driverrepo,modbus,mpc,state— SQLite URI and unix-socket issues onwin32). The only delta is
internal/arpnow reportingokinstead of"no test files". No regressions introduced.
Origin of this PR
Found while answering "does FTW work with IPv6?".
arp.Lookupis IPv4-only, so adevice reached over IPv6 gets no MAC and falls back to
ep:identity — which,given SLAAC privacy addresses rotate by design, is actively unstable. That IPv6
story is broader (bare IPv6 literals also break MQTT/HA-bridge/CTEK/Nova via
%s:%dstring concatenation instead ofnet.JoinHostPort, and scan/mDNS areIPv4-only) and is not addressed here.