fix(enrich): skip events whose timestamp will not parse - #58
Open
Sarcastic-Soul wants to merge 1 commit into
Open
fix(enrich): skip events whose timestamp will not parse#58Sarcastic-Soul wants to merge 1 commit into
Sarcastic-Soul wants to merge 1 commit into
Conversation
enrich_events guards frame rows against unparseable timestamps, with a
comment saying why, but the event loop below had no equivalent check.
parse_epoch returns 0.0 on garbage, and nearest_index(frame_epochs, 0.0)
resolves to index 0 -- so the event was attributed to the window's FIRST
frame. The SQL window is a string comparison, so a partly-malformed stamp
("2026-07-04T99:99:99", or a truncated write) passes it and reaches the
parser.
Observed: a click labeled "Merge" made in Chrome attributed to Mail, with
frame_dt_ms of 56 years and confidence still reported as high because the
event carried a native label.
Skip those rows instead. An event that cannot be placed in time has no
honest frame to belong to, and the package's contract is that unresolvable
data stays unresolved rather than being invented.
Fixes nossa-y#53
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #53
The bug
enrich_eventsalready guards frame rows against unparseable timestamps, and the comment spells out the reason:The event loop 40 lines below had no equivalent:
parse_epochreturns0.0for anything it can't read, andnearest_index(frame_epochs, 0.0)resolves to index 0 — the first frame of the window.It's worth noting why such a row reaches the parser at all: the query filters with
WHERE timestamp >= ? AND timestamp < ?, which on a TEXT column is a string comparison. That rejects wholly-garbage values ("not-a-timestamp"sorts outside the range) but happily admits anything sharing the date prefix — a truncated write like2026-07-04T10:00, or an out-of-range time like2026-07-04T99:99:99.What that looks like
Two frames —
Mailat 09:00,Chromeat 15:00 — and one click carrying a truncated timestamp:A click made in Chrome is reported against Mail.
frame_dt_msis 56 years, which would be a clear tell — except_confidenceonly consults that distance forresolution == "exact". This event had a native element label, so it short-circuits tohighand arrives downstream indistinguishable from a good attribution.Rare, since it needs a partly-corrupt row. But the failure is silent and lands on a real, wrong frame rather than being dropped.
The change
Mirror the guard that's already there:
Skipping rather than attributing is the deliberate choice: an event that can't be placed in time has no honest home, and the package's stated contract is that unresolvable data stays unresolved rather than being invented. Dropping is also what the frame path already does with the same input.
Tests
test_unparseable_event_timestamps_are_not_attributedintests/test_robustness.py— which already covers malformed frame timestamps, so this sits directly beside its counterpart. It builds a two-app day, inserts two unreadable event rows plus one good one, and asserts only the good event survives and nothing is parked on the first frame:Fails on
main, passes here. Full suite: 108 passed (107 before, +1).Scope
Three lines in
enrich_events. Frame loading, click resolution, and_confidenceare untouched — a well-formed event enriches exactly as before.