Skip to content

fix(enrich): skip events whose timestamp will not parse - #58

Open
Sarcastic-Soul wants to merge 1 commit into
nossa-y:mainfrom
Sarcastic-Soul:fix/skip-unparseable-event-timestamps
Open

fix(enrich): skip events whose timestamp will not parse#58
Sarcastic-Soul wants to merge 1 commit into
nossa-y:mainfrom
Sarcastic-Soul:fix/skip-unparseable-event-timestamps

Conversation

@Sarcastic-Soul

Copy link
Copy Markdown
Contributor

Fixes #53

The bug

enrich_events already guards frame rows against unparseable timestamps, and the comment spells out the reason:

frames = [
    (int(r[0]), e, r[2] or "", r[3], r[4])
    for r in frame_rows
    if (e := parse_epoch(r[1])) > 0  # malformed rows would unsort the bisect
]

The event loop 40 lines below had no equivalent:

for ts, etype, x, y, elem_name, elem_role, text_content, event_app in event_rows:
    epoch = parse_epoch(ts or "")   # 0.0 on garbage, then used as-is

parse_epoch returns 0.0 for anything it can't read, and nearest_index(frame_epochs, 0.0) resolves to index 0 — the first frame of the window.

It's worth noting why such a row reaches the parser at all: the query filters with WHERE timestamp >= ? AND timestamp < ?, which on a TEXT column is a string comparison. That rejects wholly-garbage values ("not-a-timestamp" sorts outside the range) but happily admits anything sharing the date prefix — a truncated write like 2026-07-04T10:00, or an out-of-range time like 2026-07-04T99:99:99.

What that looks like

Two frames — Mail at 09:00, Chrome at 15:00 — and one click carrying a truncated timestamp:

event_type=click  label='Merge'
  epoch=0.0  ->  attributed app='Mail'  frame_id=1  dt_ms=1783155600000
  confidence=high

A click made in Chrome is reported against Mail. frame_dt_ms is 56 years, which would be a clear tell — except _confidence only consults that distance for resolution == "exact". This event had a native element label, so it short-circuits to high and arrives downstream indistinguishable from a good attribution.

Rare, since it needs a partly-corrupt row. But the failure is silent and lands on a real, wrong frame rather than being dropped.

The change

Mirror the guard that's already there:

epoch = parse_epoch(ts or "")
if epoch <= 0:
    continue

Skipping rather than attributing is the deliberate choice: an event that can't be placed in time has no honest home, and the package's stated contract is that unresolvable data stays unresolved rather than being invented. Dropping is also what the frame path already does with the same input.

Tests

test_unparseable_event_timestamps_are_not_attributed in tests/test_robustness.py — which already covers malformed frame timestamps, so this sits directly beside its counterpart. It builds a two-app day, inserts two unreadable event rows plus one good one, and asserts only the good event survives and nothing is parked on the first frame:

assert [e.label for e in events] == ["Good"]
assert not any(e.app == "Mail" for e in events)

Fails on main, passes here. Full suite: 108 passed (107 before, +1).

Scope

Three lines in enrich_events. Frame loading, click resolution, and _confidence are untouched — a well-formed event enriches exactly as before.

enrich_events guards frame rows against unparseable timestamps, with a
comment saying why, but the event loop below had no equivalent check.
parse_epoch returns 0.0 on garbage, and nearest_index(frame_epochs, 0.0)
resolves to index 0 -- so the event was attributed to the window's FIRST
frame. The SQL window is a string comparison, so a partly-malformed stamp
("2026-07-04T99:99:99", or a truncated write) passes it and reaches the
parser.

Observed: a click labeled "Merge" made in Chrome attributed to Mail, with
frame_dt_ms of 56 years and confidence still reported as high because the
event carried a native label.

Skip those rows instead. An event that cannot be placed in time has no
honest frame to belong to, and the package's contract is that unresolvable
data stays unresolved rather than being invented.

Fixes nossa-y#53

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Events with unparseable timestamps are attributed to the window's first frame

1 participant