Skip to content

fix(patterns): count daily habits by local day, not UTC day - #56

Open
Sarcastic-Soul wants to merge 1 commit into
nossa-y:mainfrom
Sarcastic-Soul:fix/daily-habits-local-day
Open

fix(patterns): count daily habits by local day, not UTC day#56
Sarcastic-Soul wants to merge 1 commit into
nossa-y:mainfrom
Sarcastic-Soul:fix/daily-habits-local-day

Conversation

@Sarcastic-Soul

Copy link
Copy Markdown
Contributor

Fixes #51

The bug

daily_habits was the one place in the package that used SQLite's date() on the stored UTC timestamps:

SELECT date(timestamp) as day, element_name, COUNT(*) as cnt FROM ui_events

That buckets by UTC calendar day. Everything else — local_day_string, local_day_window_utc, fmt_local_hm, coverage()'s hour histogram — works in the user's local day.

For a user far enough from UTC, one local day straddles a UTC boundary and gets counted as two.

Why it matters

The threshold for a habit is if h["days"] >= 2. So this doesn't just inflate a number — it manufactures a habit out of activity that only ever happened once, which is exactly what the module docstring promises can't happen:

No scoring, no inference; a pattern is reported only when it actually repeated.

Concretely, in Asia/Kolkata (UTC+5:30), six clicks on a single local day:

UTC 2026-07-07T21:30  =  local 2026-07-08 03:00   \
UTC 2026-07-07T21:31  =  local 2026-07-08 03:01    |  one local day
UTC 2026-07-07T21:32  =  local 2026-07-08 03:02    |  (Wed 8 Jul, IST)
UTC 2026-07-08T02:30  =  local 2026-07-08 08:00    |
UTC 2026-07-08T02:31  =  local 2026-07-08 08:01    |
UTC 2026-07-08T02:32  =  local 2026-07-08 08:02   /
before after
reported 'Standup' on 2 days, 6x total (no habit)

Late-night and early-morning users are hit hardest, since that's when a local day crosses UTC. Users near UTC+0 are unaffected — probably why it went unnoticed.

The change

Select raw timestamps instead of pre-grouping in SQL, then bucket by local day in Python:

day = local_day_string(datetime.fromtimestamp(epoch, tz=timezone.utc))
per_day[(name, day)] = per_day.get((name, day), 0) + 1

This reuses the existing _time helpers rather than reaching for SQLite's 'localtime' modifier, so the answer doesn't depend on the SQLite process timezone and matches how coverage() already derives local time.

Three things move with it:

  • The per-day floor the SQL HAVING cnt >= 3 enforced is now the named constant MIN_CLICKS_PER_DAY = 3, applied in Python. Same value, same effect.
  • Unparseable timestamps are skipped. The SQL range is a string comparison, so 2026-07-08T99:99:99 passes it and reaches the parser. Without a guard those rows would all bucket onto the same 1970 local day and invent an extra day for the habit.
  • Ordering stays deterministic. The old ORDER BY element_name, day gave the stable sort an alphabetical tiebreaker; the sort key is now explicitly (-total, name), preserving that exactly.

Row volume is a little higher than the pre-aggregated query (one row per click rather than one per day/element), bounded by the same window and click filters — this detector already sits alongside action_sequences, which pulls up to 20k raw click rows.

Tests

patterns.py had no direct test coverage, so this adds tests/test_patterns.py with a local_timezone() context manager that pins and restores TZ (skipped where time.tzset() is unavailable, i.e. not on the Linux/macOS CI matrix):

  • test_one_local_day_is_not_a_habit — the reproduction above (fails on main)
  • test_habit_detection_is_timezone_relative — same capture data, one habit under TZ=UTC and none under TZ=Asia/Kolkata, pinning down that "a day" means the user's day (fails on main)
  • test_daily_habits_skips_malformed_timestamps — unparseable rows dropped, not bucketed at epoch 0 (fails on main)
  • test_two_local_days_is_a_habit — a genuine two-local-day habit is still reported, with the right count (passes on main, guards against over-narrowing)
  • test_per_day_click_floor_still_applies — a day below the click floor doesn't count (passes on main)

Full suite: 112 passed (107 before, +5).

Scope

daily_habits only. The other five detectors, and the days >= 2 / MIN_FREQUENCY thresholds themselves, are untouched.

daily_habits grouped clicks with SQLite's date(), which buckets the stored
UTC timestamps by UTC calendar day, while the rest of the package is
local-day based. For a user far enough from UTC a single local day
straddles a UTC boundary and was counted as two -- and since the habit
threshold is days >= 2, that reported a habit out of activity that never
repeated, contradicting the module's stated guarantee.

Bucket in Python from parse_epoch + local_day_string instead, matching how
coverage() already derives local time and keeping the result independent
of the SQLite process timezone. The per-day click floor the SQL HAVING
enforced moves to MIN_CLICKS_PER_DAY, and rows whose timestamp will not
parse are now skipped rather than bucketed at epoch 0.

Adds tests/test_patterns.py (the module had no direct coverage).

Fixes nossa-y#51

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

daily_habits counts UTC days, not local days — can report a habit that happened on one day

1 participant