fix(nearby): rank against true angular distance, in a bounded window

[mrosseel]

The object list's "Nearby" sort has two user-visible faults: the focused row is
often not the closest object, and the list stalls the UI while the scope slews.
They have three independent causes, so fixing any one alone leaves the symptom.

Full reasoning, measurements and trade-offs:
docs/adr/0029-nearby-ranking-correctness-and-cost.md (added here).

1. The BallTree was fed (RA, Dec); haversine means (lat, lon)

ClosestObjectsFinder built rows as [ra_rad, dec_rad] and queried with the
same ordering against metric="haversine". scikit-learn reads dimension 0 as
latitude, so RA was passed as latitude and Dec as longitude, and every
separation was computed on a swapped sphere. It is correct only between objects
sharing a meridian; the error grows with declination.

Pointing at RA 0°, Dec +60°:

object	RA	Dec	true separation	ranked distance before	rank before
A	10°	+60°	5.00°	10.00°	2nd
B	0°	+50°	10.00°	10.00°	1st
C	90°	+60°	41.41°	90.00°	3rd
D	180°	+62°	58.00°	178.00°	4th

The chart's nearby-DSO marker layer (UIChart._get_nearby_markers →
get_objects_within_radius) shares the tree, so it plotted the wrong set of
markers. Measured recall for a 5° radius query over a uniform 20 000-object sky:

pointing Dec	truly in radius	returned	correct	recall
0°	36	59	36	100% (23 false positives)
40°	34	58	28	82%
60°	46	41	32	70%
85°	41	12	7	17%

Near the equator the query over-includes but never misses, which is why this
survived: the failure is invisible exactly where casual testing happens. This is
the more valuable half of the fix — a missing chart marker is silent.

Fixed: rows and queries are [dec_rad, ra_rad].

2. Ranking the whole catalog to draw nine rows

get_closest_objects was always called with the default n=0, expanded to
n = len(objects), so every refresh asked for a full k = N ordering and
materialised an N-element object array — to draw about nine rows.

N	k = N query	k = 200 query
14 000	1.5 ms	0.09 ms
40 000	4.4 ms	0.13 ms

Replacing the BallTree with a vectorised haversine + argsort measured
0.9 ms / 4.0 ms — no real gain. The cost is inherent to producing a total
ordering; it can only be avoided, not optimised.

The larger cost sat one level up: each refresh also ran _next_target_index,
building a (catalog_code, sequence) dict over the entire new ordering in pure
Python — 7.6 ms at N = 14 000, 22 ms at N = 40 000 on a fast dev machine,
against a 33 ms frame budget, on a Pi.

Fixed: NEAREST_LIST_CAP = 200; the query is k = min(cap, N), which
bounds the cursor-tracking helper too.

Trade-off, stated plainly: a Nearby-sorted list no longer scrolls down to the
object on the far side of the sky. That ordering has no observing use, and
paying O(N) on every degree of slew to keep it available is the wrong bargain.
Catalog and RA sort still expose everything.

3. The refresh trigger used RA degrees, not sky degrees

should_refresh compared abs(ra - last_ra) > 1.0. One degree of RA spans
cos(dec) degrees on the sky — 0.17° at Dec 80°, 0.017° at Dec 89° — so the
list re-ranked for movement the user cannot see, exactly where slewing is
slowest and the stall most noticeable. RA also wraps: crossing RA 0 gave
abs(359.5 - 0.5) = 359, leaving the trigger permanently true in a band around
the meridian. It never missed a refresh, so this was a cost bug, not a
correctness one — and an invisible one.

Fixed: trigger on great-circle separation (great_circle_degrees), and
raise MAX_TIME from 2 s to 10 s. The time trigger exists to pick up
catalog/filter changes and altitude drift, not pointing changes; at 15°/hour of
sky rotation a 2 s cadence buys nothing.

4. The cursor followed the old object while slewing

update() called _next_target_index after every Nearby refresh to hold the
cursor on the previously selected object. For a filter-driven rebuild that
is right, and it is why the helper exists (the filter-freshness ADR). For a pointing-driven
re-rank it inverts intent: the user slews in order to change what is nearest,
and the cursor drifted down the list away from what they just pointed at. This
is the second half of "the focused thing is not the nearest thing" — a policy
fault, not a bug; the code did what it said.

Fixed: while the cursor sits on the top row it keeps following the pointing.
Once the user scrolls off the top they are browsing, and it pins to the selected
object exactly as before; scrolling back to the top re-arms the behaviour. No
new state — "has not scrolled" is _current_item_index == 0.

5. Smaller faults in the same call path

• SortOrder.RA was never implemented. sort() had branches for NEAREST
  and CATALOG_SEQUENCE only, so choosing RA left the list in whatever order it
  already had, and update()'s two-way label rendered it as "Nearby". Now
  sorted, with both labels routed through one _sort_order_label helper.
• Redundant full-catalog query. mm_change_sort called nearby_refresh()
  before sort(), ranking against a stale or empty tree that sort() then
  immediately redid. Dropped.
• Unreachable message. Nearby.refresh() returns [] when there is no
  pointing, never None, but the caller tested is None before showing "No
  Solve Yet" — so the user saw "No objects match filter" instead. Now tested via
  Nearby.has_pointing().
• Uncached index rebuild. sort() rebuilt the BallTree every call while
  UIChart already guards the identical rebuild on catalog_filter.dirty_time.
  The object list now uses the same guard, invalidated explicitly by
  refresh_object_list.
• Honest type hint. get_closest_objects returns a NumPy object array, not
  a List.

Tests

tests/test_nearby.py placed every object at ra = 0 and its docstring called
the [ra, dec] ordering "the pre-existing convention" — that is the one line on
which the bug cannot show, so the suite encoded the bug as intent.

Rewritten to place objects off a shared meridian and at high declination, and to
assert against an independently computed great-circle separation, so the axis
order is pinned rather than assumed. Added coverage for get_closest_objects
ordering, the n cap, and great_circle_degrees (including the RA wrap and
over-the-pole cases).

Checked against the old code: two of the new tests fail on it.

Verification

• ruff@0.4.8 check + format: pass
• mypy PiFinder: pass (152 files)
• pytest -m "smoke or unit": 1148 passed. Two failures are pre-existing macOS
  path-length limits (AF_UNIX path too long, File name too long) in
  test_sd_notify.py and test_solver_shmem.py, unrelated to this change.
• pytest -m integration tests/test_ui_modules.py: 216 passed, 2 skipped

Behaviour changes users may notice

• Nearby lists are truncated to 200 objects.
• Objects ranked "near" only because of the swapped metric will move or vanish
  from the list. Anyone who learned the old ordering will see it change — that
  is the point.

🤖 Generated with Claude Code