Fix SDE search: migrate to the current endpoint and honor min_score

[pushwithak]

Summary

sde_search_tool and repository_search_tool return zero results for many queries against the SDE API as it is deployed today, for two reasons this PR fixes:

• Retired endpoint — both tools defaulted to an old SDE distribution. They now default to the current host (SDE_BASE_URL-overridable).
• Missing min_score — neither tool sent min_score. The SDE endpoints apply a server-side default of 0.55 when the field is omitted, which is above the relevance score most documents receive (hybrid search scores most hits around 0.01), so the whole result set is silently dropped. Both tools now send min_score on every request, default it to 0.0, and expose it as config.

Changes

• sde_search — searches /api/search; current-host default via a single DEFAULT_SDE_BASE_URL constant (resolved lazily so SDE_BASE_URL set after import is honored); min_score on every request; request URL is rstrip'd so a trailing-slash SDE_BASE_URL (shared with code_signals) can't produce host//api/search.
• repository_search — searches the code-specific /api/code/search endpoint. SDE_BASE_URL is treated as a bare host (consistent with sde_search and code_signals) with /api/code/search appended per request, routed through the shared DEFAULT_SDE_BASE_URL constant; min_score=0.0 on every request. Because results now actually flow, the GitHub-enrichment path runs for the first time and is hardened alongside the fix:
  • the SDE call is async (httpx.AsyncClient, one client reused across pages) instead of blocking requests.post;
  • HTTP errors are raised (raise_for_status) and transient failures retried, instead of an error body silently becoming "no results";
  • a result URL that isn't github.com/owner/repo (an org page, another host) returns empty metadata / null reliability score instead of raising IndexError and sinking the whole query via asyncio.gather;
  • results with an invalid URL are skipped while still backfilling to the requested count, and a single enrichment failure keeps its result (empty metadata) rather than failing the whole batch.
• utils — first_commit_date stays a str so repository metadata serializes cleanly (a None there failed MCP output validation once results started flowing).

Test plan

• uv run pytest tests/tools/test_sde_search.py tests/tools/code_search/ — passes (unit + live functional)
• Unit guards pin each tool's host default, that min_score is on every request, and that a host-form SDE_BASE_URL (with a trailing slash) produces the right endpoint URL
• A non-owner/repo URL returns None (no IndexError) and enrichment returns empty metadata for it
• Verified against the live endpoint: "UF universal format weather radar .uf reader python reflectivity" returns 0 results with min_score omitted vs 385 with min_score=0.0
• MCP server registers sde_search_tool and repository_search_tool

[sanzog03]

Code review findings from a high-effort automated pass. Two correctness items (one confirmed, one plausible) and one cleanup — see inline comments.

[sanzog03]

🟡 Correctness (plausible): dropping the explicit first_commit_date = None can flip reliability_score from 0.0 to null.

The field now stays at its declared default "" on success. Since None != "", a successfully-fetched repo can satisfy is_null_metadata where it previously never could. For a repo whose other fields also land at defaults (notably created_at == ""), is_null_metadata flips False → True, so calculate_reliability_score short-circuits to None instead of 0.0 — changing a reported reliability_score from 0.0 to null.

Worth confirming is_null_metadata's definition doesn't now include first_commit_date == "" as a trigger for the empty-default case.

[pushwithak]

is_null_metadata only returns True if every field is at its default, including created_at == "". On a successful fetch created_at is always populated from repo.created_at.isoformat(), so it stays False and the score is computed (a zero-star repo scores e.g. 35.0, not null). created_at == "" only happens on the failure path, where null is correct. So no regression from leaving first_commit_date at its "" default.

[sanzog03]

Follow-up review of the fix commit. One correctness item on the new env placeholder — see inline.

[sanzog03]

🔴 Correctness: this bogus token placeholder actively breaks scoring — it's worse than leaving the var unset.

A user who copies .env.example verbatim (the documented setup path) keeps GITHUB_ACCESS_TOKEN="xxxxxxxxxx". repository_search_tool then authenticates every GitHub call with that invalid token → PyGithub raises 401 inside fetch_github_metadata → the exception is swallowed and an empty RepositoryMetadata is returned → is_null_metadata is True → calculate_reliability_score returns None. Every repository_search result comes back with reliability_score: null.

That's strictly worse than leaving the var unset, where unauthenticated requests (60/hr) succeed and produce real scores — and the comment just above even promises the token prevents null scores, so the placeholder misleads.

Suggest shipping it empty or commented out, e.g.:

# GITHUB_ACCESS_TOKEN=""

so an unconfigured copy falls back to working unauthenticated requests instead of failing auth.

[pushwithak]

Confirmed and fixed. Verified the mechanism: with GITHUB_ACCESS_TOKEN="xxxxxxxxxx", fetch_github_metadata builds Auth.Token("xxxxxxxxxx") → 401 → swallowed → empty metadata → reliability_score: null for every result. Unset and empty both leave auth=None and return real scores (stars=3, score=63.64 for veda-config-ghg). Commented the variable out in .env.example so a verbatim copy falls back to unauthenticated requests, and corrected the note — the previous one blamed the token's absence when it's the invalid placeholder that nulls scores.