Skip to content

feat(posthog-integration): detect existing PostHog, surface the other tricks - #1066

Open
joshtronic wants to merge 11 commits into
PostHog:mainfrom
joshtronic:feat/detect-existing-setup
Open

feat(posthog-integration): detect existing PostHog, surface the other tricks#1066
joshtronic wants to merge 11 commits into
PostHog:mainfrom
joshtronic:feat/detect-existing-setup

Conversation

@joshtronic

@joshtronic joshtronic commented Aug 7, 2026

Copy link
Copy Markdown

Problem

If you run the wizard on a project that already has PostHog you need to walk through the full integration flow again. The original issue quotes a user who bailed out of fear it would overwrite their setup. I also ran into this issue and fortunately, in my testing re-runs mostly don't do anything (destructive or otherwise). No-op still takes ~8 minutes due to auth, skill installation, and running of the agent. Adjacent problem is that the additional wizard tools (or "tricks") aren't made apparent to the user, first run or subsequent runs.

Part of #616.

Changes

  • Detection step scans package.json files for PostHog SDK using the existing findPackageJsons() (same helper revenue-analytics detection uses, monorepo-aware) and sets session.posthogSdkDetected
  • Intro leads with Looks like you already have PostHog installed. when detected; clean projects keep the "two hours in eight minutes" tagline. The PostHog ✔ detected in package.json row sits below the framework line as supporting evidence
  • New "Explore wizard tricks" menu item opens a registry-driven list of the subcommands, their descriptions, and how to run them. When PostHog is detected it is hoisted to the first option, ahead of "Continue anyway" (label borrowed from the health-check view) — exploring the other commands outranks re-running an integration the project may not need. Either way the continue path still routes into the normal (full) flow.
  • Headline and menu construction moved to an ink-free src/ui/tui/posthog-integration-intro.ts so both can be asserted in vitest, which stubs ink suite-wide
  • posthog_sdk_detected added to the standard session properties so existing events can be filtered by it

Regarding the weak signal: the notes on the issue call out that a dependency isn't proof of a working install (agreed 💯) which is why nothing gates or routes on this. A false positive costs a slightly more cautious button label. Init and/or env-key checks or post-auth server-side signals can tighten it further before anything ever routes on the flag.

On the proposed menu, the issue's UX launches the selected command, which needs an in-session program handoff that doesn't exist today. That seemed like maybe something that would be included in the v3 wizard's navigation. This ships the groundwork, but the telemetry already baked to track view changes will give a good signal on when to iterate down the road. I also left the intro's tagline untouched primarily because I couldn't come up with anything that had enough "WOW" as the "2 hours in 8 minutes" line (srsly, so good).

image image

Test plan

  • Contract test, written red-first, for detectExistingPostHog. Covers 6 cases: no package.json, dependencies without PostHog SDK, dependencies, devDependencies, nested monorepo packages, invalid installation directory (shouldn't throw)
  • Store setter test alongside the existing session-setter tests with a sessionProperties test pinning the new property
  • Headline and menu-order tests, also red-first. Order is pinned on each option's value rather than its label, so copy edits do not drag the test along; the only label asserted is the continue one, which branches on the detection
  • Running pnpm build && pnpm test && pnpm fix all came back clean
  • Live runs against a few projects via pnpm try. Projects that were already set up showed the new row, relabel, and tricks view (screenshots above). Clean project showed identical to upstream's main. Nothing gates on the flag, so --ci/headless usage is unaffected

LLM context

Friendly robots, specifically Claude Code, were utilized to help me better understand an unfamiliar codebase. Tests were generated (with my direction) with attribution on those commits. Code changes were guided, but handcrafted to help knock some ring rust off (mostly, as I think Claude committed one of my changes when I wasn't looking 👀 ). All verification runs were done by a human. Robots also played janitor and helped me identify typos.

@joshtronic
joshtronic requested a review from a team as a code owner August 7, 2026 02:43
@joshtronic joshtronic changed the title Feat/detect existing setup feat(posthog-integration): detect existing PostHog, surface the other tricks Aug 7, 2026
@gewenyu99

Copy link
Copy Markdown
Collaborator

Hey, this is incredible! I'm gonna nudge you with some more context:

  • Deterministic detection is what whacking infinite moles with a finite number of sticks feel like. We believe a haiku/GPT luna agent can do this better 10/10 times for cheaper. We actually already have this code around and are soon swapping over to it. This is a great idea, and I think we should roll this in with that bit of detection code.
  • Another migration in flight is the orchestration model for the wizard. If you look carefully at live runs today, we actually enqueue tasks dynamically, albeit for now a fixed list. There's probably more magic to be had this way, if you'd like to play around, go for it! If not, I'd be happy to take this contribution forward once you're happy with the detection related changes.

See if this gives you more ideas ;) Apologies for the highly out of dates docs around these things. They're changing like every other week.

@joshtronic
joshtronic force-pushed the feat/detect-existing-setup branch from 245925a to c050ebe Compare August 10, 2026 19:58
@joshtronic

Copy link
Copy Markdown
Author

Hey @gewenyu99, thanks for the additional context!

Totally tracks on the deterministic detection being mole-whacking nightmare fuel. The manifest scan was always meant as a cheap first signal, with the final answer being the agentic plumbing that sounds in flight on your end.

Fortunately, the detection and disclosure are decoupled already. detectExistingPostHog is the only thing that knows how the verdict happened, as it just sets posthogSdkDetected and everything downstream works off the flag. Your agentic detection can set the same flag and the rest should carry over as-is.

I just pushed a small change so the flag's comment describes the outcome rather than the mechanism. One less thing to change when the agentic detection lands.

I'm happy with this PR as-is. You can consider the detector disposable and the rest is yours to take forward whenever. I may poke at the dynamic task enqueuing this week out of curiosity, but don't let me block anything.

Appreciate you taking the time to put eyes on this PR. 🙇

@gewenyu99 gewenyu99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hellooooo friend,

I'm two weeks late but I took a look.

The way you implemented this makes perfect sense and will be easy to extend later. Good job!

Image

I think we can be more aggressive here and say something like:

  • Looks like you already have PostHog installed
  • PostHog ✔ detected in package.json
  • Explore other wizard tricks should switch places with continue anyway
  • Maybe we can wire the menu directly so that you don't need to exit and rerun the program to use the tricks.

Go for it if you've got some time. I'll pick this up another time if you don't. Thanks!

@joshtronic

Copy link
Copy Markdown
Author

Hey @gewenyu99 -- no worries on the delay. The feedback is great and appreciated. Actually stoked to get in there and wire up the tricks menu. I may also have a second PR (wrt warehouse source detection) before some travel next week :)

@joshtronic
joshtronic requested review from a team as code owners August 22, 2026 03:33
@joshtronic
joshtronic requested review from TueHaulund, ablaszkiewicz, arnohillen, cat-ph, fasyy612, hpouillot and ksvat and removed request for a team August 22, 2026 03:33
joshtronic and others added 11 commits August 22, 2026 16:19
Red-first tests for detectExistingPosthog, ahead of the implementation:
setter always fires (including the negative case), both dependency
sections count, nested monorepo packages are found, and an invalid
install dir reports false instead of throwing.

Co-Authored-By: Claude <noreply@anthropic.com>
Scan found `package.json` files for a declared PostHog SDK and store the
verdict in the session as `posthogSdkDetected`. Dependency-level signal
only, not a verified (or complete) install. Best-effort: scan failures
report false and never block detection.

Pre-work for PostHog#616. Nothing reads the flag yet. Wiring up the intro
screen is next up.

Co-Authored-By: Claude <noreply@anthropic.com>
When the detection step finds an existing PostHog SDK, the intro shows
`PostHog: detected in package.json` row, and the primary action reads
`Continue anyway` (similar phrasing as the health check screen).
Continue still routes into the normal flow as this is a disclosure and
not an actual gate.

Part of PostHog#616
If an existing PostHog SDK is detected (via `package.json`), there's a
good opportunity to raise awareness to the other wizard commands (or
"tricks"). New menu option lists out every subcommand: nam, description,
and how to invoke it directly later. No launching of the subcommands as
I wasn't sure how much of this was being addressed in the v3 TUI
improvements already. The list reads from `getSubcommandPrograms()` so
future additions will appear automagically.

Part of PostHog#616, addressing the discoverability concern in the notes.
Red-first: sessionProperties() should carry the detect step's verdict so
every capture that spreads it becomes sliceable by whether the run
started on an already-instrumented project.

Co-Authored-By: Claude <noreply@anthropic.com>
Add the detect step's verdict to `sessionProperties()`. Any event that
includes the standard session properties (setup confirmed, outro,
aborts) can be filtered by whether the run started on an
already-integrated project. Answers how often PostHog#616 actually happens
without introducing a new event.
Prep for the agentic detection swap mentioned in the PR review. The flag
comment can now outlive the current mechanism of detection. Also fixed a
small typo.
Red-first, ahead of the extraction. The intro leads with the
existing-install detection instead of burying it in a metadata row, so
pin which headline each detection state resolves to — the wiring, not the
wording, so copy edits land in one place without dragging the test along.

Lives in an ink-free module because vitest aliases `ink` to no-op stubs
suite-wide, so the screen itself can't be asserted from a render.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Projects that have PostHog SDK detected are greeted with a new headline.
Fresh projects still get the original "2 hours in 8 minutes" pitch.

Also removed an unnecessary Fragment adjacent to the actual change.
Red-first, ahead of the reorder. On a project that already has PostHog,
exploring the other commands outranks re-running an integration the
project may not need, so pin that order — and pin that a clean project's
menu is unchanged.

Order is asserted on each option's `value` rather than its label, so
copy edits stay free; the only label pinned is the continue one, which
branches on the detection.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
On projects that have the PostHog SDK detected, hoist the "Explore
wizard tricks" option to the top of the menu.

Moved menu generation out of the component to support testing the order
of the menu items.
@joshtronic
joshtronic force-pushed the feat/detect-existing-setup branch from 814ab42 to 8c0c888 Compare August 22, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants