Camera-imu alignment algorithm - #484
Open
TakKanekoGit wants to merge 48 commits into
Open
Conversation
added 27 commits
August 2, 2026 09:08
…tionalities in imu_alignment_solve.py
…rovement but not a lot (still around 1.5 deg std error)
…ty estimates are more realistic now.
…ickbots#632) * docs(optics): name the unknown optical train, the FOV gate's third rung `--camera debug` stopped solving for anyone with a stated lens. ADR 0027 relabelled the debug camera's sensor to `hq` so the archived 10.2 degree frames would be covered, but the lens half still reads live from `camera_lens`, and `resolve_lens` honours a statement whatever profile it is handed. So the gate gets derived from hq x 16mm (17.12 deg) or hq x 12mm (20.43 deg) -- trains that have never physically existed -- and every debug frame falls outside the window. Measured against the real solver: hq + no stated lens 10.33 deg [8.78, 11.88] solves (fitted 10.20) hq + stated 25mm 10.33 deg [8.78, 11.88] solves hq + stated 16mm 17.12 deg [14.55, 19.69] no solve hq + stated 12mm 20.43 deg [17.37, 23.49] no solve Same shape as the regression ADR 0029 exists to fix, mirrored: there an assumption wore a statement's confidence, here a statement is made about hardware that is not in the loop. Names the state rather than special-casing the debug camera. An **unknown optical train** is one whose frames did not come through this device's optics, and it extends the confidence ladder 0029 already established: stated -> +/-15%, assumed -> union over shipped lenses, unknown -> no gate at all. Two consequences follow from one rule -- nothing about the device may be asserted about the frames (no FOV hint), and nothing about the device may be inferred from them (no lens self-heal). The self-heal half is a live bug this records the fix for: with no `camera_lens` in config, debug mode solves at 10.20 deg, self-heal matches that to the hq's 25mm within 1.3%, and writes it into the developer's config. Attach a real imx462 afterwards and the now-stated 25mm derives 6.4 deg, nothing solves, and self-heal cannot undo it -- it only ever writes into an absence. Docs only; the code change follows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CUbR3ryYDL7RizsNhPxGkX * fix(solver): hand no FOV gate to frames from an unknown optical train `--camera debug` stopped solving for anyone with a stated lens. This is the code half of the decision recorded in the previous commit. The debug camera declares the sensor its archived frames were shot on (`hq`), but the lens half of the train is read live from `camera_lens`, so stating one pairs `hq` with glass that is not in the loop: 16mm implies 17.12 deg and 12mm 20.43, against frames that are 10.2. tetra3 prunes by implied field of view before verification, so every frame is rejected -- no solves at all, and the symptom presents as an exposure problem. Rather than special-case the debug camera, name the state and let the two consumers that must not act on it read one flag: * `CameraInterface.optical_train_known()` defaults True, so a camera has to opt *out* and a new hardware backend inherits the gate rather than silently losing it. `CameraDebug` returns False. * The camera process publishes it beside `camera_type`; `SharedStateObj` defaults it True so the boot window before the camera reports keeps its gate on real hardware. * The solver omits `fov_estimate`/`fov_max_error` entirely -- not a third window to keep consistent, and any frame from any train solves, which is what lets a developer drop their own captures into test_images/. * Lens self-heal declines to write. This half fixes a bug that has been live since 0029: a debug run on a config with no lens fits 10.20 deg, matches that to the hq's 25mm within 1.3%, and states it. Rich's own log has it firing on 2026-08-17. Attach a real imx462 afterwards and the now-stated 25mm derives 6.4 deg, nothing solves, and self-heal cannot undo it -- it only ever writes into an absence. Verified end to end, not just in unit tests: launched headless with `camera_lens: "16mm"` stated -- the exact config that produced zero solves -- and it solves RA 296.37 Dec -1.70 in Aql, fitted FOV 10.20, 21 matches, `solve_source: CAM`, with the integrator logging that self-heal is off and the config left unwritten. test_optics_solving.py now parametrises over every lens a config can name, asserting both halves: the derived gate rejects these frames for 16mm and 12mm, and no-gate solves them whatever is stated. That gap -- every existing case passed no lens key at all -- is why this shipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CUbR3ryYDL7RizsNhPxGkX --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The core algorithm for the camera-IMU alignment. This is just for development and it does not affect the rest of the codebase. Additional functionality will need to be developed to integrate this into PiFinder.
The new code is contained within:
PiFinder/develop/camera_imu_alignment/Functionality:
To track the pointing using IMU dead-reckoning, we need to know the relative orientation or alignment between the camera and IMU. The development code estimates the relative alignment from pairs of simultaneous Camera and IMU samples.