diff --git a/docs/adr/0029-fov-gate-width-follows-lens-confidence.md b/docs/adr/0029-fov-gate-width-follows-lens-confidence.md index 65d506526..a2ff5f6c3 100644 --- a/docs/adr/0029-fov-gate-width-follows-lens-confidence.md +++ b/docs/adr/0029-fov-gate-width-follows-lens-confidence.md @@ -200,3 +200,106 @@ One caveat, recorded honestly: 13.04 mm rests on a **single 12 mm sample**. The 16 mm earned its 15.61 by reproducing two independently calibrated field widths on two different sensors. A second 12 mm — ideally on an imx296, so the sensor half varies too — would put it on the same footing. + +## Amendment: a third rung — no gate when the optical train is unknown + +The ladder above has two rungs, and both assume the frames came through the +optics the device is configured with. `--camera debug` breaks that assumption: +it replays archived frames, and the train those frames were shot through is +not the train this machine is configured for. So the ladder gains a third +rung — **stated → ±15%, assumed → union over shipped lenses, unknown → no gate +at all** — and the confidence the width follows becomes confidence in the +pairing rather than only in the lens. + +### The failure + +ADR 0027 relabelled the debug camera's sensor from `imx296` to `hq`, because +the frames in `test_images/` are 10.2° and only the hq's train covers that. +That fixed the sensor half and left the lens half reading live from +`camera_lens`, which `resolve_lens` honours whatever profile it is handed. The +result is a train that has never physically existed: + +| train | derived FOV | gate | debug frames | +|---|---|---|---| +| hq + no stated lens | 10.33° | [8.78, 11.88] | solve (fitted 10.20°) | +| hq + stated `25mm` | 10.33° | [8.78, 11.88] | solve | +| hq + stated `16mm` | 17.12° | [14.55, 19.69] | **no solve** | +| hq + stated `12mm` | 20.43° | [17.37, 23.49] | **no solve** | + +Any developer who has ever opened the Lens menu, and every device whose lens +this ADR's own self-heal has written, loses `--camera debug` entirely. It is +the same shape as the regression this document exists to fix, mirrored: there +an assumption wore a statement's confidence; here a statement is made about +hardware that is not in the loop. + +`tests/test_optics_solving.py` did not catch it because every case calls +`build_optical_train("hq")` with no lens key — only ever the assumed path. + +### Why no hint, rather than the alternatives + +**Declaring a lens from the camera** (debug publishes hq + 25 mm, config +loses) keeps the gate and its mis-solve rejection, and was the first +instinct. Rejected because it needs a lens statement that must never reach +`camera_lens` — a second, differently-scoped writer for a key whose whole +meaning is "the user's claim" — and because it pins debug mode to the one +frame set we happen to ship. Dropping your own unsolvable field into +`test_images/` is the main thing debug mode is *for*, and under a declared +lens those frames are gated out exactly as the shipped ones are today. + +**Emulating the configured train** (serve frames matching the user's sensor +and lens, so `-fh` behaves like their rev4) is the right answer to a different +question and needs a real archived frame per shipped combination. The frames +we have are 10.2°, too narrow to re-project outward from; this stays open if +anyone wants it, and it wants captures, not code. + +**A database-range gate** of `[10, 30]°` is no-hint with extra steps — the +database floor is already doing that pruning — and leaves a constant to +maintain for no protection gained. + +**Rejecting non-shipped pairings** in `resolve_lens` (an hq never shipped with +a 12 mm, so that statement is not credible) would fix debug as a side effect +and close the sensor-swap deadlock too. Rejected here as blast radius: it +makes a user's explicit statement conditionally non-authoritative, which is a +direct contradiction of 0027, and it deserves its own decision rather than +riding along with a development-tool fix. + +The measured cost of no-hint is small and known. On the shipped frames all +three widths return identical RA/Dec, `Matches` and `Prob` within ~1 ms of +each other. What no-hint gives up is the upper bound that rejected confident +mis-solves in this document's noise trials — a protection whose value is that +a device under the stars never reports a wrong pointing. Nothing is under the +stars in debug mode. + +### Consequences + +**The gate is omitted, not widened.** `fov_estimate` is not passed at all when +the train is unknown, so there is no third window to keep consistent with the +other two, and any frame from any train solves. + +**Self-heal is barred under an unknown train**, and this is a live bug the +amendment fixes rather than a hazard it introduces. With no `camera_lens` in +config — the ordinary state of a development machine — `--camera debug` solves +at 10.20°, `identify_lens_from_fitted_fov` matches that to the hq's `25mm` +within 1.3%, and after three frames the integrator writes `camera_lens: 25mm` +into the developer's config. It is not wrong about the *frames*; it is a +statement about a device made from a recording. The trap springs later: attach +a real imx462 and the now-stated 25 mm derives 6.4°, nothing solves, and +self-heal cannot undo it because it only ever writes into an absence. A fitted +FOV measures the train the frames passed through, so under an unknown train it +measures nothing about this device and must not be learned from. + +**Only the gate and self-heal follow the unknown train.** The **frustum** keeps +deriving from the configured optics, deliberately: it answers "what would my +camera image", which is a question about the device and stays meaningful while +a recording plays. Propagating would also delete a frustum that is *correct* +on a dev laptop (10.33° derived against 10.20° frames) to avoid one that is +wrong only when a lens is stated. SQM is unaffected either way — only +`camera_pi` publishes radiometer samples, so the radiometric path is inert +under the debug camera. + +**A stated lens still means no solves on real hardware.** Nothing here softens +0027. The unknown train is a property of where the frames came from, not a +new escape hatch for a wrong statement — which is why the sensor-swap +deadlock (switch imx296 → hq from the Camera Type menu with a stated 16 mm, +derive 17.12°, and be unable to measure out of it) is untouched by this and +remains a documentation path (#613). diff --git a/docs/ax/camera/CONTEXT.md b/docs/ax/camera/CONTEXT.md index 85a2dce4e..421f096ec 100644 --- a/docs/ax/camera/CONTEXT.md +++ b/docs/ax/camera/CONTEXT.md @@ -123,7 +123,9 @@ the one field of view it implies, and nothing overrides it, so a wrong statement still means no solves (that is [ADR 0027](../../adr/0027-fov-gate-derived-from-optical-train.md)'s deliberate consequence, not a regression). Written by the user from the Lens menu, or -once by the device itself from a fitted FOV it is confident about. +once by the device itself from a fitted FOV it is confident about — but never +from a fitted FOV measured under an **unknown optical train**, which measures +a recording rather than this device. _Avoid_: configured lens (true of a self-healed value too, so it does not distinguish), selected lens, user lens. @@ -133,7 +135,9 @@ stated. Not a claim about the hardware — an admission that nobody has said, which is the ordinary condition of an install predating the setting. Because there is nothing to trust, the FOV gate widens to cover *every* lens that sensor has shipped with rather than centring on the fallback. An assumed lens -is a temporary state: the first confident solve turns it into a stated one. +is a temporary state: the first confident solve turns it into a stated one — +unless the train is **unknown**, in which case no solve ever ends the +assumption, because none of them measured this device. _Avoid_: default lens (reads as a preference rather than an absence of information), unset lens (the field of view is never unset — some lens is always assumed). @@ -166,6 +170,25 @@ _Avoid_: optical configuration ("configuration" already names the physical build variants — see [Positioning](../positioning/CONTEXT.md) *screen direction*), camera setup, imaging train. +**Unknown optical train**: +The state of a camera whose frames did not come through this device's optics +at all — today only the debug camera, which replays archived frames. The +device's train still resolves to *something* (the debug camera declares the +sensor its frames were shot on), but that train describes the machine, not +the frames, so two things follow: nothing about the device may be **asserted** +about the frames — the solver is handed no FOV gate rather than a derived one +— and nothing about the device may be **inferred** from them, so a fitted FOV +cannot promote an **assumed lens** to a **stated** one. This is the third rung +of the confidence ladder the FOV gate's width follows: stated → ±15%, assumed +→ the union over the sensor's shipped lenses, unknown → no gate at all (see +[ADR 0029](../../adr/0029-fov-gate-width-follows-lens-confidence.md)). +Deliberately *not* a blanket "everything derived goes dark": the **frustum** +still answers what the configured camera would image, which is a question +about the device and stays meaningful while a recording is being replayed. +_Avoid_: debug FOV, no FOV (the frames have one — nobody here knows it), +unknown lens (the lens resolves normally; it is the pairing that means +nothing), fake camera. + **Field of view**: The angular width of the **crop**, in degrees — the edge-to-edge extent, not the diagonal. Derived from the optical train. Every consumer that needs to diff --git a/python/PiFinder/camera_debug.py b/python/PiFinder/camera_debug.py index a4aa8faf1..73aa91d11 100644 --- a/python/PiFinder/camera_debug.py +++ b/python/PiFinder/camera_debug.py @@ -33,11 +33,14 @@ class CameraDebug(CameraInterface): def __init__(self, exposure_time) -> None: logger.debug("init camera debug") # Format matches PI cameras for compatibility. The sensor named here - # is not cosmetic: it half-determines the solver's FOV gate, and the - # frames in test_images/ are ~10.2 deg, which only the hq profile's - # train covers. Declaring imx296 centres the gate on 13.71 deg and - # every debug frame is rejected before verification -- no solves at - # all under `--camera debug`. See docs/adr/0027. + # is the one the frames in test_images/ were actually shot on, which + # is what makes SQM's profile lookup and the Lens menu resolve to + # something plausible. It is *not* a claim that this machine has an hq + # attached, and it does not settle the field of view on its own: the + # other half of the train comes from config, and a config that states + # a lens (16mm -> 17.12 deg, 12mm -> 20.43 deg) gates these ~10.2 deg + # frames straight out. Hence optical_train_known below. See + # docs/adr/0027 and the third-rung amendment to docs/adr/0029. self.camType = "Debug hq" self.path = utils.pifinder_dir / "test_images" self.exposure_time = exposure_time @@ -106,6 +109,18 @@ def set_camera_config( def get_cam_type(self) -> str: return self.camType + def optical_train_known(self) -> bool: + """No. These frames are a recording, not a view through this device. + + Whatever lens config states is a claim about glass that is not in the + loop, so pairing it with the sensor above produces a field of view + that describes nothing. Saying so is what keeps `--camera debug` + solving whatever the config happens to hold -- including frames a + developer drops into test_images/ from another train entirely, which + no derived gate could have anticipated. + """ + return False + def get_images(shared_state, camera_image, command_queue, console_queue, log_queue): """ diff --git a/python/PiFinder/camera_interface.py b/python/PiFinder/camera_interface.py index a63d74183..8dd4f3c17 100644 --- a/python/PiFinder/camera_interface.py +++ b/python/PiFinder/camera_interface.py @@ -287,6 +287,22 @@ def set_camera_config( def get_cam_type(self) -> str: return "foo" + def optical_train_known(self) -> bool: + """Whether these frames came through this device's own optics. + + True for anything pointed at the sky, which is why it defaults that + way: a camera has to opt *out*, so a new hardware backend inherits the + FOV gate rather than silently losing it. + + False is an **unknown optical train** (docs/ax/camera/CONTEXT.md): the + frames were captured through some other train, so the resolved one + describes this machine and not them. Two things follow, both handled + by the consumers rather than here -- the solver asserts no FOV gate, + and lens self-heal declines to infer a lens from a fitted FOV that + measured a recording. See docs/adr/0029. + """ + return True + def start_camera(self) -> None: pass @@ -308,6 +324,18 @@ def get_image_loop( shared_state.set_camera_type(camera_type) logger.info(f"Camera type set to: {camera_type}") + # Published beside the sensor because it qualifies it: the sensor + # a playback camera declares is the one its *frames* were shot on, + # which is not the same claim a live camera makes. + train_known = self.optical_train_known() + shared_state.set_optical_train_known(train_known) + if not train_known: + logger.info( + "Optical train is unknown: these frames did not come " + "through this device's optics, so the solver gets no FOV " + "gate and the lens cannot self-heal from them" + ) + # Check if auto-exposure was previously enabled in config config_exp = cfg.get_option("camera_exp") if config_exp == "auto": diff --git a/python/PiFinder/imu/__init__.py b/python/PiFinder/imu/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/PiFinder/imu/imu_align/README.md b/python/PiFinder/imu/imu_align/README.md new file mode 100644 index 000000000..9e6600214 --- /dev/null +++ b/python/PiFinder/imu/imu_align/README.md @@ -0,0 +1,221 @@ +# Camera-IMU alignment (extrinsic calibration) + +To track the pointing using IMU dead-reckoning, we need to know the relative +orientation or alignment between the camera and IMU. The development code here +estimates the alignment. + +The alignment error will introduce a "jump" when the IMU dead-reckoning hands +off to the camera solve which is (probably) approximately proportional to the +product of the camera-IMU alignment error and the angle moved under +dead-reckoning (in radians). + +See the header comments in `imu_extrinsic_calibration.py` for explanation of +the algorithm. + +## Previous studies + +In a previous study, we used recorded telemetry data to estimate the camera-IMU +alignment. The extrinsic calibration estimated an adjustment over the nominal +orientation by 1.6 degrees with an uncertainty of ±0.5 degrees. This was based +on around 1 minute of data which gave 9 samples (after outlier removal). + +With 4 minutes of data, we might be able to get it down to something around +±0.1 degrees but this might be unrealistic because it needs continual movement +and (based on simulations) the main source of error doesn't look like +nicely-behaved random noise but something else. The difference between the +moment of camera exposure and the IMU measurement could be just one issue. + +When compared to using the improved alignment with the nominal alignment, the +improvement isn't that big. It cuts the angular jump by around a half, which is +what we'd expect given the uncertainty. + +Simulations with realistic noise gave much better results. This suggests that +the accuracy of the real results may be limited by one or more of the following +following potential root causes: + +1. The alignment algorithm needs to be fed with pairs of start/end samples +with paired camera solves and IMU measurements. Outliers could introduce errors +so better selection criteria may be needed to filter out outliers. +2. The telemetry recording used the BNO055 IMU in fusion mode. This is known to +be noisy so better filtering and outlier rejections may be needed. +3. The BNO055 is an older IMU and it may be that its poorer accuracy propagates +to alignment inaccuracies. It is possible that a more modern IMU could give better +alignment results. +4. The camera and IMU samples are assumed to be from the same time instance. +Relative delays could introduce errors. Filtering of the IMU could also +introduce delays. + +## What still needs to be done + +The study showed that the camera-IMU alsignment could be estimated to ±0.5 +degrees. This is good enough to replace the nominal alignments that need to be +set in configurations. + +A rough alignment feature could be built based on the algorithm in this +directory and the sample code below. + +To improve the alignment accuracy to reduce the "jumps", the potential root +causes listed above may need to be investigated. + +## Sample code from the Jupyter notebooks + +The following is a sample code from the Jupyter notebooks that was used to +analyse the data from telemetry. It could form the basis of an implementation +in PiFinder. + + +```python +from dataclasses import dataclass +from enum import Enum +import quaternion +import numpy as np +from pathlib import Path +import json + +from astro_coords import RaDecRoll +import quaternion_transforms as qt + + +@dataclass +class ImuData: + quat: quaternion.quaternion | None = None + gyro: list | None = None + accel: list | None = None + + +@dataclass +class SolveData: + camera_ra_dec_roll: RaDecRoll | None = None + timestamp_exposure_end: float | None = None # seconds, from time.time() + imu_quat: quaternion.quaternion | None = None # Quaternion at exposure end + + +class MeasurementType(Enum): + CAMERA = 1 + IMU = 2 + + +@dataclass +class Sample: + timestamp: float | None = None # seconds, from time.time() + measurement_type: MeasurementType | None = None + data: SolveData | ImuData | None = None + + def set(self, timestamp: float, measurement_type: MeasurementType, data): + self.timestamp = timestamp + self.measurement_type = measurement_type + self.data = data + + def get(self): + return self.timestamp, self.measurement_type, self.data + + +def read_samples_from_telemetry(path: Path, n_max_samples: int | None = None) -> list[Sample]: + """ + Reads samples from a telemetry file and returns a list of Sample objects. + Each line in the telemetry file is expected to be a JSON object with the following format: + { + "t": timestamp (float, seconds from time.time()), + "e": event type (string, either "imu" or "solve"), + "q": [w, x, y, z] (quaternion for IMU measurements), + "ra": right ascension (float, degrees), + "dec": declination (float, degrees), + "roll": roll angle (float, degrees) + } + """ + samples = [] + counter = 0 + with open(path, 'r') as f: + for line in f: + d = json.loads(line) + #print(d) # For debugging (print the raw data from the telemetry file) + if d["e"] == "imu": + q = quaternion.quaternion(*d["q"]) + imu_data = ImuData(quat=q, gyro=d["gyro"], accel=d['accel']) + samples.append(Sample(timestamp=d["t"], measurement_type=MeasurementType.IMU, data=imu_data)) + elif d["e"] == "solve": + ra_dec_roll = RaDecRoll(ra=d["cam_ra"], dec=d["cam_dec"], roll=d["cam_roll"], deg=True) + solve_data = SolveData(camera_ra_dec_roll=ra_dec_roll, + timestamp_exposure_end=d["lss"], imu_quat=quaternion.quaternion(*d["iq"])) + samples.append(Sample(timestamp=d["t"], measurement_type=MeasurementType.CAMERA, data=solve_data)) + else: + continue # Skip unknown measurement types + + counter += 1 + #print(samples[-1]) # For debugging (print the stored sample) + if n_max_samples is not None: + if counter >= n_max_samples: + break + + return samples + +def get_ang_diffs(last_camera_sample: Sample, camera_sample: Sample): + ang_diff_cam = qt.get_quat_angular_diff( + last_camera_sample.data.camera_ra_dec_roll.as_quaternion(), + camera_sample.data.camera_ra_dec_roll.as_quaternion()) + ang_diff_imu = qt.get_quat_angular_diff( + last_camera_sample.data.imu_quat, + camera_sample.data.imu_quat) + + return ang_diff_cam, ang_diff_imu + +def pair_camera_imu_samples(samples: list[Sample], + max_time_diff=0.1, # [s] Maximum time difference between IMU and platesolve + min_angle_diff=np.deg2rad(5), # Reject if angle from prev. sample is less than this + verbose=False + ): + """ + Pair up solved data (RaDecRoll) with the previous IMU sample. The time + difference between the IMU and camera must be small and the angular + movement between sequential pairs must be large enough. + """ + paired_samples = [] # The result that will be returned + quarantined_samples = [] # Samples that were too close in angle but could be used later + prev_imu_idx = None + for idx, samp in enumerate(samples): + if samp.measurement_type is MeasurementType.IMU: + prev_imu_idx = idx + continue + elif samp.measurement_type is MeasurementType.CAMERA and prev_imu_idx is not None: + if not samp.data.camera_ra_dec_roll.valid: + continue # Skip if camera sample is not valid + + # Skip if IMU sample is after the camera sample or large time difference: + #imu_sample = samples[prev_imu_idx] + #time_diff = samp.timestamp - imu_sample.timestamp + #print(f"{(time_diff)*1000:.1f} ms between IMU and camera sample") + #if (time_diff < 0) or (time_diff > max_time_diff): + # continue + + if not paired_samples: + paired_samples.append(samp) + continue + + # See if we can use the oldest quarantined sample + # TODO: Also add the time difference criterion to reject old samples + if quarantined_samples: + last_camera_sample = paired_samples[-1] + q_samp = quarantined_samples[0] + ang_diff_cam, ang_diff_imu = get_ang_diffs(last_camera_sample, q_samp) + if abs(ang_diff_imu) >= min_angle_diff and abs(ang_diff_cam) >= min_angle_diff: + # Use the quarantined sample + paired_samples.append(q_samp) + quarantined_samples = quarantined_samples[1:] + + # Save pairs of data if the angular difference since the previous sample is large enough + # Note: We could re-use these by another pairing + if paired_samples: + # Skip if angular diff too small (won't be able to solve) + last_camera_sample = paired_samples[-1] + ang_diff_cam, ang_diff_imu = get_ang_diffs(last_camera_sample, samp) + #print(f"Angular difference since last sample: {np.rad2deg(ang_diff):.1f} deg") + if abs(ang_diff_imu) < min_angle_diff and abs(ang_diff_cam) < min_angle_diff or ang_diff_imu < min_angle_diff or ang_diff_cam < min_angle_diff: + quarantined_samples.append(samp) + continue + else: + paired_samples.append(samp) + + assert "Shouldn't get here" + + return paired_samples +``` \ No newline at end of file diff --git a/python/PiFinder/imu/imu_align/__init__.py b/python/PiFinder/imu/imu_align/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/PiFinder/imu/imu_align/hand_eye_solver.py b/python/PiFinder/imu/imu_align/hand_eye_solver.py new file mode 100644 index 000000000..079fb1648 --- /dev/null +++ b/python/PiFinder/imu/imu_align/hand_eye_solver.py @@ -0,0 +1,409 @@ +""" +Core solver functionalities for solving the quaternion form of the hand-eye +problem: + +q1 * q_12 = q_12 * q2 + +Where the goal is to solve for the rotation q_12. Given enough measurements of +q1 and q2, we can solve for q_12. +""" + +import logging +import numpy as np +import quaternion # Note: numpy-quaternion convention: quaternion(w, x, y, z) +from scipy.optimize import least_squares, OptimizeResult +from scipy.sparse import csr_matrix +from scipy.sparse.csgraph import connected_components +from typing import Union + +import PiFinder.pointing_model.quaternion_transforms as qt + +# Typing: +list_of_quats = list[quaternion.quaternion] +list_of_float_pairs = list[tuple[float, float]] + +logger = logging.getLogger("IMU.AlignSolver") + +N_UNKNOWN_PARAMS = 3 # Number of unknown parameters in the problem to solve + + +class HandEyeSolverDiagnostics: + lsq_result: OptimizeResult # Result from scipy.optimize.least_squares + + sample_timestamps: Union[ + list_of_float_pairs, None + ] # Timestamps of each sample-pair + sample_time_differences: Union[ + np.ndarray, None + ] # Time differences between each sample [s] + residual_norms: np.ndarray # Residual norms of each sample [rad] + rotation_angles: np.ndarray # Rotation angles of each sample [rad] + sol_cov_matrix: np.ndarray # Solution covariance matrix + sol_angle_error: float # Solution angle uncertainty [rad] + + # Optional + meta_data: dict + + def __init__( + self, + lsq_result, + q1_list: list_of_quats, + q2_list: list_of_quats, + sample_timestamps: Union[list_of_float_pairs, None] = None, + ): + if len(q1_list) != len(q2_list): + raise ValueError("q1_list and q2_list must be the same length") + if sample_timestamps is None: + self.sample_timestamps = None + self.sample_time_differences = None + else: + if len(sample_timestamps) != len(q1_list): + raise ValueError( + "sample_timestamps must be the same length as q1_list and q2_list" + ) + self.sample_timestamps = sample_timestamps.copy() + self.sample_time_differences = np.array( + [t2 - t1 for t1, t2 in self.sample_timestamps] + ) + + self.lsq_result = lsq_result + + # Residual norm per sample (collapse the 3 measurements per sample into one) + resid = lsq_result.fun.reshape( + (-1, N_UNKNOWN_PARAMS) + ) # Each row corresponds to a sample + self.residual_norms = np.linalg.norm( + resid, axis=1 + ) # Residual per sample in radians + + # Calculate rotations of each sample-pair [rad] + self.rotation_angles = np.array( + [qt.get_quat_angular_diff(q1, q2) for q1, q2 in zip(q1_list, q2_list)] + ) + + self.sol_cov_matrix, self.sol_angle_error = ( + self._calculate_solution_uncertainty(sample_timestamps) + ) + self.meta_data = {} + + def _calculate_solution_uncertainty( + self, sample_timestamps: Union[list_of_float_pairs, None] + ) -> tuple[np.ndarray, float]: + """ + Calculate the standard error of the solution: + Cov = sigma ** 2 * inv(J.T @ J) + + Alternative approach using SVD (more robust): + U, s, Vt = np.linalg.svd(result.jac, full_matrices=False) + cov_x = residuals_var * (Vt.T / s**2) @ Vt + condition_number = s[0] / s[-1] + """ + # Extract Jacobian from least_squares result + J = self.lsq_result.jac # Jacobian matrix (m_meas, n_sol) + m_meas, n_sol = J.shape + + # Calculate the inverse using "backslash": Solve: (J.T @ J) @ X = I + # NOTE: Could be speeded up using QR decomposition but this is good enough + inv_JTJ, _, _, _ = np.linalg.lstsq(J.T @ J, np.eye(n_sol), rcond=None) + + # Calculate degrees of freedom of the problem + if sample_timestamps is None: + dof = N_UNKNOWN_PARAMS * (m_meas - n_sol) # NOTE: Overestimates the DoF! + else: + n_connected_components, n_nodes = self._calculate_connected_components( + sample_timestamps + ) + dof = N_UNKNOWN_PARAMS * (n_nodes - n_connected_components - 1) + + # Calculate reduced Chi-square + rss = 2 * self.lsq_result.cost # Because cost = 0.5 * sum(residuals**2) + chi_square = rss / dof + + # Estimate uncertainty about the solution + sol_cov_matrix = chi_square * inv_JTJ + sol_angle_error = np.sqrt(np.trace(sol_cov_matrix)) # [rad] + + return sol_cov_matrix, sol_angle_error + + @staticmethod + def _calculate_connected_components( + sample_timestamps: list_of_float_pairs, + ) -> tuple[int, int]: + """ + Returns the number of connected components in the sample_timestamps + measurements. For example, if we have 5 measurement pairs (edges) from + 7 unique samples (nodes): + + [(0, 1), (0, 2), (3, 4), (4, 5), (5, 7)] + + There are 3 connected components: [(0, 1, 2), (3, 4, 5), (7,)] + """ + # Flattened along the rows & convert from float timestamps to pairs of indices (0..N) + _, inverse_idx = np.unique(np.array(sample_timestamps), return_inverse=True) + n_nodes = np.max(inverse_idx) + 1 # Number of unique samples + idx_pairs = inverse_idx.reshape(-1, 2) + + # Matrix A has a 1 in row/column pairs + rows, cols = idx_pairs.T + A = csr_matrix( + (np.ones(2 * len(idx_pairs)), (np.r_[rows, cols], np.r_[cols, rows])), + shape=(n_nodes, n_nodes), + ) + + # Return the number of connected components + return connected_components(A, directed=False, return_labels=False), n_nodes + + +def residual_rotation_vector( + x, # (3,) Trial solution (q as rotation vector) + q1_list: list_of_quats, # List of rotation quaternions + q2_list: list_of_quats, +) -> np.ndarray: + """ + For solving q_cam2imu in the quaternion form of the hand-eye problem: + q1 * q_12 = q_12 * q2 + + Calculate the esiduals at the trial solution x for least squares + optimization. + """ + # Convert trial solution (rotation vector) to quaternion + q_12 = quaternion.from_rotation_vector(x) + + n_meas = len(q1_list) + residuals = np.zeros(3 * n_meas) + for ii, (q1, q2) in enumerate(zip(q1_list, q2_list)): + q_err = (q1 * q_12) * (q_12 * q2).conjugate() # Error quaternion + # Convert to rotation vector (Lie algebra logarithm map) + residuals[(3 * ii) : (3 * ii + 3)] = quaternion.as_rotation_vector(q_err) + + return np.array(residuals) + + +def solve_rotation( + q1_list: list_of_quats, # List of rotation quaternions + q2_list: list_of_quats, + x0: Union[np.ndarray, list] = np.zeros(N_UNKNOWN_PARAMS), # Initial guess + sample_timestamps: Union[list_of_float_pairs, None] = None, +) -> tuple[Union[quaternion.quaternion, None], HandEyeSolverDiagnostics]: + """ + Solve the quaternion form of the hand-eye problem using least-squares + optimization of the rotation q_12 parameterized as a rotation vector: + + dq1 * q_12 = q_12 * dq2 + + Where q_12 is the unknown rotation that rotates q1 to q2. + + x0 is the initial guess for q_12 as a rotation vector. The default (zeros) + is the identity rotation. + + Returns None for q_12 if the solver failed to converge. + + NOTE: Possible future improvements: 1) Tune the LM parameters, 2) Calculate + the Jacobians analytically (though fast enough doing it numerically). + """ + if len(q1_list) != len(q2_list): + raise ValueError("q1_list and q2_list must be the same length") + if len(q1_list) < N_UNKNOWN_PARAMS: + raise ValueError( + f"q1_list and q2_list must have at least " + f"{N_UNKNOWN_PARAMS} elements. Got {len(q1_list)}" + ) + if len(x0) != N_UNKNOWN_PARAMS: + raise ValueError("x0 must be a length-3 vector") + + logger.debug(f"Solving for relative rotation from {len(q1_list)} sample pairs.") + result = least_squares( + residual_rotation_vector, x0, method="lm", args=(q1_list, q2_list) + ) + + # Convert estimate from rotation vector to quaternion + q_12 = quaternion.from_rotation_vector(result.x) + + diagnostics = HandEyeSolverDiagnostics( + result, q1_list, q2_list, sample_timestamps=sample_timestamps + ) + logger.debug( + f"Ran solver for relative rotation: Solution q_12={q_12}, " + f"Solution uncertainty: {np.rad2deg(diagnostics.sol_angle_error):.2f} degrees, " + f"Func evaluations: {result.nfev}, Cost = {result.cost:.4g}, " + f"Success: {result.success}, {result.message}" + ) + + if not result.success: + return None, diagnostics + + return q_12, diagnostics + + +def solve_rotation_with_outlier_removal( + q1_list: list_of_quats, # List of rotation quaternions + q2_list: list_of_quats, + x0: Union[np.ndarray, list] = np.zeros(N_UNKNOWN_PARAMS), # Initial guess + sample_timestamps: Union[list_of_float_pairs, None] = None, + mad_threshold=4.45, # TODO: Remove? Reject outlier above this multiple of MAD in first pass + n_min_samples=N_UNKNOWN_PARAMS, # Minimum number of sample pairs for a solution +) -> tuple[Union[quaternion.quaternion, None], HandEyeSolverDiagnostics]: + """ + Solve the hand-eye problem with a single pass of outlier rejection (see + solve_rotation() for details). + """ + # First pass: + q12_solution, diagnostics = solve_rotation(q1_list, q2_list, x0, sample_timestamps) + if q12_solution is None: + logger.debug("First-pass solve for imu/camera alignment failed to converge.") + return None, diagnostics + if mad_threshold is None: + return q12_solution, diagnostics + + # Second pass: + # Detect outliers above MAD threshold + median = np.median(diagnostics.residual_norms) + mad = np.median(np.abs(diagnostics.residual_norms - median)) + mean = np.mean(diagnostics.residual_norms) + sd = np.std(diagnostics.residual_norms) + logger.debug( + "After first pass: " + f"Solution uncertainty: {np.rad2deg(diagnostics.sol_angle_error):.2f} degrees " + f"MAD: {np.rad2deg(mad):.2f} degrees SD: {np.rad2deg(sd):.2f} degrees." + ) + + msk_accept = np.logical_and( + diagnostics.residual_norms < (median + mad_threshold * mad), + diagnostics.residual_norms < (mean + 3 * sd), + ) + if np.all(msk_accept): + logger.debug("No outliers. Returning solution from first-pass.") + return q12_solution, diagnostics + + logger.debug( + "Outlier removal. Keeping " + f"{np.sum(msk_accept)}/{diagnostics.residual_norms.shape[0]} samples." + ) + + if np.sum(msk_accept) < n_min_samples: + np.info( + f"Less than {n_min_samples} samples after outlier removal. " + "Not enough samples for second pass. Returning solution from first-pass." + ) + return q12_solution, diagnostics + + # Remove outliers + q1_accepted = [q for ii, q in enumerate(q1_list) if msk_accept[ii]] + q2_accepted = [q for ii, q in enumerate(q2_list) if msk_accept[ii]] + if sample_timestamps is None: + timestamps_accepted = None + else: + timestamps_accepted = [ + t for ii, t in enumerate(sample_timestamps) if msk_accept[ii] + ] + + # Solve again after outlier removal, using previous solution as the initial guess + x0 = quaternion.as_rotation_vector(q12_solution) + q12_solution_new, diagnostics_new = solve_rotation( + q1_accepted, q2_accepted, x0, timestamps_accepted + ) + # Store solutions and diagnostics from first pass + diagnostics_new.meta_data["first_pass_solution"] = q12_solution + diagnostics_new.meta_data["first_pass_diagnostics"] = diagnostics + + return q12_solution_new, diagnostics_new + + +# ------- Helper functions ------- + + +def ensure_quat_list_continuity(q_list: list_of_quats) -> list_of_quats: + """ + Ensures that consecutive quaternions in the list have consistent signs (due + to the double coverage property of quaternions where q and -q represent + same rotation). + TODO: Possibly not needed. If so, remove. + """ + q_list_out = [q_list[0]] + for q in q_list[1:]: + q = qt.ensure_quat_continuity(q_list_out[-1], q) + q_list_out.append(q) + + return q_list_out + + +def calculate_relative_rotations( + q1_list: list_of_quats, q2_list: list_of_quats +) -> list_of_quats: + """ + Calculate the relative rotation between q1_list and the corresponding q2_list: + dq[k] = q1[k].conjugate() * q2[k] + """ + return [q1.conjugate() * q2 for q1, q2 in zip(q1_list, q2_list)] + + +# ------ Simulation functions for testing & analysis -------------------------- + + +def _q_noise(noise_amp: float): + """Generates random quaternion noise. Noise amp is in radians""" + noise = np.radians(noise_amp) * np.random.randn(3) + return quaternion.from_rotation_vector(noise) + + +def _add_noise_to_quaternion_list(qs: list_of_quats, noise_amp: float): + """Adds noise to a list of quaternions. noise_amp is in radians.""" + qs_out = [] + for q in qs: + qs_out.append(_q_noise(noise_amp) * q) + + return qs_out + + +def _random_quaternions(N: int, max_rot=None) -> list_of_quats: + """ + Returns a list of N random quaternions. If max_rot is None, the quaternions + will be random. If specified, it limits the maximum swing angle from the + previous orientation. + """ + qs: list_of_quats = [] + for ii in range(N): + axis = np.random.randn(3) + axis /= np.linalg.norm(axis) + + if (max_rot is None) or (ii == 0): + angle = np.random.uniform(0, np.pi) + q = quaternion.from_rotation_vector(axis * angle) + else: + angle = np.random.uniform(0, max_rot) + dq = quaternion.from_rotation_vector(axis * angle) + q = qs[-1] * dq + + qs.append(q) + + return qs + + +def simulate_quaternion_measurements( + q_12: quaternion.quaternion, # True rel. orientations (q1 ro q2 alignment) + N: int = 100, # Number of samples to simulate + max_rot=None, # Max rotation from previous orientation + q1_noise_amp: float = np.deg2rad(0.1), # Noise amp in radians + q2_noise_amp: float = np.deg2rad(0.1), # Noise amp in radians + seed=0, # Random seed. None to disable +): + """ + Simulate camera and IMU measurements + """ + if seed is not None: + np.random.seed(seed) + + # Generate random IMU orientations + q2_true = _random_quaternions(N, max_rot=max_rot) + + # Generate corresponding camera orientations + q_21 = q_12.conjugate() + q1_true = [] + for q in q2_true: + q1_true.append(q * q_21) + + # Add noise + q1 = _add_noise_to_quaternion_list(q1_true, q1_noise_amp) + q2 = _add_noise_to_quaternion_list(q2_true, q2_noise_amp) + + return q1, q2 diff --git a/python/PiFinder/imu/imu_align/imu_alignment.py b/python/PiFinder/imu/imu_align/imu_alignment.py new file mode 100644 index 000000000..d8dc55c9c --- /dev/null +++ b/python/PiFinder/imu/imu_align/imu_alignment.py @@ -0,0 +1,399 @@ +""" +Alignment of the IMU-camera axes (extrinsic calibration) + +For dead-reckoning with the IMU, we need the rotation between the IMU and +camera axes. This is done by the quaternion q_cam2imu and its inverse +q_imu2cam. + +The goal of this module is to estimate q_cam2imu. We can do this using pairs of +camera and IMU orientation quaternions measured simultaneously. + +Required measurements +--------------------- + +The measurements we have are: + +* q_eq2cam: Quaternion rotation of the camera center relative to the equatorial + frame. +* q_x2imu: The rotation of the IMU relative to some arbibtrary reference frame + X. + +The camera and IMU measurements are paired and assumed to be simultaneous. + +Algorithm: +---------- + +We can express the rotation between successive timesteps for the camera and +IMU: + +dq_cam = q_eq2cam[k-1].conjugate() * q_eq2cam[k] dq_imu = +q_x2imu[k-1].conjugate() * q_x2imu[k] + +where * is the quaternion multiplication and .conjugate() is the quaternion +conjugate, which is equivalent to the inverse for a unit quaternion. We can +relate the changes in orientation of the camera and IMU by + +dq_cam * q_cam2imu = q_cam2imu * dq_imu + +This is the quaternion version of the hand-eye calibration problem (better +known in the matrix form: AX = XB). + +We will solve for q_cam2imu by defining the error quaternion: + +q_err = (dq_cam * q_cam2imu) * (q_cam2imu * dq_imu).conjugate() + +In the ideal case, q_err will converge to the identity quaternion (1, 0, 0, 0) +at the solution. Quaternions are defined by 4 parameters with one constraint. +We will map the quaternion to a 3-parameter rotation vector, which can be +solved more efficiently and simply. The rotation vector is the product of the +unit vector around the axis of rotation (u) and the rotation (theta): + +e = theta * u = log(q_err) + +The optimization algorith will minimize the two-norm of the error rotation +vector for k = 1..N measurements: + +sum(||e[k]||^2) + + +Assumptions & limitations +------------------------- + +1. Small rotation angles for dq_cam and dq_imu could cause numerical problems + so successive samples should be selected so that the angles are sufficiently + large. +2. The IMU will drift over time so the time between the samples used to + calculate dq_imu should be short enough for drift to be negligible. +3. The camera and IMU samples should be taken simultaneously. If the camera + moves during exposure, this will introduce an error. Error could be reduced + by used samples when the camera movement is reasonably stationary. +4. In practice, the plate solver will have worse error in roll than RA and Dec. + This is not accounted for. +5. Ideally, the camera/IMU should be rotated around all three axes but on a + mount, the rotation will likely be around two axes. This may result in a + larger uncertainty for the rotation/alignment about some axes. +""" + +from dataclasses import dataclass +from typing import Any, Union + +import logging +import numpy as np +import quaternion +import time + +from PiFinder.types.coordinates import RaDecRoll +from PiFinder.pointing_model import quaternion_transforms as qt +from PiFinder.imu.imu_align.hand_eye_solver import solve_rotation_with_outlier_removal +from PiFinder.imu.imu_align.hand_eye_solver import HandEyeSolverDiagnostics + +list_of_quats = list[quaternion.quaternion] + +logger = logging.getLogger("IMU.Align") + + +@dataclass +class CameraImuSample: + """ """ + + timestamp: float + q_cam: quaternion.quaternion + q_imu: quaternion.quaternion + + +class SampleBuffer: + """ + Buffer of samples + """ + + buffer: list + max_buffer_length: int + + def __init__(self, max_buffer_length=10): + self.max_buffer_length = max_buffer_length + self.reset_buffer() + + def reset_buffer(self): + self.buffer = [] + + @property + def len(self): + """Number of samples in buffer""" + return len(self.buffer) + + def add_sample(self, sample: Any): + if len(self.buffer) >= self.max_buffer_length: + self.buffer.pop(0) # Remove oldest sample from buffer + self.buffer.append(sample) + + def pop_sample(self, idx: int) -> int: + """Remove and return the sample at the given index""" + return self.buffer.pop(idx) + + def remove_samples(self, idx_list: set[int]): + """Remove multiple samples by indices""" + self.buffer = [ + self.buffer[i] for i in range(len(self.buffer)) if i not in idx_list + ] + + def trim_to_max_length(self): + if self.len > self.max_buffer_length: + self.buffer = self.buffer[-self.max_buffer_length :] + + +class ImuCameraAlignment: + """ + Note that max_time_diff should be kept to a few seconds at most to avoid + gyro drift over the time between samples. + """ + + candidate_buffer: SampleBuffer # Buffer of camera/IMU samples + pair_buffer: SampleBuffer # Buffer of paired samples ofcamera/IMU samples + + min_n_solve: int # Minimum number of samples for solve + max_time_diff: float # [s] Maximum time difference between pairs of samples + min_angle_diff: float # [rad] Pair samples with large enough angle difference + max_age: float # [s] Maximum age of sample compared to current time + + def __init__( + self, + candidate_buffer_length: int = 60, + min_n_solve: int = 20, + max_time_diff: float = 20.0, + min_angle_diff: float = np.deg2rad(5.0), + max_age: float = 600.0, + ): + """ + candidate_buffer_length: Should be around sample_freq * max_time_diff + + :param candidate_buffer_length: [int] Number of candidate samples to buffer + :param min_n_solve: [int] Minimum number of samples required for solve + :param max_time_diff: [s] Maximum allowed time difference between pairs of samples + :param min_angle_diff: [rad] Minimum allowed angle difference between pairs of samples + :param max_age: [s] Remove samples older than this. None to ignore + """ + self.candidate_buffer = SampleBuffer( + max_buffer_length=max(candidate_buffer_length, min_n_solve) + ) + diff_buffer_length = candidate_buffer_length + self.pair_buffer = SampleBuffer(max_buffer_length=diff_buffer_length) + + self.min_n_solve = min_n_solve + self.max_time_diff = max_time_diff + self.min_angle_diff = min_angle_diff + self.max_age = max_age + + self._samples_since_last_pair_attempt = 0 + + def add_candidate_attempt_solve( + self, timestamp: float, cam_eq: RaDecRoll, q_x2imu: quaternion.quaternion + ) -> tuple[ + Union[quaternion.quaternion, None], Union[HandEyeSolverDiagnostics, None] + ]: + """ + For general use, call this pipeline method. Add a new candidate to the + buffer. When the buffer fills up, pair samples and solve. + """ + self._add_candidate(timestamp, cam_eq, q_x2imu) + + # Pair samples: Runs periodically + if (self._samples_since_last_pair_attempt >= self.min_n_solve) or ( + self.candidate_buffer.len >= self.candidate_buffer.max_buffer_length + ): + self._purge_old_samples(timestamp) + self._purge_old_candidates() + self._pair_samples() + + # If the candidate buffer is still full after pairing, remove a + # batch of the older samples from the buffer + if self.candidate_buffer.len >= self.candidate_buffer.max_buffer_length: + remove_set = set(range(self.min_n_solve)) + self.candidate_buffer.remove_samples(remove_set) + + self._samples_since_last_pair_attempt = 0 + else: + self._samples_since_last_pair_attempt += 1 + + # Solve if there are enough samples + if self.pair_buffer.len >= self.min_n_solve: + t_start = time.time() + q_cam2imu, diagnostics = self._solve() + diagnostics.meta_data["total_solve_time"] = time.time() - t_start + + self.pair_buffer.reset_buffer() # Flush the values used for solve + self.candidate_buffer.trim_to_max_length() + + return q_cam2imu, diagnostics + else: + return None, None + + def _reset_buffers(self): + self.candidate_buffer.reset_buffer() + self.pair_buffer.reset_buffer() + + def _trim_buffers(self): + self.candidate_buffer.trim_to_max_length() + self.pair_buffer.trim_to_max_length() + + def _add_candidate( + self, timestamp: float, cam_eq: RaDecRoll, q_x2imu: quaternion.quaternion + ): + """ + Add to the candidate_buffer the camera solve & corresponding IMU sample + from integrator. + """ + if ( + timestamp is None + or cam_eq is None + or cam_eq.valid is False + or q_x2imu is None + ): + return + + # Ensure quaternion continuity from previous candidate sample + q_cam = cam_eq.as_quaternion() + if self.candidate_buffer.len == 0: + self.candidate_buffer.add_sample(CameraImuSample(timestamp, q_cam, q_x2imu)) + else: + last_candidate = self.candidate_buffer.buffer[-1] + q_cam = qt.ensure_quat_continuity(last_candidate.q_cam, q_cam) + q_imu = qt.ensure_quat_continuity(last_candidate.q_imu, q_x2imu) + self.candidate_buffer.add_sample(CameraImuSample(timestamp, q_cam, q_imu)) + + def _purge_old_samples(self, ref_time: float): + """ + Remove samples from the candidate_buffer that are older than max_age + relative to ref_time. This should be run on a schedule every + self.max_age [s]. + """ + if self.max_age is None: + return + + allowed_timestamp = ref_time - self.max_age # Purge anything older than this + + # Purge candidate_buffer: + remove_idx_list = [ + i + for i, samp in enumerate(self.candidate_buffer.buffer) + if samp.timestamp < allowed_timestamp + ] + if remove_idx_list: + self.candidate_buffer.remove_samples(set(remove_idx_list)) + + # Purge diff_buffer: + remove_idx_list = [ + i + for i, (samp1, samp2) in enumerate(self.pair_buffer.buffer) + if samp1.timestamp < allowed_timestamp + or samp2.timestamp < allowed_timestamp + ] + if remove_idx_list: + self.pair_buffer.remove_samples(set(remove_idx_list)) + + def _purge_old_candidates(self): + """ + Remove samples from candidate_buffer that are older than + self.max_time_diff from other samples in buffer because these will be + never paired. + + This should be run on a schedule every self.max_time_diff [s]. + """ + if self.candidate_buffer.len <= 1: + return + + remove_ids = set() + timestamps = np.array([samp.timestamp for samp in self.candidate_buffer.buffer]) + for isamp in range(timestamps.shape[0]): + dt = np.abs(timestamps - timestamps[isamp]) + if np.sum(dt < self.max_time_diff) <= 1: + remove_ids.add(isamp) + + if remove_ids: + self.candidate_buffer.remove_samples(remove_ids) + + def _pair_samples(self) -> int: + """ + Go through the candidate_buffer from the first sample in the buffer. + Pair two sets of camera/IMU samples from the candidate buffer that meet + the criteria and remove them from the buffer. Repeat all pairable + samples have been removed from the candidate_buffer. + """ + n_pairs = 0 + if self.candidate_buffer.len == 0: + logger.debug("No samples in candidate buffer for pairing.") + return n_pairs + + remove_ids = set() + for isamp1, samp1 in enumerate(self.candidate_buffer.buffer[:-1]): + for isamp2 in range(isamp1 + 1, self.candidate_buffer.len): + samp2 = self.candidate_buffer.buffer[isamp2] + # Check time difference between samples: + dt = samp2.timestamp - samp1.timestamp + if dt > self.max_time_diff or dt <= 0: + # 1) Samples too far apart in time (subsequent samp2 will be even newer), or + # 2) Duplicate samples or out-of-order (sample1 is newer). Remove sample1 + remove_ids.add(isamp1) + break + + # Check angle difference (from camera solve) between samples: + dtheta = qt.get_quat_angular_diff(samp1.q_cam, samp2.q_cam) + if np.abs(dtheta) < self.min_angle_diff: + continue # Samples too close in angle + + # Pair samples and remove samp1 from candidate buffer after FOR + # loops. This prevents the same pair being used again if this + # method is re-run. Note that this loop will continue pairing + # with samp1. + self.pair_buffer.add_sample((samp1, samp2)) + n_pairs += 1 + remove_ids.add(isamp1) + if self.pair_buffer.len >= self.pair_buffer.max_buffer_length: + break + if self.pair_buffer.len >= self.pair_buffer.max_buffer_length: + break + + logger.debug( + f"Created {n_pairs}-way pairs from {self.candidate_buffer.len} candidate samples." + ) + + if remove_ids: + self.candidate_buffer.remove_samples(remove_ids) + logger.debug( + f"Removed {len(remove_ids)} samples from candidate buffer. " + f"New candidate buffer length: {self.candidate_buffer.len}. " + f"Pair buffer length: {self.pair_buffer.len}." + ) + + return n_pairs # Number of successful pairings + + def _solve( + self, n_pairs=None + ) -> tuple[Union[quaternion.quaternion, None], HandEyeSolverDiagnostics]: + """ + Solve for the alignment between the camera and IMU using at least the + last n_pairs or all available pairs (if None) in diff_buffer. + """ + if n_pairs is None: + n_pairs = self.pair_buffer.len # Use all available data + if n_pairs < self.min_n_solve: + raise ValueError( + f"Oly {n_pairs} samples available for solve. Need {self.min_n_solve}." + ) + + # Generate relative rotation quaternions between paired samp1 and samp2 + # The amount of relative rotation for camera and IMU should be the same + # and this will solve the relative rotation between them. + dq_cam_list = [] + dq_imu_list = [] + sample_timestamps = [] + for samp1, samp2 in self.pair_buffer.buffer: + dq_cam_list.append(samp1.q_cam.conj() * samp2.q_cam) + dq_imu_list.append(samp1.q_imu.conj() * samp2.q_imu) + sample_timestamps.append((samp1.timestamp, samp2.timestamp)) + + # Solve + # q_cam2imu, diagnostics = solve_rotation(dq_cam_list, dq_imu_list) + q_cam2imu, diagnostics = solve_rotation_with_outlier_removal( + dq_cam_list, dq_imu_list, sample_timestamps=sample_timestamps + ) + return q_cam2imu, diagnostics diff --git a/python/PiFinder/integrator.py b/python/PiFinder/integrator.py index 713d81e5f..a123f96bf 100644 --- a/python/PiFinder/integrator.py +++ b/python/PiFinder/integrator.py @@ -68,6 +68,7 @@ SolveSource, SuccessfulSolve, ) +from PiFinder.imu.imu_align.imu_alignment import ImuCameraAlignment logger = logging.getLogger("IMU.Integrator") @@ -101,6 +102,14 @@ class LensSelfHeal: authoritative, so this only ever writes into an absence. That also means it writes at most once in the life of a device -- the write is what ends the condition it triggers on. + * **An unknown optical train never writes at all.** A fitted FOV measures + the train the frames passed through; under ``--camera debug`` that is + the train they were *recorded* on, so it says nothing about this + machine. Without this, a debug run on a config with no lens writes the + hq's ``25mm`` (the archived frames fit 10.20 deg, 1.3% off its derived + 10.33) into that developer's config -- and a real imx462 attached + afterwards then derives 6.4 deg, solves nothing, and cannot heal, + because healing only writes into an absence that no longer exists. * **A fitted FOV matching no shipped lens writes nothing.** Third-party glass leaves the lens assumed and the gate wide, which is honest: we do not know its focal length, and a wrong write would be worse than none @@ -115,8 +124,9 @@ def __init__(self, cfg, shared_state): self._shared_state = shared_state self._candidate: Optional[str] = None self._streak = 0 - # Both latch to keep a per-frame condition from logging per frame. + # All three latch to keep a per-frame condition from logging per frame. self._logged_unidentified = False + self._logged_unknown_train = False self._disabled = False def observe(self, result: SuccessfulSolve) -> None: @@ -132,6 +142,19 @@ def observe(self, result: SuccessfulSolve) -> None: self._disabled = True def _observe(self, result: SuccessfulSolve) -> None: + if not self._shared_state.optical_train_known(): + # Checked before the lens, not after: this is not "nothing to + # heal" but "nothing here can heal anything", and the streak must + # not carry across into a later run on real optics. + if not self._logged_unknown_train: + self._logged_unknown_train = True + logger.info( + "Optical train is unknown, so a fitted FOV measures the " + "recording rather than this device; lens self-heal is off" + ) + self._reset() + return + lens_key = self._shared_state.camera_lens() if lens_is_stated(lens_key): # Nothing to heal -- and after a successful write this is the @@ -229,6 +252,19 @@ def integrator( lens_self_heal = LensSelfHeal(cfg, shared_state) + # ---------- TESTING ------------ + # Initialize continual IMU/Camera alignment + # TODO: Move to a different location + logger.info("IMU/Camera alignment: Initialized with q_imu2cam: ", idr.q_imu2cam) + imu_align = ImuCameraAlignment( + candidate_buffer_length=60, + min_n_solve=20, + max_time_diff=20.0, + min_angle_diff=np.deg2rad(5.0), + max_age=600.0, + ) + # ------------------------------------------------- + while True: state_utils.sleep_for_framerate(shared_state) @@ -279,6 +315,44 @@ def integrator( lens_self_heal.observe(solve_result) estimate = _apply_successful_solve(estimate, solve_result, idr) pointing_updated = True + + # --------------- TESTING --------------- + # Add IMU/Camera samples to the buffer and attempt solve if buffer is full + # TODO: Move to a different location + new_q_cam2imu, _diag = imu_align.add_candidate_attempt_solve( + solve_result.last_solve_success, + solve_result.camera.as_radecroll(), + solve_result.imu_anchor, + ) + if new_q_cam2imu is not None: + angular_diff = qt.get_quat_angular_diff( + idr.q_imu2cam, new_q_cam2imu + ) + logger.info( + "IMU/Camera alignment: New estimate q_imu2cam: ", new_q_cam2imu + ) + logger.info( + "IMU/Camera alignment: Angular difference from previous estimate: " + f"{np.rad2deg(angular_diff):.2f} deg | " + "Solution uncertainty: " + f"{np.rad2deg(_diag.sol_angle_error):.2f} deg | " + f"Solve time: {_diag.meta_data['total_solve_time']}" + ) + # Update: + idr.q_imu2cam = new_q_cam2imu + # --------------------------------------- + + # Append plate-solve and IMU states to IMU/camera alignment buffer + # TODO: Append the following: + # solve_result.last_solve_success (timestamp) + # solve_result.camera.as_radecroll() (RaDecRoll type) + # solve_result.imu_anchor + # + # Update idr.q_imu2cam with the new estimate from IMU/camera alignment + # + # TODO: SuccessfulSolve.last_solve_success is the exposure end time. It's ambiguous... + # TODO: Move ImuDeadReckoning._q_imu2cam() to a stand-alone func in imu_dead_reckoning.py with a view to deprecating it + elif isinstance(solve_result, FailedSolve): telemetry.record_solve( solve_result, predicted=estimate.pointing.aligned.estimate @@ -383,11 +457,12 @@ def _apply_successful_solve( estimate.matched_stars = result.matched_stars estimate.matched_catID = result.matched_catID - # Reseed the dead-reckoner from the new anchor. camera/aligned are - # always present on a SuccessfulSolve, so no None-guard is needed. + # Reset the dead-reckoning from the plate-solved pointing. camera/aligned + # are always present on a SuccessfulSolve, so no None-guard is needed. q_anchor = result.imu_anchor if q_anchor is None: q_anchor = quaternion.quaternion(np.nan) + idr.solve( result.camera.as_radecroll(), result.aligned.as_radecroll(), diff --git a/python/PiFinder/pointing_model/quaternion_transforms.py b/python/PiFinder/pointing_model/quaternion_transforms.py index 7177643b4..27cfc00e1 100644 --- a/python/PiFinder/pointing_model/quaternion_transforms.py +++ b/python/PiFinder/pointing_model/quaternion_transforms.py @@ -62,6 +62,23 @@ def get_quat_angular_diff( return d_theta # In radians +def ensure_quat_continuity( + q_prev: quaternion.quaternion, q_new: quaternion.quaternion +) -> quaternion.quaternion: + """ + Ensures that consecutive quaternions to have consistent signs (due + to the double coverage property of quaternions where q and -q represent + same rotation). + """ + q0 = quaternion.as_float_array(q_prev) + q1 = quaternion.as_float_array(q_new) + + if np.dot(q0, q1) < 0: + return -q_new + else: + return q_new + + # ========== Equatorial frame functions ============================ diff --git a/python/PiFinder/solver.py b/python/PiFinder/solver.py index 9ce0b060f..2e16f86d3 100644 --- a/python/PiFinder/solver.py +++ b/python/PiFinder/solver.py @@ -979,26 +979,49 @@ def solver( train = optical_train.resolve( shared_state.camera_type(), shared_state.camera_lens() ) + # Read live for the same reason the train is: the camera + # process publishes this after the solver is already + # looping, so latching it here would miss it. + train_known = shared_state.optical_train_known() except (BrokenPipeError, ConnectionResetError) as e: logger.error(f"Lost connection to shared state manager: {e}") continue - if train is not logged_train: - logged_train = train - logger.info( - # Say which of the two it is: under an assumed lens - # the gate is wider than the stated field of view - # implies, and a reader diagnosing "why did it solve - # / not solve" needs to know the lens is a fallback - # rather than something the device was told. - "Optical train: %s %s lens on %s, field of view " - "%.2f deg, FOV gate [%.2f, %.2f]", - "stated" if train.lens_stated else "assumed", - train.lens.menu_label, - shared_state.camera_type(), - train.fov_degrees, - *_fov_gate_bounds(train), - ) - _warn_if_outside_solver_database(t3, train) + if (train, train_known) != logged_train: + logged_train = (train, train_known) + if not train_known: + # The resolved train is still worth printing -- it is + # what SQM and the frustum are using -- but saying it + # without saying it describes the device rather than + # the frames is how somebody concludes the gate is + # wrong when there is no gate. + logger.info( + "Optical train: %s %s lens on %s (%.2f deg), but " + "these frames did not come through it -- solving " + "with no FOV gate", + "stated" if train.lens_stated else "assumed", + train.lens.menu_label, + shared_state.camera_type(), + train.fov_degrees, + ) + else: + logger.info( + # Say which of the two it is: under an assumed + # lens the gate is wider than the stated field of + # view implies, and a reader diagnosing "why did + # it solve / not solve" needs to know the lens is + # a fallback rather than something the device was + # told. + "Optical train: %s %s lens on %s, field of view " + "%.2f deg, FOV gate [%.2f, %.2f]", + "stated" if train.lens_stated else "assumed", + train.lens.menu_label, + shared_state.camera_type(), + train.fov_degrees, + *_fov_gate_bounds(train), + ) + # Only meaningful against a gate we are actually + # going to hand over. + _warn_if_outside_solver_database(t3, train) # Every camera frame already carries a tiny radiometer sample # reduced in the camera process. Collect all of them and publish @@ -1076,12 +1099,22 @@ def solver( # view before verification and rejects survivors after # fitting, so this window has to describe the actual # hardware or nothing solves. See docs/adr/0027. - fov_estimate, fov_max_error = train.solver_fov_params() + # + # Under an **unknown optical train** there is nothing + # to derive it from -- the frames came through some + # other optics -- so no gate is passed at all rather + # than a wrong one. Omitting costs the upper bound + # that rejects confident mis-solves, which is a trade + # only defensible because nothing is being pointed at + # the sky. See the third-rung amendment to 0029. + if train_known: + ( + _solver_args["fov_estimate"], + _solver_args["fov_max_error"], + ) = train.solver_fov_params() solution = t3.solve_from_centroids( centroids, (512, 512), - fov_estimate=fov_estimate, - fov_max_error=fov_max_error, match_max_error=0.005, return_matches=True, # Required for SQM calculation target_pixel=shared_state.target_pixel(), diff --git a/python/PiFinder/state.py b/python/PiFinder/state.py index 555b6dd04..c0111eb4d 100644 --- a/python/PiFinder/state.py +++ b/python/PiFinder/state.py @@ -309,6 +309,11 @@ def __init__(self) -> None: # None means "not stated", which resolves to the sensor's shipped lens # -- that is what lets installs predating this setting keep working. self.__camera_lens = config.Config().get_option("camera_lens") + # Whether the frames arriving actually came through the optics the two + # halves above describe. True until a camera says otherwise, so the + # window before the camera process reports behaves like the hardware + # case rather than silently dropping the FOV gate on every boot. + self.__optical_train_known = True # Degrees the camera process rotates the solve/display image relative # to the stored raw frame (PIL CCW). None until the camera reports. self.__solve_image_rotation = None @@ -396,6 +401,18 @@ def set_camera_lens(self, v: Optional[str]): """ self.__camera_lens = v + def optical_train_known(self) -> bool: + """False when the frames did not come through this device's optics. + + See ``CameraInterface.optical_train_known``. Read alongside + ``camera_type``/``camera_lens`` rather than instead of them: the train + still resolves, it just does not describe the frames. + """ + return self.__optical_train_known + + def set_optical_train_known(self, v: bool): + self.__optical_train_known = bool(v) + def sats(self): return self.__sats diff --git a/python/PiFinder/types/coordinates.py b/python/PiFinder/types/coordinates.py index 1ed9da611..0a6ca23b3 100644 --- a/python/PiFinder/types/coordinates.py +++ b/python/PiFinder/types/coordinates.py @@ -32,7 +32,7 @@ def __init__(self, ra: float, dec: float, roll: float, deg=False): @classmethod def from_quaternion(cls, q_eq: quaternion.quaternion): ra, dec, roll = q_eq2radec(q_eq) - return cls(ra, dec, roll) + return cls(ra=ra, dec=dec, roll=roll) def reset(self): """Reset to unset state""" diff --git a/python/tests/test_camera_interface.py b/python/tests/test_camera_interface.py index 034ac5823..e17d134af 100644 --- a/python/tests/test_camera_interface.py +++ b/python/tests/test_camera_interface.py @@ -120,3 +120,46 @@ def test_recovers_after_stuck_capture_clears(self): assert cam.capture_calls == 2 assert cam._capture_thread is not wedged_thread assert cam._capture_thread is None + + +@pytest.mark.unit +class TestOpticalTrainKnown: + """Which cameras are entitled to a derived FOV gate. + + The default direction is the whole point: a camera has to opt *out*, so a + new hardware backend inherits the gate rather than silently losing it and + the mis-solve protection with it. See the third-rung amendment to + docs/adr/0029-fov-gate-width-follows-lens-confidence.md. + """ + + def test_a_camera_pointed_at_the_sky_defaults_to_known(self): + assert _ScriptedCamera().optical_train_known() is True + + def test_the_debug_camera_declares_its_train_unknown(self): + """It replays a recording, so config's lens describes absent glass. + + This is what keeps `--camera debug` solving on a config that states a + lens -- hq x 16mm gates [14.55, 19.69] and hq x 12mm [17.37, 23.49], + against frames that are 10.2 deg. + """ + from PiFinder.camera_debug import CameraDebug + + assert CameraDebug(exposure_time=400000).optical_train_known() is False + + def test_the_debug_camera_still_declares_the_sensor_it_recorded_on(self): + # Unknown is about the *pairing*, not the sensor: SQM's profile lookup + # and the Lens menu still need a plausible half to resolve from. + from PiFinder.camera_debug import CameraDebug + + assert CameraDebug(exposure_time=400000).get_cam_type() == "Debug hq" + + def test_shared_state_assumes_known_before_any_camera_reports(self): + """The boot window every run passes through must keep its gate. + + The camera process publishes this after the solver is already + looping, so a False default would drop the FOV gate on real hardware + for the first few frames of every boot. + """ + from PiFinder.state import SharedStateObj + + assert SharedStateObj().optical_train_known() is True diff --git a/python/tests/test_imu_align.py b/python/tests/test_imu_align.py new file mode 100644 index 000000000..d9c3ce6f0 --- /dev/null +++ b/python/tests/test_imu_align.py @@ -0,0 +1,59 @@ +import numpy as np +import quaternion # Note: numpy-quaternion convention: quaternion(w, x, y, z) +from PiFinder.imu.imu_align.hand_eye_solver import ( + solve_rotation, + simulate_quaternion_measurements, + HandEyeSolverDiagnostics, +) + + +def test_simulate_quaternion_measurements(): + N = 100 # Number of samples to simulate + + # Set the true camera-from-body rotation + true_rotvec = np.radians([10, -5, 20]) + q_12_true = quaternion.from_rotation_vector(true_rotvec) + + # Simulate measurements: + q1, q2 = simulate_quaternion_measurements( + q_12_true, + N=N, + q1_noise_amp=np.deg2rad(0.1), + q2_noise_amp=np.deg2rad(0.1), + seed=0, + ) + assert len(q1) == N + assert len(q2) == N + + +def test_solve_rotation(): + """ + The main block simulates pairs of q1 and q2 measurements and solves + for the q_12 for the quaternion form of the hand-eye problem: + + q1 * q_12 = q_12 * q2 + """ + # Set the true camera-from-body rotation + true_rotvec = np.array([1, 1, 1]) / np.sqrt(3) * np.deg2rad(30) + q_12_true = quaternion.from_rotation_vector(true_rotvec) + + # Simulate measurements: + sigma = np.deg2rad(0.1) + q1, q2 = simulate_quaternion_measurements( + q_12_true, + N=100, + q1_noise_amp=sigma, + q2_noise_amp=sigma, + seed=0, + ) + + # solve + q_12_est, diagnostics = solve_rotation(q1, q2) + assert isinstance(q_12_est, quaternion.quaternion) + assert isinstance(diagnostics, HandEyeSolverDiagnostics) + + # Check error + # q_error = q_12_est.conjugate() * q_12_true + # error_rad = np.linalg.norm(quaternion.as_rotation_vector(q_error)) + # print(f"Uncertainty: {np.rad2deg(diagnostics.sol_angle_error):.3f} degrees.") + # assert error_rad < np.deg2rad(1.0), f"error_rad too large. Got {error_rad:.3f} rad" diff --git a/python/tests/test_lens_self_heal.py b/python/tests/test_lens_self_heal.py index b05eb0682..8b637c763 100644 --- a/python/tests/test_lens_self_heal.py +++ b/python/tests/test_lens_self_heal.py @@ -42,9 +42,10 @@ def get_option(self, option, default=None): class FakeSharedState: - def __init__(self, camera_type="imx462", camera_lens=None): + def __init__(self, camera_type="imx462", camera_lens=None, train_known=True): self._camera_type = camera_type self._camera_lens = camera_lens + self._train_known = train_known self.published = [] def camera_type(self): @@ -53,6 +54,12 @@ def camera_type(self): def camera_lens(self): return self._camera_lens + def optical_train_known(self): + return self._train_known + + def set_optical_train_known(self, value): + self._train_known = value + def set_camera_lens(self, value): self._camera_lens = value self.published.append(value) @@ -76,6 +83,11 @@ def _solve(fov): TWELVE_MM_ON_IMX462 = build_optical_train("imx462", "12mm").fov_degrees SIXTEEN_MM_ON_IMX462 = build_optical_train("imx462", "16mm").fov_degrees +# What tetra3 fits the archived frames in test_images/ at -- the frames the +# debug camera replays. Measured, not derived; test_optics_solving.py asserts +# the same figure against the real solver. +DEBUG_FRAME_FITTED_FOV = 10.20 + def _feed(healer, fov, times): for _ in range(times): @@ -248,6 +260,66 @@ def test_an_unidentifiable_fit_logs_once_not_once_per_frame(self, caplog): assert len(caplog.records) == 1 assert "matches no lens" in caplog.records[0].getMessage() + def test_an_unknown_optical_train_never_promotes(self): + """The exact `--camera debug` case, which used to write. + + The archived frames fit 10.20 deg, which is 1.3% off the hq's derived + 10.33 -- comfortably inside LENS_IDENTIFY_TOLERANCE, so this is not a + measurement self-heal would reject on its merits. It has to be + declined on provenance: the fit measures the train the frames were + *recorded* on, and the developer's config is about a different + machine. Writing 25mm here is what leaves a real imx462 attached + afterwards deriving 6.4 deg and unable to heal its way back. + """ + cfg = FakeConfig() + state = FakeSharedState(camera_type="hq", train_known=False) + healer = LensSelfHeal(cfg, state) + + _feed(healer, DEBUG_FRAME_FITTED_FOV, LENS_IDENTIFY_CONSECUTIVE * 4) + + assert "camera_lens" not in cfg.options + assert state.published == [] + assert cfg.calls == [] + + def test_the_same_fit_would_have_been_promoted_on_real_optics(self): + # Pins the test above to provenance rather than to the number: change + # only train_known and the identical measurement writes. + cfg = FakeConfig() + state = FakeSharedState(camera_type="hq", train_known=True) + healer = LensSelfHeal(cfg, state) + + _feed(healer, DEBUG_FRAME_FITTED_FOV, LENS_IDENTIFY_CONSECUTIVE) + + assert cfg.options["camera_lens"] == "25mm" + + def test_an_unknown_train_logs_once_not_once_per_frame(self, caplog): + cfg = FakeConfig() + state = FakeSharedState(camera_type="hq", train_known=False) + healer = LensSelfHeal(cfg, state) + + with caplog.at_level("INFO", logger="IMU.Integrator"): + _feed(healer, DEBUG_FRAME_FITTED_FOV, 25) + + assert len(caplog.records) == 1 + assert "Optical train is unknown" in caplog.records[0].getMessage() + + def test_a_run_does_not_survive_the_train_going_unknown(self): + """Agreement counted on real optics must not be spent on a recording. + + Ordering matters here: the check sits ahead of the stated-lens branch + precisely so it resets the streak rather than falling through it. + """ + cfg, state = FakeConfig(), FakeSharedState(camera_type="hq") + healer = LensSelfHeal(cfg, state) + + _feed(healer, DEBUG_FRAME_FITTED_FOV, LENS_IDENTIFY_CONSECUTIVE - 1) + state.set_optical_train_known(False) + _feed(healer, DEBUG_FRAME_FITTED_FOV, 1) + state.set_optical_train_known(True) + _feed(healer, DEBUG_FRAME_FITTED_FOV, 1) + + assert "camera_lens" not in cfg.options + @pytest.mark.unit class TestSelfHealIsNeverFatal: diff --git a/python/tests/test_optics.py b/python/tests/test_optics.py index 767b25248..5477ecb45 100644 --- a/python/tests/test_optics.py +++ b/python/tests/test_optics.py @@ -617,7 +617,15 @@ def test_the_debug_frames_field_of_view_sits_inside_the_hq_gate(self): """The frames in test_images/ measure ~10.2 degrees when solved. That is inside hq + 25mm's window and outside imx296 + 16mm's, which - is the entire reason the debug camera's declared sensor changed. + is why the debug camera's declared sensor changed under ADR 0027. + + It is no longer what makes `--camera debug` solve, though, and reading + it that way is how the regression got missed: this fits only because + no lens is stated. State one and the same sensor derives 17.12 or + 20.43 degrees. The gate is now omitted entirely under an **unknown + optical train** -- see TestOpticalTrainKnown in + test_camera_interface.py and the no-gate cases in + test_optics_solving.py. """ measured_debug_frame_fov = 10.2 diff --git a/python/tests/test_optics_solving.py b/python/tests/test_optics_solving.py index 757e57aad..4e119b208 100644 --- a/python/tests/test_optics_solving.py +++ b/python/tests/test_optics_solving.py @@ -14,7 +14,7 @@ from PIL import Image from PiFinder import utils -from PiFinder.optics import build_optical_train +from PiFinder.optics import LENSES, build_optical_train pytestmark = pytest.mark.integration @@ -113,3 +113,83 @@ def test_a_mis_stated_train_rejects_a_perfectly_good_frame( """ solution = _solve(solver, debug_centroids[frame], build_optical_train("imx296")) assert solution.get("RA") is None + + +# Every lens a config can name. The debug camera's sensor is fixed at hq, so +# these are exactly the trains `--camera debug` can find itself resolving -- +# and only one of them is the one the frames were shot through. +STATEABLE_LENSES = tuple(LENSES) + + +def _solve_without_a_gate(solver, centroids): + """What the solver does under an **unknown optical train**. + + Mirrors `solver.py`'s branch rather than re-deriving anything: when the + frames did not come through this device's optics there is nothing to + derive a gate from, so `fov_estimate`/`fov_max_error` are not passed. + """ + return solver.solve_from_centroids( + centroids, + (512, 512), + match_max_error=0.005, + ) + + +@pytest.mark.parametrize("frame", DEBUG_FRAMES) +@pytest.mark.parametrize("lens_key", STATEABLE_LENSES) +def test_a_stated_lens_gates_out_the_debug_frames( + solver, debug_centroids, frame, lens_key +): + """The regression itself, asserted rather than described. + + ADR 0027 fixed the debug camera's *sensor* and left its lens reading live + from config, so stating one pairs hq with glass that is not in the loop: + 16mm implies 17.12 deg and 12mm 20.43, against 10.2 deg frames. Only the + 25mm -- the lens these frames were actually shot through -- still solves, + which is precisely why nobody noticed until a config stated something. + """ + train = build_optical_train("hq", lens_key) + solution = _solve(solver, debug_centroids[frame], train) + + if lens_key == "25mm": + assert solution.get("RA") is not None + else: + assert solution.get("RA") is None, ( + f"{lens_key} on hq derives {train.fov_degrees:.2f} deg; " + f"if this now solves the gate is no longer doing its job" + ) + + +@pytest.mark.parametrize("frame", DEBUG_FRAMES) +@pytest.mark.parametrize("lens_key", STATEABLE_LENSES) +def test_no_gate_solves_the_debug_frames_whatever_config_states( + solver, debug_centroids, frame, lens_key +): + """The fix: `--camera debug` works on any config, including yours. + + The lens is parametrised but unused by the solve on purpose -- that is + the property under test. Under an unknown optical train the config's lens + cannot reach the solver at all, so every one of these has to pass. + """ + solution = _solve_without_a_gate(solver, debug_centroids[frame]) + + assert solution.get("RA") is not None, f"failed to solve with {lens_key} stated" + assert solution["FOV"] == pytest.approx(MEASURED_DEBUG_FRAME_FOV, abs=0.1) + + +@pytest.mark.parametrize("frame", DEBUG_FRAMES) +def test_dropping_in_frames_from_another_train_still_solves( + solver, debug_centroids, frame +): + """Why no gate, rather than the camera declaring the frames' own FOV. + + A developer replacing test_images/ with frames off their own device is + the main use of debug mode. Any declared gate -- including one centred on + 10.2 deg -- would reject those, which is the bug we are fixing wearing a + different hat. Asserted here with the imx296/12mm gate (16.38 deg) as the + stand-in for "a train nobody anticipated": it rejects these frames, and + no-gate does not. + """ + foreign = build_optical_train("imx296", "12mm") + assert _solve(solver, debug_centroids[frame], foreign).get("RA") is None + assert _solve_without_a_gate(solver, debug_centroids[frame]).get("RA") is not None diff --git a/python/uv.lock b/python/uv.lock index bda020730..c0ba0910c 100644 --- a/python/uv.lock +++ b/python/uv.lock @@ -1,3 +1,3 @@ version = 1 revision = 3 -requires-python = ">=3.13" +requires-python = ">=3.9" diff --git a/release_notes/2.6.2.md b/release_notes/2.6.3.md similarity index 100% rename from release_notes/2.6.2.md rename to release_notes/2.6.3.md