From 97760ef096e60fbcb0d55647c033c4746a10840b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 10:25:25 +0000 Subject: [PATCH 01/27] Add ARCHITECTURE.md documenting module structure and integration patterns Captures the two connector generations (generic kafka-connect-rest-source framework used by Fitbit vs. the newer oura-library + thin Connect glue split used by Oura), the runtime polling/auth/conversion flow, config and Docker/CI setup, and a checklist for adding a new vendor integration such as Huawei following the Oura pattern. --- ARCHITECTURE.md | 251 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 251 insertions(+) create mode 100644 ARCHITECTURE.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 00000000..0d716c53 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,251 @@ +# Architecture + +This document describes how RADAR-REST-Connector is put together, so that future contributors +(human or agent) can orient themselves quickly and add new device/API integrations (e.g. Huawei +Health Kit) consistently with the existing patterns. + +## What this repo is + +A multi-module Gradle project providing Kafka Connect **source connectors** that poll third-party +REST APIs (wearable vendor APIs) on behalf of RADAR-base study participants and publish the +resulting data as Avro records on Kafka topics. It currently ships two concrete connectors — +**Fitbit** and **Oura** — built on top of a shared, generic REST-polling framework. + +``` +RADAR-REST-Connector/ +├── kafka-connect-rest-source/ # Generic Kafka Connect REST-source framework (Java) +├── kafka-connect-fitbit-source/ # Fitbit connector (Java), oldest/original implementation +├── oura-library/ # Oura domain logic: routes, converters, requests (Kotlin, no Kafka Connect deps) +├── kafka-connect-oura-source/ # Oura Kafka Connect glue (Java+Kotlin), wraps oura-library +├── docker/ # Docker Compose config templates, launch/ensure scripts, log4j +├── scripts/REDCAP-FITBIT-AUTH-AUTO/ # Standalone Python helper for REDCap-driven Fitbit auth +└── docker-compose.yml # Full local Kafka stack + both connectors, for manual testing +``` + +Root Gradle config (`build.gradle.kts`, `settings.gradle.kts`, `gradle/libs.versions.toml`) uses +the `org.radarbase.radar-kotlin` / `radar-root-project` plugins (from `radar-commons`) for shared +build conventions (Kotlin/Java toolchain, Sentry, versioning). All dependency versions are +centralized in `gradle/libs.versions.toml` (a Gradle version catalog) — add new deps there, not +inline in module build files. + +Avro **schemas are not defined in this repo**. They come from the external `radar-schemas-commons` +artifact (`org.radarbase:radar-schemas-commons`, versioned in the catalog), generated from the +[RADAR-Schemas](https://github.com/RADAR-base/RADAR-Schemas) repository. Adding a new data type +therefore requires a schema to exist there first (e.g. `org.radarcns.connector.oura.OuraDailyActivity`), +before this repo can build an Avro record for it. + +## Two architectural generations + +The repo contains two different design generations. Understand both before adding Huawei, and +prefer the **Oura pattern** for new integrations — it is the more recent, more testable design. + +### 1. Generic `kafka-connect-rest-source` framework (used by Fitbit) + +Located at `kafka-connect-rest-source/src/main/java/org/radarbase/connect/rest/`. Defines a small +set of interfaces meant to be generic across arbitrary REST APIs: + +- `AbstractRestSourceConnector` — Kafka Connect `SourceConnector` base class. Loads + `RestSourceConnectorConfig` from properties and hands out `RestSourceTask` as the task class. +- `RestSourceConnectorConfig` — `AbstractConfig` wrapper exposing the generic + `rest.source.*` properties (base URL, poll interval, topic selector class, payload converter + class, request generator class — all pluggable via `ConfigDef.Type.CLASS`). +- `RequestGenerator` (`request/`) — produces a `Stream` of `RestRequest`s to issue and knows when + the next request is due (`getTimeOfNextRequest()`), driven by the Kafka Connect offset storage + (`setOffsetStorageReader`). +- `RequestRoute` / `PollingRequestRoute` (`request/`) — one "route" = one logical polling + endpoint/data type. Routes own their own per-user polling cadence, backoff, and offset state, and + are notified of `requestSucceeded` / `requestEmpty` / `requestFailed`. +- `PayloadToSourceRecordConverter` (`converter/`) — turns a raw HTTP response body into one or more + Kafka Connect `SourceRecord`s. +- `RestSourceTask` — the actual Kafka Connect `SourceTask`. Its `poll()` loop: sleep until the next + request is due, iterate `requestGenerator.requests()`, execute the first request that yields + records, return them. + +This module is intentionally protocol-agnostic; it has no notion of "Fitbit" or OAuth. It's a +reasonable place to fix or extend genuinely generic REST-polling behavior (e.g. topic selection, +generic retry semantics), but new device integrations do **not** need to hook into it directly — +see the Oura pattern below. + +### 2. Fitbit connector (`kafka-connect-fitbit-source`) — first concrete integration + +Built directly on the generic framework above, entirely in Java: + +- `FitbitSourceConnector extends AbstractRestSourceConnector` — schedules a periodic + (`application.loop.interval.ms`) user-repository refresh; if the user set changes, requests task + reconfiguration (`context.requestTaskReconfiguration()`). Divides users across `tasks.max` tasks + by hashing `user.getVersionedId()`. +- `FitbitRequestGenerator extends RequestGeneratorRouter` — builds the list of enabled + `RequestRoute`s (one per Fitbit data type: sleep, activity log, resting heart rate, and — if + `fitbit.api.intraday=true` — steps, heart rate, HRV, breathing rate, skin temperature, calories, + SpO2) and an OkHttp client per user with a `TokenAuthenticator` (auto-refreshes on HTTP 401). +- `route/Fitbit*Route` — one class per data type, extending `FitbitPollingRoute`, which implements + a fairly elaborate polling algorithm: don't poll more than once per `pollInterval`; walk history + back to `HISTORICAL_TIME_DAYS`; avoid re-reading the last `LOOKBACK_TIME` to tolerate + late-arriving data from other devices; back off per-user on HTTP 429 (`TOP_OF_HOUR` or + `ROLLING_WINDOW` cooldown strategy) and after `fitbit.request.max.forbidden` consecutive HTTP 403s. +- `converter/Fitbit*AvroConverter` — one class per data type, converts JSON to the corresponding + Avro record from `radar-schemas-commons`. +- `user/UserRepository` (interface) + implementations: + - `YamlUserRepository` — reads per-user YAML files from a directory (`docker/fitbit-user.yml.template` + shows the format: id, projectId, userId, sourceId, startDate/endDate, externalUserId, OAuth2 + access/refresh tokens). + - `ServiceUserRepository` (Kotlin) — talks to an external "rest-source-authorizer" webservice + (typically fronted by ManagementPortal) for user lists and token refresh/storage, using + OAuth2 client-credentials auth. + - `firebase/FirebaseUserRepository` / `CovidCollabFirebaseUserRepository` — legacy + Firestore-backed repository for a specific historical deployment. +- `request/TokenAuthenticator` — OkHttp `Authenticator` that refreshes the access token via the + `UserRepository` on a 401 and retries the request. + +### 3. Oura connector (`oura-library` + `kafka-connect-oura-source`) — newer pattern + +This is the template to follow for a **new** vendor integration such as Huawei. It splits cleanly +into: + +- **`oura-library`** (pure Kotlin, no Kafka Connect / OkHttp-Authenticator coupling to Connect + internals) — all vendor-specific domain logic, independently unit-testable and in principle + reusable outside Kafka Connect: + - `user/User`, `user/UserRepository` — user model and repository interface (`get`, `stream`, + `getAccessToken`); note `refreshAccessToken` lives on the connector-side implementation, not + here, in current Oura code (asymmetry vs. Fitbit — be aware when implementing). + - `route/Route` (interface) + `route/OuraRoute` (abstract base) + `route/Oura*Route` (one per + data type: daily activity, readiness, sleep, SpO2, heart rate, personal info, sessions, + workouts, tags, ring configuration, stress, VO2 max, resilience, cardiovascular age, enhanced + tags, rest-mode periods, sleep-time recommendations, etc.) — each route knows its API sub-path + and builds one or more `RestRequest`s covering a `[start, end)` window, chunked by + `maxIntervalPerRequest`. + - `route/OuraRouteFactory` — central list of all routes (used mainly by tests / defaults; the + connector module actually builds its own filtered list based on config flags — see below). + - `converter/OuraDataConverter` (→ `RecordConverter`) + `converter/Oura*Converter` — one per data + type, parses the JSON response into `TopicData(topic, key, value, offset)` where `value` is a + generated Avro `SpecificRecord` from `radar-schemas-commons` (e.g. `OuraDailyActivity`). This + is the direct analogue of Fitbit's `Fitbit*AvroConverter`, but returns plain data objects + instead of Kafka Connect `SourceRecord`s directly — the Connect-specific wrapping happens in + the connector module. + - `request/OuraRequestGenerator` — the polling brain: for each `(route, user)` pair, computes the + offset to resume from (via `OuraOffsetManager`), decides whether to use a large "historical" + chunk (`HISTORICAL_QUERY_RANGE` = 1 year, once `timeSinceStart > HISTORICAL_DATA_THRESHOLD` = 1 + year) or normal recent-data chunking, and interprets HTTP responses + (`handleResponse`/`requestSuccessful`/`requestFailed`) into typed `OuraResult`/`OuraError` + sealed hierarchies with per-(route,user) backoff bookkeeping (`routeNextRequest` map) for 429 / + 403 / 401 / 400 / 422 / 404 / other. + - `request/OuraOffsetManager` (interface) — abstraction for reading/writing per-(route,user) + offsets; the Kafka Connect implementation is `KafkaOffsetManager` in the connector module. + - `offset/Offset`, `offset/Offsets` — plain offset value types. +- **`kafka-connect-oura-source`** (Java, some Kotlin) — the thin Kafka Connect glue: + - `OuraSourceConnector` — same role as `FitbitSourceConnector`: periodic user refresh, + reconfiguration on user-set change, hash-based task partitioning. + - `OuraSourceTask` — Kafka Connect `SourceTask`. Builds the enabled `Route` list from config + flags, constructs `OuraRequestGenerator`, and in `poll()` round-robins across routes + (`getRotatedRoutes()`, so one slow/rate-limited route doesn't starve the others), executes one + HTTP request via a shared `OkHttpClient`, and converts the resulting `TopicData` list into + Kafka Connect `SourceRecord`s using `AvroData` (Confluent's Kotlin/Avro↔Connect-schema bridge). + - `offset/KafkaOffsetManager` — `OuraOffsetManager` backed by Kafka Connect's + `OffsetStorageReader`. + - `user/OuraUserRepository` (abstract) / `OuraServiceUserRepository` — the concrete + "rest-source-authorizer" HTTP client, built on **Ktor** (not OkHttp) with `radar-commons`'s + `CachedSet`/`CachedValue`/`clientCredentials` helpers for user list caching and per-user OAuth2 + token caching/refresh. This is the modern replacement for Fitbit's + `ServiceUserRepository`/`TokenAuthenticator` combo, and is the pattern to copy for Huawei. + - `OuraRestSourceConnectorConfig` — `ConfigDef` with one `oura..enabled` boolean and topic + name per data type, plus `oura.user.repository.*` connection settings. + +**Key structural difference from Fitbit**: Oura does *not* extend the generic +`kafka-connect-rest-source` interfaces (`RequestRoute`, `PollingRequestRoute`, +`PayloadToSourceRecordConverter`) at all — `OuraSourceTask` implements Kafka Connect's `SourceTask` +directly and drives `oura-library`'s own `Route`/`RequestGenerator`/`RecordConverter` abstractions. +This was a deliberate move to (a) get domain logic under unit test without spinning up Kafka +Connect, and (b) avoid the generic framework's assumptions (e.g. its polling-interval math) that +didn't fit Oura's simpler historical/recent chunking model. + +## Runtime data flow (both connectors, conceptually) + +```mermaid +sequenceDiagram + participant connector as SourceConnector + participant task as SourceTask + participant userRepo as User Repository (rest-source-authorizer) + participant api as Vendor API (Fitbit/Oura/…) + participant kafka as Kafka + + connector ->> userRepo: Poll for users/config changes (periodic) + connector ->> connector: Partition users across tasks.max tasks + loop poll() + task ->> task: Determine next due (route, user) request + task ->> userRepo: Get/refresh OAuth2 access token + task ->> api: GET data for date range + api -->> task: JSON response + task ->> task: Convert JSON -> Avro SourceRecord(s) + task ->> kafka: Return records (Connect framework produces them) + task ->> task: Update in-memory + Connect offset state + end +``` + +User authentication/authorization data (OAuth2 tokens, study/user/source IDs, start/end dates) is +**not** stored in this repo. In production it's served by a "rest-source-authorizer" webservice +(part of RADAR-base, typically backed by ManagementPortal); for local/manual testing, Fitbit also +supports flat YAML files under `docker/users/` (`YamlUserRepository`). + +## Configuration model + +Every connector exposes its settings as a Kafka Connect `ConfigDef` (`org.apache.kafka.common.config`), +loaded from a Java `.properties` file (see `docker/source-fitbit.properties.template` and +`docker/source-oura.properties.template`) referenced by `connector.class`, `name`, `tasks.max`, +plus vendor-specific keys, e.g.: + +- `.api.client` / `.api.secret` — OAuth2 app credentials. +- `.user.repository.class` — pluggable `UserRepository` implementation. +- `.user.repository.url` / `.client.id` / `.client.secret` / `.oauth2.token.url` — + rest-source-authorizer connection details. +- `..topic` / `.enabled` — per-data-type Kafka topic name and on/off switch, so + studies can disable data types they don't need. + +The full current list for Fitbit is documented in `README.md`; Oura's config lives in +`OuraRestSourceConnectorConfig` (no README table yet — check the class directly). + +## Docker / deployment + +Each connector module has its own multi-stage `Dockerfile` (Gradle build stage → base image +`confluentinc/cp-kafka-connect-base`), publishing built jars plus third-party deps into +`$CONNECT_PLUGIN_PATH//`. `docker/launch` and `docker/ensure` are modified Confluent +entrypoint scripts (env-var → properties translation, Kafka-readiness wait). `docker-compose.yml` +spins up a full local Zookeeper+Kafka+SchemaRegistry+REST-proxy cluster plus both connectors for +manual end-to-end testing (`docker-compose up -d --build`, inspect with +`kafka-avro-console-consumer`). Sentry error monitoring is wired in via `radarKotlin { sentryEnabled = true }` +and configured purely through `SENTRY_DSN`/`SENTRY_*` env vars — see README "Sentry monitoring". + +## Testing + +- `kafka-connect-rest-source/src/test`, `kafka-connect-fitbit-source/src/test`, + `kafka-connect-oura-source/src/test` currently only contain config-parsing tests + (`*ConnectorConfigTest`) plus one task test — test coverage of the actual polling/conversion + logic is thin. `wiremock` and `mockito` are on the version catalog for HTTP-level testing but not + yet exercised much; `oura-library`'s pure-Kotlin design makes it the easiest place to add real + unit tests for new routes/converters without Kafka Connect scaffolding. +- CI (`.github/workflows/main.yml`) runs `./gradlew assemble` and `./gradlew check` on every push/PR + to `master`/`dev`, then builds (and on `push`, publishes) multi-arch Docker images per connector + module via a matrix job. `release.yml` does the same on GitHub Release publish, additionally + uploading built jars as release assets, tagged `vX.Y.Z` from `gradle.properties`/version catalog. + +## Adding a new vendor integration (e.g. Huawei) + +Follow the **Oura pattern**, not the Fitbit one: + +1. New Gradle module `huawei-library` (pure Kotlin, mirrors `oura-library`): `user/`, `route/`, + `converter/`, `request/`, `offset/` packages. No Kafka Connect or OkHttp-Connect-specific types + here — keep it independently testable. +2. New Gradle module `kafka-connect-huawei-source` (mirrors `kafka-connect-oura-source`): + `HuaweiSourceConnector`, `HuaweiSourceTask`, `HuaweiRestSourceConnectorConfig`, + `offset/KafkaOffsetManager`, `user/HuaweiServiceUserRepository` (Ktor-based + rest-source-authorizer client, copy `OuraServiceUserRepository`'s structure), plus a + `Dockerfile`. +3. Register both modules in `settings.gradle.kts`; add any new dependency versions to + `gradle/libs.versions.toml` first. +4. Confirm (or add) the required Avro schemas in the external RADAR-Schemas project and bump the + `radarSchemas` version in the catalog once published — this repo cannot invent schemas locally. +5. One `Route`/`Converter` pair per Huawei data type you plan to support, each independently + togglable via a `huawei..enabled` config flag, matching the Oura/Fitbit convention. +6. Add `docker/source-huawei.properties.template`, a `docker-compose.yml` service entry, and a + README config table, following the Fitbit/Oura sections as templates. +7. Add the new Docker image to the `IMAGES` matrix in both `.github/workflows/main.yml` and + `release.yml`. From 211c3d565f1972ac351c29c33790e78d1528b220 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 10:51:46 +0000 Subject: [PATCH 02/27] Scaffold huawei-library and kafka-connect-huawei-source modules Registers the two new Gradle modules, pins radar-schemas-commons 0.9.0-SNAPSHOT (huawei_schemas branch not yet released) as a separate version-catalog entry so existing modules keep the stable release, and adds the user/UserRepository domain types mirroring oura-library's pattern. --- gradle/libs.versions.toml | 4 ++ huawei-library/build.gradle | 64 +++++++++++++++++++ .../org/radarbase/huawei/user/HuaweiUser.kt | 28 ++++++++ .../kotlin/org/radarbase/huawei/user/User.kt | 22 +++++++ .../huawei/user/UserNotAuthorizedException.kt | 5 ++ .../radarbase/huawei/user/UserRepository.kt | 33 ++++++++++ kafka-connect-huawei-source/Dockerfile | 62 ++++++++++++++++++ kafka-connect-huawei-source/build.gradle.kts | 56 ++++++++++++++++ settings.gradle.kts | 2 + 9 files changed, 276 insertions(+) create mode 100644 huawei-library/build.gradle create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt create mode 100644 kafka-connect-huawei-source/Dockerfile create mode 100644 kafka-connect-huawei-source/build.gradle.kts diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 1bad2750..bf8a325a 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -16,6 +16,9 @@ sentryOpenTelemetryAgent = "8.36.0" okhttp = "4.12.0" firebaseAdmin = "9.8.0" radarSchemas = "0.8.16" +# Huawei connector schemas are not yet released; pin to the published snapshot until a stable +# radar-schemas-commons release containing the huawei_schemas branch is cut. +radarSchemasHuawei = "0.9.0-SNAPSHOT" # @pin Upgrade to 3.x.x requires kotlin v2 minimum ktor = "2.3.13" wiremock = "3.0.1" @@ -28,6 +31,7 @@ lz4 = "1.10.1" lz4 = { module = "at.yawk.lz4:lz4-java", version.ref = "lz4" } radar-commons-kotlin = { module = "org.radarbase:radar-commons-kotlin", version.ref = "radarCommons" } radar-schemas-commons = { module = "org.radarbase:radar-schemas-commons", version.ref = "radarSchemas" } +radar-schemas-commons-huawei = { module = "org.radarbase:radar-schemas-commons", version.ref = "radarSchemasHuawei" } kafka-connect-api = "org.apache.kafka:connect-api:7.8.1-ce" kafka-connect-avro-converter = { module = "io.confluent:kafka-connect-avro-converter", version.ref = "confluent" } okhttp = { module = "com.squareup.okhttp3:okhttp", version.ref = "okhttp" } diff --git a/huawei-library/build.gradle b/huawei-library/build.gradle new file mode 100644 index 00000000..ce391e17 --- /dev/null +++ b/huawei-library/build.gradle @@ -0,0 +1,64 @@ + +group = 'org.radarbase' +version = '0.0.1' + +apply plugin: 'maven-publish' + +repositories { + // Use jcenter for resolving dependencies. + // You can declare any Maven/Ivy/file repository here. + mavenCentral() + + // radar-schemas-commons huawei_schemas is only published as a snapshot; declare the + // candidate snapshot hosts here so the huawei-library build can resolve it regardless of + // which one the RADAR-Schemas release pipeline currently targets. + maven { + url = uri("https://central.sonatype.com/repository/maven-snapshots/") + } + maven { + url = uri("https://s01.oss.sonatype.org/content/repositories/snapshots/") + } + maven { + url = uri("https://maven.pkg.github.com/RADAR-base/RADAR-Schemas") + credentials { + username = project.findProperty("public.gpr.user") ?: System.getenv("GPR_USER") + password = project.findProperty("public.gpr.token") ?: System.getenv("GPR_TOKEN") + } + } +} + +dependencies { + // Use the Kotlin JDK 8 standard library. + implementation libs.kotlin.stdlib + + implementation libs.okhttp + + implementation libs.radar.schemas.commons.huawei + + implementation libs.jackson.annotations + + implementation libs.jackson.databind + + implementation libs.avro + + implementation libs.jackson.datatype.jsr310 + + // Use the Kotlin test library. + testImplementation libs.kotlin.test + + // Use the Kotlin JUnit integration. + testImplementation libs.kotlin.test.junit +} + +project.afterEvaluate { + publishing { + publications { + library(MavenPublication) { + setGroupId "$group" + setArtifactId "huawei-library" + version "$version" + from components.java + } + } + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt new file mode 100644 index 00000000..7d1521e3 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt @@ -0,0 +1,28 @@ +package org.radarbase.huawei.user + +import com.fasterxml.jackson.annotation.JsonIgnoreProperties +import com.fasterxml.jackson.annotation.JsonProperty +import org.radarcns.kafka.ObservationKey +import java.time.Instant + +@JsonIgnoreProperties(ignoreUnknown = true) +data class HuaweiUser( + @JsonProperty("id") override val id: String, + @JsonProperty("createdAt") override val createdAt: Instant, + @JsonProperty("projectId") override val projectId: String, + @JsonProperty("userId") override val userId: String, + @JsonProperty("humanReadableUserId") override val humanReadableUserId: String?, + @JsonProperty("sourceId") override val sourceId: String, + @JsonProperty("externalId") override val externalId: String?, + @JsonProperty("isAuthorized") override val isAuthorized: Boolean, + @JsonProperty("startDate") override val startDate: Instant, + @JsonProperty("endDate") override val endDate: Instant? = null, + @JsonProperty("version") override val version: String? = null, + @JsonProperty("serviceUserId") override val serviceUserId: String? = null, +) : User { + override val observationKey: ObservationKey = ObservationKey(projectId, userId, sourceId) + override val versionedId: String = "$id${version?.let { "#$it" } ?: ""}" + + fun isComplete() = + isAuthorized && (endDate == null || startDate.isBefore(endDate)) && serviceUserId != null +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt new file mode 100644 index 00000000..b84dfe76 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt @@ -0,0 +1,22 @@ +package org.radarbase.huawei.user + +import org.radarcns.kafka.ObservationKey +import java.time.Instant + +interface User { + val id: String + val projectId: String + val userId: String + val sourceId: String + val externalId: String? + val startDate: Instant + val endDate: Instant? + val createdAt: Instant + val humanReadableUserId: String? + val serviceUserId: String? + val version: String? + val isAuthorized: Boolean + + val observationKey: ObservationKey + val versionedId: String +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt new file mode 100644 index 00000000..1bd513b0 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt @@ -0,0 +1,5 @@ +package org.radarbase.huawei.user + +class UserNotAuthorizedException(message: String) : Exception(message) { + constructor(user: User) : this("User ${user.id} is not authorized") +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt new file mode 100644 index 00000000..6f26b74b --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt @@ -0,0 +1,33 @@ +package org.radarbase.huawei.user + +import java.io.IOException + +/** User repository for Huawei Health Kit users. */ +interface UserRepository { + /** + * Get specified user. + * + * @throws IOException if the user cannot be retrieved from the repository. + */ + @Throws(IOException::class) + operator fun get(key: String): User? + + /** + * Get all relevant users. + * + * @throws IOException if the list cannot be retrieved from the repository. + */ + @Throws(IOException::class) + fun stream(): Sequence + + /** + * Get the current access token of given user. + * + * @throws IOException if the new access token cannot be retrieved from the repository. + * @throws UserNotAuthorizedException if the refresh token is no longer valid. Manual action + * should be taken to get a new refresh token. + * @throws NoSuchElementException if the user does not exist in this repository. + */ + @Throws(IOException::class, UserNotAuthorizedException::class) + fun getAccessToken(user: User): String +} diff --git a/kafka-connect-huawei-source/Dockerfile b/kafka-connect-huawei-source/Dockerfile new file mode 100644 index 00000000..e01bcb7f --- /dev/null +++ b/kafka-connect-huawei-source/Dockerfile @@ -0,0 +1,62 @@ +# Copyright 2018 The Hyve +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM --platform=$BUILDPLATFORM gradle:8.14-jdk17 AS builder + +RUN mkdir /code +WORKDIR /code + +ENV GRADLE_USER_HOME=/code/.gradlecache \ + GRADLE_OPTS="-Dorg.gradle.vfs.watch=false -Djdk.lang.Process.launchMechanism=vfork" + +COPY ./gradle/libs.versions.toml /code/gradle/ +COPY ./build.gradle.kts ./settings.gradle.kts ./gradle.properties /code/ +COPY kafka-connect-huawei-source/build.gradle.kts /code/kafka-connect-huawei-source/ +COPY huawei-library/build.gradle /code/huawei-library/ + +RUN gradle downloadDependencies copyDependencies + +COPY ./kafka-connect-huawei-source/src/ /code/kafka-connect-huawei-source/src +COPY ./huawei-library/src/ /code/huawei-library/src + +RUN gradle jar + +FROM confluentinc/cp-kafka-connect-base:7.8.7 + +USER appuser + +LABEL org.opencontainers.image.authors="yatharth.ranjan@kcl.ac.uk" + +LABEL description="Kafka Huawei Health Kit REST API Source connector" + +ENV CONNECT_PLUGIN_PATH="/usr/share/java/kafka-connect/plugins" \ + WAIT_FOR_KAFKA="1" + +# To isolate the classpath from the plugin path as recommended +COPY --from=builder /code/kafka-connect-huawei-source/build/third-party/*.jar ${CONNECT_PLUGIN_PATH}/kafka-connect-huawei-source/ +COPY --from=builder /code/huawei-library/build/third-party/*.jar ${CONNECT_PLUGIN_PATH}/kafka-connect-huawei-source/ + +COPY --from=builder /code/kafka-connect-huawei-source/build/libs/*.jar ${CONNECT_PLUGIN_PATH}/kafka-connect-huawei-source/ +COPY --from=builder /code/huawei-library/build/libs/*.jar ${CONNECT_PLUGIN_PATH}/kafka-connect-huawei-source/ + +# Load topics validator +COPY --chown=appuser:appuser ./docker/ensure /etc/confluent/docker/ensure + +# Load modified launcher +COPY --chown=appuser:appuser ./docker/launch /etc/confluent/docker/launch + +# Overwrite the log4j configuration to include Sentry monitoring. +COPY ./docker/log4j.properties.template /etc/confluent/docker/log4j.properties.template +# Copy Sentry monitoring jars. +COPY --from=builder /code/kafka-connect-huawei-source/build/third-party/sentry-* /etc/kafka-connect/jars diff --git a/kafka-connect-huawei-source/build.gradle.kts b/kafka-connect-huawei-source/build.gradle.kts new file mode 100644 index 00000000..f3169abe --- /dev/null +++ b/kafka-connect-huawei-source/build.gradle.kts @@ -0,0 +1,56 @@ +description = "Kafka connector for Huawei Health Kit API source" + +repositories { + // radar-schemas-commons huawei_schemas is only published as a snapshot; declare the + // candidate snapshot hosts here so the build can resolve it regardless of which one the + // RADAR-Schemas release pipeline currently targets. + maven { + url = uri("https://central.sonatype.com/repository/maven-snapshots/") + } + maven { + url = uri("https://s01.oss.sonatype.org/content/repositories/snapshots/") + } + maven { + url = uri("https://maven.pkg.github.com/RADAR-base/RADAR-Schemas") + credentials { + username = project.findProperty("public.gpr.user") as String? ?: System.getenv("GPR_USER") + password = project.findProperty("public.gpr.token") as String? ?: System.getenv("GPR_TOKEN") + } + } +} + +dependencies { + + /* The entries in the block below are added here to force the version of + * transitive dependencies and mitigate reported vulnerabilities + */ + implementation(libs.netty.handler.proxy) + implementation(libs.netty.handler) + + api(project(":huawei-library")) + api(libs.kafka.connect.avro.converter) + api(libs.radar.schemas.commons.huawei) + implementation(libs.radar.commons.kotlin) + + api(libs.okhttp) + implementation(platform(libs.jackson.bom)) + implementation(libs.jackson.dataformat.yaml) + implementation(libs.jackson.datatype.jsr310) + implementation(libs.kotlin.stdlib) + + implementation(libs.ktor.client.auth) + implementation(libs.ktor.client.content.negotiation) + implementation(libs.ktor.serialization.jackson) + implementation(libs.ktor.client.cio) + implementation(libs.ktor.serialization.kotlinx.json) + implementation(libs.jackson.module.kotlin) + + // Included in connector runtime + compileOnly(libs.kafka.connect.api) + compileOnly(platform(libs.jackson.bom)) + compileOnly(libs.jackson.databind) + + testImplementation(libs.kafka.connect.api) + testImplementation(libs.wiremock) + testImplementation(libs.mockito.core) +} diff --git a/settings.gradle.kts b/settings.gradle.kts index 64f23944..260feaed 100644 --- a/settings.gradle.kts +++ b/settings.gradle.kts @@ -3,6 +3,8 @@ include(":kafka-connect-fitbit-source") include(":kafka-connect-rest-source") include(":kafka-connect-oura-source") include(":oura-library") +include(":kafka-connect-huawei-source") +include(":huawei-library") pluginManagement { repositories { From d51979e39a398c187a512bef93957d21f75976eb Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 11:05:47 +0000 Subject: [PATCH 03/27] Implement huawei-library: routes, converters, and request generation Adds the domain logic for the Huawei Health Kit connector, mirroring oura-library's split of pure-Kotlin logic from Kafka Connect glue: - request/: RestRequest, RequestGenerator, HuaweiRequestGenerator (offset tracking, per-route backoff on 401/403/429/etc), HuaweiResult/HuaweiError. - route/: HuaweiRoute base (OAuth2-authorized GET/POST + time-range chunking), three concrete route kinds covering the Health Kit Data API's endpoints - HuaweiSampleSetRoute (POST sampleSet:polymerize, raw or groupByTime-aggregated), HuaweiHealthRecordRoute (GET healthRecords), HuaweiActivityRecordRoute (GET activityRecords) - and HuaweiRouteFactory, a single registry mapping all ~54 Huawei data types from the radar-huawei-connector schema spec to their endpoint, dataTypeName, and Avro record builder. - converter/: FieldValues (typed accessor for Huawei's field-value sample point format) and generic converters that turn API responses into TopicData for the registered Avro records. Field-value key names are best-effort (Huawei HiHealth Field naming convention); this was verified to compile against a locally-published radar-schemas-commons 0.9.0-SNAPSHOT (huawei_schemas branch) since none of the real snapshot hosts are reachable from this sandbox - flagged in comments for verification against a live API response. --- huawei-library/build.gradle | 5 + .../radarbase/huawei/converter/FieldValues.kt | 58 ++ .../HuaweiActivityRecordConverter.kt | 85 +++ .../huawei/converter/HuaweiDataConverter.kt | 41 ++ .../converter/HuaweiHealthRecordConverter.kt | 49 ++ .../converter/HuaweiSampleSetConverter.kt | 53 ++ .../huawei/converter/RecordConverter.kt | 19 + .../huawei/converter/SequenceExtensions.kt | 11 + .../radarbase/huawei/converter/TopicData.kt | 11 + .../org/radarbase/huawei/offset/Offset.kt | 11 + .../org/radarbase/huawei/offset/Offsets.kt | 5 + .../huawei/request/HuaweiOffsetManager.kt | 13 + .../huawei/request/HuaweiRequestGenerator.kt | 190 ++++++ .../radarbase/huawei/request/HuaweiResult.kt | 64 ++ .../huawei/request/RequestGenerator.kt | 19 + .../radarbase/huawei/request/RestRequest.kt | 14 + .../request/TooManyRequestsException.kt | 3 + .../huawei/route/HuaweiActivityRecordRoute.kt | 46 ++ .../huawei/route/HuaweiHealthRecordRoute.kt | 58 ++ .../org/radarbase/huawei/route/HuaweiRoute.kt | 62 ++ .../huawei/route/HuaweiRouteDefinition.kt | 19 + .../huawei/route/HuaweiRouteFactory.kt | 566 ++++++++++++++++++ .../huawei/route/HuaweiSampleSetRoute.kt | 75 +++ .../org/radarbase/huawei/route/Route.kt | 23 + kafka-connect-huawei-source/build.gradle.kts | 5 + 25 files changed, 1505 insertions(+) create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt diff --git a/huawei-library/build.gradle b/huawei-library/build.gradle index ce391e17..882ad1e3 100644 --- a/huawei-library/build.gradle +++ b/huawei-library/build.gradle @@ -9,6 +9,11 @@ repositories { // You can declare any Maven/Ivy/file repository here. mavenCentral() + // Prefer a locally-published snapshot (e.g. built by hand from the RADAR-Schemas + // huawei_schemas branch via `./gradlew :radar-schemas-commons:publishToMavenLocal`) before + // falling back to remote snapshot hosts. + mavenLocal() + // radar-schemas-commons huawei_schemas is only published as a snapshot; declare the // candidate snapshot hosts here so the huawei-library build can resolve it regardless of // which one the RADAR-Schemas release pipeline currently targets. diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt new file mode 100644 index 00000000..57386986 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt @@ -0,0 +1,58 @@ +package org.radarbase.huawei.converter + +import com.fasterxml.jackson.databind.JsonNode + +/** + * Typed accessor for a single Huawei Health Kit sample point's field values. + * + * The Health Kit Data API (`sampleSet:polymerize`) represents each field of a sample point using + * the same typed-value wrapper as the on-device HiHealth SDK's `Field`/`Value` model: a list of + * objects shaped like `{"fieldName": "steps_delta", "integerValue": 123}` (or `floatValue`, + * `longValue`, `stringValue` depending on the field's declared type). This class also tolerates a + * flattened `{"fieldName": value, ...}` object, in case a particular endpoint or API version + * returns the simplified shape, so a single parser can be reused across all sample-set based + * routes. + * + * Field name constants follow Huawei's public `Field` identifiers (e.g. `steps_delta`, `calories`, + * `avg`, `max`, `min`), as documented for the on-device and REST Health Kit APIs. + */ +class FieldValues private constructor(private val values: Map) { + + fun getInt(field: String): Int? = values[field]?.let { if (it.isNull) null else it.asInt() } + + fun getLong(field: String): Long? = values[field]?.let { if (it.isNull) null else it.asLong() } + + fun getDouble(field: String): Double? = values[field]?.let { if (it.isNull) null else it.asDouble() } + + fun getFloat(field: String): Float? = getDouble(field)?.toFloat() + + fun getString(field: String): String? = values[field]?.let { if (it.isNull) null else it.asText() } + + companion object { + private const val FIELD_NAME_KEY = "fieldName" + private val VALUE_KEYS = listOf("integerValue", "floatValue", "longValue", "stringValue", "value") + + fun from(node: JsonNode?): FieldValues { + if (node == null || node.isMissingNode || node.isNull) { + return FieldValues(emptyMap()) + } + if (node.isArray) { + val map = LinkedHashMap() + node.forEach { entry -> + val name = entry.get(FIELD_NAME_KEY)?.asText() ?: return@forEach + val value = VALUE_KEYS.firstNotNullOfOrNull { key -> entry.get(key) } + if (value != null) { + map[name] = value + } + } + return FieldValues(map) + } + if (node.isObject) { + val map = LinkedHashMap() + node.properties().forEach { (name, value) -> map[name] = value } + return FieldValues(map) + } + return FieldValues(emptyMap()) + } + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt new file mode 100644 index 00000000..ebe7d40f --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt @@ -0,0 +1,85 @@ +package org.radarbase.huawei.converter + +import com.fasterxml.jackson.databind.JsonNode +import org.radarbase.huawei.user.User +import org.radarcns.connector.huawei.HuaweiActivityRecord +import java.time.Instant + +/** + * Converts `GET /healthkit/v1/activityRecords` responses into [HuaweiActivityRecord]s. + * + * Field names below follow the Huawei Health Kit `ActivityRecord`/`Device`/`ActivitySummary` + * model (activity record id, name, description, time zone, activity type, device manufacturer and + * type, and a nested activity summary with pace/data/section statistics). Nested JSON structures + * that map to free-form Avro `string` fields (pace map, data summary, section summary) are kept as + * their raw JSON text, since their internal shape varies by activity type. + */ +class HuaweiActivityRecordConverter( + private val topic: String = "connect_huawei_activity_record", +) : HuaweiDataConverter { + + override fun processRecords(root: JsonNode, user: User): Sequence> { + val timeReceived = Instant.now() + val records = root.get("activityRecords") ?: root.get("records") ?: return emptySequence() + return records.asSequence() + .mapCatching { record -> + val startTime = record.epochInstant("startTime") + ?: error("Huawei activity record is missing startTime") + TopicData( + topic = topic, + key = user.observationKey, + offset = startTime.epochSecond, + value = record.toActivityRecord(startTime, timeReceived), + ) + } + } + + private fun JsonNode.toActivityRecord( + startTime: Instant, + timeReceived: Instant, + ): HuaweiActivityRecord { + val device = this.get("device") + val summary = this.get("activitySummary") + return HuaweiActivityRecord.newBuilder().apply { + time = startTime.toEpoch() + this.timeReceived = timeReceived.toEpoch() + endTime = epochInstant("endTime")?.toEpoch() + activityRecordId = textOrNull("id") ?: textOrNull("activityRecordId") + name = textOrNull("name") + description = textOrNull("description") + timeZone = textOrNull("timeZone") + activityTypeId = textOrNull("activityType") ?: textOrNull("activityTypeId") + activeTimeMillis = longOrNull("activeTime") ?: longOrNull("activeTimeMillis") + isKeepGoing = boolOrNull("isKeepGoing") + deviceManufacturer = device?.textOrNull("manufacturer") + deviceType = device?.intOrNull("type") + activitySummaryAvgPace = summary?.doubleOrNull("avgPace") + activitySummaryBestPace = summary?.doubleOrNull("bestPace") + activitySummaryPaceMap = summary?.get("paceMap")?.toString() + activitySummaryDataSummary = summary?.get("dataSummary")?.toString() + activitySummarySectionSummary = summary?.get("sectionSummary")?.toString() + }.build() + } + + private fun JsonNode.epochInstant(field: String): Instant? { + val value = this.get(field) ?: return null + if (value.isNull) return null + val millis = if (value.isTextual) value.asText().toLongOrNull() else value.asLong() + return millis?.let { Instant.ofEpochMilli(it) } + } + + private fun JsonNode.textOrNull(field: String): String? = + this.get(field)?.takeUnless { it.isNull }?.asText() + + private fun JsonNode.intOrNull(field: String): Int? = + this.get(field)?.takeUnless { it.isNull }?.asInt() + + private fun JsonNode.longOrNull(field: String): Long? = + this.get(field)?.takeUnless { it.isNull }?.asLong() + + private fun JsonNode.doubleOrNull(field: String): Double? = + this.get(field)?.takeUnless { it.isNull }?.asDouble() + + private fun JsonNode.boolOrNull(field: String): Boolean? = + this.get(field)?.takeUnless { it.isNull }?.asBoolean() +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt new file mode 100644 index 00000000..6b095587 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt @@ -0,0 +1,41 @@ +package org.radarbase.huawei.converter + +import com.fasterxml.jackson.databind.JsonNode +import okhttp3.Headers +import org.radarbase.huawei.request.HuaweiRequestGenerator.Companion.JSON_READER +import org.radarbase.huawei.request.RestRequest +import org.radarbase.huawei.user.User +import java.time.Instant + +/** + * Converts a Huawei Health Kit HTTP JSON response body to zero or more [TopicData] records. + */ +interface HuaweiDataConverter : RecordConverter { + /** Process the JSON records generated by given request. */ + fun processRecords( + root: JsonNode, + user: User, + ): Sequence> + + override fun convert( + request: RestRequest, + headers: Headers, + data: ByteArray, + ): List { + val node = JSON_READER.readTree(data) + + return this.processRecords(node, request.user) + .mapNotNull { r -> + r.fold( + { it }, + { + logger.error("Data conversion failed.. " + it.message) + null + }, + ) + } + .toList() + } + + fun Instant.toEpoch(): Double = this.toEpochMilli() / 1000.0 +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt new file mode 100644 index 00000000..013307bc --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt @@ -0,0 +1,49 @@ +package org.radarbase.huawei.converter + +import com.fasterxml.jackson.databind.JsonNode +import org.apache.avro.specific.SpecificRecord +import org.radarbase.huawei.user.User +import java.time.Instant + +private fun JsonNode.epochInstant(field: String): Instant? { + val value = this.get(field) ?: return null + if (value.isNull) return null + val millis = if (value.isTextual) value.asText().toLongOrNull() else value.asLong() + return millis?.let { Instant.ofEpochMilli(it) } +} + +/** + * Generic converter for `GET /healthkit/v1/healthRecords` responses: iterates every record + * returned for the requested `subDataTypeName` and builds one Avro record per entry via + * [buildRecord]. + */ +class HuaweiHealthRecordConverter( + private val topic: String, + private val buildRecord: ( + fields: FieldValues, + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + ) -> SpecificRecord, +) : HuaweiDataConverter { + + override fun processRecords(root: JsonNode, user: User): Sequence> { + val timeReceived = Instant.now() + val records = root.get("healthRecords") ?: root.get("records") ?: return emptySequence() + return records.asSequence() + .mapCatching { record -> + val startTime = record.epochInstant("startTime") + ?: error("Huawei health record is missing startTime") + val endTime = record.epochInstant("endTime") + val fieldValues = FieldValues.from( + record.get("value") ?: record.get("fieldValues") ?: record.get("field"), + ) + TopicData( + topic = topic, + key = user.observationKey, + offset = startTime.epochSecond, + value = buildRecord(fieldValues, startTime, endTime, timeReceived), + ) + } + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt new file mode 100644 index 00000000..6e2929f4 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt @@ -0,0 +1,53 @@ +package org.radarbase.huawei.converter + +import com.fasterxml.jackson.databind.JsonNode +import org.apache.avro.specific.SpecificRecord +import org.radarbase.huawei.user.User +import java.time.Instant + +private fun JsonNode.epochInstant(field: String): Instant? { + val value = this.get(field) ?: return null + if (value.isNull) return null + val millis = if (value.isTextual) value.asText().toLongOrNull() else value.asLong() + return millis?.let { Instant.ofEpochMilli(it) } +} + +/** + * Generic converter for `sampleSet:polymerize` responses: iterates every sample point of every + * data-type group in the response and builds one Avro record per point via [buildRecord]. + * + * This single converter is reused for the large majority of Huawei Health Kit data types, since + * they all share the same `sampleSet[].samplePoints[]` response envelope and differ only in which + * Avro record type their field values are mapped onto. + */ +class HuaweiSampleSetConverter( + private val topic: String, + private val buildRecord: ( + fields: FieldValues, + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + ) -> SpecificRecord, +) : HuaweiDataConverter { + + override fun processRecords(root: JsonNode, user: User): Sequence> { + val timeReceived = Instant.now() + val sampleSets = root.get("sampleSet") ?: root.get("sampleSets") ?: return emptySequence() + return sampleSets.asSequence() + .flatMap { group -> + (group.get("samplePoints") ?: group.get("samplePoint"))?.asSequence() ?: emptySequence() + } + .mapCatching { point -> + val startTime = point.epochInstant("startTime") + ?: error("Huawei sample point is missing startTime") + val endTime = point.epochInstant("endTime") + val fieldValues = FieldValues.from(point.get("value") ?: point.get("fieldValues")) + TopicData( + topic = topic, + key = user.observationKey, + offset = startTime.epochSecond, + value = buildRecord(fieldValues, startTime, endTime, timeReceived), + ) + } + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt new file mode 100644 index 00000000..f9e9e16b --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt @@ -0,0 +1,19 @@ +package org.radarbase.huawei.converter + +import okhttp3.Headers +import org.radarbase.huawei.request.RestRequest +import org.slf4j.LoggerFactory +import java.io.IOException + +interface RecordConverter { + @Throws(IOException::class) + fun convert( + request: RestRequest, + headers: Headers, + data: ByteArray, + ): List + + companion object { + var logger = LoggerFactory.getLogger(RecordConverter::class.java) + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt new file mode 100644 index 00000000..fe1dc73f --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt @@ -0,0 +1,11 @@ +package org.radarbase.huawei.converter + +import org.slf4j.LoggerFactory + +val logger = LoggerFactory.getLogger("org.radarbase.huawei.converter.SequenceExtensions") + +internal fun Sequence.mapCatching(fn: (T) -> S): Sequence> = map { t -> + runCatching { + fn(t) + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt new file mode 100644 index 00000000..a537af98 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt @@ -0,0 +1,11 @@ +package org.radarbase.huawei.converter + +import org.apache.avro.specific.SpecificRecord + +/** Single value for a topic. */ +data class TopicData( + val topic: String, + val key: SpecificRecord, + val value: SpecificRecord, + val offset: Long, +) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt new file mode 100644 index 00000000..9da597c0 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt @@ -0,0 +1,11 @@ +package org.radarbase.huawei.offset + +import org.radarbase.huawei.route.Route +import org.radarbase.huawei.user.User +import java.time.Instant + +data class Offset( + val user: User, + val route: Route, + val offset: Instant, +) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt new file mode 100644 index 00000000..88c67afb --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt @@ -0,0 +1,5 @@ +package org.radarbase.huawei.offset + +data class Offsets( + val offsets: List, +) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt new file mode 100644 index 00000000..03b23c34 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt @@ -0,0 +1,13 @@ +package org.radarbase.huawei.request + +import org.radarbase.huawei.offset.Offset +import org.radarbase.huawei.route.Route +import org.radarbase.huawei.user.User +import java.time.Instant + +interface HuaweiOffsetManager { + + fun getOffset(route: Route, user: User): Offset? + + fun updateOffsets(route: Route, user: User, offset: Instant) +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt new file mode 100644 index 00000000..b2d7da11 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt @@ -0,0 +1,190 @@ +package org.radarbase.huawei.request + +import com.fasterxml.jackson.core.JsonFactory +import com.fasterxml.jackson.databind.ObjectMapper +import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule +import okhttp3.Response +import org.radarbase.huawei.converter.TopicData +import org.radarbase.huawei.route.Route +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserRepository +import org.slf4j.LoggerFactory +import java.io.IOException +import java.time.Duration +import java.time.Instant + +class HuaweiRequestGenerator( + private val userRepository: UserRepository, + private val huaweiOffsetManager: HuaweiOffsetManager, + val routes: List, +) : RequestGenerator { + private val routeNextRequest: MutableMap = mutableMapOf() + + var nextRequestTime: Instant = Instant.MIN + + override fun requests(user: User, max: Int): Sequence = + routes.asSequence() + .flatMap { route -> + if (routeReady(user, route)) { + generateRequests(route, user) + } else { + logger.info( + "Skip {} for {}: route in backoff until {}", + route, + user.versionedId, + routeNextRequest[routeKey(route, user)], + ) + emptySequence() + } + } + + override fun requests(route: Route, max: Int): Sequence = + userRepository.stream() + .flatMap { user -> + if (routeReady(user, route)) { + generateRequests(route, user) + } else { + logger.info( + "Skip {} for {}: route in backoff until {}", + route, + user.versionedId, + routeNextRequest[routeKey(route, user)], + ) + emptySequence() + } + } + + override fun requests(route: Route, user: User, max: Int): Sequence = + if (routeReady(user, route)) { + generateRequests(route, user) + } else { + logger.info( + "Skip {} for {}: route in backoff until {}", + route, + user.versionedId, + routeNextRequest[routeKey(route, user)], + ) + emptySequence() + } + + fun generateRequests(route: Route, user: User): Sequence { + val offset = huaweiOffsetManager.getOffset(route, user) + val startDate = user.startDate + val startOffset: Instant = if (offset == null) { + logger.info("No offsets found for $user, using the start date.") + startDate + } else { + offset.offset.coerceAtLeast(startDate) + } + val endDate = user.endDate?.coerceAtMost(Instant.now()) ?: Instant.now() + if (!startOffset.isBefore(endDate)) { + logger.info( + "Skip {} for {}: interval empty (startOffset={} >= endDate={})", + route, + user.versionedId, + startOffset, + endDate, + ) + return emptySequence() + } + return route.generateRequests(user, startOffset, endDate, USER_MAX_REQUESTS) + } + + fun handleResponse(req: RestRequest, response: Response): HuaweiResult> { + return if (response.isSuccessful) { + HuaweiResult.Success(requestSuccessful(req, response)) + } else { + try { + HuaweiResult.Error(requestFailed(req, response)) + } catch (e: TooManyRequestsException) { + HuaweiResult.Success(emptyList()) + } + } + } + + override fun requestSuccessful(request: RestRequest, response: Response): List { + logger.debug("Request successful: {}..", request.request) + val body = response.body + val data = body?.bytes() ?: ByteArray(0) + val records = request.route.converters.flatMap { it.convert(request, response.headers, data) } + val offset = records.maxByOrNull { it.offset }?.offset + val key = routeKey(request.route, request.user) + if (offset != null) { + val maxOffsetTime = Instant.ofEpochSecond(offset) + val nextOffset = maxOffsetTime.plus(OFFSET_BUFFER).coerceAtLeast(request.endDate) + huaweiOffsetManager.updateOffsets(request.route, request.user, nextOffset) + } else { + huaweiOffsetManager.updateOffsets(request.route, request.user, request.endDate) + } + routeNextRequest[key] = Instant.now().plus(SUCCESS_BACK_OFF_TIME) + return records + } + + override fun requestFailed(request: RestRequest, response: Response): HuaweiError { + val key = routeKey(request.route, request.user) + return when (response.code) { + 429 -> { + logger.info("Too many requests, rate limit reached. Backing off...") + nextRequestTime = Instant.now() + BACK_OFF_TIME + routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) + HuaweiRateLimitError("Rate limit reached..", TooManyRequestsException(), "429") + } + 403 -> { + logger.warn("User {} does not have access to this Huawei Health Kit data type.", request.user) + routeNextRequest[key] = Instant.now().plus(USER_BACK_OFF_TIME) + HuaweiAccessForbiddenError( + "Huawei Health Kit scope not granted or data not available..", + IOException("Forbidden"), + "403", + ) + } + 401 -> { + logger.warn("User {} access token is expired, malformed, or revoked.", request.user) + routeNextRequest[key] = Instant.now().plus(USER_BACK_OFF_TIME) + HuaweiUnauthorizedAccessError( + "Access token expired or revoked..", + IOException("Unauthorized"), + "401", + ) + } + 400 -> { + logger.warn("Client exception for request {}", request) + routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) + HuaweiClientException("Client unsupported or unauthorized..", IOException("Invalid client"), "400") + } + 422 -> { + logger.warn("Request failed (validation error): {}, {}", request, response) + routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) + HuaweiValidationError(response.body?.string() ?: "validation error", IOException("Validation error"), "422") + } + 404 -> { + logger.warn("Not found: {}", request) + routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) + HuaweiNotFoundError(response.body?.string() ?: "not found", IOException("Data not found"), "404") + } + else -> { + logger.warn("Request failed: {}, {}", request, response) + routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) + HuaweiGenericError(response.body?.string() ?: "unknown error", IOException("Unknown error"), "500") + } + } + } + + private fun routeReady(user: User, route: Route): Boolean { + val key = routeKey(route, user) + return routeNextRequest[key]?.let { Instant.now() > it } ?: true + } + + private fun routeKey(route: Route, user: User): String = user.versionedId + "#" + route + + companion object { + private val logger = LoggerFactory.getLogger(HuaweiRequestGenerator::class.java) + private val BACK_OFF_TIME = Duration.ofMinutes(10L) + private val USER_BACK_OFF_TIME = Duration.ofHours(12L) + private val SUCCESS_BACK_OFF_TIME = Duration.ofSeconds(10L) + private val OFFSET_BUFFER = Duration.ofHours(1) + private const val USER_MAX_REQUESTS = 1000 + val JSON_FACTORY = JsonFactory() + val JSON_READER = ObjectMapper(JSON_FACTORY).registerModule(JavaTimeModule()).reader() + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt new file mode 100644 index 00000000..8a4fa587 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt @@ -0,0 +1,64 @@ +package org.radarbase.huawei.request + +sealed class HuaweiResult { + data class Success(val value: T) : HuaweiResult() + data class Error(val error: HuaweiError) : HuaweiResult() +} + +sealed interface HuaweiError + +sealed class HuaweiErrorBase( + val message: String, + val cause: Exception? = null, + val code: String, +) : HuaweiError + +class HuaweiRateLimitError(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( + message, + cause, + code, +) + +class HuaweiClientException(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( + message, + cause, + code, +) + +class HuaweiUnauthorizedAccessError( + message: String, + cause: Exception? = null, + code: String, +) : HuaweiErrorBase( + message, + cause, + code, +) + +class HuaweiAccessForbiddenError( + message: String, + cause: Exception? = null, + code: String, +) : HuaweiErrorBase( + message, + cause, + code, +) + +class HuaweiValidationError(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( + message, + cause, + code, +) + +class HuaweiGenericError(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( + message, + cause, + code, +) + +class HuaweiNotFoundError(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( + message, + cause, + code, +) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt new file mode 100644 index 00000000..39bb60e9 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt @@ -0,0 +1,19 @@ +package org.radarbase.huawei.request + +import okhttp3.Response +import org.radarbase.huawei.converter.TopicData +import org.radarbase.huawei.route.Route +import org.radarbase.huawei.user.User + +interface RequestGenerator { + + fun requests(user: User, max: Int): Sequence + + fun requests(route: Route, user: User, max: Int): Sequence + + fun requests(route: Route, max: Int): Sequence + + fun requestSuccessful(request: RestRequest, response: Response): List + + fun requestFailed(request: RestRequest, response: Response): HuaweiError +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt new file mode 100644 index 00000000..502ecc26 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt @@ -0,0 +1,14 @@ +package org.radarbase.huawei.request + +import okhttp3.Request +import org.radarbase.huawei.route.HuaweiRoute +import org.radarbase.huawei.user.User +import java.time.Instant + +data class RestRequest( + val request: Request, + val user: User, + val route: HuaweiRoute, + val startDate: Instant, + val endDate: Instant, +) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt new file mode 100644 index 00000000..3dc5aa9c --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt @@ -0,0 +1,3 @@ +package org.radarbase.huawei.request + +class TooManyRequestsException : RuntimeException() diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt new file mode 100644 index 00000000..63318b23 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt @@ -0,0 +1,46 @@ +package org.radarbase.huawei.route + +import org.radarbase.huawei.converter.HuaweiActivityRecordConverter +import org.radarbase.huawei.converter.HuaweiDataConverter +import org.radarbase.huawei.request.RestRequest +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserRepository +import java.time.Duration +import java.time.Instant + +/** + * Route backed by `GET /healthkit/v1/activityRecords`, covering the Huawei Health Kit Activity + * Records API (workout / physical-activity sessions). + */ +class HuaweiActivityRecordRoute( + userRepository: UserRepository, + private val topic: String = "connect_huawei_activity_record", + maxIntervalPerRequest: Duration = Duration.ofDays(30L), +) : HuaweiRoute(userRepository, maxIntervalPerRequest) { + + override val converters: List = listOf(HuaweiActivityRecordConverter(topic)) + + override fun toString(): String = "huawei_activity_record" + + override fun generateRequests( + user: User, + start: Instant, + end: Instant, + max: Int, + ): Sequence = chunkedRanges(start, end, max).map { (rangeStart, rangeEnd) -> + RestRequest( + request = createGetRequest( + user, + "activityRecords", + mapOf( + "startTime" to rangeStart.toEpochMilli().toString(), + "endTime" to rangeEnd.toEpochMilli().toString(), + ), + ), + user = user, + route = this, + startDate = rangeStart, + endDate = rangeEnd, + ) + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt new file mode 100644 index 00000000..ae2fa2b5 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt @@ -0,0 +1,58 @@ +package org.radarbase.huawei.route + +import org.apache.avro.specific.SpecificRecord +import org.radarbase.huawei.converter.FieldValues +import org.radarbase.huawei.converter.HuaweiDataConverter +import org.radarbase.huawei.converter.HuaweiHealthRecordConverter +import org.radarbase.huawei.request.RestRequest +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserRepository +import java.time.Duration +import java.time.Instant + +/** + * Route backed by `GET /healthkit/v1/healthRecords`, used for the `health.record.*` data types + * (ambulatory blood pressure sessions, heart rate alerts, hyperthermia, low SpO2 alerts, + * menstrual cycle phases, and comprehensive sleep records). + */ +open class HuaweiHealthRecordRoute( + userRepository: UserRepository, + private val subDataTypeName: String, + private val topic: String, + maxIntervalPerRequest: Duration = Duration.ofDays(30L), + buildRecord: ( + fields: FieldValues, + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + ) -> SpecificRecord, +) : HuaweiRoute(userRepository, maxIntervalPerRequest) { + + override val converters: List = + listOf(HuaweiHealthRecordConverter(topic, buildRecord)) + + override fun toString(): String = "huawei_" + topic.removePrefix("connect_huawei_") + + override fun generateRequests( + user: User, + start: Instant, + end: Instant, + max: Int, + ): Sequence = chunkedRanges(start, end, max).map { (rangeStart, rangeEnd) -> + RestRequest( + request = createGetRequest( + user, + "healthRecords", + mapOf( + "subDataTypeName" to subDataTypeName, + "startTime" to rangeStart.toEpochMilli().toString(), + "endTime" to rangeEnd.toEpochMilli().toString(), + ), + ), + user = user, + route = this, + startDate = rangeStart, + endDate = rangeEnd, + ) + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt new file mode 100644 index 00000000..8fd5bb01 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt @@ -0,0 +1,62 @@ +package org.radarbase.huawei.route + +import okhttp3.HttpUrl.Companion.toHttpUrl +import okhttp3.MediaType.Companion.toMediaType +import okhttp3.Request +import okhttp3.RequestBody.Companion.toRequestBody +import org.radarbase.huawei.converter.HuaweiDataConverter +import org.radarbase.huawei.request.RestRequest +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserRepository +import java.time.Duration +import java.time.Instant + +/** + * Base class for all Huawei Health Kit routes. + * + * Handles OAuth2-authorized request construction (both `GET` with query parameters and `POST` + * with a JSON body, since the Health Kit Data API mixes both styles across its endpoints) and + * generic time-range chunking, shared by all concrete route types. + */ +abstract class HuaweiRoute( + private val userRepository: UserRepository, + override val maxIntervalPerRequest: Duration = DEFAULT_INTERVAL_PER_REQUEST, +) : Route { + abstract val converters: List + + protected fun createGetRequest(user: User, path: String, queryParams: Map): Request { + val accessToken = userRepository.getAccessToken(user) + val urlBuilder = "$HUAWEI_API_BASE_URL/$path".toHttpUrl().newBuilder() + queryParams.forEach { (key, value) -> urlBuilder.addQueryParameter(key, value) } + return Request.Builder() + .url(urlBuilder.build()) + .header("Authorization", "Bearer $accessToken") + .get() + .build() + } + + protected fun createPostRequest(user: User, path: String, jsonBody: String): Request { + val accessToken = userRepository.getAccessToken(user) + return Request.Builder() + .url("$HUAWEI_API_BASE_URL/$path".toHttpUrl()) + .header("Authorization", "Bearer $accessToken") + .post(jsonBody.toRequestBody(JSON_MEDIA_TYPE)) + .build() + } + + /** Split `[start, end)` into consecutive windows of at most [maxIntervalPerRequest], capped at [max] windows. */ + protected fun chunkedRanges(start: Instant, end: Instant, max: Int): Sequence> = + generateSequence(start) { it + maxIntervalPerRequest } + .takeWhile { it < end } + .take(max) + .map { rangeStart -> rangeStart to (rangeStart + maxIntervalPerRequest).coerceAtMost(end) } + + override fun generateRequests(user: User, start: Instant, end: Instant): Sequence = + generateRequests(user, start, end, Int.MAX_VALUE) + + companion object { + const val HUAWEI_API_BASE_URL = "https://health-api.cloud.huawei.com/healthkit/v1" + private val JSON_MEDIA_TYPE = "application/json; charset=utf-8".toMediaType() + private val DEFAULT_INTERVAL_PER_REQUEST = Duration.ofDays(30L) + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt new file mode 100644 index 00000000..5a311e47 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt @@ -0,0 +1,19 @@ +package org.radarbase.huawei.route + +import org.radarbase.huawei.user.UserRepository + +/** + * A single registered Huawei Health Kit data type: a short config key (used to build + * `huawei..enabled` / `huawei..topic` connector properties), the default Kafka topic + * name, and a factory for the [HuaweiRoute] that queries it. + * + * Using one shared registry (see [HuaweiRouteFactory]) for both the Kafka Connect config + * definition and the set of routes actually polled avoids hand-duplicating each of the ~54 Huawei + * data types across a `ConfigDef` and a route-construction switch. + */ +data class HuaweiRouteDefinition( + val key: String, + val defaultTopic: String, + val enabledByDefault: Boolean = true, + val build: (UserRepository, topic: String) -> HuaweiRoute, +) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt new file mode 100644 index 00000000..2e1df2fa --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -0,0 +1,566 @@ +package org.radarbase.huawei.route + +import org.apache.avro.specific.SpecificRecord +import org.radarbase.huawei.converter.FieldValues +import org.radarcns.connector.huawei.HuaweiActiveHours +import org.radarcns.connector.huawei.HuaweiCgmBloodGlucose +import org.radarcns.connector.huawei.HuaweiContinuousActivityStatistics +import org.radarcns.connector.huawei.HuaweiContinuousAltitudeStatistics +import org.radarcns.connector.huawei.HuaweiContinuousBloodGlucoseStatistics +import org.radarcns.connector.huawei.HuaweiContinuousBodyBloodPressureStatistics +import org.radarcns.connector.huawei.HuaweiContinuousBreatheRateStatistics +import org.radarcns.connector.huawei.HuaweiContinuousCaloriesBurnt +import org.radarcns.connector.huawei.HuaweiContinuousCaloriesBurntTotal +import org.radarcns.connector.huawei.HuaweiContinuousDistanceDelta +import org.radarcns.connector.huawei.HuaweiContinuousDistanceTotal +import org.radarcns.connector.huawei.HuaweiContinuousEcgDetail +import org.radarcns.connector.huawei.HuaweiContinuousExerciseIntensity +import org.radarcns.connector.huawei.HuaweiContinuousExerciseIntensityV2 +import org.radarcns.connector.huawei.HuaweiContinuousExerciseIntensityV2Statistics +import org.radarcns.connector.huawei.HuaweiContinuousSleepFragment +import org.radarcns.connector.huawei.HuaweiContinuousSpo2Statistics +import org.radarcns.connector.huawei.HuaweiContinuousStepsDelta +import org.radarcns.connector.huawei.HuaweiContinuousStepsTotal +import org.radarcns.connector.huawei.HuaweiDailyActivitySummary +import org.radarcns.connector.huawei.HuaweiEmotion +import org.radarcns.connector.huawei.HuaweiHealthRecordDynamicBp +import org.radarcns.connector.huawei.HuaweiHealthRecordHeartRateAlert +import org.radarcns.connector.huawei.HuaweiHealthRecordHyperthermia +import org.radarcns.connector.huawei.HuaweiHealthRecordLowSpo2Alert +import org.radarcns.connector.huawei.HuaweiHealthRecordMenstrualCycle +import org.radarcns.connector.huawei.HuaweiHealthRecordSleep +import org.radarcns.connector.huawei.HuaweiHeartRateVariability +import org.radarcns.connector.huawei.HuaweiRestingCaloriesStatistics +import org.radarcns.connector.huawei.HuaweiSleepOnOffBedRecord +import org.radarcns.connector.huawei.HuaweiSleepRespiratoryDetail +import org.radarcns.connector.huawei.HuaweiSleepRespiratoryEvent +import org.radarcns.connector.huawei.HuaweiStatistics +import org.radarcns.connector.huawei.HuaweiVo2Max +import java.time.Instant + +/** + * Registry of every Huawei Health Kit data type this connector supports, mapping each one to the + * Kafka Connect REST endpoint (`sampleSet:polymerize`, `healthRecords`, or `activityRecords`) and + * Avro record type documented in the `radar-huawei-connector` schema specification + * (RADAR-base/RADAR-Schemas, `huawei_schemas` branch). + * + * Huawei `dataTypeName`/`subDataTypeName` values below are taken verbatim from that + * specification's `doc` strings (prefixed with the vendor namespace `com.huawei.`), which in turn + * describe the Huawei Health Kit REST Data API's own data type identifiers. + * + * Field-value key names used in the record builders are Huawei Health Kit `Field` identifiers + * (snake_case, matching the on-device HiHealth SDK's public `Field.FIELD_*` constant family, e.g. + * `steps_delta`, `calories`, `avg`/`max`/`min`/`last`). Where a field is not among Huawei's widely + * documented constants, the snake_case form of the Avro field's own name is used as a best-effort + * default (see [snake]) — verify against a live API response and adjust the key strings in this + * file if Huawei's actual response uses different names. + */ +object HuaweiRouteFactory { + + private const val VENDOR_PREFIX = "com.huawei." + + private fun Instant.toEpoch(): Double = toEpochMilli() / 1000.0 + + /** Best-effort camelCase -> snake_case conversion for deriving a Huawei field key from an Avro field name. */ + private fun snake(name: String): String = + SNAKE_CASE_BOUNDARY.replace(name) { "${it.groupValues[1]}_${it.groupValues[2]}" }.lowercase() + + private val SNAKE_CASE_BOUNDARY = Regex("([a-z0-9])([A-Z])") + + private fun HuaweiStatistics.Builder.populateCommon( + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + fields: FieldValues, + ) { + time = startTime.toEpoch() + this.timeReceived = timeReceived.toEpoch() + this.endTime = endTime?.toEpoch() + avg = fields.getDouble("avg") + max = fields.getDouble("max") + min = fields.getDouble("min") + last = fields.getDouble("last") + count = fields.getInt("count") + } + + /** Data types that reuse the generic [HuaweiStatistics] schema: (config key, Huawei data type name, default topic). */ + private val genericStatisticsTypes = listOf( + Triple("continuous_body_fat_rate_statistics", "continuous.body.fat.rate.statistics", "connect_huawei_continuous_body_fat_rate_statistics"), + Triple("continuous_body_temperature_rest_statistics", "continuous.body.temperature.rest.statistics", "connect_huawei_continuous_body_temperature_rest_statistics"), + Triple("continuous_body_temperature_statistics", "continuous.body.temperature.statistics", "connect_huawei_continuous_body_temperature_statistics"), + Triple("continuous_calories_bmr_statistics", "continuous.calories.bmr.statistics", "connect_huawei_continuous_calories_bmr_statistics"), + Triple("continuous_exercise_heart_rate_statistics", "continuous.exercise_heart_rate.statistics", "connect_huawei_continuous_exercise_heart_rate_statistics"), + Triple("continuous_heart_rate_statistics", "continuous.heart_rate.statistics", "connect_huawei_continuous_heart_rate_statistics"), + Triple("continuous_power_statistics", "continuous.power.statistics", "connect_huawei_continuous_power_statistics"), + Triple("continuous_skin_temperature_statistics", "continuous.skin.temperature.statistics", "connect_huawei_continuous_skin_temperature_statistics"), + Triple("continuous_speed_statistics", "continuous.speed.statistics", "connect_huawei_continuous_speed_statistics"), + Triple("continuous_steps_rate_statistics", "continuous.steps.rate.statistics", "connect_huawei_continuous_steps_rate_statistics"), + Triple("continuous_stroke_rate_statistics", "continuous.stroke_rate.statistics", "connect_huawei_continuous_stroke_rate_statistics"), + Triple("instantaneous_resting_heart_rate_statistics", "instantaneous.resting_heart_rate.statistics", "connect_huawei_instantaneous_resting_heart_rate_statistics"), + Triple("instantaneous_stress_statistics", "instantaneous.stress.statistics", "connect_huawei_instantaneous_stress_statistics"), + Triple("vo2max_statistics", "vo2max.statistics", "connect_huawei_vo2max_statistics"), + ) + + /** Full registry of Huawei Health Kit data types supported by this connector. */ + val definitions: List = buildList { + add( + HuaweiRouteDefinition("activity_record", "connect_huawei_activity_record") { repo, topic -> + HuaweiActivityRecordRoute(repo, topic) + }, + ) + + // cgm_blood_glucose (+ .statistics variant) + add(sampleSetDefinition("cgm_blood_glucose", "cgm_blood_glucose", "connect_huawei_cgm_blood_glucose") { f, start, _, received -> + HuaweiCgmBloodGlucose.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch() + level = f.getDouble("level") + avg = f.getInt("avg"); max = f.getInt("max"); min = f.getInt("min"); last = f.getInt("last") + }.build() + }) + add(sampleSetDefinition("cgm_blood_glucose_statistics", "cgm_blood_glucose.statistics", "connect_huawei_cgm_blood_glucose_statistics") { f, start, _, received -> + HuaweiCgmBloodGlucose.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch() + level = f.getDouble("level") + avg = f.getInt("avg"); max = f.getInt("max"); min = f.getInt("min"); last = f.getInt("last") + }.build() + }) + + add(sampleSetDefinition("daily_activity_summary", "daily_activity_summary", "connect_huawei_daily_activity_summary") { f, start, end, received -> + HuaweiDailyActivitySummary.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + steps = f.getInt("steps") + activeCalories = f.getInt("calories") + exerciseTime = f.getInt("exercise_time") + activeHours = f.getInt("active_hours") + stepsGoal = f.getInt("steps_target") + activeCaloriesGoal = f.getInt("calories_target") + exerciseTimeGoal = f.getInt("exercise_time_target") + activeHoursGoal = f.getInt("active_hours_target") + }.build() + }) + + add(sampleSetDefinition("active_hours", "active_hours", "connect_huawei_active_hours") { f, start, end, received -> + f.toActiveHours(start, end, received) + }) + add(sampleSetDefinition("active_hours_statistics", "active_hours.statistics", "connect_huawei_active_hours_statistics") { f, start, end, received -> + f.toActiveHours(start, end, received) + }) + + add(sampleSetDefinition("continuous_activity_fragment", "continuous.activity.fragment", "connect_huawei_continuous_activity_fragment") { f, start, end, received -> + f.toContinuousActivityStatistics(start, end, received) + }) + add(sampleSetDefinition("continuous_activity_statistics", "continuous.activity.statistics", "connect_huawei_continuous_activity_statistics") { f, start, end, received -> + f.toContinuousActivityStatistics(start, end, received) + }) + + add(sampleSetDefinition("continuous_altitude_statistics", "continuous.altitude.statistics", "connect_huawei_continuous_altitude_statistics") { f, start, end, received -> + HuaweiContinuousAltitudeStatistics.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + avg = f.getDouble("avg"); max = f.getDouble("max"); min = f.getDouble("min") + ascentTotal = f.getDouble("ascent_total") + descentTotal = f.getDouble("descent_total") + }.build() + }) + + add(sampleSetDefinition("continuous_blood_glucose_statistics", "continuous.blood_glucose.statistics", "connect_huawei_continuous_blood_glucose_statistics") { f, start, end, received -> + HuaweiContinuousBloodGlucoseStatistics.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + avg = f.getDouble("avg"); max = f.getDouble("max"); min = f.getDouble("min") + correlationWithMealtime = f.getInt("correlate_mealtime") + meal = f.getInt("meal") + correlationWithSleepState = f.getInt("correlate_sleep") + sampleSource = f.getInt("sample_source") + }.build() + }) + + add(sampleSetDefinition("continuous_breathe_rate_statistics", "continuous.breathe_rate.statistics", "connect_huawei_continuous_breathe_rate_statistics") { f, start, end, received -> + HuaweiContinuousBreatheRateStatistics.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + maxBreatheRate = f.getInt("max_breathe_rate") + minBreatheRate = f.getInt("min_breathe_rate") + avgBreatheRate = f.getInt("avg_breathe_rate") + minBreathrateBaseline = f.getInt("min_breathrate_baseline") + maxBreathrateBaseline = f.getInt("max_breathrate_baseline") + }.build() + }) + + add(sampleSetDefinition("continuous_body_blood_pressure_statistics", "continuous.body.blood_pressure.statistics", "connect_huawei_continuous_body_blood_pressure_statistics") { f, start, end, received -> + HuaweiContinuousBodyBloodPressureStatistics.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + systolicPressureAvg = f.getDouble("systolic_pressure_avg") + systolicPressureMax = f.getDouble("systolic_pressure_max") + systolicPressureMin = f.getDouble("systolic_pressure_min") + diastolicPressureAvg = f.getDouble("diastolic_pressure_avg") + diastolicPressureMax = f.getDouble("diastolic_pressure_max") + diastolicPressureMin = f.getDouble("diastolic_pressure_min") + sphygmusAvg = f.getDouble("sphygmus_avg") + sphygmusMax = f.getDouble("sphygmus_max") + sphygmusMin = f.getDouble("sphygmus_min") + sphygmusLast = f.getDouble("sphygmus_last") + }.build() + }) + + genericStatisticsTypes.forEach { (key, dataType, topic) -> + add( + sampleSetDefinition(key, dataType, topic) { f, start, end, received -> + HuaweiStatistics.newBuilder().apply { populateCommon(start, end, received, f) }.build() + }, + ) + } + + add(sampleSetDefinition("continuous_calories_burnt", "continuous.calories.burnt", "connect_huawei_continuous_calories_burnt") { f, start, end, received -> + HuaweiContinuousCaloriesBurnt.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + calories = f.getDouble("calories") + }.build() + }) + add(sampleSetDefinition("continuous_calories_consumed", "continuous.calories.consumed", "connect_huawei_continuous_calories_consumed") { f, start, end, received -> + HuaweiContinuousCaloriesBurnt.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + calories = f.getDouble("calories") + }.build() + }) + add(sampleSetDefinition("continuous_calories_burnt_total", "continuous.calories.burnt.total", "connect_huawei_continuous_calories_burnt_total") { f, start, end, received -> + HuaweiContinuousCaloriesBurntTotal.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + caloriesTotal = f.getDouble("calories_total") + }.build() + }) + + add(sampleSetDefinition("continuous_distance_delta", "continuous.distance.delta", "connect_huawei_continuous_distance_delta") { f, start, end, received -> + HuaweiContinuousDistanceDelta.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + distanceDelta = f.getDouble("distance_delta") + }.build() + }) + add(sampleSetDefinition("continuous_distance_total", "continuous.distance.total", "connect_huawei_continuous_distance_total") { f, start, end, received -> + HuaweiContinuousDistanceTotal.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + distance = f.getDouble("distance_total") + }.build() + }) + + add(sampleSetDefinition("continuous_ecg_detail", "continuous.ecg_detail", "connect_huawei_continuous_ecg_detail") { f, start, end, received -> + HuaweiContinuousEcgDetail.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + ecgRecordId = f.getString("record_id") + averageHeartRate = f.getInt("avg_heart_rate") + ecgArrhythmiaType = f.getInt("arrhythmia_type") + ecgArrhythmiaResult = f.getInt("arrhythmia_result") + userSymptom = f.getString("user_symptom") + samplingFrequency = f.getInt("sampling_frequency") + voltageData = f.getString("voltage_data") + }.build() + }) + + add(sampleSetDefinition("continuous_exercise_intensity", "continuous.exercise_intensity", "connect_huawei_continuous_exercise_intensity") { f, start, end, received -> + f.toContinuousExerciseIntensity(start, end, received) + }) + add(sampleSetDefinition("continuous_exercise_intensity_statistics", "continuous.exercise_intensity.statistics", "connect_huawei_continuous_exercise_intensity_statistics") { f, start, end, received -> + f.toContinuousExerciseIntensity(start, end, received) + }) + + add(sampleSetDefinition("continuous_exercise_intensity_v2", "continuous.exercise_intensity.v2", "connect_huawei_continuous_exercise_intensity_v2") { f, start, end, received -> + HuaweiContinuousExerciseIntensityV2.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + exerciseType = f.getInt("exercise_type") + }.build() + }) + add(sampleSetDefinition("continuous_exercise_intensity_v2_statistics", "continuous.exercise_intensity.v2.statistics", "connect_huawei_continuous_exercise_intensity_v2_statistics") { f, start, end, received -> + HuaweiContinuousExerciseIntensityV2Statistics.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + zone1Duration = f.getInt("zone1_duration") + zone2Duration = f.getInt("zone2_duration") + zone3Duration = f.getInt("zone3_duration") + zone4Duration = f.getInt("zone4_duration") + zone5Duration = f.getInt("zone5_duration") + }.build() + }) + + add(sampleSetDefinition("continuous_sleep_fragment", "continuous.sleep.fragment", "connect_huawei_continuous_sleep_fragment") { f, start, end, received -> + HuaweiContinuousSleepFragment.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + sleepState = f.getInt("sleep_state") + }.build() + }) + + add(sampleSetDefinition("continuous_spo2_statistics", "continuous.spo2.statistics", "connect_huawei_continuous_spo2_statistics") { f, start, end, received -> + HuaweiContinuousSpo2Statistics.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + saturationAvg = f.getDouble("avg") + saturationMax = f.getDouble("max") + saturationMin = f.getDouble("min") + saturationLast = f.getDouble("last") + }.build() + }) + + add(sampleSetDefinition("continuous_steps_delta", "continuous.steps.delta", "connect_huawei_continuous_steps_delta") { f, start, end, received -> + HuaweiContinuousStepsDelta.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + stepsDelta = f.getInt("steps_delta") + }.build() + }) + add(sampleSetDefinition("continuous_steps_total", "continuous.steps.total", "connect_huawei_continuous_steps_total") { f, start, end, received -> + HuaweiContinuousStepsTotal.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + steps = f.getInt("steps") + duration = f.getInt("duration") + }.build() + }) + + add(sampleSetDefinition("emotion", "emotion", "connect_huawei_emotion") { f, start, _, received -> + HuaweiEmotion.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch() + emotionStatus = f.getInt("emotion") + }.build() + }) + + add(healthRecordDefinition("health_record_dynamic_bp", "health.record.dynamic_bp", "connect_huawei_health_record_dynamic_bp") { f, start, end, received -> + f.toHealthRecordDynamicBp(start, end, received) + }) + add(healthRecordDefinition("health_record_bradycardia", "health.record.bradycardia", "connect_huawei_health_record_bradycardia") { f, start, end, received -> + f.toHealthRecordHeartRateAlert(start, end, received) + }) + add(healthRecordDefinition("health_record_tachycardia", "health.record.tachycardia", "connect_huawei_health_record_tachycardia") { f, start, end, received -> + f.toHealthRecordHeartRateAlert(start, end, received) + }) + add(healthRecordDefinition("health_record_hyperthermia", "health.record.hyperthermia", "connect_huawei_health_record_hyperthermia") { f, start, end, received -> + HuaweiHealthRecordHyperthermia.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + highBodyTemperatureAlarm = f.getFloat("high_body_temperature_alarm") + }.build() + }) + add(healthRecordDefinition("health_record_low_spo2_alert", "health.record.lowSpo2Alert", "connect_huawei_health_record_low_spo2_alert") { f, start, end, received -> + HuaweiHealthRecordLowSpo2Alert.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + threshold = f.getFloat("threshold") + maxSpO2 = f.getFloat("max_spo2") + minSpO2 = f.getFloat("min_spo2") + }.build() + }) + add(healthRecordDefinition("health_record_menstrual_cycle", "health.record.menstrual_cycle", "connect_huawei_health_record_menstrual_cycle") { f, start, end, received -> + HuaweiHealthRecordMenstrualCycle.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + recordday = f.getInt("record_day") + status = f.getInt("status") + substatus = f.getInt("sub_status") + remarks = f.getString("remarks") + timezone = f.getString("timezone") + }.build() + }) + add(healthRecordDefinition("health_record_sleep", "health.record.sleep", "connect_huawei_health_record_sleep") { f, start, end, received -> + HuaweiHealthRecordSleep.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + fallAsleepTime = f.getLong("fall_asleep_time") + wakeupTime = f.getLong("wakeup_time") + lightSleepTime = f.getInt("light_sleep_time") + deepSleepTime = f.getInt("deep_sleep_time") + dreamTime = f.getInt("dream_time") + awakeTime = f.getInt("awake_time") + allSleepTime = f.getInt("all_sleep_time") + wakeupCount = f.getInt("wakeup_count") + deepSleepPart = f.getInt("deep_sleep_part") + sleepScore = f.getInt("sleep_score") + sleepLatency = f.getInt("sleep_latency") + sleepEfficiency = f.getInt("sleep_efficiency") + goBedTime = f.getLong("go_bed_time") + sleepType = f.getInt("sleep_type") + prepareSleepTime = f.getLong("prepare_sleep_time") + offBedTime = f.getLong("off_bed_time") + }.build() + }) + + add(sampleSetDefinition("heart_rate_variability", "heart_rate_variability", "connect_huawei_heart_rate_variability") { f, start, _, received -> + HuaweiHeartRateVariability.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch() + heartRateVariabilityRmssd = f.getInt("heart_rate_variability_rmssd") + }.build() + }) + + add(sampleSetDefinition("resting_calories_statistics", "resting_calories.statistics", "connect_huawei_resting_calories_statistics") { f, start, end, received -> + HuaweiRestingCaloriesStatistics.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + predictedCalories = f.getFloat("predicted_calories") + totalCalories = f.getFloat("total_calories") + }.build() + }) + + add(sampleSetDefinition("sleep_on_off_bed_record", "sleep.on_off_bed_record", "connect_huawei_sleep_on_off_bed_record") { f, start, _, received -> + HuaweiSleepOnOffBedRecord.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch() + onOffBedState = f.getInt("on_off_bed_state") + }.build() + }) + + add(sampleSetDefinition("sleep_respiratory_detail", "sleep_respiratory_detail", "connect_huawei_sleep_respiratory_detail") { f, start, end, received -> + HuaweiSleepRespiratoryDetail.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + type = f.getInt("type") + value = f.getDouble("value") + }.build() + }) + add(sampleSetDefinition("sleep_respiratory_event", "sleep_respiratory_event", "connect_huawei_sleep_respiratory_event") { f, start, end, received -> + HuaweiSleepRespiratoryEvent.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + eventname = f.getInt("event_name") + }.build() + }) + + add(sampleSetDefinition("vo2max", "vo2max", "connect_huawei_vo2max") { f, start, _, received -> + HuaweiVo2Max.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch() + vo2max = f.getInt("vo2max") + }.build() + }) + } + + private fun FieldValues.toActiveHours( + start: Instant, + end: Instant?, + received: Instant, + ): HuaweiActiveHours = HuaweiActiveHours.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + activeHours = getInt("active_hours") + moderateIntensityMinutes = getInt("moderate_intensity_minutes") + highIntensityMinutes = getInt("high_intensity_minutes") + }.build() + + private fun FieldValues.toContinuousActivityStatistics( + start: Instant, + end: Instant?, + received: Instant, + ): HuaweiContinuousActivityStatistics = HuaweiContinuousActivityStatistics.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + typeOfActivity = getInt("activity_type") + span = getInt("span") + fragments = getInt("fragments") + }.build() + + private fun FieldValues.toContinuousExerciseIntensity( + start: Instant, + end: Instant?, + received: Instant, + ): HuaweiContinuousExerciseIntensity = HuaweiContinuousExerciseIntensity.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + intensity = getDouble("intensity") + span = getInt("span") + }.build() + + private fun FieldValues.toHealthRecordHeartRateAlert( + start: Instant, + end: Instant?, + received: Instant, + ): HuaweiHealthRecordHeartRateAlert = HuaweiHealthRecordHeartRateAlert.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + threshold = getDouble("threshold") + avgHeartRate = getDouble("avg_heart_rate") + maxHeartRate = getDouble("max_heart_rate") + minHeartRate = getDouble("min_heart_rate") + }.build() + + /** + * The 24h ambulatory blood pressure monitoring record has ~80 numeric fields, all following + * the same `` naming (e.g. `avgSystolicBpAll`, `maxHeartRateWake`). + * [snake] derives each Huawei field key mechanically from the Avro field name to avoid + * hand-transcribing ~80 near-identical key strings. + */ + private fun FieldValues.toHealthRecordDynamicBp( + start: Instant, + end: Instant?, + received: Instant, + ): HuaweiHealthRecordDynamicBp { + val f = this + fun i(name: String) = f.getInt(snake(name)) + fun d(name: String) = f.getDouble(snake(name)) + fun l(name: String) = f.getLong(snake(name)) + return HuaweiHealthRecordDynamicBp.newBuilder().apply { + time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + planId = f.getString(snake("planId")) + planStartTime = l("planStartTime") + planEndTime = l("planEndTime") + planActualTime = l("planActualTime") + planStatus = i("planStatus") + gasBagType = i("gasBagType") + sleepStartTime = l("sleepStartTime") + sleepEndTime = l("sleepEndTime") + + validCntAll = i("validCntAll"); cntAll = i("cntAll") + maxSystolicBpAll = i("maxSystolicBpAll"); maxDiastolicBpAll = i("maxDiastolicBpAll"); maxHeartRateAll = i("maxHeartRateAll") + midSystolicBpAll = i("midSystolicBpAll"); midDiastolicBpAll = i("midDiastolicBpAll"); midHeartRateAll = i("midHeartRateAll") + minSystolicBpAll = i("minSystolicBpAll"); minDiastolicBpAll = i("minDiastolicBpAll"); minHeartRateAll = i("minHeartRateAll") + avgSystolicBpAll = i("avgSystolicBpAll"); avgDiastolicBpAll = i("avgDiastolicBpAll"); avgHeartRateAll = i("avgHeartRateAll") + stdSystolicBpAll = i("stdSystolicBpAll"); stdDiastolicBpAll = i("stdDiastolicBpAll"); stdHeartRateAll = i("stdHeartRateAll") + coefSystolicBpAll = d("coefSystolicBpAll"); coefDiastolicBpAll = d("coefDiastolicBpAll"); coefHeartRateAll = d("coefHeartRateAll") + loadSystolicBpAll = d("loadSystolicBpAll"); loadDiastolicBpAll = d("loadDiastolicBpAll") + dropSystolicBpAll = d("dropSystolicBpAll"); dropDiastolicBpAll = d("dropDiastolicBpAll") + peakSystolicBpAll = i("peakSystolicBpAll"); peakDiastolicBpAll = i("peakDiastolicBpAll") + + validCntWake = i("validCntWake"); cntWake = i("cntWake") + maxSystolicBpWake = i("maxSystolicBpWake"); maxDiastolicBpWake = i("maxDiastolicBpWake"); maxHeartRateWake = i("maxHeartRateWake") + midSystolicBpWake = i("midSystolicBpWake"); midDiastolicBpWake = i("midDiastolicBpWake"); midHeartRateWake = i("midHeartRateWake") + minSystolicBpWake = i("minSystolicBpWake"); minDiastolicBpWake = i("minDiastolicBpWake"); minHeartRateWake = i("minHeartRateWake") + avgSystolicBpWake = i("avgSystolicBpWake"); avgDiastolicBpWake = i("avgDiastolicBpWake"); avgHeartRateWake = i("avgHeartRateWake") + stdSystolicBpWake = i("stdSystolicBpWake"); stdDiastolicBpWake = i("stdDiastolicBpWake"); stdHeartRateWake = i("stdHeartRateWake") + coefSystolicBpWake = d("coefSystolicBpWake"); coefDiastolicBpWake = d("coefDiastolicBpWake"); coefHeartRateWake = d("coefHeartRateWake") + loadSystolicBpWake = d("loadSystolicBpWake"); loadDiastolicBpWake = d("loadDiastolicBpWake") + + validCntSleep = i("validCntSleep"); cntSleep = i("cntSleep") + maxSystolicBpSleep = i("maxSystolicBpSleep"); maxDiastolicBpSleep = i("maxDiastolicBpSleep"); maxHeartRateSleep = i("maxHeartRateSleep") + midSystolicBpSleep = i("midSystolicBpSleep"); midDiastolicBpSleep = i("midDiastolicBpSleep"); midHeartRateSleep = i("midHeartRateSleep") + minSystolicBpSleep = i("minSystolicBpSleep"); minDiastolicBpSleep = i("minDiastolicBpSleep"); minHeartRateSleep = i("minHeartRateSleep") + avgSystolicBpSleep = i("avgSystolicBpSleep"); avgDiastolicBpSleep = i("avgDiastolicBpSleep"); avgHeartRateSleep = i("avgHeartRateSleep") + stdSystolicBpSleep = i("stdSystolicBpSleep"); stdDiastolicBpSleep = i("stdDiastolicBpSleep"); stdHeartRateSleep = i("stdHeartRateSleep") + coefSystolicBpSleep = d("coefSystolicBpSleep"); coefDiastolicBpSleep = d("coefDiastolicBpSleep"); coefHeartRateSleep = d("coefHeartRateSleep") + loadSystolicBpSleep = d("loadSystolicBpSleep"); loadDiastolicBpSleep = d("loadDiastolicBpSleep") + + validCntWakeTwo = i("validCntWakeTwo"); cntWakeTwo = i("cntWakeTwo") + maxSystolicBpWakeTwo = i("maxSystolicBpWakeTwo"); maxDiastolicBpWakeTwo = i("maxDiastolicBpWakeTwo"); maxHeartRateWakeTwo = i("maxHeartRateWakeTwo") + midSystolicBpWakeTwo = i("midSystolicBpWakeTwo"); midDiastolicBpWakeTwo = i("midDiastolicBpWakeTwo"); midHeartRateWakeTwo = i("midHeartRateWakeTwo") + minSystolicBpWakeTwo = i("minSystolicBpWakeTwo"); minDiastolicBpWakeTwo = i("minDiastolicBpWakeTwo"); minHeartRateWakeTwo = i("minHeartRateWakeTwo") + avgSystolicBpWakeTwo = i("avgSystolicBpWakeTwo"); avgDiastolicBpWakeTwo = i("avgDiastolicBpWakeTwo"); avgHeartRateWakeTwo = i("avgHeartRateWakeTwo") + stdSystolicBpWakeTwo = i("stdSystolicBpWakeTwo"); stdDiastolicBpWakeTwo = i("stdDiastolicBpWakeTwo"); stdHeartRateWakeTwo = i("stdHeartRateWakeTwo") + coefSystolicBpWakeTwo = d("coefSystolicBpWakeTwo"); coefDiastolicBpWakeTwo = d("coefDiastolicBpWakeTwo"); coefHeartRateWakeTwo = d("coefHeartRateWakeTwo") + + extendData = f.getString("extend_data") + }.build() + } + + private fun sampleSetDefinition( + key: String, + dataTypeSuffix: String, + defaultTopic: String, + buildRecord: ( + fields: FieldValues, + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + ) -> SpecificRecord, + ): HuaweiRouteDefinition = HuaweiRouteDefinition(key, defaultTopic) { repo, topic -> + HuaweiSampleSetRoute( + userRepository = repo, + dataTypeName = VENDOR_PREFIX + dataTypeSuffix, + topic = topic, + groupByTimeUnit = if (dataTypeSuffix.endsWith(".statistics")) "day" else null, + buildRecord = buildRecord, + ) + } + + private fun healthRecordDefinition( + key: String, + subDataTypeName: String, + defaultTopic: String, + buildRecord: ( + fields: FieldValues, + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + ) -> SpecificRecord, + ): HuaweiRouteDefinition = HuaweiRouteDefinition(key, defaultTopic) { repo, topic -> + HuaweiHealthRecordRoute( + userRepository = repo, + subDataTypeName = VENDOR_PREFIX + subDataTypeName, + topic = topic, + buildRecord = buildRecord, + ) + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt new file mode 100644 index 00000000..ddea128f --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt @@ -0,0 +1,75 @@ +package org.radarbase.huawei.route + +import com.fasterxml.jackson.databind.ObjectMapper +import org.apache.avro.specific.SpecificRecord +import org.radarbase.huawei.converter.FieldValues +import org.radarbase.huawei.converter.HuaweiDataConverter +import org.radarbase.huawei.converter.HuaweiSampleSetConverter +import org.radarbase.huawei.request.RestRequest +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserRepository +import java.time.Duration +import java.time.Instant + +/** + * Route backed by `POST /healthkit/v1/sampleSet:polymerize`, which covers the large majority of + * Huawei Health Kit data types (all `continuous.*`, `instantaneous.*`, `cgm_blood_glucose`, + * `active_hours`, `daily_activity_summary`, `emotion`, `heart_rate_variability`, `vo2max`, + * `resting_calories.statistics`, `sleep.on_off_bed_record`, and `sleep_respiratory_*` types). + * + * When [groupByTimeUnit] is set, the request aggregates sample points into buckets of that size — + * this is how Huawei's `.statistics` data types are queried. When it is `null`, the endpoint + * returns raw, un-aggregated sample points for [dataTypeName] over the requested time range. + */ +open class HuaweiSampleSetRoute( + userRepository: UserRepository, + private val dataTypeName: String, + private val topic: String, + private val groupByTimeUnit: String? = null, + maxIntervalPerRequest: Duration = Duration.ofDays(30L), + buildRecord: ( + fields: FieldValues, + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + ) -> SpecificRecord, +) : HuaweiRoute(userRepository, maxIntervalPerRequest) { + + override val converters: List = + listOf(HuaweiSampleSetConverter(topic, buildRecord)) + + override fun toString(): String = "huawei_" + topic.removePrefix("connect_huawei_") + + override fun generateRequests( + user: User, + start: Instant, + end: Instant, + max: Int, + ): Sequence = chunkedRanges(start, end, max).map { (rangeStart, rangeEnd) -> + RestRequest( + request = createPostRequest(user, "sampleSet:polymerize", buildRequestBody(rangeStart, rangeEnd)), + user = user, + route = this, + startDate = rangeStart, + endDate = rangeEnd, + ) + } + + private fun buildRequestBody(start: Instant, end: Instant): String { + val root = MAPPER.createObjectNode() + root.putArray("polymerizeWith").addObject().put("dataTypeName", dataTypeName) + root.put("startTime", start.toEpochMilli()) + root.put("endTime", end.toEpochMilli()) + if (groupByTimeUnit != null) { + val groupPeriod = root.putObject("groupByTime").putObject("groupPeriod") + groupPeriod.put("unit", groupByTimeUnit) + groupPeriod.put("value", 1) + groupPeriod.put("timeZone", "+0000") + } + return MAPPER.writeValueAsString(root) + } + + companion object { + private val MAPPER = ObjectMapper() + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt new file mode 100644 index 00000000..39e2cf71 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt @@ -0,0 +1,23 @@ +package org.radarbase.huawei.route + +import org.radarbase.huawei.request.RestRequest +import org.radarbase.huawei.user.User +import java.time.Duration +import java.time.Instant + +interface Route { + + fun generateRequests(user: User, start: Instant, end: Instant): Sequence + + fun generateRequests(user: User, start: Instant, end: Instant, max: Int): Sequence + + /** + * This is how it would appear in the offsets + */ + override fun toString(): String + + /** + * The duration of data to request in a single request of this route. + */ + val maxIntervalPerRequest: Duration +} diff --git a/kafka-connect-huawei-source/build.gradle.kts b/kafka-connect-huawei-source/build.gradle.kts index f3169abe..05fcfa3c 100644 --- a/kafka-connect-huawei-source/build.gradle.kts +++ b/kafka-connect-huawei-source/build.gradle.kts @@ -1,6 +1,11 @@ description = "Kafka connector for Huawei Health Kit API source" repositories { + // Prefer a locally-published snapshot (e.g. built by hand from the RADAR-Schemas + // huawei_schemas branch via `./gradlew :radar-schemas-commons:publishToMavenLocal`) before + // falling back to remote snapshot hosts. + mavenLocal() + // radar-schemas-commons huawei_schemas is only published as a snapshot; declare the // candidate snapshot hosts here so the build can resolve it regardless of which one the // RADAR-Schemas release pipeline currently targets. From 8b394ed5b1f547ceb8bd93687bb201f85eaebd9d Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 11:13:31 +0000 Subject: [PATCH 04/27] Implement kafka-connect-huawei-source Kafka Connect glue Mirrors kafka-connect-oura-source: HuaweiSourceConnector (periodic user refresh + task reconfiguration), HuaweiSourceTask (round-robin polling across routes, JSON->Avro->SourceRecord conversion via AvroData), KafkaOffsetManager, and HuaweiServiceUserRepository (Ktor-based rest-source-authorizer client with OAuth2 client-credentials auth and cached user/token lookups). HuaweiRestSourceConnectorConfig is written in Kotlin and loop-generates its ~110 per-data-type `huawei..enabled`/`huawei..topic` ConfigDef entries from huawei-library's HuaweiRouteFactory.definitions registry, rather than hand-duplicating a static field per topic as Fitbit/Oura do - the same registry also drives which routes HuaweiSourceTask actually builds, so the connector config and the set of polled routes can never drift out of sync. Could not compile this module in this sandbox: packages.confluent.io (needed for kafka-connect-api/kafka-connect-avro-converter) is blocked by the sandbox's egress policy - confirmed this is pre-existing and applies equally to kafka-connect-oura-source, not something introduced here. huawei-library (the part of this change with real logic to verify) does compile cleanly against a locally-published radar-schemas-commons 0.9.0-SNAPSHOT. --- .../huawei/AbstractRestSourceConnector.java | 57 ++++ .../huawei/HuaweiRestSourceConnectorConfig.kt | 259 +++++++++++++++ .../rest/huawei/HuaweiSourceConnector.java | 142 ++++++++ .../connect/rest/huawei/HuaweiSourceTask.java | 204 ++++++++++++ .../huawei/offset/KafkaOffsetManager.java | 55 +++ .../huawei/user/HttpResponseException.java | 33 ++ .../user/HuaweiServiceUserRepository.kt | 312 ++++++++++++++++++ .../rest/huawei/user/HuaweiUserRepository.kt | 36 ++ .../connect/rest/huawei/user/HuaweiUsers.java | 40 +++ .../huawei/user/OAuth2UserCredentials.java | 79 +++++ .../connect/rest/huawei/util/VersionUtil.java | 32 ++ 11 files changed, 1249 insertions(+) create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java new file mode 100644 index 00000000..ad09e22a --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java @@ -0,0 +1,57 @@ +package org.radarbase.connect.rest.huawei; + +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import org.apache.kafka.connect.connector.Task; +import org.apache.kafka.connect.source.SourceConnector; +import org.radarbase.connect.rest.huawei.util.VersionUtil; + +@SuppressWarnings("unused") +public abstract class AbstractRestSourceConnector extends SourceConnector { + protected HuaweiRestSourceConnectorConfig config; + + @Override + public String version() { + return VersionUtil.getVersion(); + } + + @Override + public Class taskClass() { + return HuaweiSourceTask.class; + } + + @Override + public List> taskConfigs(int maxTasks) { + return Collections.nCopies(maxTasks, new HashMap<>(config.originalsStrings())); + } + + @Override + public void start(Map props) { + config = getConfig(props); + } + + public abstract HuaweiRestSourceConnectorConfig getConfig(Map conf); + + @Override + public void stop() { + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt new file mode 100644 index 00000000..8178a949 --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt @@ -0,0 +1,259 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.radarbase.connect.rest.huawei + +import okhttp3.HttpUrl +import okhttp3.HttpUrl.Companion.toHttpUrlOrNull +import org.apache.kafka.common.config.AbstractConfig +import org.apache.kafka.common.config.ConfigDef +import org.apache.kafka.common.config.ConfigDef.Importance +import org.apache.kafka.common.config.ConfigDef.NonEmptyString +import org.apache.kafka.common.config.ConfigDef.Type +import org.apache.kafka.common.config.ConfigDef.Width +import org.apache.kafka.common.config.ConfigException +import org.apache.kafka.connect.errors.ConnectException +import org.radarbase.connect.rest.huawei.user.HuaweiServiceUserRepository +import org.radarbase.connect.rest.huawei.user.HuaweiUserRepository +import org.radarbase.huawei.route.HuaweiRouteFactory +import java.net.MalformedURLException +import java.net.URL +import java.time.Duration + +/** + * Kafka Connect configuration for the Huawei Health Kit source connector. + * + * Every data type registered in [HuaweiRouteFactory.definitions] gets a `huawei..enabled` + * boolean and a `huawei..topic` string config, generated from that single shared registry + * instead of ~110 hand-duplicated `ConfigDef` entries (one connector, one config, one canonical + * list of Huawei data types). + */ +class HuaweiRestSourceConnectorConfig( + config: ConfigDef, + parsedConfig: MutableMap, + doLog: Boolean, +) : AbstractConfig(config, parsedConfig, doLog) { + + constructor(parsedConfig: MutableMap, doLog: Boolean) : this(conf(), parsedConfig, doLog) + + private var userRepository: HuaweiUserRepository? = null + + fun getHuaweiUsers(): List = getList(HUAWEI_USERS_CONFIG) + + fun getHuaweiClient(): String = getString(HUAWEI_API_CLIENT_CONFIG) + + fun getHuaweiClientSecret(): String = getPassword(HUAWEI_API_SECRET_CONFIG).value() + + fun getUserRepository(reuse: HuaweiUserRepository?): HuaweiUserRepository { + val repo = if (reuse != null && reuse.javaClass == getClass(HUAWEI_USER_REPOSITORY_CONFIG)) { + reuse + } else { + createUserRepository() + } + repo.initialize(this) + userRepository = repo + return repo + } + + fun getUserRepository(): HuaweiUserRepository { + val repo = checkNotNull(userRepository) { "User repository has not been initialized" } + repo.initialize(this) + return repo + } + + @Suppress("UNCHECKED_CAST") + private fun createUserRepository(): HuaweiUserRepository = try { + (getClass(HUAWEI_USER_REPOSITORY_CONFIG) as Class) + .getDeclaredConstructor() + .newInstance() + } catch (e: ReflectiveOperationException) { + throw ConnectException("Invalid class. $e") + } + + fun getHuaweiUserRepositoryUrl(): HttpUrl { + var urlString = getString(HUAWEI_USER_REPOSITORY_URL_CONFIG).trim() + if (urlString.isNotEmpty() && urlString.last() != '/') { + urlString += "/" + } + return urlString.toHttpUrlOrNull() + ?: throw ConfigException( + HUAWEI_USER_REPOSITORY_URL_CONFIG, + urlString, + "User repository URL $urlString cannot be parsed as URL.", + ) + } + + fun getPollIntervalPerUser(): Duration = Duration.ofSeconds(getInt(HUAWEI_USER_POLL_INTERVAL_CONFIG).toLong()) + + fun getHuaweiUserRepositoryClientId(): String = getString(HUAWEI_USER_REPOSITORY_CLIENT_ID_CONFIG) + + fun getHuaweiUserRepositoryClientSecret(): String = + getPassword(HUAWEI_USER_REPOSITORY_CLIENT_SECRET_CONFIG).value() + + fun getHuaweiUserRepositoryTokenUrl(): URL? { + val value = getString(HUAWEI_USER_REPOSITORY_TOKEN_URL_CONFIG) + if (value.isNullOrEmpty()) { + return null + } + return try { + URL(value) + } catch (e: MalformedURLException) { + throw ConfigException("Huawei user repository token URL is invalid.") + } + } + + /** + * The (config key -> Kafka topic) pairs of every Huawei data type that is enabled in this + * configuration. + */ + fun enabledTopics(): Map = + HuaweiRouteFactory.definitions + .filter { getBoolean(enabledKey(it.key)) } + .associate { it.key to getString(topicKey(it.key)) } + + companion object { + private const val SOURCE_POLL_INTERVAL_CONFIG = "rest.source.poll.interval.ms" + private const val SOURCE_POLL_INTERVAL_DOC = "How often to poll the source URL." + private const val SOURCE_POLL_INTERVAL_DISPLAY = "Polling interval" + private const val SOURCE_POLL_INTERVAL_DEFAULT = 60000L + + const val SOURCE_URL_CONFIG = "rest.source.base.url" + private const val SOURCE_URL_DOC = "Base URL for REST source connector." + private const val SOURCE_URL_DISPLAY = "Base URL for REST source connector." + const val SOURCE_URL_DEFAULT = "https://health-api.cloud.huawei.com/healthkit/v1" + + const val HUAWEI_USERS_CONFIG = "huawei.users" + private const val HUAWEI_USERS_DOC = + "The user ID of Huawei users to include in polling, separated by commas. " + + "Non existing user names will be ignored. " + + "If empty, all users in the user directory will be used." + private const val HUAWEI_USERS_DISPLAY = "Huawei users" + + const val HUAWEI_API_CLIENT_CONFIG = "huawei.api.client" + private const val HUAWEI_API_CLIENT_DOC = "Client ID for the Huawei Health Kit API" + private const val HUAWEI_API_CLIENT_DISPLAY = "Huawei API client ID" + + const val HUAWEI_API_SECRET_CONFIG = "huawei.api.secret" + private const val HUAWEI_API_SECRET_DOC = "Secret for the Huawei API client set in huawei.api.client." + private const val HUAWEI_API_SECRET_DISPLAY = "Huawei API client secret" + + const val HUAWEI_USER_REPOSITORY_CONFIG = "huawei.user.repository.class" + private const val HUAWEI_USER_REPOSITORY_DOC = "Class for managing users and authentication." + private const val HUAWEI_USER_REPOSITORY_DISPLAY = "User repository class" + + const val HUAWEI_USER_POLL_INTERVAL_CONFIG = "huawei.user.poll.interval" + private const val HUAWEI_USER_POLL_INTERVAL_DOC = + "Polling interval per Huawei user per request route in seconds." + private const val HUAWEI_USER_POLL_INTERVAL_DEFAULT = 150 + private const val HUAWEI_USER_POLL_INTERVAL_DISPLAY = "Per-user per-route polling interval." + + const val HUAWEI_USER_REPOSITORY_URL_CONFIG = "huawei.user.repository.url" + private const val HUAWEI_USER_REPOSITORY_URL_DOC = + "URL for webservice containing user credentials. Only used if a webservice-based " + + "user repository is configured." + private const val HUAWEI_USER_REPOSITORY_URL_DISPLAY = "User repository URL" + private const val HUAWEI_USER_REPOSITORY_URL_DEFAULT = "" + + const val HUAWEI_USER_REPOSITORY_CLIENT_ID_CONFIG = "huawei.user.repository.client.id" + private const val HUAWEI_USER_REPOSITORY_CLIENT_ID_DOC = "Client ID for connecting to the service repository." + private const val HUAWEI_USER_REPOSITORY_CLIENT_ID_DISPLAY = "Client ID for user repository." + + const val HUAWEI_USER_REPOSITORY_CLIENT_SECRET_CONFIG = "huawei.user.repository.client.secret" + private const val HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DOC = + "Client secret for connecting to the service repository." + private const val HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DISPLAY = "Client Secret for user repository." + + const val HUAWEI_USER_REPOSITORY_TOKEN_URL_CONFIG = "huawei.user.repository.oauth2.token.url" + private const val HUAWEI_USER_REPOSITORY_TOKEN_URL_DOC = "OAuth 2.0 token url for retrieving client credentials." + private const val HUAWEI_USER_REPOSITORY_TOKEN_URL_DISPLAY = "OAuth 2.0 token URL." + + private fun enabledKey(key: String) = "huawei.$key.enabled" + private fun topicKey(key: String) = "huawei.$key.topic" + + @JvmStatic + fun conf(): ConfigDef { + val group = "Huawei" + var order = 0 + + val def = ConfigDef() + .define( + SOURCE_POLL_INTERVAL_CONFIG, Type.LONG, SOURCE_POLL_INTERVAL_DEFAULT, Importance.LOW, + SOURCE_POLL_INTERVAL_DOC, group, ++order, Width.SHORT, SOURCE_POLL_INTERVAL_DISPLAY, + ) + .define( + SOURCE_URL_CONFIG, Type.STRING, SOURCE_URL_DEFAULT, Importance.HIGH, + SOURCE_URL_DOC, group, ++order, Width.SHORT, SOURCE_URL_DISPLAY, + ) + .define( + HUAWEI_USERS_CONFIG, Type.LIST, emptyList(), Importance.HIGH, + HUAWEI_USERS_DOC, group, ++order, Width.SHORT, HUAWEI_USERS_DISPLAY, + ) + .define( + HUAWEI_API_CLIENT_CONFIG, Type.STRING, ConfigDef.NO_DEFAULT_VALUE, NonEmptyString(), + Importance.HIGH, HUAWEI_API_CLIENT_DOC, group, ++order, Width.SHORT, HUAWEI_API_CLIENT_DISPLAY, + ) + .define( + HUAWEI_API_SECRET_CONFIG, Type.PASSWORD, ConfigDef.NO_DEFAULT_VALUE, Importance.HIGH, + HUAWEI_API_SECRET_DOC, group, ++order, Width.SHORT, HUAWEI_API_SECRET_DISPLAY, + ) + .define( + HUAWEI_USER_POLL_INTERVAL_CONFIG, Type.INT, HUAWEI_USER_POLL_INTERVAL_DEFAULT, Importance.MEDIUM, + HUAWEI_USER_POLL_INTERVAL_DOC, group, ++order, Width.SHORT, HUAWEI_USER_POLL_INTERVAL_DISPLAY, + ) + .define( + HUAWEI_USER_REPOSITORY_CONFIG, Type.CLASS, HuaweiServiceUserRepository::class.java, + Importance.MEDIUM, HUAWEI_USER_REPOSITORY_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_DISPLAY, + ) + .define( + HUAWEI_USER_REPOSITORY_URL_CONFIG, Type.STRING, HUAWEI_USER_REPOSITORY_URL_DEFAULT, + Importance.LOW, HUAWEI_USER_REPOSITORY_URL_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_URL_DISPLAY, + ) + .define( + HUAWEI_USER_REPOSITORY_CLIENT_ID_CONFIG, Type.STRING, "", Importance.MEDIUM, + HUAWEI_USER_REPOSITORY_CLIENT_ID_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_CLIENT_ID_DISPLAY, + ) + .define( + HUAWEI_USER_REPOSITORY_CLIENT_SECRET_CONFIG, Type.PASSWORD, "", Importance.MEDIUM, + HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DISPLAY, + ) + .define( + HUAWEI_USER_REPOSITORY_TOKEN_URL_CONFIG, Type.STRING, "", Importance.MEDIUM, + HUAWEI_USER_REPOSITORY_TOKEN_URL_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_TOKEN_URL_DISPLAY, + ) + + HuaweiRouteFactory.definitions.forEach { d -> + val label = d.key.replace('_', ' ') + def.define( + enabledKey(d.key), Type.BOOLEAN, d.enabledByDefault, Importance.LOW, + "Enable or disable Huawei $label", group, ++order, Width.SHORT, + "Huawei $label enabled", + ) + def.define( + topicKey(d.key), Type.STRING, d.defaultTopic, Importance.LOW, + "Kafka topic for Huawei $label", group, ++order, Width.SHORT, + "Huawei $label topic", + ) + } + + return def + } + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java new file mode 100644 index 00000000..63c9debf --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java @@ -0,0 +1,142 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.connect.rest.huawei; + +import java.io.IOException; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; +import java.util.stream.Collectors; +import org.apache.kafka.common.config.ConfigDef; +import org.apache.kafka.common.config.ConfigException; +import org.radarbase.huawei.user.User; +import org.radarbase.connect.rest.huawei.user.HuaweiUserRepository; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import kotlin.sequences.SequencesKt; +import kotlin.sequences.Sequence; +import kotlin.streams.jdk8.StreamsKt; + +import static org.radarbase.connect.rest.huawei.HuaweiRestSourceConnectorConfig.HUAWEI_USERS_CONFIG; + +public class HuaweiSourceConnector extends AbstractRestSourceConnector { + + private static final Logger logger = LoggerFactory.getLogger(HuaweiSourceConnector.class); + private ScheduledExecutorService executor; + private Set configuredUsers; + private HuaweiUserRepository repository; + + @Override + public void start(Map props) { + logger.info("Starting Huawei source connector"); + super.start(props); + executor = Executors.newSingleThreadScheduledExecutor(); + + executor.scheduleAtFixedRate(() -> { + if (repository.hasPendingUpdates()) { + try { + logger.info("Requesting latest user details..."); + repository.applyPendingUpdates(); + Set newUsers = + SequencesKt.toSet(getConfig(props, false).getUserRepository(repository).stream()); + if (configuredUsers != null && !newUsers.equals(configuredUsers)) { + logger.info("User info mismatch found. Requesting reconfiguration..."); + reconfigure(); + } + } catch (IOException e) { + logger.warn("Failed to refresh users: {}", e.toString()); + } + } else { + logger.info("No pending updates found. Not attempting to refresh users."); + } + }, 0, 5, TimeUnit.MINUTES); + } + + @Override + public void stop() { + super.stop(); + executor.shutdown(); + + configuredUsers = null; + } + + private HuaweiRestSourceConnectorConfig getConfig(Map conf, boolean doLog) { + return new HuaweiRestSourceConnectorConfig(conf, doLog); + } + + @Override + public HuaweiRestSourceConnectorConfig getConfig(Map conf) { + HuaweiRestSourceConnectorConfig connectorConfig = getConfig(conf, true); + repository = connectorConfig.getUserRepository(repository); + return connectorConfig; + } + + @Override + public ConfigDef config() { + return HuaweiRestSourceConnectorConfig.conf(); + } + + @Override + public List> taskConfigs(int maxTasks) { + return configureTasks(maxTasks); + } + + private List> configureTasks(int maxTasks) { + Map baseConfig = config.originalsStrings(); + HuaweiRestSourceConnectorConfig huaweiConfig = getConfig(baseConfig); + if (repository == null) { + repository = huaweiConfig.getUserRepository(null); + } + // Divide the users over tasks + try { + Sequence ids = SequencesKt.map(huaweiConfig.getUserRepository(repository).stream(), User::getVersionedId); + List> userTasks = StreamsKt.asStream(ids) + // group users based on their hashCode, in principle, this allows for more efficient + // reconfigurations for a fixed number of tasks, since that allows existing tasks to + // only handle small modifications users to handle. + .collect(Collectors.groupingBy( + u -> Math.abs(u.hashCode()) % maxTasks, + Collectors.joining(","))) + .values().stream() + .map(u -> { + Map taskConfig = new HashMap<>(baseConfig); + taskConfig.put(HUAWEI_USERS_CONFIG, u); + return taskConfig; + }) + .collect(Collectors.toList()); + this.configuredUsers = SequencesKt.toSet(huaweiConfig.getUserRepository().stream()); + logger.info("Received userTask Configs {}", userTasks); + return userTasks; + } catch (Exception ex) { + throw new ConfigException("Cannot read users", ex); + } + } + + public void reconfigure() { + new Thread(() -> { + logger.info("Requesting reconfiguration"); + context.requestTaskReconfiguration(); + logger.info("Requested reconfiguration"); + }).start(); + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java new file mode 100644 index 00000000..e3e6520e --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java @@ -0,0 +1,204 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.connect.rest.huawei; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import org.apache.kafka.connect.data.SchemaAndValue; +import org.apache.kafka.connect.errors.ConnectException; +import org.apache.kafka.connect.source.SourceRecord; +import org.apache.kafka.connect.source.SourceTask; +import org.apache.kafka.connect.storage.OffsetStorageReader; +import org.radarbase.connect.rest.huawei.offset.KafkaOffsetManager; +import org.radarbase.connect.rest.huawei.user.HuaweiUserRepository; +import org.radarbase.connect.rest.huawei.util.VersionUtil; +import org.radarbase.huawei.converter.TopicData; +import org.radarbase.huawei.request.HuaweiRequestGenerator; +import org.radarbase.huawei.request.HuaweiResult; +import org.radarbase.huawei.request.HuaweiResult.Success; +import org.radarbase.huawei.request.HuaweiResult.Error; +import org.radarbase.huawei.request.HuaweiErrorBase; +import org.radarbase.huawei.request.RestRequest; +import org.radarbase.huawei.route.HuaweiRouteDefinition; +import org.radarbase.huawei.route.HuaweiRouteFactory; +import org.radarbase.huawei.route.Route; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.radarbase.huawei.user.User; +import io.confluent.connect.avro.AvroData; +import kotlin.streams.jdk8.StreamsKt; +import okhttp3.OkHttpClient; +import okhttp3.Response; + +public class HuaweiSourceTask extends SourceTask { + private static final Logger logger = LoggerFactory.getLogger(HuaweiSourceTask.class); + + private OkHttpClient baseClient; + private HuaweiUserRepository userRepository; + private List routes; + private HuaweiRequestGenerator huaweiRequestGenerator; + private final AvroData avroData = new AvroData(20); + private KafkaOffsetManager offsetManager; + private static final String TIMESTAMP_OFFSET_KEY = "timestamp"; + private static final long TIMEOUT = 60000L; + private int routeStartIndex = 0; + + public void initialize(HuaweiRestSourceConnectorConfig config, OffsetStorageReader offsetStorageReader) { + this.baseClient = new OkHttpClient(); + + this.userRepository = config.getUserRepository(); + this.offsetManager = new KafkaOffsetManager(offsetStorageReader); + this.routes = getRoutes(config); + this.huaweiRequestGenerator = new HuaweiRequestGenerator(this.userRepository, this.offsetManager, this.routes); + this.offsetManager.initialize(getPartitions()); + } + + private List getRoutes(HuaweiRestSourceConnectorConfig config) { + Map enabledTopics = config.enabledTopics(); + List result = new ArrayList<>(); + for (HuaweiRouteDefinition definition : HuaweiRouteFactory.INSTANCE.getDefinitions()) { + String topic = enabledTopics.get(definition.getKey()); + if (topic != null) { + result.add(definition.getBuild().invoke(userRepository, topic)); + } + } + return result; + } + + public List> getPartitions() { + try { + return StreamsKt.asStream(userRepository.stream()) + .flatMap(u -> this.routes.stream().map(r -> getPartition(r.toString(), u))) + .collect(Collectors.toList()); + } catch (Exception e) { + logger.warn("Failed to initialize user partitions.."); + return Collections.emptyList(); + } + } + + public Map getPartition(String route, User user) { + Map partition = new HashMap<>(4); + partition.put("user", user.getVersionedId()); + partition.put("route", route); + return partition; + } + + public Stream requests() { + if (this.routes == null || this.routes.isEmpty()) { + return Stream.empty(); + } + + // Rotate routes so that all routes are requested in a round-robin manner + List rotatedRoutes = getRotatedRoutes(); + return rotatedRoutes.stream() + .flatMap((Route r) -> StreamsKt.asStream(huaweiRequestGenerator.requests(r, 100))); + } + + private List getRotatedRoutes() { + List rotatedRoutes = new ArrayList<>(this.routes); + Collections.rotate(rotatedRoutes, routeStartIndex % this.routes.size()); + routeStartIndex = (routeStartIndex + 1) % this.routes.size(); + return rotatedRoutes; + } + + public Stream handleRequest(RestRequest req) throws IOException { + try (Response response = baseClient.newCall(req.getRequest()).execute()) { + HuaweiResult> result = this.huaweiRequestGenerator.handleResponse(req, response); + if (result instanceof HuaweiResult.Success) { + Success> success = (Success>) result; + return success.getValue().stream().map(r -> { + SchemaAndValue avro = avroData.toConnectData(r.getValue().getSchema(), r.getValue()); + SchemaAndValue key = avroData.toConnectData(r.getKey().getSchema(), r.getKey()); + Map partition = getPartition(req.getRoute().toString(), req.getUser()); + Map offset = Collections.singletonMap(TIMESTAMP_OFFSET_KEY, r.getOffset()); + + return new SourceRecord(partition, offset, r.getTopic(), + key.schema(), key.value(), avro.schema(), avro.value()); + }); + } else { + HuaweiErrorBase e = (HuaweiErrorBase) ((HuaweiResult.Error) result).getError(); + logger.warn("Failed to make request: {} {} {}", e.getMessage(), e.getCause(), e.getCode()); + return Stream.empty(); + } + } + } + + @Override + public void start(Map map) { + HuaweiRestSourceConnectorConfig connectorConfig; + try { + Class connector = Class.forName(map.get("connector.class")); + Object connectorInst = connector.getConstructor().newInstance(); + connectorConfig = ((HuaweiSourceConnector) connectorInst).getConfig(map); + } catch (ClassNotFoundException e) { + throw new ConnectException("Connector " + map.get("connector.class") + " not found", e); + } catch (ReflectiveOperationException e) { + throw new ConnectException("Connector " + map.get("connector.class") + + " could not be instantiated", e); + } + this.initialize(connectorConfig, context.offsetStorageReader()); + } + + @Override + public List poll() throws InterruptedException { + long requestsGenerated = 0; + List sourceRecords = Collections.emptyList(); + + do { + Thread.sleep(TIMEOUT); + + Iterator requestIterator = this.requests().iterator(); + + while (sourceRecords.isEmpty() && requestIterator.hasNext()) { + RestRequest request = requestIterator.next(); + + logger.info("Requesting for user {}, url: {}", request.getUser().getUserId(), request.getRequest().url()); + requestsGenerated++; + + try { + sourceRecords = this.handleRequest(request) + .collect(Collectors.toList()); + } catch (IOException ex) { + logger.warn("Failed to make request: {}", ex.toString()); + } + } + } while (sourceRecords.isEmpty()); + + logger.info("Processed {} records from {} URLs", sourceRecords.size(), requestsGenerated); + + return sourceRecords; + } + + @Override + public void stop() { + logger.debug("Stopping source task"); + } + + @Override + public String version() { + return VersionUtil.getVersion(); + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java new file mode 100644 index 00000000..dcc044ef --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java @@ -0,0 +1,55 @@ +package org.radarbase.connect.rest.huawei.offset; + +import java.time.Duration; +import java.time.Instant; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import static java.time.temporal.ChronoUnit.NANOS; +import org.apache.kafka.connect.storage.OffsetStorageReader; +import org.radarbase.huawei.offset.Offset; +import org.radarbase.huawei.request.HuaweiOffsetManager; +import org.radarbase.huawei.route.Route; +import org.radarbase.huawei.user.User; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class KafkaOffsetManager implements HuaweiOffsetManager { + private static final Logger logger = LoggerFactory.getLogger(KafkaOffsetManager.class); + private static final String TIMESTAMP_OFFSET_KEY = "timestamp"; + private static final Duration ONE_NANO = NANOS.getDuration(); + + private final OffsetStorageReader offsetStorageReader; + private Map offsets; + + public KafkaOffsetManager(OffsetStorageReader offsetStorageReader) { + this.offsetStorageReader = offsetStorageReader; + } + + public void initialize(List> partitions) { + if (this.offsetStorageReader != null) { + this.offsets = this.offsetStorageReader.offsets(partitions).entrySet().stream() + .filter(e -> e.getValue() != null && e.getValue().containsKey(TIMESTAMP_OFFSET_KEY)) + .collect(Collectors.toMap( + e -> e.getKey().get("user") + "-" + e.getKey().get("route"), + e -> Instant.ofEpochSecond(((Number) e.getValue().get(TIMESTAMP_OFFSET_KEY)).longValue()))); + } else { + logger.warn("Offset storage reader is null, will resume from an empty state."); + } + } + + @Override + public Offset getOffset(Route route, User user) { + Instant offset = offsets.getOrDefault(getOffsetKey(route, user), user.getStartDate().minus(ONE_NANO)); + return new Offset(user, route, offset); + } + + @Override + public void updateOffsets(Route route, User user, Instant offset) { + offsets.put(getOffsetKey(route, user), offset); + } + + private String getOffsetKey(Route route, User user) { + return user.getVersionedId() + "-" + route.toString(); + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java new file mode 100644 index 00000000..c4f8c30c --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java @@ -0,0 +1,33 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.connect.rest.huawei.user; + +import java.io.IOException; + +public class HttpResponseException extends IOException { + private final int statusCode; + + public HttpResponseException(String message, int statusCode) { + super(message); + this.statusCode = statusCode; + } + + public int getStatusCode() { + return statusCode; + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt new file mode 100644 index 00000000..8bf64c2a --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt @@ -0,0 +1,312 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.radarbase.connect.rest.huawei.user + +import com.fasterxml.jackson.databind.ObjectMapper +import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule +import com.fasterxml.jackson.module.kotlin.readValue +import com.fasterxml.jackson.module.kotlin.registerKotlinModule +import io.ktor.client.HttpClient +import io.ktor.client.engine.cio.CIO +import io.ktor.client.plugins.HttpTimeout +import io.ktor.client.plugins.auth.Auth +import io.ktor.client.plugins.auth.providers.BasicAuthCredentials +import io.ktor.client.plugins.auth.providers.basic +import io.ktor.client.plugins.contentnegotiation.ContentNegotiation +import io.ktor.client.plugins.defaultRequest +import io.ktor.client.request.HttpRequestBuilder +import io.ktor.client.request.request +import io.ktor.client.request.setBody +import io.ktor.client.request.url +import io.ktor.client.statement.bodyAsText +import io.ktor.client.statement.request +import io.ktor.http.ContentType +import io.ktor.http.HttpMethod +import io.ktor.http.HttpStatusCode +import io.ktor.http.URLBuilder +import io.ktor.http.Url +import io.ktor.http.contentLength +import io.ktor.http.contentType +import io.ktor.http.isSuccess +import io.ktor.http.takeFrom +import io.ktor.serialization.jackson.jackson +import io.ktor.serialization.kotlinx.json.json +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.runBlocking +import kotlinx.coroutines.withContext +import kotlinx.serialization.json.Json +import org.radarbase.connect.rest.huawei.HuaweiRestSourceConnectorConfig +import org.radarbase.kotlin.coroutines.CacheConfig +import org.radarbase.kotlin.coroutines.CachedSet +import org.radarbase.kotlin.coroutines.CachedValue +import org.radarbase.ktor.auth.ClientCredentialsConfig +import org.radarbase.ktor.auth.clientCredentials +import org.radarbase.huawei.user.HuaweiUser +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserNotAuthorizedException +import org.slf4j.LoggerFactory +import java.io.IOException +import java.util.concurrent.ConcurrentHashMap +import kotlin.streams.asSequence +import kotlin.time.Duration.Companion.days +import kotlin.time.Duration.Companion.hours +import kotlin.time.Duration.Companion.minutes +import kotlin.time.Duration.Companion.seconds + +/** + * User repository backed by the RADAR-base "rest-source-authorizer" webservice, mirroring + * [org.radarbase.connect.rest.oura.user.OuraServiceUserRepository]. Retrieves the list of Huawei + * users configured for a study (`GET users?source-type=Huawei`) and their Huawei Health Kit OAuth2 + * access/refresh tokens (`users//token`). + */ +@Suppress("unused") +class HuaweiServiceUserRepository : HuaweiUserRepository() { + private lateinit var userCache: CachedSet + private lateinit var client: HttpClient + private val credentialCaches = ConcurrentHashMap>() + private val credentialCacheConfig = + CacheConfig(refreshDuration = 1.days, retryDuration = 1.minutes) + private val mapper = ObjectMapper().registerKotlinModule().registerModule(JavaTimeModule()) + + @Throws(IOException::class) + override fun get(key: String): User = + runBlocking(Dispatchers.Default) { + makeRequest { url("users/$key") } + } + + override fun initialize(config: HuaweiRestSourceConnectorConfig) { + val containedUsers = config.getHuaweiUsers().toHashSet() + + client = + createClient( + baseUrl = config.getHuaweiUserRepositoryUrl(), + tokenUrl = config.getHuaweiUserRepositoryTokenUrl()?.let { URLBuilder(it.toString()).build() }, + clientId = config.getHuaweiUserRepositoryClientId(), + clientSecret = config.getHuaweiUserRepositoryClientSecret(), + scope = "SUBJECT.READ MEASUREMENT.CREATE", + audience = "res_restAuthorizer", + ) + + userCache = + CachedSet( + CacheConfig(refreshDuration = 1.hours, retryDuration = 1.minutes), + ) { + makeRequest { url("users?source-type=Huawei") } + .users + .toHashSet() + .filterTo(HashSet()) { u -> + u.isComplete() && + (containedUsers.isEmpty() || u.versionedId in containedUsers) + } + } + } + + private fun createClient( + baseUrl: Url, + tokenUrl: Url?, + clientId: String?, + clientSecret: String?, + scope: String?, + audience: String?, + ): HttpClient = + HttpClient(CIO) { + if (tokenUrl != null) { + install(Auth) { + clientCredentials( + ClientCredentialsConfig( + tokenUrl.toString(), + clientId, + clientSecret, + scope, + audience, + ).copyWithEnv("MANAGEMENT_PORTAL"), + baseUrl.host, + ) + } + install(ContentNegotiation) { + json( + Json { + ignoreUnknownKeys = true + }, + ) + } + } else if (clientId != null && clientSecret != null) { + install(Auth) { + basic { + credentials { + BasicAuthCredentials(username = clientId, password = clientSecret) + } + realm = "Access to the '/' path" + sendWithoutRequest { + it.url.host == baseUrl.host + } + } + } + } + + defaultRequest { + url.takeFrom(baseUrl) + } + + install(ContentNegotiation) { + jackson { + registerModule(JavaTimeModule()) // support java.time.* types + } + } + + install(HttpTimeout) { + connectTimeoutMillis = 60.seconds.inWholeMilliseconds + requestTimeoutMillis = 90.seconds.inWholeMilliseconds + } + } + + override fun stream(): Sequence = + runBlocking(Dispatchers.Default) { + val valueInCache = + userCache.getFromCache() + .takeIf { it is CachedValue.CacheValue } + ?.getOrThrow() + + (valueInCache ?: userCache.get()) + .stream() + .filter { it.isComplete() } + .asSequence() + } + + @Throws(IOException::class, UserNotAuthorizedException::class) + override fun getAccessToken(user: User): String { + if (!user.isAuthorized) { + throw UserNotAuthorizedException("User is not authorized") + } + return runBlocking(Dispatchers.Default) { + credentialCache(user) + .get { !it.isAccessTokenExpired } + .value + .accessToken + } + } + + @Throws(IOException::class, UserNotAuthorizedException::class) + override fun refreshAccessToken(user: User): String { + if (!user.isAuthorized) { + throw UserNotAuthorizedException("User is not authorized") + } + return runBlocking(Dispatchers.Default) { + val token = + requestAccessToken(user) { + url("users/${user.id}/token") + method = HttpMethod.Post + setBody("{}") + contentType(ContentType.Application.Json) + } + credentialCache(user).set(token) + token.accessToken + } + } + + private suspend fun credentialCache(user: User): CachedValue = + credentialCaches.computeIfAbsent(user.id) { + CachedValue(credentialCacheConfig) { + requestAccessToken(user) { url("users/${user.id}/token") } + } + } + + @Throws(UserNotAuthorizedException::class, IOException::class) + private suspend fun requestAccessToken( + user: User, + builder: HttpRequestBuilder.() -> Unit, + ): OAuth2UserCredentials = + try { + makeRequest(builder) + } catch (ex: HttpResponseException) { + if (ex.statusCode == 407) { + credentialCaches -= user.id + throw UserNotAuthorizedException(ex.message) + } + throw ex + } + + override fun hasPendingUpdates(): Boolean = + runBlocking(Dispatchers.Default) { + userCache.isStale(1.hours) + } + + @Throws(IOException::class) + override fun applyPendingUpdates() { + logger.info("Requesting user information from webservice") + + runBlocking(Dispatchers.Default) { + userCache.get() + } + } + + private suspend inline fun makeRequest( + crossinline builder: HttpRequestBuilder.() -> Unit, + ): T = + withContext(Dispatchers.IO) { + val requestBuilder = HttpRequestBuilder() + builder(requestBuilder) + logger.info("Making HTTP request: ${requestBuilder.method} ${requestBuilder.url}") + + val response = client.request(builder) + logger.info("Response status: ${response.status}") + val contentLength = response.contentLength() + val transferEncoding = response.headers["Transfer-Encoding"] + val hasBody = (contentLength != null && contentLength > 0) || + (transferEncoding != null && transferEncoding.contains("chunked")) + val responseBody = try { + response.bodyAsText() + } catch (e: Exception) { + "Error reading body: ${e.message}" + } + + if (response.status == HttpStatusCode.NotFound) { + logger.error("HTTP 404 Not Found: ${response.request.url}") + throw NoSuchElementException("URL " + response.request.url + " does not exist") + } else if (!response.status.isSuccess()) { + val message = "HTTP ${response.status.value} error: $responseBody" + logger.error(message) + throw HttpResponseException(message, response.status.value) + } else if (!hasBody) { + logger.warn( + "HTTP ${response.status.value} OK but no body content. Returning empty result.", + ) + @Suppress("UNCHECKED_CAST") + return@withContext when (T::class) { + String::class -> "" as T + List::class -> emptyList() as T + else -> mapper.readValue("{}") + } + } + + try { + val result = mapper.readValue(responseBody) + logger.info("Successfully parsed response as ${T::class.simpleName}") + result + } catch (e: Exception) { + logger.error( + "Failed to parse response body as ${T::class.simpleName}: ${e.message}", + ) + logger.error("Response body that failed to parse: $responseBody") + throw e + } + } + + companion object { + private val logger = LoggerFactory.getLogger(HuaweiServiceUserRepository::class.java) + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt new file mode 100644 index 00000000..010a5b5a --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt @@ -0,0 +1,36 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.radarbase.connect.rest.huawei.user + +import org.radarbase.connect.rest.huawei.HuaweiRestSourceConnectorConfig +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserNotAuthorizedException +import org.radarbase.huawei.user.UserRepository +import java.io.IOException + +@Suppress("unused") +abstract class HuaweiUserRepository : UserRepository { + abstract fun initialize(config: HuaweiRestSourceConnectorConfig) + + @Throws(IOException::class, UserNotAuthorizedException::class) + abstract fun refreshAccessToken(user: User): String + + @Throws(IOException::class) + abstract fun applyPendingUpdates() + + abstract fun hasPendingUpdates(): Boolean +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java new file mode 100644 index 00000000..e01aeff8 --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java @@ -0,0 +1,40 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.connect.rest.huawei.user; + +import com.fasterxml.jackson.annotation.JsonCreator; +import com.fasterxml.jackson.annotation.JsonIgnoreProperties; +import com.fasterxml.jackson.annotation.JsonProperty; + +import java.util.ArrayList; +import java.util.List; +import org.radarbase.huawei.user.HuaweiUser; + +@JsonIgnoreProperties(ignoreUnknown = true) +public class HuaweiUsers { + private final List users; + + @JsonCreator + public HuaweiUsers(@JsonProperty("users") List users) { + this.users = new ArrayList<>(users); + } + + public List getUsers() { + return users; + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java new file mode 100644 index 00000000..88bad3b8 --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java @@ -0,0 +1,79 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.connect.rest.huawei.user; + +import com.fasterxml.jackson.annotation.JsonIgnore; +import com.fasterxml.jackson.annotation.JsonIgnoreProperties; +import com.fasterxml.jackson.annotation.JsonProperty; +import com.fasterxml.jackson.annotation.JsonSetter; +import java.time.Duration; +import java.time.Instant; + +@JsonIgnoreProperties(ignoreUnknown = true) +public class OAuth2UserCredentials { + private static final Duration DEFAULT_EXPIRY = Duration.ofHours(1); + private static final Duration EXPIRY_TIME_MARGIN = Duration.ofMinutes(5); + + @JsonProperty + private String accessToken; + @JsonProperty + private String refreshToken; + @JsonProperty + private Instant expiresAt; + + public OAuth2UserCredentials() { + } + + public OAuth2UserCredentials(String refreshToken, String accessToken, Long expiresIn) { + this.refreshToken = refreshToken; + this.accessToken = accessToken; + this.expiresAt = getExpiresAt(expiresIn != null && expiresIn > 0L + ? Duration.ofSeconds(expiresIn) : DEFAULT_EXPIRY); + } + + public String getAccessToken() { + return accessToken; + } + + @JsonSetter + public void setAccessToken(String accessToken) { + this.accessToken = accessToken; + if (expiresAt == null) { + expiresAt = getExpiresAt(DEFAULT_EXPIRY); + } + } + + public boolean hasRefreshToken() { + return refreshToken != null && !refreshToken.isEmpty(); + } + + public String getRefreshToken() { + return refreshToken; + } + + protected static Instant getExpiresAt(Duration expiresIn) { + return Instant.now() + .plus(expiresIn) + .minus(EXPIRY_TIME_MARGIN); + } + + @JsonIgnore + public boolean isAccessTokenExpired() { + return expiresAt == null || Instant.now().isAfter(expiresAt); + } +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java new file mode 100644 index 00000000..8c23ac79 --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java @@ -0,0 +1,32 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.connect.rest.huawei.util; + +public final class VersionUtil { + private VersionUtil() { + // utility class + } + + public static String getVersion() { + try { + return VersionUtil.class.getPackage().getImplementationVersion(); + } catch (Exception ex) { + return "0.0.0.0"; + } + } +} From bf058911b03d0f61ce85518c0efb161d10749e3a Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 11:16:29 +0000 Subject: [PATCH 05/27] Wire up Huawei connector in docker-compose, README, and CI matrices Adds docker/source-huawei.properties.template, a radar-huawei-connector docker-compose service, a README section, and registers the kafka-connect-huawei-source image in both CI workflow matrices. Updates ARCHITECTURE.md to document the Huawei module and the route-registry config-generation pattern it introduces for connectors with a large number of data types. --- .github/workflows/main.yml | 5 + .github/workflows/release.yml | 5 + ARCHITECTURE.md | 120 +++++++++++++++++------ README.md | 28 +++++- docker-compose.yml | 47 +++++++++ docker/source-huawei.properties.template | 12 +++ 6 files changed, 183 insertions(+), 34 deletions(-) create mode 100644 docker/source-huawei.properties.template diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 9be81f61..e5397368 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -22,6 +22,11 @@ env: 'build_file': 'kafka-connect-oura-source/Dockerfile', 'authors': 'Pim van Nierop , Pauline Conde ', 'description': 'RADAR-base Oura connector application' + },{ + 'name': 'kafka-connect-huawei-source', + 'build_file': 'kafka-connect-huawei-source/Dockerfile', + 'authors': 'Yatharth Ranjan ', + 'description': 'RADAR-base Huawei Health Kit connector application' }] jobs: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c70528ad..861df14e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -18,6 +18,11 @@ env: 'build_file': 'kafka-connect-oura-source/Dockerfile', 'authors': 'Pauline Conde , Yatharth Ranjan ', 'description': 'RADAR-base Oura connector application' + },{ + 'name': 'kafka-connect-huawei-source', + 'build_file': 'kafka-connect-huawei-source/Dockerfile', + 'authors': 'Yatharth Ranjan ', + 'description': 'RADAR-base Huawei Health Kit connector application' }] jobs: diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 0d716c53..cc623526 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,15 +1,17 @@ # Architecture This document describes how RADAR-REST-Connector is put together, so that future contributors -(human or agent) can orient themselves quickly and add new device/API integrations (e.g. Huawei -Health Kit) consistently with the existing patterns. +(human or agent) can orient themselves quickly and add new device/API integrations consistently +with the existing patterns. ## What this repo is A multi-module Gradle project providing Kafka Connect **source connectors** that poll third-party REST APIs (wearable vendor APIs) on behalf of RADAR-base study participants and publish the -resulting data as Avro records on Kafka topics. It currently ships two concrete connectors — -**Fitbit** and **Oura** — built on top of a shared, generic REST-polling framework. +resulting data as Avro records on Kafka topics. It currently ships three concrete connectors — +**Fitbit**, **Oura**, and **Huawei Health Kit** — the latter two built on the "library + thin +Connect glue" pattern described below; Fitbit predates that pattern and uses the older, generic +`kafka-connect-rest-source` framework instead. ``` RADAR-REST-Connector/ @@ -17,6 +19,8 @@ RADAR-REST-Connector/ ├── kafka-connect-fitbit-source/ # Fitbit connector (Java), oldest/original implementation ├── oura-library/ # Oura domain logic: routes, converters, requests (Kotlin, no Kafka Connect deps) ├── kafka-connect-oura-source/ # Oura Kafka Connect glue (Java+Kotlin), wraps oura-library +├── huawei-library/ # Huawei domain logic: routes, converters, requests (Kotlin, no Kafka Connect deps) +├── kafka-connect-huawei-source/ # Huawei Kafka Connect glue (Java+Kotlin), wraps huawei-library ├── docker/ # Docker Compose config templates, launch/ensure scripts, log4j ├── scripts/REDCAP-FITBIT-AUTH-AUTO/ # Standalone Python helper for REDCap-driven Fitbit auth └── docker-compose.yml # Full local Kafka stack + both connectors, for manual testing @@ -158,6 +162,41 @@ This was a deliberate move to (a) get domain logic under unit test without spinn Connect, and (b) avoid the generic framework's assumptions (e.g. its polling-interval math) that didn't fit Oura's simpler historical/recent chunking model. +### 4. Huawei Health Kit connector (`huawei-library` + `kafka-connect-huawei-source`) + +Structurally identical to the Oura pattern above (pure-Kotlin domain library + thin Connect glue +module), but with two differences worth knowing about: + +- **Three request "shapes" instead of one.** The Huawei Health Kit Data API doesn't have a single + uniform per-route request shape like Oura's `GET .../{subPath}?start_date=...&end_date=...`. It + exposes `POST /healthkit/v1/sampleSet:polymerize` (raw sample points, or day-aggregated + statistics when a `groupByTime` block is added to the JSON body) for most data types, plus two + GET endpoints — `activityRecords` and `healthRecords` — for workout sessions and clinical-style + records (blood pressure sessions, heart-rate alerts, menstrual cycle, sleep). `HuaweiRoute` is + the shared abstract base (OAuth2-authorized request building + time-range chunking); + `HuaweiSampleSetRoute`, `HuaweiHealthRecordRoute`, and `HuaweiActivityRecordRoute` are the three + concrete route kinds. +- **A single route registry drives both the route list and the Connect config**, instead of + Oura/Fitbit's one-hand-written-`ConfigDef`-entry-per-data-type approach. Huawei has ~54 data + types (see the `radar-huawei-connector` schema spec in RADAR-Schemas, + `specifications/connector/radar-huawei-connector-1.0.0.yml`), several of which reuse the same + Avro schema (`HuaweiStatistics` alone backs 14 different `*.statistics` topics) — hand-duplicating + a `ConfigDef.define(...)` block and a route-construction branch per type, Fitbit/Oura-style, + would mean ~110 near-identical static fields. Instead, `huawei-library`'s + `route/HuaweiRouteFactory.definitions` is a `List` (config key, default + topic, default enabled, and a `(UserRepository, topic) -> HuaweiRoute` builder) — the single + source of truth for "what Huawei data types exist." `HuaweiRestSourceConnectorConfig.conf()` + loops over it to generate `huawei..enabled`/`huawei..topic` `ConfigDef` entries, and + `HuaweiSourceTask.getRoutes()` loops over the same list filtered by that config to build the + actual `Route` instances — so the config and the polled routes can't drift out of sync. If you + add a data type to a future connector with a similarly large surface, prefer this registry + pattern over copy-pasting Oura's per-type `ConfigDef` blocks. + +Field-value key names inside `HuaweiRouteFactory`'s record builders (what JSON key a given Avro +field is read from) are a best-effort mapping to Huawei's documented `Field` naming convention — +verify them against a real Health Kit API response and adjust before relying on this in +production; see the KDoc at the top of that file. + ## Runtime data flow (both connectors, conceptually) ```mermaid @@ -201,7 +240,10 @@ plus vendor-specific keys, e.g.: studies can disable data types they don't need. The full current list for Fitbit is documented in `README.md`; Oura's config lives in -`OuraRestSourceConnectorConfig` (no README table yet — check the class directly). +`OuraRestSourceConnectorConfig` (no README table yet — check the class directly). Huawei's +per-data-type keys are generated from `HuaweiRouteFactory.definitions` (see below) rather than +hand-written — check that list, or a running connector's `GET /connectors//config`, for the +current set. ## Docker / deployment @@ -209,43 +251,59 @@ Each connector module has its own multi-stage `Dockerfile` (Gradle build stage `confluentinc/cp-kafka-connect-base`), publishing built jars plus third-party deps into `$CONNECT_PLUGIN_PATH//`. `docker/launch` and `docker/ensure` are modified Confluent entrypoint scripts (env-var → properties translation, Kafka-readiness wait). `docker-compose.yml` -spins up a full local Zookeeper+Kafka+SchemaRegistry+REST-proxy cluster plus both connectors for -manual end-to-end testing (`docker-compose up -d --build`, inspect with +spins up a full local Zookeeper+Kafka+SchemaRegistry+REST-proxy cluster plus all three connectors +for manual end-to-end testing (`docker-compose up -d --build`, inspect with `kafka-avro-console-consumer`). Sentry error monitoring is wired in via `radarKotlin { sentryEnabled = true }` and configured purely through `SENTRY_DSN`/`SENTRY_*` env vars — see README "Sentry monitoring". ## Testing - `kafka-connect-rest-source/src/test`, `kafka-connect-fitbit-source/src/test`, - `kafka-connect-oura-source/src/test` currently only contain config-parsing tests - (`*ConnectorConfigTest`) plus one task test — test coverage of the actual polling/conversion - logic is thin. `wiremock` and `mockito` are on the version catalog for HTTP-level testing but not - yet exercised much; `oura-library`'s pure-Kotlin design makes it the easiest place to add real - unit tests for new routes/converters without Kafka Connect scaffolding. + `kafka-connect-oura-source/src/test`, `kafka-connect-huawei-source/src/test` currently only + contain config-parsing tests (`*ConnectorConfigTest`) plus one task test — test coverage of the + actual polling/conversion logic is thin. `wiremock` and `mockito` are on the version catalog for + HTTP-level testing but not yet exercised much; the `oura-library`/`huawei-library` pure-Kotlin + design makes those the easiest place to add real unit tests for new routes/converters without + Kafka Connect scaffolding. - CI (`.github/workflows/main.yml`) runs `./gradlew assemble` and `./gradlew check` on every push/PR to `master`/`dev`, then builds (and on `push`, publishes) multi-arch Docker images per connector module via a matrix job. `release.yml` does the same on GitHub Release publish, additionally uploading built jars as release assets, tagged `vX.Y.Z` from `gradle.properties`/version catalog. - -## Adding a new vendor integration (e.g. Huawei) - -Follow the **Oura pattern**, not the Fitbit one: - -1. New Gradle module `huawei-library` (pure Kotlin, mirrors `oura-library`): `user/`, `route/`, - `converter/`, `request/`, `offset/` packages. No Kafka Connect or OkHttp-Connect-specific types - here — keep it independently testable. -2. New Gradle module `kafka-connect-huawei-source` (mirrors `kafka-connect-oura-source`): - `HuaweiSourceConnector`, `HuaweiSourceTask`, `HuaweiRestSourceConnectorConfig`, - `offset/KafkaOffsetManager`, `user/HuaweiServiceUserRepository` (Ktor-based - rest-source-authorizer client, copy `OuraServiceUserRepository`'s structure), plus a - `Dockerfile`. +- **Sandbox note:** in a network-restricted environment (no access to `packages.confluent.io`, or + to whichever host actually serves a given `-SNAPSHOT` dependency), only the pure-Kotlin library + modules (`oura-library`, `huawei-library`) may be compilable — the `kafka-connect-*-source` + glue modules depend on `io.confluent:kafka-connect-avro-converter` / + `org.apache.kafka:connect-api` from Confluent's Maven repo and won't resolve. If you hit this, + it's an environment limitation, not a code problem: check whether the library module alone + compiles before concluding the code is broken, and consider publishing a needed `-SNAPSHOT` + dependency to `mavenLocal()` (e.g. `gradle :radar-schemas-commons:publishToMavenLocal` from a + RADAR-Schemas checkout) to verify domain logic against the real generated classes. + +## Adding a new vendor integration + +Follow the **Oura/Huawei pattern**, not the Fitbit one — see the Huawei section above for a +worked example, including the route-registry technique for connectors with a large number of +data types: + +1. New Gradle module `-library` (pure Kotlin, mirrors `oura-library`/`huawei-library`): + `user/`, `route/`, `converter/`, `request/`, `offset/` packages. No Kafka Connect or + OkHttp-Connect-specific types here — keep it independently testable. +2. New Gradle module `kafka-connect--source` (mirrors `kafka-connect-oura-source`/ + `kafka-connect-huawei-source`): `SourceConnector`, `SourceTask`, + `RestSourceConnectorConfig`, `offset/KafkaOffsetManager`, + `user/ServiceUserRepository` (Ktor-based rest-source-authorizer client, copy + `OuraServiceUserRepository`'s/`HuaweiServiceUserRepository`'s structure), plus a `Dockerfile`. 3. Register both modules in `settings.gradle.kts`; add any new dependency versions to - `gradle/libs.versions.toml` first. + `gradle/libs.versions.toml` first. If the vendor's schemas are only available as a `-SNAPSHOT`, + add a separate version-catalog entry for it (see `radarSchemasHuawei`) so it doesn't force + every other module onto an unreleased version. 4. Confirm (or add) the required Avro schemas in the external RADAR-Schemas project and bump the - `radarSchemas` version in the catalog once published — this repo cannot invent schemas locally. -5. One `Route`/`Converter` pair per Huawei data type you plan to support, each independently - togglable via a `huawei..enabled` config flag, matching the Oura/Fitbit convention. -6. Add `docker/source-huawei.properties.template`, a `docker-compose.yml` service entry, and a - README config table, following the Fitbit/Oura sections as templates. + catalog version once published — this repo cannot invent schemas locally. +5. One `Route`/`Converter` per vendor data type. For a small number of data types, per-type classes + (Oura's approach) are fine; for a large or schema-reuse-heavy surface (Huawei's ~54 types + sharing a handful of Avro schemas), prefer a single registry (`HuaweiRouteFactory.definitions`) + that both the `ConfigDef` builder and the route-construction code iterate over. +6. Add `docker/source-.properties.template`, a `docker-compose.yml` service entry, and a + README section, following the Fitbit/Oura/Huawei sections as templates. 7. Add the new Docker image to the `IMAGES` matrix in both `.github/workflows/main.yml` and `release.yml`. diff --git a/README.md b/README.md index 585c87e0..09b941e1 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # Kafka Connect REST Source and Fitbit Source This project contains a Kafka Connect source connector for a general REST API, for -specific Fitbit and Oura devices. The documentation of the Kafka Connect REST source still needs to -be done. +specific Fitbit, Oura, and Huawei Health Kit devices. The documentation of the Kafka Connect REST +source still needs to be done. @@ -10,6 +10,7 @@ be done. * [Fitbit source connector](#fitbit-source-connector) * [Installation](#installation) * [Usage](#usage) + * [Huawei Health Kit source connector](#huawei-health-kit-source-connector) * [Sentry monitoring](#sentry-monitoring) * [Contributing](#contributing) @@ -208,9 +209,30 @@ sequenceDiagram connector ->> connector: Update offset times ``` +## Huawei Health Kit source connector + +The `kafka-connect-huawei-source` module polls the +[Huawei Health Kit Data API](https://developer.huawei.com/consumer/en/doc/HMSCore-References/rest-overview-0000001254420693) +for the data types documented in the +[`radar-huawei-connector` schema specification](https://github.com/RADAR-base/RADAR-Schemas/blob/huawei_schemas/specifications/connector/radar-huawei-connector-1.0.0.yml) +(RADAR-Schemas, `huawei_schemas` branch) — activity records, continuous/instantaneous sample +statistics (steps, distance, calories, heart rate, SpO2, blood pressure, breathing rate, ECG, +sleep stages, and more), health records (ambulatory blood pressure, heart rate alerts, +hyperthermia, low SpO2 alerts, menstrual cycle, sleep), and daily summaries. It follows the same +`rest.source.*`, `huawei.api.client`/`huawei.api.secret`, and `huawei.user.repository.*` +configuration conventions as the Fitbit and Oura connectors above, plus one +`huawei..enabled` / `huawei..topic` pair per Huawei data type — see +`org.radarbase.huawei.route.HuaweiRouteFactory` for the full list of `` keys and their +default topic names, and `docker/source-huawei.properties.template` for a minimal example. + +This connector requires a +[published `radar-schemas-commons` build containing the `huawei_schemas` branch](https://github.com/RADAR-base/RADAR-Schemas/tree/huawei_schemas) +(currently `0.9.0-SNAPSHOT`) to be resolvable from one of the repositories declared in +`huawei-library/build.gradle` / `kafka-connect-huawei-source/build.gradle.kts`. + ## Sentry monitoring -To enable Sentry monitoring for the generic REST, Fitbit, or Oura source connector service: +To enable Sentry monitoring for the generic REST, Fitbit, Oura, or Huawei source connector service: 1. Set a `SENTRY_DSN` environment variable that points to the desired Sentry DSN. 2. (Optional) Set the `SENTRY_LOG_LEVEL` environment variable to control the minimum log level of diff --git a/docker-compose.yml b/docker-compose.yml index 53c62257..6243286b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -4,6 +4,7 @@ version: "2.4" volumes: fitbit-logs: {} oura-logs: {} + huawei-logs: {} services: #---------------------------------------------------------------------------# @@ -231,3 +232,49 @@ services: # SENTRY_DSN: 'https://000000000000.ingest.de.sentry.io/000000000000' # SENTRY_ATTACHSTACKTRACE: true # SENTRY_STACKTRACE_APP_PACKAGES: io.confluent.connect,org.radarbase.connect.rest + + #---------------------------------------------------------------------------# + # RADAR Huawei connector # + #---------------------------------------------------------------------------# + radar-huawei-connector: + build: + context: . + dockerfile: ./kafka-connect-huawei-source/Dockerfile + image: radarbase/radar-connect-huawei-source + restart: on-failure + volumes: + - ./docker/source-huawei.properties:/etc/kafka-connect/source-huawei.properties + - ./docker/users:/var/lib/kafka-connect-huawei-source/users + - huawei-logs:/var/lib/kafka-connect-huawei-source/logs + depends_on: + - zookeeper-1 + - zookeeper-2 + - zookeeper-3 + - kafka-1 + - kafka-2 + - kafka-3 + - schema-registry-1 + environment: + CONNECT_BOOTSTRAP_SERVERS: PLAINTEXT://kafka-1:9092,PLAINTEXT://kafka-2:9092,PLAINTEXT://kafka-3:9092 + CONNECT_REST_PORT: 8083 + CONNECT_GROUP_ID: "default" + CONNECT_CONFIG_STORAGE_TOPIC: "default.config" + CONNECT_OFFSET_STORAGE_TOPIC: "default.offsets" + CONNECT_STATUS_STORAGE_TOPIC: "default.status" + CONNECT_KEY_CONVERTER: "io.confluent.connect.avro.AvroConverter" + CONNECT_VALUE_CONVERTER: "io.confluent.connect.avro.AvroConverter" + CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry-1:8081" + CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry-1:8081" + CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter" + CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter" + CONNECT_OFFSET_STORAGE_FILE_FILENAME: "/var/lib/kafka-connect-huawei-source/logs/connect.offsets" + CONNECT_REST_ADVERTISED_HOST_NAME: "radar-huawei-connector" + CONNECT_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181 + CONNECTOR_PROPERTY_FILE_PREFIX: "source-huawei" + KAFKA_HEAP_OPTS: "-Xms256m -Xmx768m" + KAFKA_BROKERS: 3 + CONNECT_LOG4J_LOGGERS: "org.reflections=ERROR" + # SENTRY_LOG_LEVEL: 'ERROR' + # SENTRY_DSN: 'https://000000000000.ingest.de.sentry.io/000000000000' + # SENTRY_ATTACHSTACKTRACE: true + # SENTRY_STACKTRACE_APP_PACKAGES: io.confluent.connect,org.radarbase.connect.rest diff --git a/docker/source-huawei.properties.template b/docker/source-huawei.properties.template new file mode 100644 index 00000000..1dd5b63e --- /dev/null +++ b/docker/source-huawei.properties.template @@ -0,0 +1,12 @@ +name=radar-huawei-source +connector.class=org.radarbase.connect.rest.huawei.HuaweiSourceConnector +tasks.max=4 +rest.source.base.url=https://health-api.cloud.huawei.com/healthkit/v1 +rest.source.poll.interval.ms=5000 +huawei.api.client=? +huawei.api.secret=? +huawei.user.repository.class=org.radarbase.connect.rest.huawei.user.HuaweiServiceUserRepository +huawei.user.repository.url=http://localhost:8080/ +huawei.user.repository.client.id=radar_huawei_connector +huawei.user.repository.client.secret= +huawei.user.repository.oauth2.token.url= From 089a0912751fe39c4491883d4ed682b83ed88bbf Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 11:19:59 +0000 Subject: [PATCH 06/27] Add tests for huawei-library and the connector config huawei-library tests (verified passing in this sandbox against a locally published radar-schemas-commons 0.9.0-SNAPSHOT): - FieldValuesTest: typed-value array and flattened-object parsing. - HuaweiRouteFactoryTest: builds every one of the ~54 registered route definitions, feeds each a fixture payload shaped for its endpoint kind, and asserts the converter produces exactly one record on the expected topic without throwing - this caught two real bugs during development (a bad endTime field on HuaweiCgmBloodGlucose, and a private extension function shadowing HuaweiDataConverter's). kafka-connect-huawei-source gets a config test mirroring the existing *RestSourceConnectorConfigTest convention, plus checks that enabledTopics() reflects the shared HuaweiRouteFactory.definitions registry and honors per-type huawei..enabled overrides. Could not run this one in-sandbox (see prior commit: packages.confluent.io is blocked here for every kafka-connect-*-source module, pre-existing and unrelated to this change). Also switches huawei-library's JUnit integration from kotlin-test-junit (JUnit4) to kotlin-test-junit5, since oura-library's copy-pasted dependency block conflicts with the JUnit Platform the radar-kotlin Gradle plugin configures the test task with - previously unnoticed only because no *-library module had tests yet. --- gradle/libs.versions.toml | 1 + huawei-library/build.gradle | 5 +- .../huawei/converter/FieldValuesTest.kt | 47 +++++ .../huawei/route/HuaweiRouteFactoryTest.kt | 180 ++++++++++++++++++ kafka-connect-huawei-source/build.gradle.kts | 1 + .../HuaweiRestSourceConnectorConfigTest.kt | 63 ++++++ 6 files changed, 295 insertions(+), 2 deletions(-) create mode 100644 huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt create mode 100644 huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt create mode 100644 kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index bf8a325a..9d14af12 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -59,6 +59,7 @@ mockito-core = { module = "org.mockito:mockito-core", version.ref = "mockito" } wiremock = { module = "com.github.tomakehurst:wiremock", version.ref = "wiremock" } kotlin-test = { module = "org.jetbrains.kotlin:kotlin-test", version.ref = "kotlin" } kotlin-test-junit = { module = "org.jetbrains.kotlin:kotlin-test-junit", version.ref = "kotlin" } +kotlin-test-junit5 = { module = "org.jetbrains.kotlin:kotlin-test-junit5", version.ref = "kotlin" } [plugins] radar-root-project = { id = "org.radarbase.radar-root-project", version.ref = "radarCommons" } diff --git a/huawei-library/build.gradle b/huawei-library/build.gradle index 882ad1e3..d35e82e9 100644 --- a/huawei-library/build.gradle +++ b/huawei-library/build.gradle @@ -51,8 +51,9 @@ dependencies { // Use the Kotlin test library. testImplementation libs.kotlin.test - // Use the Kotlin JUnit integration. - testImplementation libs.kotlin.test.junit + // Use the Kotlin JUnit 5 integration (matches the JUnit Platform the radar-kotlin + // Gradle plugin configures the `test` task with). + testImplementation libs.kotlin.test.junit5 } project.afterEvaluate { diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt new file mode 100644 index 00000000..cdca893f --- /dev/null +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt @@ -0,0 +1,47 @@ +package org.radarbase.huawei.converter + +import com.fasterxml.jackson.databind.ObjectMapper +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertNull + +class FieldValuesTest { + private val mapper = ObjectMapper() + + @Test + fun `parses typed-value array shape`() { + val node = mapper.readTree( + """ + [ + {"fieldName": "steps", "integerValue": 123}, + {"fieldName": "distance", "floatValue": 4.5}, + {"fieldName": "note", "stringValue": "hello"} + ] + """.trimIndent(), + ) + val fields = FieldValues.from(node) + + assertEquals(123, fields.getInt("steps")) + assertEquals(4.5, fields.getDouble("distance")) + assertEquals("hello", fields.getString("note")) + assertNull(fields.getInt("missing")) + } + + @Test + fun `parses flattened object shape`() { + val node = mapper.readTree("""{"avg": 1.5, "max": 3, "min": null}""") + val fields = FieldValues.from(node) + + assertEquals(1.5, fields.getDouble("avg")) + assertEquals(3, fields.getInt("max")) + assertNull(fields.getInt("min")) + } + + @Test + fun `handles missing or null root node`() { + val fields = FieldValues.from(null) + + assertNull(fields.getInt("anything")) + assertNull(fields.getString("anything")) + } +} diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt new file mode 100644 index 00000000..b6437590 --- /dev/null +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -0,0 +1,180 @@ +package org.radarbase.huawei.route + +import com.fasterxml.jackson.databind.ObjectMapper +import com.fasterxml.jackson.databind.node.ArrayNode +import com.fasterxml.jackson.databind.node.ObjectNode +import org.apache.avro.Schema +import org.radarbase.huawei.user.HuaweiUser +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserRepository +import org.radarcns.connector.huawei.HuaweiHealthRecordDynamicBp +import java.time.Instant +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * Exercises every [HuaweiRouteFactory] definition end to end: builds the route, feeds a + * generously-populated fixture payload shaped like the endpoint it targets (`sampleSet:polymerize`, + * `healthRecords`, or `activityRecords`), and asserts the converter produces exactly one record on + * the definition's own topic without throwing. This is the main regression test against typos in + * the ~90 hand-written Huawei field-value key strings (and the Avro builder calls around them). + */ +class HuaweiRouteFactoryTest { + private val mapper = ObjectMapper() + private val fakeUser: User = HuaweiUser( + id = "u1", + createdAt = Instant.now(), + projectId = "p", + userId = "u", + humanReadableUserId = null, + sourceId = "s", + externalId = "ext", + isAuthorized = true, + startDate = Instant.parse("2024-01-01T00:00:00Z"), + ) + private val fakeUserRepository = object : UserRepository { + override fun get(key: String): User = fakeUser + override fun stream(): Sequence = sequenceOf(fakeUser) + override fun getAccessToken(user: User): String = "token" + } + + @Test + fun `definitions have unique keys and topics`() { + val keys = HuaweiRouteFactory.definitions.map { it.key } + val topics = HuaweiRouteFactory.definitions.map { it.defaultTopic } + + assertEquals(keys.size, keys.toSet().size, "duplicate route definition keys: $keys") + assertEquals(topics.size, topics.toSet().size, "duplicate route definition topics: $topics") + } + + @Test + fun `every definition converts a fixture payload without error`() { + val failures = mutableListOf() + + HuaweiRouteFactory.definitions.forEach { definition -> + val route = definition.build(fakeUserRepository, definition.defaultTopic) + try { + val payload = fixtureFor(route) + val records = route.converters.single().processRecords(payload, fakeUser).toList() + val successes = records.mapNotNull { it.getOrNull() } + + if (successes.size != 1) { + failures += "${definition.key}: expected 1 record, got ${successes.size} " + + "(errors: ${records.mapNotNull { it.exceptionOrNull() }})" + } else if (successes.first().topic != definition.defaultTopic) { + failures += "${definition.key}: unexpected topic ${successes.first().topic}" + } + } catch (e: Exception) { + failures += "${definition.key}: threw ${e}" + } + } + + assertTrue(failures.isEmpty(), "Failures:\n" + failures.joinToString("\n")) + } + + private fun fixtureFor(route: HuaweiRoute) = when (route) { + is HuaweiActivityRecordRoute -> activityRecordFixture() + is HuaweiHealthRecordRoute -> healthRecordFixture() + is HuaweiSampleSetRoute -> sampleSetFixture() + else -> error("Unknown route type: ${route::class}") + } + + private fun sampleSetFixture(): ObjectNode { + val root = mapper.createObjectNode() + val sampleSet = root.putArray("sampleSet") + val group = sampleSet.addObject() + val samplePoints = group.putArray("samplePoints") + val point = samplePoints.addObject() + point.put("startTime", START_MILLIS) + point.put("endTime", END_MILLIS) + point.set("value", genericValueArray()) + return root + } + + private fun healthRecordFixture(): ObjectNode { + val root = mapper.createObjectNode() + val records = root.putArray("healthRecords") + val record = records.addObject() + record.put("startTime", START_MILLIS) + record.put("endTime", END_MILLIS) + record.set("value", genericValueArray()) + return root + } + + private fun activityRecordFixture(): ObjectNode { + val root = mapper.createObjectNode() + val records = root.putArray("activityRecords") + val record = records.addObject() + record.put("startTime", START_MILLIS) + record.put("endTime", END_MILLIS) + record.put("id", "activity-1") + record.put("name", "Run") + record.put("description", "Morning run") + record.put("timeZone", "Europe/London") + record.put("activityType", "1") + record.put("activeTime", 1000L) + record.put("isKeepGoing", false) + val device = record.putObject("device") + device.put("manufacturer", "Huawei") + device.put("type", 1) + val summary = record.putObject("activitySummary") + summary.put("avgPace", 300.0) + summary.put("bestPace", 250.0) + summary.putObject("paceMap") + summary.putArray("dataSummary") + summary.putArray("sectionSummary") + return root + } + + /** One value entry per literal field-value key used across [HuaweiRouteFactory], plus every + * (snake-cased) field of [HuaweiHealthRecordDynamicBp] - covering both the ad hoc key names + * used for most data types and the mechanically-derived ones used for the 24h ABPM record. */ + private fun genericValueArray(): ArrayNode { + val array = mapper.createArrayNode() + (LITERAL_FIELD_KEYS + dynamicBpFieldKeys()).distinct().forEach { key -> + val entry = array.addObject() + entry.put("fieldName", key) + entry.put("integerValue", 1) + entry.put("floatValue", 1.5) + entry.put("stringValue", "test") + } + return array + } + + private fun dynamicBpFieldKeys(): List = + (HuaweiHealthRecordDynamicBp::class.java.getField("SCHEMA$").get(null) as Schema).fields + .map { it.name() } + .filterNot { it in setOf("time", "timeReceived", "endTime") } + .map(::snake) + + private fun snake(name: String): String = + Regex("([a-z0-9])([A-Z])").replace(name) { "${it.groupValues[1]}_${it.groupValues[2]}" }.lowercase() + + companion object { + private const val START_MILLIS = 1704067200000L // 2024-01-01T00:00:00Z + private const val END_MILLIS = 1704070800000L // 2024-01-01T01:00:00Z + + private val LITERAL_FIELD_KEYS = listOf( + "active_hours", "active_hours_target", "all_sleep_time", "arrhythmia_result", + "arrhythmia_type", "ascent_total", "avg", "avg_breathe_rate", "avg_heart_rate", + "awake_time", "calories", "calories_target", "calories_total", "correlate_mealtime", + "correlate_sleep", "count", "deep_sleep_part", "deep_sleep_time", "descent_total", + "diastolic_pressure_avg", "diastolic_pressure_max", "diastolic_pressure_min", + "distance_delta", "distance_total", "dream_time", "duration", "emotion", "event_name", + "exercise_time", "exercise_time_target", "exercise_type", "extend_data", + "fall_asleep_time", "go_bed_time", "heart_rate_variability_rmssd", + "high_body_temperature_alarm", "last", "level", "light_sleep_time", "max", + "max_breathe_rate", "max_breathrate_baseline", "max_spo2", "meal", "min", + "min_breathe_rate", "min_breathrate_baseline", "min_spo2", "off_bed_time", + "on_off_bed_state", "predicted_calories", "prepare_sleep_time", "record_day", + "record_id", "remarks", "sample_source", "sampling_frequency", "sleep_efficiency", + "sleep_latency", "sleep_score", "sleep_state", "sleep_type", "sphygmus_avg", + "sphygmus_last", "sphygmus_max", "sphygmus_min", "status", "steps", "steps_delta", + "steps_target", "sub_status", "systolic_pressure_avg", "systolic_pressure_max", + "systolic_pressure_min", "threshold", "timezone", "total_calories", "type", + "user_symptom", "value", "vo2max", "voltage_data", "wakeup_count", "wakeup_time", + "zone1_duration", "zone2_duration", "zone3_duration", "zone4_duration", "zone5_duration", + ) + } +} diff --git a/kafka-connect-huawei-source/build.gradle.kts b/kafka-connect-huawei-source/build.gradle.kts index 05fcfa3c..3bc0b307 100644 --- a/kafka-connect-huawei-source/build.gradle.kts +++ b/kafka-connect-huawei-source/build.gradle.kts @@ -58,4 +58,5 @@ dependencies { testImplementation(libs.kafka.connect.api) testImplementation(libs.wiremock) testImplementation(libs.mockito.core) + testImplementation(libs.kotlin.test) } diff --git a/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt b/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt new file mode 100644 index 00000000..dd7ae19a --- /dev/null +++ b/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt @@ -0,0 +1,63 @@ +/* + * Copyright 2018 The Hyve + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.connect.rest.huawei + +import org.junit.jupiter.api.Test +import org.radarbase.huawei.route.HuaweiRouteFactory +import kotlin.test.assertEquals + +class HuaweiRestSourceConnectorConfigTest { + + @Test + fun conf() { + println(HuaweiRestSourceConnectorConfig.conf().toHtmlTable()) + } + + @Test + fun `enabled topics default to every registered data type`() { + val config = HuaweiRestSourceConnectorConfig( + mutableMapOf( + "huawei.api.client" to "client", + "huawei.api.secret" to "secret", + ), + false, + ) + + val enabled = config.enabledTopics() + + assertEquals(HuaweiRouteFactory.definitions.size, enabled.size) + HuaweiRouteFactory.definitions.forEach { definition -> + assertEquals(definition.defaultTopic, enabled[definition.key]) + } + } + + @Test + fun `a data type can be disabled via config`() { + val definition = HuaweiRouteFactory.definitions.first() + val config = HuaweiRestSourceConnectorConfig( + mutableMapOf( + "huawei.api.client" to "client", + "huawei.api.secret" to "secret", + "huawei.${definition.key}.enabled" to "false", + ), + false, + ) + + assertEquals(null, config.enabledTopics()[definition.key]) + } +} From 9267bc83f7bf2e2934fc66cb272459fef1d56920 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 11:22:14 +0000 Subject: [PATCH 07/27] Fix ktlint style violations in huawei-library `./gradlew check` runs ktlint over every module; auto-formatted the mechanical issues (semicolons, wrapping) and manually split a handful of lines ktlint couldn't auto-correct. huawei-library:check is now fully green (ktlint + all 5 tests) in this sandbox. --- .../radarbase/huawei/converter/FieldValues.kt | 11 +- .../converter/HuaweiSampleSetConverter.kt | 3 +- .../huawei/request/HuaweiRequestGenerator.kt | 37 +- .../radarbase/huawei/request/HuaweiResult.kt | 62 +- .../huawei/route/HuaweiActivityRecordRoute.kt | 3 +- .../org/radarbase/huawei/route/HuaweiRoute.kt | 28 +- .../huawei/route/HuaweiRouteFactory.kt | 1129 ++++++++++++----- .../huawei/route/HuaweiSampleSetRoute.kt | 6 +- .../huawei/route/HuaweiRouteFactoryTest.kt | 9 +- 9 files changed, 882 insertions(+), 406 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt index 57386986..f0db5d6a 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt @@ -22,15 +22,20 @@ class FieldValues private constructor(private val values: Map) fun getLong(field: String): Long? = values[field]?.let { if (it.isNull) null else it.asLong() } - fun getDouble(field: String): Double? = values[field]?.let { if (it.isNull) null else it.asDouble() } + fun getDouble(field: String): Double? = values[field]?.let { + if (it.isNull) null else it.asDouble() + } fun getFloat(field: String): Float? = getDouble(field)?.toFloat() - fun getString(field: String): String? = values[field]?.let { if (it.isNull) null else it.asText() } + fun getString(field: String): String? = values[field]?.let { + if (it.isNull) null else it.asText() + } companion object { private const val FIELD_NAME_KEY = "fieldName" - private val VALUE_KEYS = listOf("integerValue", "floatValue", "longValue", "stringValue", "value") + private val VALUE_KEYS = + listOf("integerValue", "floatValue", "longValue", "stringValue", "value") fun from(node: JsonNode?): FieldValues { if (node == null || node.isMissingNode || node.isNull) { diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt index 6e2929f4..e5c6d566 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt @@ -35,7 +35,8 @@ class HuaweiSampleSetConverter( val sampleSets = root.get("sampleSet") ?: root.get("sampleSets") ?: return emptySequence() return sampleSets.asSequence() .flatMap { group -> - (group.get("samplePoints") ?: group.get("samplePoint"))?.asSequence() ?: emptySequence() + val points = group.get("samplePoints") ?: group.get("samplePoint") + points?.asSequence() ?: emptySequence() } .mapCatching { point -> val startTime = point.epochInstant("startTime") diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt index b2d7da11..2a733024 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt @@ -106,7 +106,13 @@ class HuaweiRequestGenerator( logger.debug("Request successful: {}..", request.request) val body = response.body val data = body?.bytes() ?: ByteArray(0) - val records = request.route.converters.flatMap { it.convert(request, response.headers, data) } + val records = request.route.converters.flatMap { + it.convert( + request, + response.headers, + data, + ) + } val offset = records.maxByOrNull { it.offset }?.offset val key = routeKey(request.route, request.user) if (offset != null) { @@ -130,7 +136,10 @@ class HuaweiRequestGenerator( HuaweiRateLimitError("Rate limit reached..", TooManyRequestsException(), "429") } 403 -> { - logger.warn("User {} does not have access to this Huawei Health Kit data type.", request.user) + logger.warn( + "User {} does not have access to this Huawei Health Kit data type.", + request.user, + ) routeNextRequest[key] = Instant.now().plus(USER_BACK_OFF_TIME) HuaweiAccessForbiddenError( "Huawei Health Kit scope not granted or data not available..", @@ -150,22 +159,38 @@ class HuaweiRequestGenerator( 400 -> { logger.warn("Client exception for request {}", request) routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) - HuaweiClientException("Client unsupported or unauthorized..", IOException("Invalid client"), "400") + HuaweiClientException( + "Client unsupported or unauthorized..", + IOException("Invalid client"), + "400", + ) } 422 -> { logger.warn("Request failed (validation error): {}, {}", request, response) routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) - HuaweiValidationError(response.body?.string() ?: "validation error", IOException("Validation error"), "422") + HuaweiValidationError( + response.body?.string() ?: "validation error", + IOException("Validation error"), + "422", + ) } 404 -> { logger.warn("Not found: {}", request) routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) - HuaweiNotFoundError(response.body?.string() ?: "not found", IOException("Data not found"), "404") + HuaweiNotFoundError( + response.body?.string() ?: "not found", + IOException("Data not found"), + "404", + ) } else -> { logger.warn("Request failed: {}, {}", request, response) routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) - HuaweiGenericError(response.body?.string() ?: "unknown error", IOException("Unknown error"), "500") + HuaweiGenericError( + response.body?.string() ?: "unknown error", + IOException("Unknown error"), + "500", + ) } } } diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt index 8a4fa587..fba9e551 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt @@ -13,52 +13,44 @@ sealed class HuaweiErrorBase( val code: String, ) : HuaweiError -class HuaweiRateLimitError(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( - message, - cause, - code, -) +class HuaweiRateLimitError( + message: String, + cause: Exception? = null, + code: String, +) : HuaweiErrorBase(message, cause, code) -class HuaweiClientException(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( - message, - cause, - code, -) +class HuaweiClientException( + message: String, + cause: Exception? = null, + code: String, +) : HuaweiErrorBase(message, cause, code) class HuaweiUnauthorizedAccessError( message: String, cause: Exception? = null, code: String, -) : HuaweiErrorBase( - message, - cause, - code, -) +) : HuaweiErrorBase(message, cause, code) class HuaweiAccessForbiddenError( message: String, cause: Exception? = null, code: String, -) : HuaweiErrorBase( - message, - cause, - code, -) +) : HuaweiErrorBase(message, cause, code) -class HuaweiValidationError(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( - message, - cause, - code, -) +class HuaweiValidationError( + message: String, + cause: Exception? = null, + code: String, +) : HuaweiErrorBase(message, cause, code) -class HuaweiGenericError(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( - message, - cause, - code, -) +class HuaweiGenericError( + message: String, + cause: Exception? = null, + code: String, +) : HuaweiErrorBase(message, cause, code) -class HuaweiNotFoundError(message: String, cause: Exception? = null, code: String) : HuaweiErrorBase( - message, - cause, - code, -) +class HuaweiNotFoundError( + message: String, + cause: Exception? = null, + code: String, +) : HuaweiErrorBase(message, cause, code) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt index 63318b23..07a4f586 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt @@ -18,7 +18,8 @@ class HuaweiActivityRecordRoute( maxIntervalPerRequest: Duration = Duration.ofDays(30L), ) : HuaweiRoute(userRepository, maxIntervalPerRequest) { - override val converters: List = listOf(HuaweiActivityRecordConverter(topic)) + override val converters: List = + listOf(HuaweiActivityRecordConverter(topic)) override fun toString(): String = "huawei_activity_record" diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt index 8fd5bb01..33a47dea 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt @@ -24,7 +24,11 @@ abstract class HuaweiRoute( ) : Route { abstract val converters: List - protected fun createGetRequest(user: User, path: String, queryParams: Map): Request { + protected fun createGetRequest( + user: User, + path: String, + queryParams: Map, + ): Request { val accessToken = userRepository.getAccessToken(user) val urlBuilder = "$HUAWEI_API_BASE_URL/$path".toHttpUrl().newBuilder() queryParams.forEach { (key, value) -> urlBuilder.addQueryParameter(key, value) } @@ -44,15 +48,27 @@ abstract class HuaweiRoute( .build() } - /** Split `[start, end)` into consecutive windows of at most [maxIntervalPerRequest], capped at [max] windows. */ - protected fun chunkedRanges(start: Instant, end: Instant, max: Int): Sequence> = + /** + * Split `[start, end)` into consecutive windows of at most [maxIntervalPerRequest], capped at + * [max] windows. + */ + protected fun chunkedRanges( + start: Instant, + end: Instant, + max: Int, + ): Sequence> = generateSequence(start) { it + maxIntervalPerRequest } .takeWhile { it < end } .take(max) - .map { rangeStart -> rangeStart to (rangeStart + maxIntervalPerRequest).coerceAtMost(end) } + .map { rangeStart -> + rangeStart to (rangeStart + maxIntervalPerRequest).coerceAtMost(end) + } - override fun generateRequests(user: User, start: Instant, end: Instant): Sequence = - generateRequests(user, start, end, Int.MAX_VALUE) + override fun generateRequests( + user: User, + start: Instant, + end: Instant, + ): Sequence = generateRequests(user, start, end, Int.MAX_VALUE) companion object { const val HUAWEI_API_BASE_URL = "https://health-api.cloud.huawei.com/healthkit/v1" diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index 2e1df2fa..5d7e2032 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -63,7 +63,9 @@ object HuaweiRouteFactory { /** Best-effort camelCase -> snake_case conversion for deriving a Huawei field key from an Avro field name. */ private fun snake(name: String): String = - SNAKE_CASE_BOUNDARY.replace(name) { "${it.groupValues[1]}_${it.groupValues[2]}" }.lowercase() + SNAKE_CASE_BOUNDARY.replace( + name, + ) { "${it.groupValues[1]}_${it.groupValues[2]}" }.lowercase() private val SNAKE_CASE_BOUNDARY = Regex("([a-z0-9])([A-Z])") @@ -85,333 +87,693 @@ object HuaweiRouteFactory { /** Data types that reuse the generic [HuaweiStatistics] schema: (config key, Huawei data type name, default topic). */ private val genericStatisticsTypes = listOf( - Triple("continuous_body_fat_rate_statistics", "continuous.body.fat.rate.statistics", "connect_huawei_continuous_body_fat_rate_statistics"), - Triple("continuous_body_temperature_rest_statistics", "continuous.body.temperature.rest.statistics", "connect_huawei_continuous_body_temperature_rest_statistics"), - Triple("continuous_body_temperature_statistics", "continuous.body.temperature.statistics", "connect_huawei_continuous_body_temperature_statistics"), - Triple("continuous_calories_bmr_statistics", "continuous.calories.bmr.statistics", "connect_huawei_continuous_calories_bmr_statistics"), - Triple("continuous_exercise_heart_rate_statistics", "continuous.exercise_heart_rate.statistics", "connect_huawei_continuous_exercise_heart_rate_statistics"), - Triple("continuous_heart_rate_statistics", "continuous.heart_rate.statistics", "connect_huawei_continuous_heart_rate_statistics"), - Triple("continuous_power_statistics", "continuous.power.statistics", "connect_huawei_continuous_power_statistics"), - Triple("continuous_skin_temperature_statistics", "continuous.skin.temperature.statistics", "connect_huawei_continuous_skin_temperature_statistics"), - Triple("continuous_speed_statistics", "continuous.speed.statistics", "connect_huawei_continuous_speed_statistics"), - Triple("continuous_steps_rate_statistics", "continuous.steps.rate.statistics", "connect_huawei_continuous_steps_rate_statistics"), - Triple("continuous_stroke_rate_statistics", "continuous.stroke_rate.statistics", "connect_huawei_continuous_stroke_rate_statistics"), - Triple("instantaneous_resting_heart_rate_statistics", "instantaneous.resting_heart_rate.statistics", "connect_huawei_instantaneous_resting_heart_rate_statistics"), - Triple("instantaneous_stress_statistics", "instantaneous.stress.statistics", "connect_huawei_instantaneous_stress_statistics"), + Triple( + "continuous_body_fat_rate_statistics", + "continuous.body.fat.rate.statistics", + "connect_huawei_continuous_body_fat_rate_statistics", + ), + Triple( + "continuous_body_temperature_rest_statistics", + "continuous.body.temperature.rest.statistics", + "connect_huawei_continuous_body_temperature_rest_statistics", + ), + Triple( + "continuous_body_temperature_statistics", + "continuous.body.temperature.statistics", + "connect_huawei_continuous_body_temperature_statistics", + ), + Triple( + "continuous_calories_bmr_statistics", + "continuous.calories.bmr.statistics", + "connect_huawei_continuous_calories_bmr_statistics", + ), + Triple( + "continuous_exercise_heart_rate_statistics", + "continuous.exercise_heart_rate.statistics", + "connect_huawei_continuous_exercise_heart_rate_statistics", + ), + Triple( + "continuous_heart_rate_statistics", + "continuous.heart_rate.statistics", + "connect_huawei_continuous_heart_rate_statistics", + ), + Triple( + "continuous_power_statistics", + "continuous.power.statistics", + "connect_huawei_continuous_power_statistics", + ), + Triple( + "continuous_skin_temperature_statistics", + "continuous.skin.temperature.statistics", + "connect_huawei_continuous_skin_temperature_statistics", + ), + Triple( + "continuous_speed_statistics", + "continuous.speed.statistics", + "connect_huawei_continuous_speed_statistics", + ), + Triple( + "continuous_steps_rate_statistics", + "continuous.steps.rate.statistics", + "connect_huawei_continuous_steps_rate_statistics", + ), + Triple( + "continuous_stroke_rate_statistics", + "continuous.stroke_rate.statistics", + "connect_huawei_continuous_stroke_rate_statistics", + ), + Triple( + "instantaneous_resting_heart_rate_statistics", + "instantaneous.resting_heart_rate.statistics", + "connect_huawei_instantaneous_resting_heart_rate_statistics", + ), + Triple( + "instantaneous_stress_statistics", + "instantaneous.stress.statistics", + "connect_huawei_instantaneous_stress_statistics", + ), Triple("vo2max_statistics", "vo2max.statistics", "connect_huawei_vo2max_statistics"), ) /** Full registry of Huawei Health Kit data types supported by this connector. */ val definitions: List = buildList { add( - HuaweiRouteDefinition("activity_record", "connect_huawei_activity_record") { repo, topic -> + HuaweiRouteDefinition( + "activity_record", + "connect_huawei_activity_record", + ) { repo, topic -> HuaweiActivityRecordRoute(repo, topic) }, ) // cgm_blood_glucose (+ .statistics variant) - add(sampleSetDefinition("cgm_blood_glucose", "cgm_blood_glucose", "connect_huawei_cgm_blood_glucose") { f, start, _, received -> - HuaweiCgmBloodGlucose.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch() - level = f.getDouble("level") - avg = f.getInt("avg"); max = f.getInt("max"); min = f.getInt("min"); last = f.getInt("last") - }.build() - }) - add(sampleSetDefinition("cgm_blood_glucose_statistics", "cgm_blood_glucose.statistics", "connect_huawei_cgm_blood_glucose_statistics") { f, start, _, received -> - HuaweiCgmBloodGlucose.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch() - level = f.getDouble("level") - avg = f.getInt("avg"); max = f.getInt("max"); min = f.getInt("min"); last = f.getInt("last") - }.build() - }) - - add(sampleSetDefinition("daily_activity_summary", "daily_activity_summary", "connect_huawei_daily_activity_summary") { f, start, end, received -> - HuaweiDailyActivitySummary.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - steps = f.getInt("steps") - activeCalories = f.getInt("calories") - exerciseTime = f.getInt("exercise_time") - activeHours = f.getInt("active_hours") - stepsGoal = f.getInt("steps_target") - activeCaloriesGoal = f.getInt("calories_target") - exerciseTimeGoal = f.getInt("exercise_time_target") - activeHoursGoal = f.getInt("active_hours_target") - }.build() - }) - - add(sampleSetDefinition("active_hours", "active_hours", "connect_huawei_active_hours") { f, start, end, received -> - f.toActiveHours(start, end, received) - }) - add(sampleSetDefinition("active_hours_statistics", "active_hours.statistics", "connect_huawei_active_hours_statistics") { f, start, end, received -> - f.toActiveHours(start, end, received) - }) - - add(sampleSetDefinition("continuous_activity_fragment", "continuous.activity.fragment", "connect_huawei_continuous_activity_fragment") { f, start, end, received -> - f.toContinuousActivityStatistics(start, end, received) - }) - add(sampleSetDefinition("continuous_activity_statistics", "continuous.activity.statistics", "connect_huawei_continuous_activity_statistics") { f, start, end, received -> - f.toContinuousActivityStatistics(start, end, received) - }) - - add(sampleSetDefinition("continuous_altitude_statistics", "continuous.altitude.statistics", "connect_huawei_continuous_altitude_statistics") { f, start, end, received -> - HuaweiContinuousAltitudeStatistics.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - avg = f.getDouble("avg"); max = f.getDouble("max"); min = f.getDouble("min") - ascentTotal = f.getDouble("ascent_total") - descentTotal = f.getDouble("descent_total") - }.build() - }) - - add(sampleSetDefinition("continuous_blood_glucose_statistics", "continuous.blood_glucose.statistics", "connect_huawei_continuous_blood_glucose_statistics") { f, start, end, received -> - HuaweiContinuousBloodGlucoseStatistics.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - avg = f.getDouble("avg"); max = f.getDouble("max"); min = f.getDouble("min") - correlationWithMealtime = f.getInt("correlate_mealtime") - meal = f.getInt("meal") - correlationWithSleepState = f.getInt("correlate_sleep") - sampleSource = f.getInt("sample_source") - }.build() - }) - - add(sampleSetDefinition("continuous_breathe_rate_statistics", "continuous.breathe_rate.statistics", "connect_huawei_continuous_breathe_rate_statistics") { f, start, end, received -> - HuaweiContinuousBreatheRateStatistics.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - maxBreatheRate = f.getInt("max_breathe_rate") - minBreatheRate = f.getInt("min_breathe_rate") - avgBreatheRate = f.getInt("avg_breathe_rate") - minBreathrateBaseline = f.getInt("min_breathrate_baseline") - maxBreathrateBaseline = f.getInt("max_breathrate_baseline") - }.build() - }) - - add(sampleSetDefinition("continuous_body_blood_pressure_statistics", "continuous.body.blood_pressure.statistics", "connect_huawei_continuous_body_blood_pressure_statistics") { f, start, end, received -> - HuaweiContinuousBodyBloodPressureStatistics.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - systolicPressureAvg = f.getDouble("systolic_pressure_avg") - systolicPressureMax = f.getDouble("systolic_pressure_max") - systolicPressureMin = f.getDouble("systolic_pressure_min") - diastolicPressureAvg = f.getDouble("diastolic_pressure_avg") - diastolicPressureMax = f.getDouble("diastolic_pressure_max") - diastolicPressureMin = f.getDouble("diastolic_pressure_min") - sphygmusAvg = f.getDouble("sphygmus_avg") - sphygmusMax = f.getDouble("sphygmus_max") - sphygmusMin = f.getDouble("sphygmus_min") - sphygmusLast = f.getDouble("sphygmus_last") - }.build() - }) + add( + sampleSetDefinition( + "cgm_blood_glucose", + "cgm_blood_glucose", + "connect_huawei_cgm_blood_glucose", + ) { f, start, _, received -> + HuaweiCgmBloodGlucose.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + level = f.getDouble("level") + avg = f.getInt("avg") + max = f.getInt("max") + min = f.getInt("min") + last = f.getInt("last") + }.build() + }, + ) + add( + sampleSetDefinition( + "cgm_blood_glucose_statistics", + "cgm_blood_glucose.statistics", + "connect_huawei_cgm_blood_glucose_statistics", + ) { f, start, _, received -> + HuaweiCgmBloodGlucose.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + level = f.getDouble("level") + avg = f.getInt("avg") + max = f.getInt("max") + min = f.getInt("min") + last = f.getInt("last") + }.build() + }, + ) + + add( + sampleSetDefinition( + "daily_activity_summary", + "daily_activity_summary", + "connect_huawei_daily_activity_summary", + ) { f, start, end, received -> + HuaweiDailyActivitySummary.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + steps = f.getInt("steps") + activeCalories = f.getInt("calories") + exerciseTime = f.getInt("exercise_time") + activeHours = f.getInt("active_hours") + stepsGoal = f.getInt("steps_target") + activeCaloriesGoal = f.getInt("calories_target") + exerciseTimeGoal = f.getInt("exercise_time_target") + activeHoursGoal = f.getInt("active_hours_target") + }.build() + }, + ) + + add( + sampleSetDefinition( + "active_hours", + "active_hours", + "connect_huawei_active_hours", + ) { f, start, end, received -> + f.toActiveHours(start, end, received) + }, + ) + add( + sampleSetDefinition( + "active_hours_statistics", + "active_hours.statistics", + "connect_huawei_active_hours_statistics", + ) { f, start, end, received -> + f.toActiveHours(start, end, received) + }, + ) + + add( + sampleSetDefinition( + "continuous_activity_fragment", + "continuous.activity.fragment", + "connect_huawei_continuous_activity_fragment", + ) { f, start, end, received -> + f.toContinuousActivityStatistics(start, end, received) + }, + ) + add( + sampleSetDefinition( + "continuous_activity_statistics", + "continuous.activity.statistics", + "connect_huawei_continuous_activity_statistics", + ) { f, start, end, received -> + f.toContinuousActivityStatistics(start, end, received) + }, + ) + + add( + sampleSetDefinition( + "continuous_altitude_statistics", + "continuous.altitude.statistics", + "connect_huawei_continuous_altitude_statistics", + ) { f, start, end, received -> + HuaweiContinuousAltitudeStatistics.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + avg = f.getDouble("avg") + max = f.getDouble("max") + min = f.getDouble("min") + ascentTotal = f.getDouble("ascent_total") + descentTotal = f.getDouble("descent_total") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_blood_glucose_statistics", + "continuous.blood_glucose.statistics", + "connect_huawei_continuous_blood_glucose_statistics", + ) { f, start, end, received -> + HuaweiContinuousBloodGlucoseStatistics.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + avg = f.getDouble("avg") + max = f.getDouble("max") + min = f.getDouble("min") + correlationWithMealtime = f.getInt("correlate_mealtime") + meal = f.getInt("meal") + correlationWithSleepState = f.getInt("correlate_sleep") + sampleSource = f.getInt("sample_source") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_breathe_rate_statistics", + "continuous.breathe_rate.statistics", + "connect_huawei_continuous_breathe_rate_statistics", + ) { f, start, end, received -> + HuaweiContinuousBreatheRateStatistics.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + maxBreatheRate = f.getInt("max_breathe_rate") + minBreatheRate = f.getInt("min_breathe_rate") + avgBreatheRate = f.getInt("avg_breathe_rate") + minBreathrateBaseline = f.getInt("min_breathrate_baseline") + maxBreathrateBaseline = f.getInt("max_breathrate_baseline") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_body_blood_pressure_statistics", + "continuous.body.blood_pressure.statistics", + "connect_huawei_continuous_body_blood_pressure_statistics", + ) { f, start, end, received -> + HuaweiContinuousBodyBloodPressureStatistics.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + systolicPressureAvg = f.getDouble("systolic_pressure_avg") + systolicPressureMax = f.getDouble("systolic_pressure_max") + systolicPressureMin = f.getDouble("systolic_pressure_min") + diastolicPressureAvg = f.getDouble("diastolic_pressure_avg") + diastolicPressureMax = f.getDouble("diastolic_pressure_max") + diastolicPressureMin = f.getDouble("diastolic_pressure_min") + sphygmusAvg = f.getDouble("sphygmus_avg") + sphygmusMax = f.getDouble("sphygmus_max") + sphygmusMin = f.getDouble("sphygmus_min") + sphygmusLast = f.getDouble("sphygmus_last") + }.build() + }, + ) genericStatisticsTypes.forEach { (key, dataType, topic) -> add( sampleSetDefinition(key, dataType, topic) { f, start, end, received -> - HuaweiStatistics.newBuilder().apply { populateCommon(start, end, received, f) }.build() + HuaweiStatistics.newBuilder().apply { + populateCommon( + start, + end, + received, + f, + ) + }.build() }, ) } - add(sampleSetDefinition("continuous_calories_burnt", "continuous.calories.burnt", "connect_huawei_continuous_calories_burnt") { f, start, end, received -> - HuaweiContinuousCaloriesBurnt.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - calories = f.getDouble("calories") - }.build() - }) - add(sampleSetDefinition("continuous_calories_consumed", "continuous.calories.consumed", "connect_huawei_continuous_calories_consumed") { f, start, end, received -> - HuaweiContinuousCaloriesBurnt.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - calories = f.getDouble("calories") - }.build() - }) - add(sampleSetDefinition("continuous_calories_burnt_total", "continuous.calories.burnt.total", "connect_huawei_continuous_calories_burnt_total") { f, start, end, received -> - HuaweiContinuousCaloriesBurntTotal.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - caloriesTotal = f.getDouble("calories_total") - }.build() - }) - - add(sampleSetDefinition("continuous_distance_delta", "continuous.distance.delta", "connect_huawei_continuous_distance_delta") { f, start, end, received -> - HuaweiContinuousDistanceDelta.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - distanceDelta = f.getDouble("distance_delta") - }.build() - }) - add(sampleSetDefinition("continuous_distance_total", "continuous.distance.total", "connect_huawei_continuous_distance_total") { f, start, end, received -> - HuaweiContinuousDistanceTotal.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - distance = f.getDouble("distance_total") - }.build() - }) - - add(sampleSetDefinition("continuous_ecg_detail", "continuous.ecg_detail", "connect_huawei_continuous_ecg_detail") { f, start, end, received -> - HuaweiContinuousEcgDetail.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - ecgRecordId = f.getString("record_id") - averageHeartRate = f.getInt("avg_heart_rate") - ecgArrhythmiaType = f.getInt("arrhythmia_type") - ecgArrhythmiaResult = f.getInt("arrhythmia_result") - userSymptom = f.getString("user_symptom") - samplingFrequency = f.getInt("sampling_frequency") - voltageData = f.getString("voltage_data") - }.build() - }) - - add(sampleSetDefinition("continuous_exercise_intensity", "continuous.exercise_intensity", "connect_huawei_continuous_exercise_intensity") { f, start, end, received -> - f.toContinuousExerciseIntensity(start, end, received) - }) - add(sampleSetDefinition("continuous_exercise_intensity_statistics", "continuous.exercise_intensity.statistics", "connect_huawei_continuous_exercise_intensity_statistics") { f, start, end, received -> - f.toContinuousExerciseIntensity(start, end, received) - }) - - add(sampleSetDefinition("continuous_exercise_intensity_v2", "continuous.exercise_intensity.v2", "connect_huawei_continuous_exercise_intensity_v2") { f, start, end, received -> - HuaweiContinuousExerciseIntensityV2.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - exerciseType = f.getInt("exercise_type") - }.build() - }) - add(sampleSetDefinition("continuous_exercise_intensity_v2_statistics", "continuous.exercise_intensity.v2.statistics", "connect_huawei_continuous_exercise_intensity_v2_statistics") { f, start, end, received -> - HuaweiContinuousExerciseIntensityV2Statistics.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - zone1Duration = f.getInt("zone1_duration") - zone2Duration = f.getInt("zone2_duration") - zone3Duration = f.getInt("zone3_duration") - zone4Duration = f.getInt("zone4_duration") - zone5Duration = f.getInt("zone5_duration") - }.build() - }) - - add(sampleSetDefinition("continuous_sleep_fragment", "continuous.sleep.fragment", "connect_huawei_continuous_sleep_fragment") { f, start, end, received -> - HuaweiContinuousSleepFragment.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - sleepState = f.getInt("sleep_state") - }.build() - }) - - add(sampleSetDefinition("continuous_spo2_statistics", "continuous.spo2.statistics", "connect_huawei_continuous_spo2_statistics") { f, start, end, received -> - HuaweiContinuousSpo2Statistics.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - saturationAvg = f.getDouble("avg") - saturationMax = f.getDouble("max") - saturationMin = f.getDouble("min") - saturationLast = f.getDouble("last") - }.build() - }) - - add(sampleSetDefinition("continuous_steps_delta", "continuous.steps.delta", "connect_huawei_continuous_steps_delta") { f, start, end, received -> - HuaweiContinuousStepsDelta.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - stepsDelta = f.getInt("steps_delta") - }.build() - }) - add(sampleSetDefinition("continuous_steps_total", "continuous.steps.total", "connect_huawei_continuous_steps_total") { f, start, end, received -> - HuaweiContinuousStepsTotal.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - steps = f.getInt("steps") - duration = f.getInt("duration") - }.build() - }) - - add(sampleSetDefinition("emotion", "emotion", "connect_huawei_emotion") { f, start, _, received -> - HuaweiEmotion.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch() - emotionStatus = f.getInt("emotion") - }.build() - }) - - add(healthRecordDefinition("health_record_dynamic_bp", "health.record.dynamic_bp", "connect_huawei_health_record_dynamic_bp") { f, start, end, received -> - f.toHealthRecordDynamicBp(start, end, received) - }) - add(healthRecordDefinition("health_record_bradycardia", "health.record.bradycardia", "connect_huawei_health_record_bradycardia") { f, start, end, received -> - f.toHealthRecordHeartRateAlert(start, end, received) - }) - add(healthRecordDefinition("health_record_tachycardia", "health.record.tachycardia", "connect_huawei_health_record_tachycardia") { f, start, end, received -> - f.toHealthRecordHeartRateAlert(start, end, received) - }) - add(healthRecordDefinition("health_record_hyperthermia", "health.record.hyperthermia", "connect_huawei_health_record_hyperthermia") { f, start, end, received -> - HuaweiHealthRecordHyperthermia.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - highBodyTemperatureAlarm = f.getFloat("high_body_temperature_alarm") - }.build() - }) - add(healthRecordDefinition("health_record_low_spo2_alert", "health.record.lowSpo2Alert", "connect_huawei_health_record_low_spo2_alert") { f, start, end, received -> - HuaweiHealthRecordLowSpo2Alert.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - threshold = f.getFloat("threshold") - maxSpO2 = f.getFloat("max_spo2") - minSpO2 = f.getFloat("min_spo2") - }.build() - }) - add(healthRecordDefinition("health_record_menstrual_cycle", "health.record.menstrual_cycle", "connect_huawei_health_record_menstrual_cycle") { f, start, end, received -> - HuaweiHealthRecordMenstrualCycle.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - recordday = f.getInt("record_day") - status = f.getInt("status") - substatus = f.getInt("sub_status") - remarks = f.getString("remarks") - timezone = f.getString("timezone") - }.build() - }) - add(healthRecordDefinition("health_record_sleep", "health.record.sleep", "connect_huawei_health_record_sleep") { f, start, end, received -> - HuaweiHealthRecordSleep.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - fallAsleepTime = f.getLong("fall_asleep_time") - wakeupTime = f.getLong("wakeup_time") - lightSleepTime = f.getInt("light_sleep_time") - deepSleepTime = f.getInt("deep_sleep_time") - dreamTime = f.getInt("dream_time") - awakeTime = f.getInt("awake_time") - allSleepTime = f.getInt("all_sleep_time") - wakeupCount = f.getInt("wakeup_count") - deepSleepPart = f.getInt("deep_sleep_part") - sleepScore = f.getInt("sleep_score") - sleepLatency = f.getInt("sleep_latency") - sleepEfficiency = f.getInt("sleep_efficiency") - goBedTime = f.getLong("go_bed_time") - sleepType = f.getInt("sleep_type") - prepareSleepTime = f.getLong("prepare_sleep_time") - offBedTime = f.getLong("off_bed_time") - }.build() - }) - - add(sampleSetDefinition("heart_rate_variability", "heart_rate_variability", "connect_huawei_heart_rate_variability") { f, start, _, received -> - HuaweiHeartRateVariability.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch() - heartRateVariabilityRmssd = f.getInt("heart_rate_variability_rmssd") - }.build() - }) - - add(sampleSetDefinition("resting_calories_statistics", "resting_calories.statistics", "connect_huawei_resting_calories_statistics") { f, start, end, received -> - HuaweiRestingCaloriesStatistics.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - predictedCalories = f.getFloat("predicted_calories") - totalCalories = f.getFloat("total_calories") - }.build() - }) - - add(sampleSetDefinition("sleep_on_off_bed_record", "sleep.on_off_bed_record", "connect_huawei_sleep_on_off_bed_record") { f, start, _, received -> - HuaweiSleepOnOffBedRecord.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch() - onOffBedState = f.getInt("on_off_bed_state") - }.build() - }) - - add(sampleSetDefinition("sleep_respiratory_detail", "sleep_respiratory_detail", "connect_huawei_sleep_respiratory_detail") { f, start, end, received -> - HuaweiSleepRespiratoryDetail.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - type = f.getInt("type") - value = f.getDouble("value") - }.build() - }) - add(sampleSetDefinition("sleep_respiratory_event", "sleep_respiratory_event", "connect_huawei_sleep_respiratory_event") { f, start, end, received -> - HuaweiSleepRespiratoryEvent.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() - eventname = f.getInt("event_name") - }.build() - }) - - add(sampleSetDefinition("vo2max", "vo2max", "connect_huawei_vo2max") { f, start, _, received -> - HuaweiVo2Max.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch() - vo2max = f.getInt("vo2max") - }.build() - }) + add( + sampleSetDefinition( + "continuous_calories_burnt", + "continuous.calories.burnt", + "connect_huawei_continuous_calories_burnt", + ) { f, start, end, received -> + HuaweiContinuousCaloriesBurnt.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + calories = f.getDouble("calories") + }.build() + }, + ) + add( + sampleSetDefinition( + "continuous_calories_consumed", + "continuous.calories.consumed", + "connect_huawei_continuous_calories_consumed", + ) { f, start, end, received -> + HuaweiContinuousCaloriesBurnt.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + calories = f.getDouble("calories") + }.build() + }, + ) + add( + sampleSetDefinition( + "continuous_calories_burnt_total", + "continuous.calories.burnt.total", + "connect_huawei_continuous_calories_burnt_total", + ) { f, start, end, received -> + HuaweiContinuousCaloriesBurntTotal.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + caloriesTotal = f.getDouble("calories_total") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_distance_delta", + "continuous.distance.delta", + "connect_huawei_continuous_distance_delta", + ) { f, start, end, received -> + HuaweiContinuousDistanceDelta.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + distanceDelta = f.getDouble("distance_delta") + }.build() + }, + ) + add( + sampleSetDefinition( + "continuous_distance_total", + "continuous.distance.total", + "connect_huawei_continuous_distance_total", + ) { f, start, end, received -> + HuaweiContinuousDistanceTotal.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + distance = f.getDouble("distance_total") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_ecg_detail", + "continuous.ecg_detail", + "connect_huawei_continuous_ecg_detail", + ) { f, start, end, received -> + HuaweiContinuousEcgDetail.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + ecgRecordId = f.getString("record_id") + averageHeartRate = f.getInt("avg_heart_rate") + ecgArrhythmiaType = f.getInt("arrhythmia_type") + ecgArrhythmiaResult = f.getInt("arrhythmia_result") + userSymptom = f.getString("user_symptom") + samplingFrequency = f.getInt("sampling_frequency") + voltageData = f.getString("voltage_data") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_exercise_intensity", + "continuous.exercise_intensity", + "connect_huawei_continuous_exercise_intensity", + ) { f, start, end, received -> + f.toContinuousExerciseIntensity(start, end, received) + }, + ) + add( + sampleSetDefinition( + "continuous_exercise_intensity_statistics", + "continuous.exercise_intensity.statistics", + "connect_huawei_continuous_exercise_intensity_statistics", + ) { f, start, end, received -> + f.toContinuousExerciseIntensity(start, end, received) + }, + ) + + add( + sampleSetDefinition( + "continuous_exercise_intensity_v2", + "continuous.exercise_intensity.v2", + "connect_huawei_continuous_exercise_intensity_v2", + ) { f, start, end, received -> + HuaweiContinuousExerciseIntensityV2.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + exerciseType = f.getInt("exercise_type") + }.build() + }, + ) + add( + sampleSetDefinition( + "continuous_exercise_intensity_v2_statistics", + "continuous.exercise_intensity.v2.statistics", + "connect_huawei_continuous_exercise_intensity_v2_statistics", + ) { f, start, end, received -> + HuaweiContinuousExerciseIntensityV2Statistics.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + zone1Duration = f.getInt("zone1_duration") + zone2Duration = f.getInt("zone2_duration") + zone3Duration = f.getInt("zone3_duration") + zone4Duration = f.getInt("zone4_duration") + zone5Duration = f.getInt("zone5_duration") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_sleep_fragment", + "continuous.sleep.fragment", + "connect_huawei_continuous_sleep_fragment", + ) { f, start, end, received -> + HuaweiContinuousSleepFragment.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + sleepState = f.getInt("sleep_state") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_spo2_statistics", + "continuous.spo2.statistics", + "connect_huawei_continuous_spo2_statistics", + ) { f, start, end, received -> + HuaweiContinuousSpo2Statistics.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + saturationAvg = f.getDouble("avg") + saturationMax = f.getDouble("max") + saturationMin = f.getDouble("min") + saturationLast = f.getDouble("last") + }.build() + }, + ) + + add( + sampleSetDefinition( + "continuous_steps_delta", + "continuous.steps.delta", + "connect_huawei_continuous_steps_delta", + ) { f, start, end, received -> + HuaweiContinuousStepsDelta.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + stepsDelta = f.getInt("steps_delta") + }.build() + }, + ) + add( + sampleSetDefinition( + "continuous_steps_total", + "continuous.steps.total", + "connect_huawei_continuous_steps_total", + ) { f, start, end, received -> + HuaweiContinuousStepsTotal.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + steps = f.getInt("steps") + duration = f.getInt("duration") + }.build() + }, + ) + + add( + sampleSetDefinition( + "emotion", + "emotion", + "connect_huawei_emotion", + ) { f, start, _, received -> + HuaweiEmotion.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + emotionStatus = f.getInt("emotion") + }.build() + }, + ) + + add( + healthRecordDefinition( + "health_record_dynamic_bp", + "health.record.dynamic_bp", + "connect_huawei_health_record_dynamic_bp", + ) { f, start, end, received -> + f.toHealthRecordDynamicBp(start, end, received) + }, + ) + add( + healthRecordDefinition( + "health_record_bradycardia", + "health.record.bradycardia", + "connect_huawei_health_record_bradycardia", + ) { f, start, end, received -> + f.toHealthRecordHeartRateAlert(start, end, received) + }, + ) + add( + healthRecordDefinition( + "health_record_tachycardia", + "health.record.tachycardia", + "connect_huawei_health_record_tachycardia", + ) { f, start, end, received -> + f.toHealthRecordHeartRateAlert(start, end, received) + }, + ) + add( + healthRecordDefinition( + "health_record_hyperthermia", + "health.record.hyperthermia", + "connect_huawei_health_record_hyperthermia", + ) { f, start, end, received -> + HuaweiHealthRecordHyperthermia.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + highBodyTemperatureAlarm = f.getFloat("high_body_temperature_alarm") + }.build() + }, + ) + add( + healthRecordDefinition( + "health_record_low_spo2_alert", + "health.record.lowSpo2Alert", + "connect_huawei_health_record_low_spo2_alert", + ) { f, start, end, received -> + HuaweiHealthRecordLowSpo2Alert.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + threshold = f.getFloat("threshold") + maxSpO2 = f.getFloat("max_spo2") + minSpO2 = f.getFloat("min_spo2") + }.build() + }, + ) + add( + healthRecordDefinition( + "health_record_menstrual_cycle", + "health.record.menstrual_cycle", + "connect_huawei_health_record_menstrual_cycle", + ) { f, start, end, received -> + HuaweiHealthRecordMenstrualCycle.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + recordday = f.getInt("record_day") + status = f.getInt("status") + substatus = f.getInt("sub_status") + remarks = f.getString("remarks") + timezone = f.getString("timezone") + }.build() + }, + ) + add( + healthRecordDefinition( + "health_record_sleep", + "health.record.sleep", + "connect_huawei_health_record_sleep", + ) { f, start, end, received -> + HuaweiHealthRecordSleep.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + fallAsleepTime = f.getLong("fall_asleep_time") + wakeupTime = f.getLong("wakeup_time") + lightSleepTime = f.getInt("light_sleep_time") + deepSleepTime = f.getInt("deep_sleep_time") + dreamTime = f.getInt("dream_time") + awakeTime = f.getInt("awake_time") + allSleepTime = f.getInt("all_sleep_time") + wakeupCount = f.getInt("wakeup_count") + deepSleepPart = f.getInt("deep_sleep_part") + sleepScore = f.getInt("sleep_score") + sleepLatency = f.getInt("sleep_latency") + sleepEfficiency = f.getInt("sleep_efficiency") + goBedTime = f.getLong("go_bed_time") + sleepType = f.getInt("sleep_type") + prepareSleepTime = f.getLong("prepare_sleep_time") + offBedTime = f.getLong("off_bed_time") + }.build() + }, + ) + + add( + sampleSetDefinition( + "heart_rate_variability", + "heart_rate_variability", + "connect_huawei_heart_rate_variability", + ) { f, start, _, received -> + HuaweiHeartRateVariability.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + heartRateVariabilityRmssd = f.getInt("heart_rate_variability_rmssd") + }.build() + }, + ) + + add( + sampleSetDefinition( + "resting_calories_statistics", + "resting_calories.statistics", + "connect_huawei_resting_calories_statistics", + ) { f, start, end, received -> + HuaweiRestingCaloriesStatistics.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + predictedCalories = f.getFloat("predicted_calories") + totalCalories = f.getFloat("total_calories") + }.build() + }, + ) + + add( + sampleSetDefinition( + "sleep_on_off_bed_record", + "sleep.on_off_bed_record", + "connect_huawei_sleep_on_off_bed_record", + ) { f, start, _, received -> + HuaweiSleepOnOffBedRecord.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + onOffBedState = f.getInt("on_off_bed_state") + }.build() + }, + ) + + add( + sampleSetDefinition( + "sleep_respiratory_detail", + "sleep_respiratory_detail", + "connect_huawei_sleep_respiratory_detail", + ) { f, start, end, received -> + HuaweiSleepRespiratoryDetail.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + type = f.getInt("type") + value = f.getDouble("value") + }.build() + }, + ) + add( + sampleSetDefinition( + "sleep_respiratory_event", + "sleep_respiratory_event", + "connect_huawei_sleep_respiratory_event", + ) { f, start, end, received -> + HuaweiSleepRespiratoryEvent.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + eventname = f.getInt("event_name") + }.build() + }, + ) + + add( + sampleSetDefinition( + "vo2max", + "vo2max", + "connect_huawei_vo2max", + ) { f, start, _, received -> + HuaweiVo2Max.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + vo2max = f.getInt("vo2max") + }.build() + }, + ) } private fun FieldValues.toActiveHours( @@ -419,7 +781,9 @@ object HuaweiRouteFactory { end: Instant?, received: Instant, ): HuaweiActiveHours = HuaweiActiveHours.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() activeHours = getInt("active_hours") moderateIntensityMinutes = getInt("moderate_intensity_minutes") highIntensityMinutes = getInt("high_intensity_minutes") @@ -430,7 +794,9 @@ object HuaweiRouteFactory { end: Instant?, received: Instant, ): HuaweiContinuousActivityStatistics = HuaweiContinuousActivityStatistics.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() typeOfActivity = getInt("activity_type") span = getInt("span") fragments = getInt("fragments") @@ -441,7 +807,9 @@ object HuaweiRouteFactory { end: Instant?, received: Instant, ): HuaweiContinuousExerciseIntensity = HuaweiContinuousExerciseIntensity.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() intensity = getDouble("intensity") span = getInt("span") }.build() @@ -451,7 +819,9 @@ object HuaweiRouteFactory { end: Instant?, received: Instant, ): HuaweiHealthRecordHeartRateAlert = HuaweiHealthRecordHeartRateAlert.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() threshold = getDouble("threshold") avgHeartRate = getDouble("avg_heart_rate") maxHeartRate = getDouble("max_heart_rate") @@ -474,7 +844,9 @@ object HuaweiRouteFactory { fun d(name: String) = f.getDouble(snake(name)) fun l(name: String) = f.getLong(snake(name)) return HuaweiHealthRecordDynamicBp.newBuilder().apply { - time = start.toEpoch(); timeReceived = received.toEpoch(); endTime = end?.toEpoch() + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() planId = f.getString(snake("planId")) planStartTime = l("planStartTime") planEndTime = l("planEndTime") @@ -484,42 +856,99 @@ object HuaweiRouteFactory { sleepStartTime = l("sleepStartTime") sleepEndTime = l("sleepEndTime") - validCntAll = i("validCntAll"); cntAll = i("cntAll") - maxSystolicBpAll = i("maxSystolicBpAll"); maxDiastolicBpAll = i("maxDiastolicBpAll"); maxHeartRateAll = i("maxHeartRateAll") - midSystolicBpAll = i("midSystolicBpAll"); midDiastolicBpAll = i("midDiastolicBpAll"); midHeartRateAll = i("midHeartRateAll") - minSystolicBpAll = i("minSystolicBpAll"); minDiastolicBpAll = i("minDiastolicBpAll"); minHeartRateAll = i("minHeartRateAll") - avgSystolicBpAll = i("avgSystolicBpAll"); avgDiastolicBpAll = i("avgDiastolicBpAll"); avgHeartRateAll = i("avgHeartRateAll") - stdSystolicBpAll = i("stdSystolicBpAll"); stdDiastolicBpAll = i("stdDiastolicBpAll"); stdHeartRateAll = i("stdHeartRateAll") - coefSystolicBpAll = d("coefSystolicBpAll"); coefDiastolicBpAll = d("coefDiastolicBpAll"); coefHeartRateAll = d("coefHeartRateAll") - loadSystolicBpAll = d("loadSystolicBpAll"); loadDiastolicBpAll = d("loadDiastolicBpAll") - dropSystolicBpAll = d("dropSystolicBpAll"); dropDiastolicBpAll = d("dropDiastolicBpAll") - peakSystolicBpAll = i("peakSystolicBpAll"); peakDiastolicBpAll = i("peakDiastolicBpAll") - - validCntWake = i("validCntWake"); cntWake = i("cntWake") - maxSystolicBpWake = i("maxSystolicBpWake"); maxDiastolicBpWake = i("maxDiastolicBpWake"); maxHeartRateWake = i("maxHeartRateWake") - midSystolicBpWake = i("midSystolicBpWake"); midDiastolicBpWake = i("midDiastolicBpWake"); midHeartRateWake = i("midHeartRateWake") - minSystolicBpWake = i("minSystolicBpWake"); minDiastolicBpWake = i("minDiastolicBpWake"); minHeartRateWake = i("minHeartRateWake") - avgSystolicBpWake = i("avgSystolicBpWake"); avgDiastolicBpWake = i("avgDiastolicBpWake"); avgHeartRateWake = i("avgHeartRateWake") - stdSystolicBpWake = i("stdSystolicBpWake"); stdDiastolicBpWake = i("stdDiastolicBpWake"); stdHeartRateWake = i("stdHeartRateWake") - coefSystolicBpWake = d("coefSystolicBpWake"); coefDiastolicBpWake = d("coefDiastolicBpWake"); coefHeartRateWake = d("coefHeartRateWake") - loadSystolicBpWake = d("loadSystolicBpWake"); loadDiastolicBpWake = d("loadDiastolicBpWake") - - validCntSleep = i("validCntSleep"); cntSleep = i("cntSleep") - maxSystolicBpSleep = i("maxSystolicBpSleep"); maxDiastolicBpSleep = i("maxDiastolicBpSleep"); maxHeartRateSleep = i("maxHeartRateSleep") - midSystolicBpSleep = i("midSystolicBpSleep"); midDiastolicBpSleep = i("midDiastolicBpSleep"); midHeartRateSleep = i("midHeartRateSleep") - minSystolicBpSleep = i("minSystolicBpSleep"); minDiastolicBpSleep = i("minDiastolicBpSleep"); minHeartRateSleep = i("minHeartRateSleep") - avgSystolicBpSleep = i("avgSystolicBpSleep"); avgDiastolicBpSleep = i("avgDiastolicBpSleep"); avgHeartRateSleep = i("avgHeartRateSleep") - stdSystolicBpSleep = i("stdSystolicBpSleep"); stdDiastolicBpSleep = i("stdDiastolicBpSleep"); stdHeartRateSleep = i("stdHeartRateSleep") - coefSystolicBpSleep = d("coefSystolicBpSleep"); coefDiastolicBpSleep = d("coefDiastolicBpSleep"); coefHeartRateSleep = d("coefHeartRateSleep") - loadSystolicBpSleep = d("loadSystolicBpSleep"); loadDiastolicBpSleep = d("loadDiastolicBpSleep") - - validCntWakeTwo = i("validCntWakeTwo"); cntWakeTwo = i("cntWakeTwo") - maxSystolicBpWakeTwo = i("maxSystolicBpWakeTwo"); maxDiastolicBpWakeTwo = i("maxDiastolicBpWakeTwo"); maxHeartRateWakeTwo = i("maxHeartRateWakeTwo") - midSystolicBpWakeTwo = i("midSystolicBpWakeTwo"); midDiastolicBpWakeTwo = i("midDiastolicBpWakeTwo"); midHeartRateWakeTwo = i("midHeartRateWakeTwo") - minSystolicBpWakeTwo = i("minSystolicBpWakeTwo"); minDiastolicBpWakeTwo = i("minDiastolicBpWakeTwo"); minHeartRateWakeTwo = i("minHeartRateWakeTwo") - avgSystolicBpWakeTwo = i("avgSystolicBpWakeTwo"); avgDiastolicBpWakeTwo = i("avgDiastolicBpWakeTwo"); avgHeartRateWakeTwo = i("avgHeartRateWakeTwo") - stdSystolicBpWakeTwo = i("stdSystolicBpWakeTwo"); stdDiastolicBpWakeTwo = i("stdDiastolicBpWakeTwo"); stdHeartRateWakeTwo = i("stdHeartRateWakeTwo") - coefSystolicBpWakeTwo = d("coefSystolicBpWakeTwo"); coefDiastolicBpWakeTwo = d("coefDiastolicBpWakeTwo"); coefHeartRateWakeTwo = d("coefHeartRateWakeTwo") + validCntAll = i("validCntAll") + cntAll = i("cntAll") + maxSystolicBpAll = i("maxSystolicBpAll") + maxDiastolicBpAll = i("maxDiastolicBpAll") + maxHeartRateAll = i("maxHeartRateAll") + midSystolicBpAll = i("midSystolicBpAll") + midDiastolicBpAll = i("midDiastolicBpAll") + midHeartRateAll = i("midHeartRateAll") + minSystolicBpAll = i("minSystolicBpAll") + minDiastolicBpAll = i("minDiastolicBpAll") + minHeartRateAll = i("minHeartRateAll") + avgSystolicBpAll = i("avgSystolicBpAll") + avgDiastolicBpAll = i("avgDiastolicBpAll") + avgHeartRateAll = i("avgHeartRateAll") + stdSystolicBpAll = i("stdSystolicBpAll") + stdDiastolicBpAll = i("stdDiastolicBpAll") + stdHeartRateAll = i("stdHeartRateAll") + coefSystolicBpAll = d("coefSystolicBpAll") + coefDiastolicBpAll = d("coefDiastolicBpAll") + coefHeartRateAll = d("coefHeartRateAll") + loadSystolicBpAll = d("loadSystolicBpAll") + loadDiastolicBpAll = d("loadDiastolicBpAll") + dropSystolicBpAll = d("dropSystolicBpAll") + dropDiastolicBpAll = d("dropDiastolicBpAll") + peakSystolicBpAll = i("peakSystolicBpAll") + peakDiastolicBpAll = i("peakDiastolicBpAll") + + validCntWake = i("validCntWake") + cntWake = i("cntWake") + maxSystolicBpWake = i("maxSystolicBpWake") + maxDiastolicBpWake = i("maxDiastolicBpWake") + maxHeartRateWake = i("maxHeartRateWake") + midSystolicBpWake = i("midSystolicBpWake") + midDiastolicBpWake = i("midDiastolicBpWake") + midHeartRateWake = i("midHeartRateWake") + minSystolicBpWake = i("minSystolicBpWake") + minDiastolicBpWake = i("minDiastolicBpWake") + minHeartRateWake = i("minHeartRateWake") + avgSystolicBpWake = i("avgSystolicBpWake") + avgDiastolicBpWake = i("avgDiastolicBpWake") + avgHeartRateWake = i("avgHeartRateWake") + stdSystolicBpWake = i("stdSystolicBpWake") + stdDiastolicBpWake = i("stdDiastolicBpWake") + stdHeartRateWake = i("stdHeartRateWake") + coefSystolicBpWake = d("coefSystolicBpWake") + coefDiastolicBpWake = d("coefDiastolicBpWake") + coefHeartRateWake = d("coefHeartRateWake") + loadSystolicBpWake = d("loadSystolicBpWake") + loadDiastolicBpWake = d("loadDiastolicBpWake") + + validCntSleep = i("validCntSleep") + cntSleep = i("cntSleep") + maxSystolicBpSleep = i("maxSystolicBpSleep") + maxDiastolicBpSleep = i("maxDiastolicBpSleep") + maxHeartRateSleep = i("maxHeartRateSleep") + midSystolicBpSleep = i("midSystolicBpSleep") + midDiastolicBpSleep = i("midDiastolicBpSleep") + midHeartRateSleep = i("midHeartRateSleep") + minSystolicBpSleep = i("minSystolicBpSleep") + minDiastolicBpSleep = i("minDiastolicBpSleep") + minHeartRateSleep = i("minHeartRateSleep") + avgSystolicBpSleep = i("avgSystolicBpSleep") + avgDiastolicBpSleep = i("avgDiastolicBpSleep") + avgHeartRateSleep = i("avgHeartRateSleep") + stdSystolicBpSleep = i("stdSystolicBpSleep") + stdDiastolicBpSleep = i("stdDiastolicBpSleep") + stdHeartRateSleep = i("stdHeartRateSleep") + coefSystolicBpSleep = d("coefSystolicBpSleep") + coefDiastolicBpSleep = d("coefDiastolicBpSleep") + coefHeartRateSleep = d("coefHeartRateSleep") + loadSystolicBpSleep = d("loadSystolicBpSleep") + loadDiastolicBpSleep = d("loadDiastolicBpSleep") + + validCntWakeTwo = i("validCntWakeTwo") + cntWakeTwo = i("cntWakeTwo") + maxSystolicBpWakeTwo = i("maxSystolicBpWakeTwo") + maxDiastolicBpWakeTwo = i("maxDiastolicBpWakeTwo") + maxHeartRateWakeTwo = i("maxHeartRateWakeTwo") + midSystolicBpWakeTwo = i("midSystolicBpWakeTwo") + midDiastolicBpWakeTwo = i("midDiastolicBpWakeTwo") + midHeartRateWakeTwo = i("midHeartRateWakeTwo") + minSystolicBpWakeTwo = i("minSystolicBpWakeTwo") + minDiastolicBpWakeTwo = i("minDiastolicBpWakeTwo") + minHeartRateWakeTwo = i("minHeartRateWakeTwo") + avgSystolicBpWakeTwo = i("avgSystolicBpWakeTwo") + avgDiastolicBpWakeTwo = i("avgDiastolicBpWakeTwo") + avgHeartRateWakeTwo = i("avgHeartRateWakeTwo") + stdSystolicBpWakeTwo = i("stdSystolicBpWakeTwo") + stdDiastolicBpWakeTwo = i("stdDiastolicBpWakeTwo") + stdHeartRateWakeTwo = i("stdHeartRateWakeTwo") + coefSystolicBpWakeTwo = d("coefSystolicBpWakeTwo") + coefDiastolicBpWakeTwo = d("coefDiastolicBpWakeTwo") + coefHeartRateWakeTwo = d("coefHeartRateWakeTwo") extendData = f.getString("extend_data") }.build() diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt index ddea128f..91b207fa 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt @@ -47,7 +47,11 @@ open class HuaweiSampleSetRoute( max: Int, ): Sequence = chunkedRanges(start, end, max).map { (rangeStart, rangeEnd) -> RestRequest( - request = createPostRequest(user, "sampleSet:polymerize", buildRequestBody(rangeStart, rangeEnd)), + request = createPostRequest( + user, + "sampleSet:polymerize", + buildRequestBody(rangeStart, rangeEnd), + ), user = user, route = this, startDate = rangeStart, diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt index b6437590..a109ac9b 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -66,7 +66,7 @@ class HuaweiRouteFactoryTest { failures += "${definition.key}: unexpected topic ${successes.first().topic}" } } catch (e: Exception) { - failures += "${definition.key}: threw ${e}" + failures += "${definition.key}: threw $e" } } @@ -149,7 +149,9 @@ class HuaweiRouteFactoryTest { .map(::snake) private fun snake(name: String): String = - Regex("([a-z0-9])([A-Z])").replace(name) { "${it.groupValues[1]}_${it.groupValues[2]}" }.lowercase() + Regex("([a-z0-9])([A-Z])") + .replace(name) { "${it.groupValues[1]}_${it.groupValues[2]}" } + .lowercase() companion object { private const val START_MILLIS = 1704067200000L // 2024-01-01T00:00:00Z @@ -174,7 +176,8 @@ class HuaweiRouteFactoryTest { "steps_target", "sub_status", "systolic_pressure_avg", "systolic_pressure_max", "systolic_pressure_min", "threshold", "timezone", "total_calories", "type", "user_symptom", "value", "vo2max", "voltage_data", "wakeup_count", "wakeup_time", - "zone1_duration", "zone2_duration", "zone3_duration", "zone4_duration", "zone5_duration", + "zone1_duration", "zone2_duration", "zone3_duration", "zone4_duration", + "zone5_duration", ) } } From 9dac4847bbf37bafbc73f6ef78db871a59ccd1dd Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 11:24:00 +0000 Subject: [PATCH 08/27] Fix ktlint style violations in kafka-connect-huawei-source ktlint is static analysis and doesn't need dependency resolution to run, so this module's Kotlin sources could be linted even though they can't be compiled in this sandbox. Wraps long ConfigDef.define() calls and doc-string constants, and simplifies the getUserRepository()/ initialize() flow slightly in the process. --- .../huawei/HuaweiRestSourceConnectorConfig.kt | 158 ++++++++++++++---- .../user/HuaweiServiceUserRepository.kt | 10 +- 2 files changed, 130 insertions(+), 38 deletions(-) diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt index 8178a949..4adbd949 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt @@ -47,7 +47,11 @@ class HuaweiRestSourceConnectorConfig( doLog: Boolean, ) : AbstractConfig(config, parsedConfig, doLog) { - constructor(parsedConfig: MutableMap, doLog: Boolean) : this(conf(), parsedConfig, doLog) + constructor(parsedConfig: MutableMap, doLog: Boolean) : this( + conf(), + parsedConfig, + doLog, + ) private var userRepository: HuaweiUserRepository? = null @@ -58,7 +62,8 @@ class HuaweiRestSourceConnectorConfig( fun getHuaweiClientSecret(): String = getPassword(HUAWEI_API_SECRET_CONFIG).value() fun getUserRepository(reuse: HuaweiUserRepository?): HuaweiUserRepository { - val repo = if (reuse != null && reuse.javaClass == getClass(HUAWEI_USER_REPOSITORY_CONFIG)) { + val configuredClass = getClass(HUAWEI_USER_REPOSITORY_CONFIG) + val repo = if (reuse != null && reuse.javaClass == configuredClass) { reuse } else { createUserRepository() @@ -96,9 +101,13 @@ class HuaweiRestSourceConnectorConfig( ) } - fun getPollIntervalPerUser(): Duration = Duration.ofSeconds(getInt(HUAWEI_USER_POLL_INTERVAL_CONFIG).toLong()) + fun getPollIntervalPerUser(): Duration = Duration.ofSeconds( + getInt(HUAWEI_USER_POLL_INTERVAL_CONFIG).toLong(), + ) - fun getHuaweiUserRepositoryClientId(): String = getString(HUAWEI_USER_REPOSITORY_CLIENT_ID_CONFIG) + fun getHuaweiUserRepositoryClientId(): String = getString( + HUAWEI_USER_REPOSITORY_CLIENT_ID_CONFIG, + ) fun getHuaweiUserRepositoryClientSecret(): String = getPassword(HUAWEI_USER_REPOSITORY_CLIENT_SECRET_CONFIG).value() @@ -147,11 +156,13 @@ class HuaweiRestSourceConnectorConfig( private const val HUAWEI_API_CLIENT_DISPLAY = "Huawei API client ID" const val HUAWEI_API_SECRET_CONFIG = "huawei.api.secret" - private const val HUAWEI_API_SECRET_DOC = "Secret for the Huawei API client set in huawei.api.client." + private const val HUAWEI_API_SECRET_DOC = + "Secret for the Huawei API client set in huawei.api.client." private const val HUAWEI_API_SECRET_DISPLAY = "Huawei API client secret" const val HUAWEI_USER_REPOSITORY_CONFIG = "huawei.user.repository.class" - private const val HUAWEI_USER_REPOSITORY_DOC = "Class for managing users and authentication." + private const val HUAWEI_USER_REPOSITORY_DOC = + "Class for managing users and authentication." private const val HUAWEI_USER_REPOSITORY_DISPLAY = "User repository class" const val HUAWEI_USER_POLL_INTERVAL_CONFIG = "huawei.user.poll.interval" @@ -168,16 +179,22 @@ class HuaweiRestSourceConnectorConfig( private const val HUAWEI_USER_REPOSITORY_URL_DEFAULT = "" const val HUAWEI_USER_REPOSITORY_CLIENT_ID_CONFIG = "huawei.user.repository.client.id" - private const val HUAWEI_USER_REPOSITORY_CLIENT_ID_DOC = "Client ID for connecting to the service repository." - private const val HUAWEI_USER_REPOSITORY_CLIENT_ID_DISPLAY = "Client ID for user repository." + private const val HUAWEI_USER_REPOSITORY_CLIENT_ID_DOC = + "Client ID for connecting to the service repository." + private const val HUAWEI_USER_REPOSITORY_CLIENT_ID_DISPLAY = + "Client ID for user repository." - const val HUAWEI_USER_REPOSITORY_CLIENT_SECRET_CONFIG = "huawei.user.repository.client.secret" + const val HUAWEI_USER_REPOSITORY_CLIENT_SECRET_CONFIG = + "huawei.user.repository.client.secret" private const val HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DOC = "Client secret for connecting to the service repository." - private const val HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DISPLAY = "Client Secret for user repository." + private const val HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DISPLAY = + "Client Secret for user repository." - const val HUAWEI_USER_REPOSITORY_TOKEN_URL_CONFIG = "huawei.user.repository.oauth2.token.url" - private const val HUAWEI_USER_REPOSITORY_TOKEN_URL_DOC = "OAuth 2.0 token url for retrieving client credentials." + const val HUAWEI_USER_REPOSITORY_TOKEN_URL_CONFIG = + "huawei.user.repository.oauth2.token.url" + private const val HUAWEI_USER_REPOSITORY_TOKEN_URL_DOC = + "OAuth 2.0 token url for retrieving client credentials." private const val HUAWEI_USER_REPOSITORY_TOKEN_URL_DISPLAY = "OAuth 2.0 token URL." private fun enabledKey(key: String) = "huawei.$key.enabled" @@ -190,52 +207,125 @@ class HuaweiRestSourceConnectorConfig( val def = ConfigDef() .define( - SOURCE_POLL_INTERVAL_CONFIG, Type.LONG, SOURCE_POLL_INTERVAL_DEFAULT, Importance.LOW, - SOURCE_POLL_INTERVAL_DOC, group, ++order, Width.SHORT, SOURCE_POLL_INTERVAL_DISPLAY, + SOURCE_POLL_INTERVAL_CONFIG, + Type.LONG, + SOURCE_POLL_INTERVAL_DEFAULT, + Importance.LOW, + SOURCE_POLL_INTERVAL_DOC, + group, + ++order, + Width.SHORT, + SOURCE_POLL_INTERVAL_DISPLAY, ) .define( - SOURCE_URL_CONFIG, Type.STRING, SOURCE_URL_DEFAULT, Importance.HIGH, - SOURCE_URL_DOC, group, ++order, Width.SHORT, SOURCE_URL_DISPLAY, + SOURCE_URL_CONFIG, + Type.STRING, + SOURCE_URL_DEFAULT, + Importance.HIGH, + SOURCE_URL_DOC, + group, + ++order, + Width.SHORT, + SOURCE_URL_DISPLAY, ) .define( - HUAWEI_USERS_CONFIG, Type.LIST, emptyList(), Importance.HIGH, - HUAWEI_USERS_DOC, group, ++order, Width.SHORT, HUAWEI_USERS_DISPLAY, + HUAWEI_USERS_CONFIG, + Type.LIST, + emptyList(), + Importance.HIGH, + HUAWEI_USERS_DOC, + group, + ++order, + Width.SHORT, + HUAWEI_USERS_DISPLAY, ) .define( - HUAWEI_API_CLIENT_CONFIG, Type.STRING, ConfigDef.NO_DEFAULT_VALUE, NonEmptyString(), - Importance.HIGH, HUAWEI_API_CLIENT_DOC, group, ++order, Width.SHORT, HUAWEI_API_CLIENT_DISPLAY, + HUAWEI_API_CLIENT_CONFIG, + Type.STRING, + ConfigDef.NO_DEFAULT_VALUE, + NonEmptyString(), + Importance.HIGH, + HUAWEI_API_CLIENT_DOC, + group, + ++order, + Width.SHORT, + HUAWEI_API_CLIENT_DISPLAY, ) .define( - HUAWEI_API_SECRET_CONFIG, Type.PASSWORD, ConfigDef.NO_DEFAULT_VALUE, Importance.HIGH, - HUAWEI_API_SECRET_DOC, group, ++order, Width.SHORT, HUAWEI_API_SECRET_DISPLAY, + HUAWEI_API_SECRET_CONFIG, + Type.PASSWORD, + ConfigDef.NO_DEFAULT_VALUE, + Importance.HIGH, + HUAWEI_API_SECRET_DOC, + group, + ++order, + Width.SHORT, + HUAWEI_API_SECRET_DISPLAY, ) .define( - HUAWEI_USER_POLL_INTERVAL_CONFIG, Type.INT, HUAWEI_USER_POLL_INTERVAL_DEFAULT, Importance.MEDIUM, - HUAWEI_USER_POLL_INTERVAL_DOC, group, ++order, Width.SHORT, HUAWEI_USER_POLL_INTERVAL_DISPLAY, + HUAWEI_USER_POLL_INTERVAL_CONFIG, + Type.INT, + HUAWEI_USER_POLL_INTERVAL_DEFAULT, + Importance.MEDIUM, + HUAWEI_USER_POLL_INTERVAL_DOC, + group, + ++order, + Width.SHORT, + HUAWEI_USER_POLL_INTERVAL_DISPLAY, ) .define( - HUAWEI_USER_REPOSITORY_CONFIG, Type.CLASS, HuaweiServiceUserRepository::class.java, - Importance.MEDIUM, HUAWEI_USER_REPOSITORY_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_CONFIG, + Type.CLASS, + HuaweiServiceUserRepository::class.java, + Importance.MEDIUM, + HUAWEI_USER_REPOSITORY_DOC, + group, + ++order, + Width.SHORT, HUAWEI_USER_REPOSITORY_DISPLAY, ) .define( - HUAWEI_USER_REPOSITORY_URL_CONFIG, Type.STRING, HUAWEI_USER_REPOSITORY_URL_DEFAULT, - Importance.LOW, HUAWEI_USER_REPOSITORY_URL_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_URL_CONFIG, + Type.STRING, + HUAWEI_USER_REPOSITORY_URL_DEFAULT, + Importance.LOW, + HUAWEI_USER_REPOSITORY_URL_DOC, + group, + ++order, + Width.SHORT, HUAWEI_USER_REPOSITORY_URL_DISPLAY, ) .define( - HUAWEI_USER_REPOSITORY_CLIENT_ID_CONFIG, Type.STRING, "", Importance.MEDIUM, - HUAWEI_USER_REPOSITORY_CLIENT_ID_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_CLIENT_ID_CONFIG, + Type.STRING, + "", + Importance.MEDIUM, + HUAWEI_USER_REPOSITORY_CLIENT_ID_DOC, + group, + ++order, + Width.SHORT, HUAWEI_USER_REPOSITORY_CLIENT_ID_DISPLAY, ) .define( - HUAWEI_USER_REPOSITORY_CLIENT_SECRET_CONFIG, Type.PASSWORD, "", Importance.MEDIUM, - HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_CLIENT_SECRET_CONFIG, + Type.PASSWORD, + "", + Importance.MEDIUM, + HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DOC, + group, + ++order, + Width.SHORT, HUAWEI_USER_REPOSITORY_CLIENT_SECRET_DISPLAY, ) .define( - HUAWEI_USER_REPOSITORY_TOKEN_URL_CONFIG, Type.STRING, "", Importance.MEDIUM, - HUAWEI_USER_REPOSITORY_TOKEN_URL_DOC, group, ++order, Width.SHORT, + HUAWEI_USER_REPOSITORY_TOKEN_URL_CONFIG, + Type.STRING, + "", + Importance.MEDIUM, + HUAWEI_USER_REPOSITORY_TOKEN_URL_DOC, + group, + ++order, + Width.SHORT, HUAWEI_USER_REPOSITORY_TOKEN_URL_DISPLAY, ) diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt index 8bf64c2a..5bdfc642 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt @@ -50,14 +50,14 @@ import kotlinx.coroutines.runBlocking import kotlinx.coroutines.withContext import kotlinx.serialization.json.Json import org.radarbase.connect.rest.huawei.HuaweiRestSourceConnectorConfig +import org.radarbase.huawei.user.HuaweiUser +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserNotAuthorizedException import org.radarbase.kotlin.coroutines.CacheConfig import org.radarbase.kotlin.coroutines.CachedSet import org.radarbase.kotlin.coroutines.CachedValue import org.radarbase.ktor.auth.ClientCredentialsConfig import org.radarbase.ktor.auth.clientCredentials -import org.radarbase.huawei.user.HuaweiUser -import org.radarbase.huawei.user.User -import org.radarbase.huawei.user.UserNotAuthorizedException import org.slf4j.LoggerFactory import java.io.IOException import java.util.concurrent.ConcurrentHashMap @@ -90,11 +90,13 @@ class HuaweiServiceUserRepository : HuaweiUserRepository() { override fun initialize(config: HuaweiRestSourceConnectorConfig) { val containedUsers = config.getHuaweiUsers().toHashSet() + val tokenUrl = config.getHuaweiUserRepositoryTokenUrl() + ?.let { URLBuilder(it.toString()).build() } client = createClient( baseUrl = config.getHuaweiUserRepositoryUrl(), - tokenUrl = config.getHuaweiUserRepositoryTokenUrl()?.let { URLBuilder(it.toString()).build() }, + tokenUrl = tokenUrl, clientId = config.getHuaweiUserRepositoryClientId(), clientSecret = config.getHuaweiUserRepositoryClientSecret(), scope = "SUBJECT.READ MEASUREMENT.CREATE", From a6ec537d92ee309a30d15e6c7950b25aa7bac4bc Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 13:14:21 +0000 Subject: [PATCH 09/27] Add Onsentia affiliation to Huawei connector author metadata Adds yatharthranjan@onsentia.com alongside the existing KCL address in the Huawei Docker image label and CI workflow author fields. --- .github/workflows/main.yml | 2 +- .github/workflows/release.yml | 4 ++-- kafka-connect-huawei-source/Dockerfile | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index e5397368..dde9bc90 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -25,7 +25,7 @@ env: },{ 'name': 'kafka-connect-huawei-source', 'build_file': 'kafka-connect-huawei-source/Dockerfile', - 'authors': 'Yatharth Ranjan ', + 'authors': 'Yatharth Ranjan , Yatharth Ranjan ', 'description': 'RADAR-base Huawei Health Kit connector application' }] diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 861df14e..6f43aa67 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -16,12 +16,12 @@ env: },{ 'name': 'kafka-connect-oura-source', 'build_file': 'kafka-connect-oura-source/Dockerfile', - 'authors': 'Pauline Conde , Yatharth Ranjan ', + 'authors': 'Pauline Conde , Yatharth Ranjan , Yatharth Ranjan ', 'description': 'RADAR-base Oura connector application' },{ 'name': 'kafka-connect-huawei-source', 'build_file': 'kafka-connect-huawei-source/Dockerfile', - 'authors': 'Yatharth Ranjan ', + 'authors': 'Yatharth Ranjan , Yatharth Ranjan ', 'description': 'RADAR-base Huawei Health Kit connector application' }] diff --git a/kafka-connect-huawei-source/Dockerfile b/kafka-connect-huawei-source/Dockerfile index e01bcb7f..330759cf 100644 --- a/kafka-connect-huawei-source/Dockerfile +++ b/kafka-connect-huawei-source/Dockerfile @@ -36,7 +36,7 @@ FROM confluentinc/cp-kafka-connect-base:7.8.7 USER appuser -LABEL org.opencontainers.image.authors="yatharth.ranjan@kcl.ac.uk" +LABEL org.opencontainers.image.authors="yatharth.ranjan@kcl.ac.uk, yatharthranjan@onsentia.com" LABEL description="Kafka Huawei Health Kit REST API Source connector" From 2af0bb943980cb0a0471a2d3dbbf2636517fc52d Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 13:45:40 +0000 Subject: [PATCH 10/27] Fix CI type errors in HuaweiServiceUserRepository - Wrap the okhttp3.HttpUrl from getHuaweiUserRepositoryUrl() in URLBuilder(...).build() before passing it as createClient's Ktor Url parameter, matching how the token URL was already handled (and how OuraServiceUserRepository does it). - Throwable.message is nullable; fall back to a default string before passing it to UserNotAuthorizedException's non-null constructor. These only surfaced in real CI since packages.confluent.io is blocked in this sandbox, so kafka-connect-huawei-source couldn't be compiled here - verified via ktlint (which needs no dependency resolution) that the fix introduces no new style issues. --- .../connect/rest/huawei/user/HuaweiServiceUserRepository.kt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt index 5bdfc642..ae86aea6 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt @@ -95,7 +95,7 @@ class HuaweiServiceUserRepository : HuaweiUserRepository() { client = createClient( - baseUrl = config.getHuaweiUserRepositoryUrl(), + baseUrl = URLBuilder(config.getHuaweiUserRepositoryUrl().toString()).build(), tokenUrl = tokenUrl, clientId = config.getHuaweiUserRepositoryClientId(), clientSecret = config.getHuaweiUserRepositoryClientSecret(), @@ -237,7 +237,7 @@ class HuaweiServiceUserRepository : HuaweiUserRepository() { } catch (ex: HttpResponseException) { if (ex.statusCode == 407) { credentialCaches -= user.id - throw UserNotAuthorizedException(ex.message) + throw UserNotAuthorizedException(ex.message ?: "User is not authorized") } throw ex } From 59cdbc56a51dac4b0781015b9184c01dad35ea52 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 14 Jul 2026 14:05:36 +0000 Subject: [PATCH 11/27] Fix ktlint line-length violation in build.gradle.kts ktlintKotlinScriptCheck also lints .gradle.kts files; wraps the two GitHub Packages credential lines that exceeded 100 chars. --- kafka-connect-huawei-source/build.gradle.kts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kafka-connect-huawei-source/build.gradle.kts b/kafka-connect-huawei-source/build.gradle.kts index 3bc0b307..d2e0a528 100644 --- a/kafka-connect-huawei-source/build.gradle.kts +++ b/kafka-connect-huawei-source/build.gradle.kts @@ -18,8 +18,10 @@ repositories { maven { url = uri("https://maven.pkg.github.com/RADAR-base/RADAR-Schemas") credentials { - username = project.findProperty("public.gpr.user") as String? ?: System.getenv("GPR_USER") - password = project.findProperty("public.gpr.token") as String? ?: System.getenv("GPR_TOKEN") + username = project.findProperty("public.gpr.user") as String? + ?: System.getenv("GPR_USER") + password = project.findProperty("public.gpr.token") as String? + ?: System.getenv("GPR_TOKEN") } } } From d7dbf15937aeacbe9433c9d713a68a2ac908d2d5 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Thu, 23 Jul 2026 15:19:49 +0000 Subject: [PATCH 12/27] Add Onsentia copyright header to Huawei connector source files Applies a standard Apache-2.0 copyright header (Copyright 2026 Onsentia) to every .kt/.java file in huawei-library and kafka-connect-huawei-source - the two modules added for the Huawei Health Kit integration. Files that had copied The Hyve's 2018 header from the Oura pattern get it replaced; files with no header get one prepended. Pre-existing Fitbit/Oura files are left untouched since their copyright belongs to their original authors. --- .../radarbase/huawei/converter/FieldValues.kt | 17 +++++++++++++++++ .../converter/HuaweiActivityRecordConverter.kt | 17 +++++++++++++++++ .../huawei/converter/HuaweiDataConverter.kt | 17 +++++++++++++++++ .../converter/HuaweiHealthRecordConverter.kt | 17 +++++++++++++++++ .../converter/HuaweiSampleSetConverter.kt | 17 +++++++++++++++++ .../huawei/converter/RecordConverter.kt | 17 +++++++++++++++++ .../huawei/converter/SequenceExtensions.kt | 17 +++++++++++++++++ .../org/radarbase/huawei/converter/TopicData.kt | 17 +++++++++++++++++ .../org/radarbase/huawei/offset/Offset.kt | 17 +++++++++++++++++ .../org/radarbase/huawei/offset/Offsets.kt | 17 +++++++++++++++++ .../huawei/request/HuaweiOffsetManager.kt | 17 +++++++++++++++++ .../huawei/request/HuaweiRequestGenerator.kt | 17 +++++++++++++++++ .../radarbase/huawei/request/HuaweiResult.kt | 17 +++++++++++++++++ .../huawei/request/RequestGenerator.kt | 17 +++++++++++++++++ .../org/radarbase/huawei/request/RestRequest.kt | 17 +++++++++++++++++ .../huawei/request/TooManyRequestsException.kt | 17 +++++++++++++++++ .../huawei/route/HuaweiActivityRecordRoute.kt | 17 +++++++++++++++++ .../huawei/route/HuaweiHealthRecordRoute.kt | 17 +++++++++++++++++ .../org/radarbase/huawei/route/HuaweiRoute.kt | 17 +++++++++++++++++ .../huawei/route/HuaweiRouteDefinition.kt | 17 +++++++++++++++++ .../huawei/route/HuaweiRouteFactory.kt | 17 +++++++++++++++++ .../huawei/route/HuaweiSampleSetRoute.kt | 17 +++++++++++++++++ .../kotlin/org/radarbase/huawei/route/Route.kt | 17 +++++++++++++++++ .../org/radarbase/huawei/user/HuaweiUser.kt | 17 +++++++++++++++++ .../kotlin/org/radarbase/huawei/user/User.kt | 17 +++++++++++++++++ .../huawei/user/UserNotAuthorizedException.kt | 17 +++++++++++++++++ .../org/radarbase/huawei/user/UserRepository.kt | 17 +++++++++++++++++ .../huawei/converter/FieldValuesTest.kt | 17 +++++++++++++++++ .../huawei/route/HuaweiRouteFactoryTest.kt | 17 +++++++++++++++++ .../huawei/AbstractRestSourceConnector.java | 2 +- .../huawei/HuaweiRestSourceConnectorConfig.kt | 2 +- .../rest/huawei/HuaweiSourceConnector.java | 2 +- .../connect/rest/huawei/HuaweiSourceTask.java | 2 +- .../rest/huawei/offset/KafkaOffsetManager.java | 17 +++++++++++++++++ .../rest/huawei/user/HttpResponseException.java | 2 +- .../huawei/user/HuaweiServiceUserRepository.kt | 2 +- .../rest/huawei/user/HuaweiUserRepository.kt | 2 +- .../connect/rest/huawei/user/HuaweiUsers.java | 2 +- .../rest/huawei/user/OAuth2UserCredentials.java | 2 +- .../connect/rest/huawei/util/VersionUtil.java | 2 +- .../HuaweiRestSourceConnectorConfigTest.kt | 2 +- 41 files changed, 521 insertions(+), 11 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt index f0db5d6a..2fb3d0ea 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import com.fasterxml.jackson.databind.JsonNode diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt index ebe7d40f..43e4bd9e 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import com.fasterxml.jackson.databind.JsonNode diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt index 6b095587..473bd938 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import com.fasterxml.jackson.databind.JsonNode diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt index 013307bc..505836f1 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import com.fasterxml.jackson.databind.JsonNode diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt index e5c6d566..e44ab0c5 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import com.fasterxml.jackson.databind.JsonNode diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt index f9e9e16b..71e40465 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import okhttp3.Headers diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt index fe1dc73f..eb25544a 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import org.slf4j.LoggerFactory diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt index a537af98..af32d046 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import org.apache.avro.specific.SpecificRecord diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt index 9da597c0..6ef10272 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.offset import org.radarbase.huawei.route.Route diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt index 88c67afb..ff34db97 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.offset data class Offsets( diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt index 03b23c34..8c40ea5f 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.request import org.radarbase.huawei.offset.Offset diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt index 2a733024..07d7d4b9 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.request import com.fasterxml.jackson.core.JsonFactory diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt index fba9e551..8e4e5648 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.request sealed class HuaweiResult { diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt index 39bb60e9..08ad8f15 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.request import okhttp3.Response diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt index 502ecc26..e46f3e01 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.request import okhttp3.Request diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt index 3dc5aa9c..f8b583bc 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.request class TooManyRequestsException : RuntimeException() diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt index 07a4f586..58dd8ca2 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.route import org.radarbase.huawei.converter.HuaweiActivityRecordConverter diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt index ae2fa2b5..e1823aa7 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.route import org.apache.avro.specific.SpecificRecord diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt index 33a47dea..9653cd4e 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.route import okhttp3.HttpUrl.Companion.toHttpUrl diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt index 5a311e47..d117a36e 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.route import org.radarbase.huawei.user.UserRepository diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index 5d7e2032..e4a17802 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.route import org.apache.avro.specific.SpecificRecord diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt index 91b207fa..922766ff 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.route import com.fasterxml.jackson.databind.ObjectMapper diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt index 39e2cf71..0a3b0c8f 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.route import org.radarbase.huawei.request.RestRequest diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt index 7d1521e3..06ff3933 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.user import com.fasterxml.jackson.annotation.JsonIgnoreProperties diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt index b84dfe76..9e15d7b9 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.user import org.radarcns.kafka.ObservationKey diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt index 1bd513b0..1a3ed168 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.user class UserNotAuthorizedException(message: String) : Exception(message) { diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt index 6f26b74b..da59958c 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.user import java.io.IOException diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt index cdca893f..717676b3 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.converter import com.fasterxml.jackson.databind.ObjectMapper diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt index a109ac9b..4942376b 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.huawei.route import com.fasterxml.jackson.databind.ObjectMapper diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java index ad09e22a..57dc6c05 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java @@ -1,7 +1,7 @@ package org.radarbase.connect.rest.huawei; /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt index 4adbd949..150512bf 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java index 63c9debf..cf1ec845 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java index e3e6520e..bc672d4d 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java index dcc044ef..c8b32539 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java @@ -1,3 +1,20 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + package org.radarbase.connect.rest.huawei.offset; import java.time.Duration; diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java index c4f8c30c..4b94e045 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt index ae86aea6..2e9ac67c 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt index 010a5b5a..aa24c836 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java index e01aeff8..b23f5ef6 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java index 88bad3b8..3ea2c8a1 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java index 8c23ac79..e3bf8e09 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt b/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt index dd7ae19a..04d4e1b3 100644 --- a/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt +++ b/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt @@ -1,5 +1,5 @@ /* - * Copyright 2018 The Hyve + * Copyright 2026 Onsentia * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. From ac15a680e028a8c8a33756ad7a1d7679cf52f9d3 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Thu, 23 Jul 2026 15:24:27 +0000 Subject: [PATCH 13/27] Fix copyright header in Huawei connector Dockerfile Was still copied from Oura's Dockerfile (Copyright 2018 The Hyve); updates to match the Onsentia header applied to the rest of this module's files. --- kafka-connect-huawei-source/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kafka-connect-huawei-source/Dockerfile b/kafka-connect-huawei-source/Dockerfile index 330759cf..5da90434 100644 --- a/kafka-connect-huawei-source/Dockerfile +++ b/kafka-connect-huawei-source/Dockerfile @@ -1,4 +1,4 @@ -# Copyright 2018 The Hyve +# Copyright 2026 Onsentia # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. From 401291b1d84caaf38348673d5ff887f7004ed270 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Thu, 23 Jul 2026 15:27:54 +0000 Subject: [PATCH 14/27] Add @author yatharthranjan to Huawei connector source docs Adds an @author yatharthranjan KDoc/Javadoc tag to the primary class/interface/object of every .kt/.java file in huawei-library and kafka-connect-huawei-source: appended to existing class-level doc comments where present, added as a new minimal doc comment otherwise. --- .../main/kotlin/org/radarbase/huawei/converter/FieldValues.kt | 2 ++ .../huawei/converter/HuaweiActivityRecordConverter.kt | 2 ++ .../org/radarbase/huawei/converter/HuaweiDataConverter.kt | 2 ++ .../radarbase/huawei/converter/HuaweiHealthRecordConverter.kt | 2 ++ .../org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt | 2 ++ .../kotlin/org/radarbase/huawei/converter/RecordConverter.kt | 3 +++ .../org/radarbase/huawei/converter/SequenceExtensions.kt | 3 +++ .../main/kotlin/org/radarbase/huawei/converter/TopicData.kt | 3 +++ .../src/main/kotlin/org/radarbase/huawei/offset/Offset.kt | 3 +++ .../src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt | 3 +++ .../kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt | 3 +++ .../org/radarbase/huawei/request/HuaweiRequestGenerator.kt | 3 +++ .../main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt | 3 +++ .../kotlin/org/radarbase/huawei/request/RequestGenerator.kt | 3 +++ .../main/kotlin/org/radarbase/huawei/request/RestRequest.kt | 3 +++ .../org/radarbase/huawei/request/TooManyRequestsException.kt | 3 +++ .../org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt | 2 ++ .../org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt | 2 ++ .../src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt | 2 ++ .../kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt | 2 ++ .../kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt | 2 ++ .../kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt | 2 ++ .../src/main/kotlin/org/radarbase/huawei/route/Route.kt | 3 +++ .../src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt | 3 +++ .../src/main/kotlin/org/radarbase/huawei/user/User.kt | 3 +++ .../org/radarbase/huawei/user/UserNotAuthorizedException.kt | 3 +++ .../main/kotlin/org/radarbase/huawei/user/UserRepository.kt | 3 +++ .../kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt | 3 +++ .../org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt | 2 ++ .../connect/rest/huawei/AbstractRestSourceConnector.java | 3 +++ .../connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt | 2 ++ .../radarbase/connect/rest/huawei/HuaweiSourceConnector.java | 3 +++ .../org/radarbase/connect/rest/huawei/HuaweiSourceTask.java | 3 +++ .../connect/rest/huawei/offset/KafkaOffsetManager.java | 3 +++ .../connect/rest/huawei/user/HttpResponseException.java | 3 +++ .../connect/rest/huawei/user/HuaweiServiceUserRepository.kt | 2 ++ .../radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt | 3 +++ .../org/radarbase/connect/rest/huawei/user/HuaweiUsers.java | 3 +++ .../connect/rest/huawei/user/OAuth2UserCredentials.java | 3 +++ .../org/radarbase/connect/rest/huawei/util/VersionUtil.java | 3 +++ .../connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt | 3 +++ 41 files changed, 109 insertions(+) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt index 2fb3d0ea..2bb49b49 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/FieldValues.kt @@ -32,6 +32,8 @@ import com.fasterxml.jackson.databind.JsonNode * * Field name constants follow Huawei's public `Field` identifiers (e.g. `steps_delta`, `calories`, * `avg`, `max`, `min`), as documented for the on-device and REST Health Kit APIs. + * + * @author yatharthranjan */ class FieldValues private constructor(private val values: Map) { diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt index 43e4bd9e..26248540 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt @@ -30,6 +30,8 @@ import java.time.Instant * type, and a nested activity summary with pace/data/section statistics). Nested JSON structures * that map to free-form Avro `string` fields (pace map, data summary, section summary) are kept as * their raw JSON text, since their internal shape varies by activity type. + * + * @author yatharthranjan */ class HuaweiActivityRecordConverter( private val topic: String = "connect_huawei_activity_record", diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt index 473bd938..51d79be4 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDataConverter.kt @@ -26,6 +26,8 @@ import java.time.Instant /** * Converts a Huawei Health Kit HTTP JSON response body to zero or more [TopicData] records. + * + * @author yatharthranjan */ interface HuaweiDataConverter : RecordConverter { /** Process the JSON records generated by given request. */ diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt index 505836f1..b68c660a 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt @@ -33,6 +33,8 @@ private fun JsonNode.epochInstant(field: String): Instant? { * Generic converter for `GET /healthkit/v1/healthRecords` responses: iterates every record * returned for the requested `subDataTypeName` and builds one Avro record per entry via * [buildRecord]. + * + * @author yatharthranjan */ class HuaweiHealthRecordConverter( private val topic: String, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt index e44ab0c5..192ad7d5 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiSampleSetConverter.kt @@ -36,6 +36,8 @@ private fun JsonNode.epochInstant(field: String): Instant? { * This single converter is reused for the large majority of Huawei Health Kit data types, since * they all share the same `sampleSet[].samplePoints[]` response envelope and differ only in which * Avro record type their field values are mapped onto. + * + * @author yatharthranjan */ class HuaweiSampleSetConverter( private val topic: String, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt index 71e40465..ca82e213 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/RecordConverter.kt @@ -22,6 +22,9 @@ import org.radarbase.huawei.request.RestRequest import org.slf4j.LoggerFactory import java.io.IOException +/** + * @author yatharthranjan + */ interface RecordConverter { @Throws(IOException::class) fun convert( diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt index eb25544a..62746512 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/SequenceExtensions.kt @@ -19,6 +19,9 @@ package org.radarbase.huawei.converter import org.slf4j.LoggerFactory +/** + * @author yatharthranjan + */ val logger = LoggerFactory.getLogger("org.radarbase.huawei.converter.SequenceExtensions") internal fun Sequence.mapCatching(fn: (T) -> S): Sequence> = map { t -> diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt index af32d046..aadd6a09 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/TopicData.kt @@ -20,6 +20,9 @@ package org.radarbase.huawei.converter import org.apache.avro.specific.SpecificRecord /** Single value for a topic. */ +/** + * @author yatharthranjan + */ data class TopicData( val topic: String, val key: SpecificRecord, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt index 6ef10272..791b0065 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offset.kt @@ -21,6 +21,9 @@ import org.radarbase.huawei.route.Route import org.radarbase.huawei.user.User import java.time.Instant +/** + * @author yatharthranjan + */ data class Offset( val user: User, val route: Route, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt index ff34db97..1c31c8b7 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/offset/Offsets.kt @@ -17,6 +17,9 @@ package org.radarbase.huawei.offset +/** + * @author yatharthranjan + */ data class Offsets( val offsets: List, ) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt index 8c40ea5f..0709396f 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiOffsetManager.kt @@ -22,6 +22,9 @@ import org.radarbase.huawei.route.Route import org.radarbase.huawei.user.User import java.time.Instant +/** + * @author yatharthranjan + */ interface HuaweiOffsetManager { fun getOffset(route: Route, user: User): Offset? diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt index 07d7d4b9..6891ffb0 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt @@ -30,6 +30,9 @@ import java.io.IOException import java.time.Duration import java.time.Instant +/** + * @author yatharthranjan + */ class HuaweiRequestGenerator( private val userRepository: UserRepository, private val huaweiOffsetManager: HuaweiOffsetManager, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt index 8e4e5648..0f9dda6d 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiResult.kt @@ -17,6 +17,9 @@ package org.radarbase.huawei.request +/** + * @author yatharthranjan + */ sealed class HuaweiResult { data class Success(val value: T) : HuaweiResult() data class Error(val error: HuaweiError) : HuaweiResult() diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt index 08ad8f15..bf48a457 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RequestGenerator.kt @@ -22,6 +22,9 @@ import org.radarbase.huawei.converter.TopicData import org.radarbase.huawei.route.Route import org.radarbase.huawei.user.User +/** + * @author yatharthranjan + */ interface RequestGenerator { fun requests(user: User, max: Int): Sequence diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt index e46f3e01..ef199dbf 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/RestRequest.kt @@ -22,6 +22,9 @@ import org.radarbase.huawei.route.HuaweiRoute import org.radarbase.huawei.user.User import java.time.Instant +/** + * @author yatharthranjan + */ data class RestRequest( val request: Request, val user: User, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt index f8b583bc..db54d069 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/TooManyRequestsException.kt @@ -17,4 +17,7 @@ package org.radarbase.huawei.request +/** + * @author yatharthranjan + */ class TooManyRequestsException : RuntimeException() diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt index 58dd8ca2..ebd75def 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt @@ -28,6 +28,8 @@ import java.time.Instant /** * Route backed by `GET /healthkit/v1/activityRecords`, covering the Huawei Health Kit Activity * Records API (workout / physical-activity sessions). + * + * @author yatharthranjan */ class HuaweiActivityRecordRoute( userRepository: UserRepository, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt index e1823aa7..4d7a8c44 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt @@ -31,6 +31,8 @@ import java.time.Instant * Route backed by `GET /healthkit/v1/healthRecords`, used for the `health.record.*` data types * (ambulatory blood pressure sessions, heart rate alerts, hyperthermia, low SpO2 alerts, * menstrual cycle phases, and comprehensive sleep records). + * + * @author yatharthranjan */ open class HuaweiHealthRecordRoute( userRepository: UserRepository, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt index 9653cd4e..02bf19b8 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt @@ -34,6 +34,8 @@ import java.time.Instant * Handles OAuth2-authorized request construction (both `GET` with query parameters and `POST` * with a JSON body, since the Health Kit Data API mixes both styles across its endpoints) and * generic time-range chunking, shared by all concrete route types. + * + * @author yatharthranjan */ abstract class HuaweiRoute( private val userRepository: UserRepository, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt index d117a36e..80a3679d 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteDefinition.kt @@ -27,6 +27,8 @@ import org.radarbase.huawei.user.UserRepository * Using one shared registry (see [HuaweiRouteFactory]) for both the Kafka Connect config * definition and the set of routes actually polled avoids hand-duplicating each of the ~54 Huawei * data types across a `ConfigDef` and a route-construction switch. + * + * @author yatharthranjan */ data class HuaweiRouteDefinition( val key: String, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index e4a17802..ad34ad9d 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -71,6 +71,8 @@ import java.time.Instant * documented constants, the snake_case form of the Avro field's own name is used as a best-effort * default (see [snake]) — verify against a live API response and adjust the key strings in this * file if Huawei's actual response uses different names. + * + * @author yatharthranjan */ object HuaweiRouteFactory { diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt index 922766ff..664e80a7 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt @@ -37,6 +37,8 @@ import java.time.Instant * When [groupByTimeUnit] is set, the request aggregates sample points into buckets of that size — * this is how Huawei's `.statistics` data types are queried. When it is `null`, the endpoint * returns raw, un-aggregated sample points for [dataTypeName] over the requested time range. + * + * @author yatharthranjan */ open class HuaweiSampleSetRoute( userRepository: UserRepository, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt index 0a3b0c8f..ba2ae506 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/Route.kt @@ -22,6 +22,9 @@ import org.radarbase.huawei.user.User import java.time.Duration import java.time.Instant +/** + * @author yatharthranjan + */ interface Route { fun generateRequests(user: User, start: Instant, end: Instant): Sequence diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt index 06ff3933..692120c8 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/HuaweiUser.kt @@ -22,6 +22,9 @@ import com.fasterxml.jackson.annotation.JsonProperty import org.radarcns.kafka.ObservationKey import java.time.Instant +/** + * @author yatharthranjan + */ @JsonIgnoreProperties(ignoreUnknown = true) data class HuaweiUser( @JsonProperty("id") override val id: String, diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt index 9e15d7b9..e2092cda 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/User.kt @@ -20,6 +20,9 @@ package org.radarbase.huawei.user import org.radarcns.kafka.ObservationKey import java.time.Instant +/** + * @author yatharthranjan + */ interface User { val id: String val projectId: String diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt index 1a3ed168..ace22016 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserNotAuthorizedException.kt @@ -17,6 +17,9 @@ package org.radarbase.huawei.user +/** + * @author yatharthranjan + */ class UserNotAuthorizedException(message: String) : Exception(message) { constructor(user: User) : this("User ${user.id} is not authorized") } diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt index da59958c..34d50e78 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/user/UserRepository.kt @@ -20,6 +20,9 @@ package org.radarbase.huawei.user import java.io.IOException /** User repository for Huawei Health Kit users. */ +/** + * @author yatharthranjan + */ interface UserRepository { /** * Get specified user. diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt index 717676b3..428f5065 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/converter/FieldValuesTest.kt @@ -22,6 +22,9 @@ import kotlin.test.Test import kotlin.test.assertEquals import kotlin.test.assertNull +/** + * @author yatharthranjan + */ class FieldValuesTest { private val mapper = ObjectMapper() diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt index 4942376b..55f3c473 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -36,6 +36,8 @@ import kotlin.test.assertTrue * `healthRecords`, or `activityRecords`), and asserts the converter produces exactly one record on * the definition's own topic without throwing. This is the main regression test against typos in * the ~90 hand-written Huawei field-value key strings (and the Avro builder calls around them). + * + * @author yatharthranjan */ class HuaweiRouteFactoryTest { private val mapper = ObjectMapper() diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java index 57dc6c05..479ac97a 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/AbstractRestSourceConnector.java @@ -25,6 +25,9 @@ import org.apache.kafka.connect.source.SourceConnector; import org.radarbase.connect.rest.huawei.util.VersionUtil; +/** + * @author yatharthranjan + */ @SuppressWarnings("unused") public abstract class AbstractRestSourceConnector extends SourceConnector { protected HuaweiRestSourceConnectorConfig config; diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt index 150512bf..b8e1a2e8 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt @@ -40,6 +40,8 @@ import java.time.Duration * boolean and a `huawei..topic` string config, generated from that single shared registry * instead of ~110 hand-duplicated `ConfigDef` entries (one connector, one config, one canonical * list of Huawei data types). + * + * @author yatharthranjan */ class HuaweiRestSourceConnectorConfig( config: ConfigDef, diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java index cf1ec845..75e8397c 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceConnector.java @@ -39,6 +39,9 @@ import static org.radarbase.connect.rest.huawei.HuaweiRestSourceConnectorConfig.HUAWEI_USERS_CONFIG; +/** + * @author yatharthranjan + */ public class HuaweiSourceConnector extends AbstractRestSourceConnector { private static final Logger logger = LoggerFactory.getLogger(HuaweiSourceConnector.class); diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java index bc672d4d..6fd00a65 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiSourceTask.java @@ -53,6 +53,9 @@ import okhttp3.OkHttpClient; import okhttp3.Response; +/** + * @author yatharthranjan + */ public class HuaweiSourceTask extends SourceTask { private static final Logger logger = LoggerFactory.getLogger(HuaweiSourceTask.class); diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java index c8b32539..8f2aec3c 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/offset/KafkaOffsetManager.java @@ -31,6 +31,9 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; +/** + * @author yatharthranjan + */ public class KafkaOffsetManager implements HuaweiOffsetManager { private static final Logger logger = LoggerFactory.getLogger(KafkaOffsetManager.class); private static final String TIMESTAMP_OFFSET_KEY = "timestamp"; diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java index 4b94e045..4a8573a9 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HttpResponseException.java @@ -19,6 +19,9 @@ import java.io.IOException; +/** + * @author yatharthranjan + */ public class HttpResponseException extends IOException { private final int statusCode; diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt index 2e9ac67c..d0c5d088 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiServiceUserRepository.kt @@ -72,6 +72,8 @@ import kotlin.time.Duration.Companion.seconds * [org.radarbase.connect.rest.oura.user.OuraServiceUserRepository]. Retrieves the list of Huawei * users configured for a study (`GET users?source-type=Huawei`) and their Huawei Health Kit OAuth2 * access/refresh tokens (`users//token`). + * + * @author yatharthranjan */ @Suppress("unused") class HuaweiServiceUserRepository : HuaweiUserRepository() { diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt index aa24c836..44efe62e 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUserRepository.kt @@ -22,6 +22,9 @@ import org.radarbase.huawei.user.UserNotAuthorizedException import org.radarbase.huawei.user.UserRepository import java.io.IOException +/** + * @author yatharthranjan + */ @Suppress("unused") abstract class HuaweiUserRepository : UserRepository { abstract fun initialize(config: HuaweiRestSourceConnectorConfig) diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java index b23f5ef6..db225b9a 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiUsers.java @@ -25,6 +25,9 @@ import java.util.List; import org.radarbase.huawei.user.HuaweiUser; +/** + * @author yatharthranjan + */ @JsonIgnoreProperties(ignoreUnknown = true) public class HuaweiUsers { private final List users; diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java index 3ea2c8a1..42f2edf8 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/OAuth2UserCredentials.java @@ -24,6 +24,9 @@ import java.time.Duration; import java.time.Instant; +/** + * @author yatharthranjan + */ @JsonIgnoreProperties(ignoreUnknown = true) public class OAuth2UserCredentials { private static final Duration DEFAULT_EXPIRY = Duration.ofHours(1); diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java index e3bf8e09..1c74b19f 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/util/VersionUtil.java @@ -17,6 +17,9 @@ package org.radarbase.connect.rest.huawei.util; +/** + * @author yatharthranjan + */ public final class VersionUtil { private VersionUtil() { // utility class diff --git a/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt b/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt index 04d4e1b3..585d8775 100644 --- a/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt +++ b/kafka-connect-huawei-source/src/test/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfigTest.kt @@ -21,6 +21,9 @@ import org.junit.jupiter.api.Test import org.radarbase.huawei.route.HuaweiRouteFactory import kotlin.test.assertEquals +/** + * @author yatharthranjan + */ class HuaweiRestSourceConnectorConfigTest { @Test From 4e23dbf5da49970b6c53d2fe25a953c75e361d33 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Tue, 28 Jul 2026 13:25:43 +0000 Subject: [PATCH 15/27] Add file-based HuaweiYamlUserRepository for local testing Lets the Huawei connector run against per-user YAML credential files under huawei.user.dir, mirroring Fitbit's YamlUserRepository, so it can be tested locally without a rest-source-authorizer webservice. Adds a docker/huawei-user.yml.template and README instructions for the flow. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- README.md | 36 +++ docker/huawei-user.yml.template | 24 ++ docker/source-huawei.properties.template | 6 + .../huawei/HuaweiRestSourceConnectorConfig.kt | 29 +++ .../rest/huawei/user/HuaweiLocalUser.kt | 104 ++++++++ .../huawei/user/HuaweiYamlUserRepository.kt | 234 ++++++++++++++++++ 6 files changed, 433 insertions(+) create mode 100644 docker/huawei-user.yml.template create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt create mode 100644 kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiYamlUserRepository.kt diff --git a/README.md b/README.md index 09b941e1..7fe83445 100644 --- a/README.md +++ b/README.md @@ -230,6 +230,42 @@ This connector requires a (currently `0.9.0-SNAPSHOT`) to be resolvable from one of the repositories declared in `huawei-library/build.gradle` / `kafka-connect-huawei-source/build.gradle.kts`. +### Testing locally + +The easiest way to try out the Huawei connector without standing up a +`rest-source-authorizer` webservice is the file-based +`org.radarbase.connect.rest.huawei.user.HuaweiYamlUserRepository`, which reads one YAML file per +user from a local directory, mirroring the Fitbit `YamlUserRepository` above. + +1. [Register a Huawei Health Kit OAuth 2.0 app](https://developer.huawei.com/consumer/en/doc/HMSCore-Guides/config-agc-0000001050170137) + and obtain an access token and refresh token for one test user by hand, using Huawei's + [OAuth 2.0 authorization code flow](https://developer.huawei.com/consumer/en/doc/HMSCore-Guides/authorization-code-0000001053629189). +2. Copy `docker/huawei-user.yml.template` to a file in `docker/users/` (e.g. `docker/users/test.yml`) + and fill in the `externalUserId`, `oauth2.accessToken`, and `oauth2.refreshToken` fields. +3. Copy `docker/source-huawei.properties.template` to `docker/source-huawei.properties`, set + `huawei.api.client` / `huawei.api.secret` to your Huawei app's client ID and secret, and change + `huawei.user.repository.class` to `org.radarbase.connect.rest.huawei.user.HuaweiYamlUserRepository` + (the `huawei.user.repository.url`/`.client.id`/`.client.secret`/`.oauth2.token.url` properties + are only used by the webservice-based repository and can be left as-is or removed). +4. Run the full stack with `docker-compose up -d --build` and inspect the connector's progress with + `docker-compose logs -f radar-huawei-connector`. +5. To inspect the data coming out of a specific route, run, for example: + + ```shell + docker-compose exec schema-registry-1 kafka-avro-console-consumer \ + --bootstrap-server kafka-1:9092,kafka-2:9092,kafka-3:9092 \ + --from-beginning \ + --topic connect_huawei_activity_record + ``` + + (replace the topic with any `huawei..topic` default from + `org.radarbase.huawei.route.HuaweiRouteFactory`). + +For a full RADAR-base deployment, use the webservice-based +`org.radarbase.connect.rest.huawei.user.HuaweiServiceUserRepository` (the default) against a +`rest-source-authorizer` instance instead, following the same pattern as the Fitbit connector's +ManagementPortal setup above. + ## Sentry monitoring To enable Sentry monitoring for the generic REST, Fitbit, Oura, or Huawei source connector service: diff --git a/docker/huawei-user.yml.template b/docker/huawei-user.yml.template new file mode 100644 index 00000000..99e5187e --- /dev/null +++ b/docker/huawei-user.yml.template @@ -0,0 +1,24 @@ +--- +# Unique user key +id: test +# Project ID to be used in org.radarcns.kafka.ObservationKey record keys +projectId: radar-test +# User ID to be used in org.radarcns.kafka.ObservationKey record keys +userId: test +# Source ID to be used in org.radarcns.kafka.ObservationKey record keys +sourceId: huawei-watch +# Date from when to collect data. +startDate: 2018-08-06T00:00:00Z +# Date until when to collect data. +endDate: 2099-01-01T00:00:00Z +# Huawei user ID as returned by the Huawei OAuth 2.0 authentication procedure +externalUserId: ? +oauth2: + # Huawei Health Kit OAuth 2.0 access token as returned by the Huawei authentication procedure + accessToken: ? + # Huawei Health Kit OAuth 2.0 refresh token as returned by the Huawei authentication procedure + refreshToken: ? + # Optional expiry time of the access token. If absent, it will be estimated to one hour + # when the source connector starts. When an authentication error occurs, a new access token will + # be fetched regardless of the value in this field. + #expiresAt: 2018-08-06T00:00:00Z diff --git a/docker/source-huawei.properties.template b/docker/source-huawei.properties.template index 1dd5b63e..69607aa6 100644 --- a/docker/source-huawei.properties.template +++ b/docker/source-huawei.properties.template @@ -5,6 +5,12 @@ rest.source.base.url=https://health-api.cloud.huawei.com/healthkit/v1 rest.source.poll.interval.ms=5000 huawei.api.client=? huawei.api.secret=? +# For local testing without a rest-source-authorizer webservice, use the file-based +# repository instead, backed by per-user YAML files under huawei.user.dir - see +# docker/huawei-user.yml.template and the "Testing locally" section in README.md. +#huawei.user.repository.class=org.radarbase.connect.rest.huawei.user.HuaweiYamlUserRepository +#huawei.user.dir=/var/lib/kafka-connect-huawei-source/users + huawei.user.repository.class=org.radarbase.connect.rest.huawei.user.HuaweiServiceUserRepository huawei.user.repository.url=http://localhost:8080/ huawei.user.repository.client.id=radar_huawei_connector diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt index b8e1a2e8..d797cce2 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/HuaweiRestSourceConnectorConfig.kt @@ -31,6 +31,8 @@ import org.radarbase.connect.rest.huawei.user.HuaweiUserRepository import org.radarbase.huawei.route.HuaweiRouteFactory import java.net.MalformedURLException import java.net.URL +import java.nio.file.Path +import java.nio.file.Paths import java.time.Duration /** @@ -90,6 +92,14 @@ class HuaweiRestSourceConnectorConfig( throw ConnectException("Invalid class. $e") } + /** + * Directory containing per-user YAML credential files, for the file-based + * [org.radarbase.connect.rest.huawei.user.HuaweiYamlUserRepository]. Only used if that + * repository is configured via [HUAWEI_USER_REPOSITORY_CONFIG]. + */ + fun getHuaweiUserCredentialsPath(): Path = + Paths.get(getString(HUAWEI_USER_CREDENTIALS_DIR_CONFIG)) + fun getHuaweiUserRepositoryUrl(): HttpUrl { var urlString = getString(HUAWEI_USER_REPOSITORY_URL_CONFIG).trim() if (urlString.isNotEmpty() && urlString.last() != '/') { @@ -173,6 +183,14 @@ class HuaweiRestSourceConnectorConfig( private const val HUAWEI_USER_POLL_INTERVAL_DEFAULT = 150 private const val HUAWEI_USER_POLL_INTERVAL_DISPLAY = "Per-user per-route polling interval." + const val HUAWEI_USER_CREDENTIALS_DIR_CONFIG = "huawei.user.dir" + private const val HUAWEI_USER_CREDENTIALS_DIR_DOC = + "Directory containing Huawei user information and credentials. Only used if a " + + "file-based user repository is configured." + private const val HUAWEI_USER_CREDENTIALS_DIR_DISPLAY = "User directory" + private const val HUAWEI_USER_CREDENTIALS_DIR_DEFAULT = + "/var/lib/kafka-connect-huawei-source/users" + const val HUAWEI_USER_REPOSITORY_URL_CONFIG = "huawei.user.repository.url" private const val HUAWEI_USER_REPOSITORY_URL_DOC = "URL for webservice containing user credentials. Only used if a webservice-based " + @@ -286,6 +304,17 @@ class HuaweiRestSourceConnectorConfig( Width.SHORT, HUAWEI_USER_REPOSITORY_DISPLAY, ) + .define( + HUAWEI_USER_CREDENTIALS_DIR_CONFIG, + Type.STRING, + HUAWEI_USER_CREDENTIALS_DIR_DEFAULT, + Importance.LOW, + HUAWEI_USER_CREDENTIALS_DIR_DOC, + group, + ++order, + Width.SHORT, + HUAWEI_USER_CREDENTIALS_DIR_DISPLAY, + ) .define( HUAWEI_USER_REPOSITORY_URL_CONFIG, Type.STRING, diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt new file mode 100644 index 00000000..895d22fb --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt @@ -0,0 +1,104 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.radarbase.connect.rest.huawei.user + +import com.fasterxml.jackson.annotation.JsonIgnoreProperties +import com.fasterxml.jackson.annotation.JsonInclude +import com.fasterxml.jackson.annotation.JsonProperty +import org.radarbase.huawei.user.User +import org.radarcns.kafka.ObservationKey +import java.time.Instant + +/** + * A single user's Huawei Health Kit credentials, read from (and written back to) a local YAML + * file by [HuaweiYamlUserRepository]. Mirrors Fitbit's `LocalUser`. See + * `docker/huawei-user.yml.template` for the expected file format. + * + * @author yatharthranjan + */ +@JsonInclude(JsonInclude.Include.NON_EMPTY) +@JsonIgnoreProperties(ignoreUnknown = true) +class HuaweiLocalUser : User { + @JsonProperty("id") + override var id: String = "" + + @JsonProperty("projectId") + override var projectId: String = "" + + @JsonProperty("userId") + override var userId: String = "" + + @JsonProperty("sourceId") + override var sourceId: String = "" + + @JsonProperty("externalUserId") + override var externalId: String? = null + + @JsonProperty("startDate") + override var startDate: Instant = Instant.parse("2017-01-01T00:00:00Z") + + @JsonProperty("endDate") + override var endDate: Instant? = Instant.parse("9999-12-31T23:59:59.999Z") + + @JsonProperty("createdAt") + override var createdAt: Instant = Instant.now() + + @JsonProperty("humanReadableUserId") + override var humanReadableUserId: String? = null + + @JsonProperty("serviceUserId") + override var serviceUserId: String? = null + + @JsonProperty("version") + override var version: String? = null + + @JsonProperty("oauth2") + var oauth2Credentials: OAuth2UserCredentials = OAuth2UserCredentials() + + @JsonProperty("isAuthorized") + var isAuthorizedOverride: Boolean? = null + + override val isAuthorized: Boolean + get() = isAuthorizedOverride + ?: (!oauth2Credentials.isAccessTokenExpired || oauth2Credentials.hasRefreshToken()) + + override val observationKey: ObservationKey + get() = ObservationKey(projectId, userId, sourceId) + + override val versionedId: String + get() = "$id${version?.let { "#$it" } ?: ""}" + + fun copy(): HuaweiLocalUser { + val copy = HuaweiLocalUser() + copy.id = id + copy.projectId = projectId + copy.userId = userId + copy.sourceId = sourceId + copy.externalId = externalId + copy.startDate = startDate + copy.endDate = endDate + copy.createdAt = createdAt + copy.humanReadableUserId = humanReadableUserId + copy.serviceUserId = serviceUserId + copy.version = version + copy.oauth2Credentials = oauth2Credentials + copy.isAuthorizedOverride = isAuthorizedOverride + return copy + } + + override fun toString(): String = "HuaweiLocalUser(id='$id', versionedId='$versionedId')" +} diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiYamlUserRepository.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiYamlUserRepository.kt new file mode 100644 index 00000000..97e09b6f --- /dev/null +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiYamlUserRepository.kt @@ -0,0 +1,234 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package org.radarbase.connect.rest.huawei.user + +import com.fasterxml.jackson.databind.JsonNode +import com.fasterxml.jackson.databind.ObjectMapper +import com.fasterxml.jackson.databind.SerializationFeature +import com.fasterxml.jackson.dataformat.yaml.YAMLFactory +import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule +import com.fasterxml.jackson.module.kotlin.registerKotlinModule +import okhttp3.FormBody +import okhttp3.Headers +import okhttp3.OkHttpClient +import okhttp3.Request +import org.radarbase.connect.rest.huawei.HuaweiRestSourceConnectorConfig +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserNotAuthorizedException +import org.slf4j.LoggerFactory +import java.io.IOException +import java.nio.file.Files +import java.nio.file.Path +import java.nio.file.StandardCopyOption +import java.time.Duration +import java.time.Instant +import java.util.Base64 +import java.util.concurrent.ConcurrentHashMap +import java.util.concurrent.atomic.AtomicReference +import java.util.concurrent.locks.ReentrantLock +import java.util.stream.Collectors + +/** + * User repository that reads (and writes refreshed tokens back to) YAML files in a local + * directory, one file per user - mirrors Fitbit's `YamlUserRepository`. This is the easiest way + * to run this connector locally without standing up a rest-source-authorizer webservice: register + * a Huawei Health Kit OAuth2 app, obtain one user's access/refresh token by hand (e.g. via + * Huawei's OAuth 2.0 authorization code flow), and drop them into a file under the directory + * configured by `huawei.user.dir` - see `docker/huawei-user.yml.template`. + * + * @author yatharthranjan + */ +@Suppress("unused") +class HuaweiYamlUserRepository : HuaweiUserRepository() { + private val client = OkHttpClient() + private val users = ConcurrentHashMap() + private val nextFetch = AtomicReference(Instant.EPOCH) + private lateinit var credentialsDir: Path + private lateinit var clientCredentials: Headers + + override fun initialize(config: HuaweiRestSourceConnectorConfig) { + credentialsDir = config.getHuaweiUserCredentialsPath() + Files.createDirectories(credentialsDir) + val credentialString = "${config.getHuaweiClient()}:${config.getHuaweiClientSecret()}" + val credentialsBase64 = Base64.getEncoder().encodeToString(credentialString.toByteArray()) + clientCredentials = Headers.headersOf("Authorization", "Basic $credentialsBase64") + } + + override operator fun get(key: String): User? { + updateUsers() + return users[key]?.locked { it.copy() } + } + + override fun stream(): Sequence { + if (nextFetch.get() == Instant.EPOCH) { + applyPendingUpdates() + } + return users.values.asSequence() + .filter { it.locked { u -> u.oauth2Credentials.hasRefreshToken() } } + .map { it.locked { u -> u.copy() } } + } + + @Throws(IOException::class, UserNotAuthorizedException::class) + override fun getAccessToken(user: User): String { + updateUsers() + val actual = users[user.id] + ?: throw NoSuchElementException("User $user is not present in this user repository.") + val current = actual.locked { u -> + if (!u.oauth2Credentials.isAccessTokenExpired) u.oauth2Credentials.accessToken else null + } + return current ?: refreshAccessToken(user) + } + + @Throws(IOException::class, UserNotAuthorizedException::class) + override fun refreshAccessToken(user: User): String { + val actual = users[user.id] + ?: throw NoSuchElementException("User $user is not present in this user repository.") + val refreshToken = actual.locked { it.oauth2Credentials.refreshToken } + val node = requestAccessToken(refreshToken) + + val expiresIn = node["expires_in"]?.asLong() + val accessToken = node["access_token"]?.asText() + ?: throw UserNotAuthorizedException("Did not get an access token") + val newRefreshToken = node["refresh_token"]?.asText() ?: refreshToken + + actual.update { u -> + u.oauth2Credentials = OAuth2UserCredentials(newRefreshToken, accessToken, expiresIn) + store(actual.path, u) + } + return accessToken + } + + override fun hasPendingUpdates(): Boolean = Instant.now().isAfter(nextFetch.get()) + + @Throws(IOException::class) + override fun applyPendingUpdates() { + forceUpdateUsers() + nextFetch.set(Instant.now().plus(FETCH_THRESHOLD)) + } + + private fun updateUsers() { + val next = nextFetch.get() + val now = Instant.now() + if (!now.isAfter(next) || !nextFetch.compareAndSet(next, now.plus(FETCH_THRESHOLD))) { + return + } + forceUpdateUsers() + } + + private fun forceUpdateUsers() { + try { + Files.walk(credentialsDir).use { walker -> + val newUsers = walker + .filter { + Files.isRegularFile(it) && + it.fileName.toString().lowercase().endsWith(".yml") + } + .map { path -> + LockedUser( + YAML_READER.readValue(path.toFile(), HuaweiLocalUser::class.java), + path, + ) + } + .collect(Collectors.toMap({ it.locked { u -> u.id } }, { it })) + users.keys.retainAll(newUsers.keys) + newUsers.forEach { (id, u) -> users.putIfAbsent(id, u) } + } + } catch (ex: IOException) { + logger.error("Failed to read user directory: {}", ex.toString()) + } + } + + private fun requestAccessToken(refreshToken: String?): JsonNode { + if (refreshToken.isNullOrEmpty()) { + throw UserNotAuthorizedException("Refresh token is not set") + } + val request = Request.Builder() + .url(HUAWEI_TOKEN_URL) + .headers(clientCredentials) + .post( + FormBody.Builder() + .add("grant_type", "refresh_token") + .add("refresh_token", refreshToken) + .build(), + ) + .build() + + client.newCall(request).execute().use { response -> + val body = response.body?.string() + return when { + response.isSuccessful && body != null -> JSON_READER.readTree(body) + response.code == 400 || response.code == 401 -> + throw UserNotAuthorizedException("Refresh token is no longer valid.") + else -> throw IOException( + "Failed to request refresh token, HTTP status ${response.code}" + + (body?.let { " and content $it" } ?: ""), + ) + } + } + } + + private fun store(path: Path, user: HuaweiLocalUser) { + try { + val temp = Files.createTempFile(user.id, ".tmp") + try { + Files.newOutputStream(temp).use { out -> YAML_WRITER.writeValue(out, user) } + Files.move(temp, path, StandardCopyOption.REPLACE_EXISTING) + } finally { + Files.deleteIfExists(temp) + } + } catch (ex: IOException) { + logger.error("Failed to store user file: {}", ex.toString()) + } + } + + /** Guards a mutable [HuaweiLocalUser] against concurrent read/refresh/store. */ + private class LockedUser(val user: HuaweiLocalUser, val path: Path) { + private val lock = ReentrantLock() + + fun locked(block: (HuaweiLocalUser) -> V): V { + lock.lock() + try { + return block(user) + } finally { + lock.unlock() + } + } + + fun update(block: (HuaweiLocalUser) -> Unit) { + lock.lock() + try { + block(user) + } finally { + lock.unlock() + } + } + } + + companion object { + private val logger = LoggerFactory.getLogger(HuaweiYamlUserRepository::class.java) + private const val HUAWEI_TOKEN_URL = "https://oauth-login.cloud.huawei.com/oauth2/v3/token" + private val FETCH_THRESHOLD = Duration.ofHours(1L) + private val YAML_MAPPER = ObjectMapper(YAMLFactory()).apply { + registerKotlinModule() + registerModule(JavaTimeModule()) + configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false) + } + private val YAML_READER = YAML_MAPPER.reader() + private val YAML_WRITER = YAML_MAPPER.writerFor(HuaweiLocalUser::class.java) + private val JSON_READER = ObjectMapper().registerModule(JavaTimeModule()).reader() + } +} From 9577b6794301e739ce546e356e6fb305ac562bbb Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Tue, 4 Aug 2026 11:51:49 +0000 Subject: [PATCH 16/27] Fix Jackson getter conflict on HuaweiLocalUser.isAuthorized isAuthorizedOverride was mapped to the same JSON key ("isAuthorized") as the isAuthorized computed property, causing Jackson to fail with "Conflicting getter definitions" when reading user YAML files. Rename the manual-override field's JSON key to "authorized". Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt index 895d22fb..a8a665ec 100644 --- a/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt +++ b/kafka-connect-huawei-source/src/main/java/org/radarbase/connect/rest/huawei/user/HuaweiLocalUser.kt @@ -69,7 +69,7 @@ class HuaweiLocalUser : User { @JsonProperty("oauth2") var oauth2Credentials: OAuth2UserCredentials = OAuth2UserCredentials() - @JsonProperty("isAuthorized") + @JsonProperty("authorized") var isAuthorizedOverride: Boolean? = null override val isAuthorized: Boolean From 5f69d0d1c46336145302bb1dadb8fcf55ad66350 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Tue, 4 Aug 2026 12:20:14 +0000 Subject: [PATCH 17/27] Log actual Huawei API error body on 400/401/403 responses The 400/401/403 branches previously logged only a hardcoded generic message, discarding the real error body Huawei's API returned. This made it impossible to diagnose the actual cause of a failed request (e.g. wrong client/app, invalid scope) from the logs. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../huawei/request/HuaweiRequestGenerator.kt | 20 +++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt index 6891ffb0..f039d2ca 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt @@ -156,31 +156,39 @@ class HuaweiRequestGenerator( HuaweiRateLimitError("Rate limit reached..", TooManyRequestsException(), "429") } 403 -> { + val body = response.body?.string() ?: "no response body" logger.warn( - "User {} does not have access to this Huawei Health Kit data type.", + "User {} does not have access to this Huawei Health Kit data type: {}", request.user, + body, ) routeNextRequest[key] = Instant.now().plus(USER_BACK_OFF_TIME) HuaweiAccessForbiddenError( - "Huawei Health Kit scope not granted or data not available..", + "Huawei Health Kit scope not granted or data not available: $body", IOException("Forbidden"), "403", ) } 401 -> { - logger.warn("User {} access token is expired, malformed, or revoked.", request.user) + val body = response.body?.string() ?: "no response body" + logger.warn( + "User {} access token is expired, malformed, or revoked: {}", + request.user, + body, + ) routeNextRequest[key] = Instant.now().plus(USER_BACK_OFF_TIME) HuaweiUnauthorizedAccessError( - "Access token expired or revoked..", + "Access token expired or revoked: $body", IOException("Unauthorized"), "401", ) } 400 -> { - logger.warn("Client exception for request {}", request) + val body = response.body?.string() ?: "no response body" + logger.warn("Client exception for request {}: {}", request, body) routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) HuaweiClientException( - "Client unsupported or unauthorized..", + "Client unsupported or unauthorized: $body", IOException("Invalid client"), "400", ) From 1ace5c11dc3a9fa725b6c6921bd79cd51dae90b1 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Tue, 4 Aug 2026 12:30:17 +0000 Subject: [PATCH 18/27] Fix wrong query param name on the healthRecords GET route HuaweiHealthRecordRoute sent the health record type identifier under a "subDataTypeName" query parameter, but Huawei's healthRecords API expects it under "dataTypeName" - the API was silently rejecting every health_record_* request with "DataTypeName is null" since it never received the parameter it actually looks for. Renamed the parameter (and the route/factory field) to dataTypeName throughout. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../radarbase/huawei/converter/HuaweiHealthRecordConverter.kt | 2 +- .../org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt | 4 ++-- .../kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt index b68c660a..671301db 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt @@ -31,7 +31,7 @@ private fun JsonNode.epochInstant(field: String): Instant? { /** * Generic converter for `GET /healthkit/v1/healthRecords` responses: iterates every record - * returned for the requested `subDataTypeName` and builds one Avro record per entry via + * returned for the requested `dataTypeName` and builds one Avro record per entry via * [buildRecord]. * * @author yatharthranjan diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt index 4d7a8c44..ef9e6740 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt @@ -36,7 +36,7 @@ import java.time.Instant */ open class HuaweiHealthRecordRoute( userRepository: UserRepository, - private val subDataTypeName: String, + private val dataTypeName: String, private val topic: String, maxIntervalPerRequest: Duration = Duration.ofDays(30L), buildRecord: ( @@ -63,7 +63,7 @@ open class HuaweiHealthRecordRoute( user, "healthRecords", mapOf( - "subDataTypeName" to subDataTypeName, + "dataTypeName" to dataTypeName, "startTime" to rangeStart.toEpochMilli().toString(), "endTime" to rangeEnd.toEpochMilli().toString(), ), diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index ad34ad9d..c93b5606 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -995,7 +995,7 @@ object HuaweiRouteFactory { private fun healthRecordDefinition( key: String, - subDataTypeName: String, + dataTypeName: String, defaultTopic: String, buildRecord: ( fields: FieldValues, @@ -1006,7 +1006,7 @@ object HuaweiRouteFactory { ): HuaweiRouteDefinition = HuaweiRouteDefinition(key, defaultTopic) { repo, topic -> HuaweiHealthRecordRoute( userRepository = repo, - subDataTypeName = VENDOR_PREFIX + subDataTypeName, + dataTypeName = VENDOR_PREFIX + dataTypeName, topic = topic, buildRecord = buildRecord, ) From 33512fc795f4f0d825fbb571f43b2134eeb3e83d Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Tue, 4 Aug 2026 14:34:12 +0000 Subject: [PATCH 19/27] Fix healthRecords route against the official REST API reference Per Huawei's official "Querying Health Records of a Data Type" spec: - The endpoint is on API version v2, not v1. - The data type query parameter is named "dataType", not "dataTypeName" (and not "subDataTypeName" as it was before that). - startTime/endTime, both in the request and in each returned record, are in nanoseconds since the epoch, not milliseconds. This was causing every health_record_* request to fail with "DataTypeName is null", and would have produced wildly wrong timestamps for any record that did come back. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../converter/HuaweiHealthRecordConverter.kt | 9 +++++---- .../huawei/route/HuaweiHealthRecordRoute.kt | 17 +++++++++++++---- .../org/radarbase/huawei/route/HuaweiRoute.kt | 4 +++- .../huawei/route/HuaweiRouteFactoryTest.kt | 6 ++++-- 4 files changed, 25 insertions(+), 11 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt index 671301db..aa522ff1 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiHealthRecordConverter.kt @@ -22,16 +22,17 @@ import org.apache.avro.specific.SpecificRecord import org.radarbase.huawei.user.User import java.time.Instant +/** Huawei's healthRecords v2 endpoint reports startTime/endTime in nanoseconds since the epoch. */ private fun JsonNode.epochInstant(field: String): Instant? { val value = this.get(field) ?: return null if (value.isNull) return null - val millis = if (value.isTextual) value.asText().toLongOrNull() else value.asLong() - return millis?.let { Instant.ofEpochMilli(it) } + val nanos = if (value.isTextual) value.asText().toLongOrNull() else value.asLong() + return nanos?.let { Instant.ofEpochSecond(it / 1_000_000_000L, it % 1_000_000_000L) } } /** - * Generic converter for `GET /healthkit/v1/healthRecords` responses: iterates every record - * returned for the requested `dataTypeName` and builds one Avro record per entry via + * Generic converter for `GET /healthkit/v2/healthRecords` responses: iterates every record + * returned for the requested `dataType` and builds one Avro record per entry via * [buildRecord]. * * @author yatharthranjan diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt index ef9e6740..9a376e36 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiHealthRecordRoute.kt @@ -28,10 +28,16 @@ import java.time.Duration import java.time.Instant /** - * Route backed by `GET /healthkit/v1/healthRecords`, used for the `health.record.*` data types + * Route backed by `GET /healthkit/v2/healthRecords`, used for the `health.record.*` data types * (ambulatory blood pressure sessions, heart rate alerts, hyperthermia, low SpO2 alerts, * menstrual cycle phases, and comprehensive sleep records). * + * Per the official Health Kit REST API reference, this endpoint is on API version `v2` (unlike + * `sampleSet:polymerize`/`activityRecords`, which are on `v1`), takes the data type under the + * `dataType` query parameter (not `dataTypeName`), and its `startTime`/`endTime` parameters (and + * the `startTime`/`endTime` fields of each returned record) are in **nanoseconds** since the + * epoch, not milliseconds. + * * @author yatharthranjan */ open class HuaweiHealthRecordRoute( @@ -63,10 +69,11 @@ open class HuaweiHealthRecordRoute( user, "healthRecords", mapOf( - "dataTypeName" to dataTypeName, - "startTime" to rangeStart.toEpochMilli().toString(), - "endTime" to rangeEnd.toEpochMilli().toString(), + "dataType" to dataTypeName, + "startTime" to rangeStart.toEpochNanos().toString(), + "endTime" to rangeEnd.toEpochNanos().toString(), ), + baseUrl = HUAWEI_API_BASE_URL_V2, ), user = user, route = this, @@ -74,4 +81,6 @@ open class HuaweiHealthRecordRoute( endDate = rangeEnd, ) } + + private fun Instant.toEpochNanos(): Long = epochSecond * 1_000_000_000L + nano.toLong() } diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt index 02bf19b8..81f94a32 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt @@ -47,9 +47,10 @@ abstract class HuaweiRoute( user: User, path: String, queryParams: Map, + baseUrl: String = HUAWEI_API_BASE_URL, ): Request { val accessToken = userRepository.getAccessToken(user) - val urlBuilder = "$HUAWEI_API_BASE_URL/$path".toHttpUrl().newBuilder() + val urlBuilder = "$baseUrl/$path".toHttpUrl().newBuilder() queryParams.forEach { (key, value) -> urlBuilder.addQueryParameter(key, value) } return Request.Builder() .url(urlBuilder.build()) @@ -91,6 +92,7 @@ abstract class HuaweiRoute( companion object { const val HUAWEI_API_BASE_URL = "https://health-api.cloud.huawei.com/healthkit/v1" + const val HUAWEI_API_BASE_URL_V2 = "https://health-api.cloud.huawei.com/healthkit/v2" private val JSON_MEDIA_TYPE = "application/json; charset=utf-8".toMediaType() private val DEFAULT_INTERVAL_PER_REQUEST = Duration.ofDays(30L) } diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt index 55f3c473..dd7aa8e3 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -115,8 +115,8 @@ class HuaweiRouteFactoryTest { val root = mapper.createObjectNode() val records = root.putArray("healthRecords") val record = records.addObject() - record.put("startTime", START_MILLIS) - record.put("endTime", END_MILLIS) + record.put("startTime", START_NANOS) + record.put("endTime", END_NANOS) record.set("value", genericValueArray()) return root } @@ -175,6 +175,8 @@ class HuaweiRouteFactoryTest { companion object { private const val START_MILLIS = 1704067200000L // 2024-01-01T00:00:00Z private const val END_MILLIS = 1704070800000L // 2024-01-01T01:00:00Z + private const val START_NANOS = START_MILLIS * 1_000_000L + private const val END_NANOS = END_MILLIS * 1_000_000L private val LITERAL_FIELD_KEYS = listOf( "active_hours", "active_hours_target", "all_sleep_time", "arrhythmia_result", From 1a1b6dfa090f08024f66e38c9daaeb6cc781bc06 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Tue, 4 Aug 2026 14:35:09 +0000 Subject: [PATCH 20/27] Report the real HTTP status code for unclassified error responses The generic error branch (405/409/500/502/503/590, and anything else not explicitly handled) hardcoded its HuaweiGenericError's code to "500" regardless of the actual response status, which was misleading for diagnostics. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../org/radarbase/huawei/request/HuaweiRequestGenerator.kt | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt index f039d2ca..5d7ce2aa 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/request/HuaweiRequestGenerator.kt @@ -212,12 +212,13 @@ class HuaweiRequestGenerator( ) } else -> { - logger.warn("Request failed: {}, {}", request, response) + val body = response.body?.string() ?: "unknown error" + logger.warn("Request failed: {}: {}", request, body) routeNextRequest[key] = Instant.now().plus(BACK_OFF_TIME) HuaweiGenericError( - response.body?.string() ?: "unknown error", + body, IOException("Unknown error"), - "500", + response.code.toString(), ) } } From 358584156aafff6d9b4a314b41f9ecccacd9b818 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Tue, 4 Aug 2026 14:43:25 +0000 Subject: [PATCH 21/27] Strip ".statistics" suffix from the dataTypeName sent to polymerize Per the official Health Kit REST API reference (Postman "HMS Core" collection, "Querying Sampling Data Statistics of Multiple Days"), the groupByTime-aggregated variant of a data type is obtained by polymerizing the underlying *raw* dataTypeName with groupByTime, not by sending a literal "*.statistics"-suffixed dataTypeName - that suffix is only RADAR-Schemas'/this connector's own label for "the daily-aggregated route", not a real Huawei data type identifier. Sending it verbatim is exactly why Huawei's API had no dataCollector for e.g. "com.huawei.continuous.heart_rate.statistics", "com.huawei.vo2max.statistics", "com.huawei.resting_calories.statistics", etc. - across every single statistics route built via sampleSetDefinition(). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index c93b5606..d98c7fa9 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -986,7 +986,11 @@ object HuaweiRouteFactory { ): HuaweiRouteDefinition = HuaweiRouteDefinition(key, defaultTopic) { repo, topic -> HuaweiSampleSetRoute( userRepository = repo, - dataTypeName = VENDOR_PREFIX + dataTypeSuffix, + // Huawei's polymerize API has no dataCollector for a literal "*.statistics" data + // type - ".statistics" is only this connector's/RADAR-Schemas' label for "the + // groupByTime-aggregated variant of the underlying raw data type", so it must be + // stripped from the dataTypeName actually sent on the wire. + dataTypeName = VENDOR_PREFIX + dataTypeSuffix.removeSuffix(".statistics"), topic = topic, groupByTimeUnit = if (dataTypeSuffix.endsWith(".statistics")) "day" else null, buildRecord = buildRecord, From d8c63885f14364c81f8328ad3cbc3c68ea118d26 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Tue, 4 Aug 2026 16:01:31 +0000 Subject: [PATCH 22/27] Query .statistics data types via sampleSet:dailyPolymerize, not polymerize Huawei's live API confirmed the root cause behind most "no default dataCollector found"/"Invalid dataTypeName" errors on statistics routes: sampleSet:polymerize does not support a groupByTime-aggregated query for every data type (confirmed live: "com.huawei.resting_calories does not support the query mode, please use dailyPolymerize API"). Per the official REST API reference for "Querying Sampling Data Statistics of Multiple Days", the day-aggregated variant of a data type must instead go through the dedicated POST /healthkit/v2/sampleSet:dailyPolymerize endpoint, which takes a startDay/endDay (yyyyMMdd) + timeZone request body and returns a differently-shaped, doubly-nested group[].sampleSet[].samplePoints[] response (with group-level times in milliseconds but sample-point times in nanoseconds). Adds HuaweiDailyPolymerizeRoute/HuaweiDailyPolymerizeConverter and routes every "*.statistics" definition through it instead of HuaweiSampleSetRoute, which now only handles raw (non-statistics) data types and has had its now-dead groupByTime support removed. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../HuaweiDailyPolymerizeConverter.kt | 70 ++++++++++++++ .../route/HuaweiDailyPolymerizeRoute.kt | 96 +++++++++++++++++++ .../org/radarbase/huawei/route/HuaweiRoute.kt | 9 +- .../huawei/route/HuaweiRouteFactory.kt | 33 ++++--- .../huawei/route/HuaweiSampleSetRoute.kt | 19 ++-- .../huawei/route/HuaweiRouteFactoryTest.kt | 17 ++++ 6 files changed, 218 insertions(+), 26 deletions(-) create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDailyPolymerizeConverter.kt create mode 100644 huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiDailyPolymerizeRoute.kt diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDailyPolymerizeConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDailyPolymerizeConverter.kt new file mode 100644 index 00000000..3e4d5d72 --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiDailyPolymerizeConverter.kt @@ -0,0 +1,70 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.huawei.converter + +import com.fasterxml.jackson.databind.JsonNode +import org.apache.avro.specific.SpecificRecord +import org.radarbase.huawei.user.User +import java.time.Instant + +/** Sample points inside `sampleSet:dailyPolymerize`'s response report their times in nanoseconds. */ +private fun JsonNode.epochNanoInstant(field: String): Instant? { + val value = this.get(field) ?: return null + if (value.isNull) return null + val nanos = if (value.isTextual) value.asText().toLongOrNull() else value.asLong() + return nanos?.let { Instant.ofEpochSecond(it / 1_000_000_000L, it % 1_000_000_000L) } +} + +/** + * Converter for `POST /healthkit/v2/sampleSet:dailyPolymerize` responses: unlike + * `sampleSet:polymerize`, each day's result is wrapped in a `group[]` entry containing its own + * `sampleSet[].samplePoints[]`, so this walks two levels of nesting instead of one before reaching + * the same `{"fieldName": ..., "value": ...}` point shape used elsewhere. + * + * @author yatharthranjan + */ +class HuaweiDailyPolymerizeConverter( + private val topic: String, + private val buildRecord: ( + fields: FieldValues, + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + ) -> SpecificRecord, +) : HuaweiDataConverter { + + override fun processRecords(root: JsonNode, user: User): Sequence> { + val timeReceived = Instant.now() + val groups = root.get("group") ?: return emptySequence() + return groups.asSequence() + .flatMap { group -> group.get("sampleSet")?.asSequence() ?: emptySequence() } + .flatMap { sampleSet -> sampleSet.get("samplePoints")?.asSequence() ?: emptySequence() } + .mapCatching { point -> + val startTime = point.epochNanoInstant("startTime") + ?: error("Huawei daily polymerize sample point is missing startTime") + val endTime = point.epochNanoInstant("endTime") + val fieldValues = FieldValues.from(point.get("value")) + TopicData( + topic = topic, + key = user.observationKey, + offset = startTime.epochSecond, + value = buildRecord(fieldValues, startTime, endTime, timeReceived), + ) + } + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiDailyPolymerizeRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiDailyPolymerizeRoute.kt new file mode 100644 index 00000000..dafbdd8c --- /dev/null +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiDailyPolymerizeRoute.kt @@ -0,0 +1,96 @@ +/* + * Copyright 2026 Onsentia + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.radarbase.huawei.route + +import com.fasterxml.jackson.databind.ObjectMapper +import org.apache.avro.specific.SpecificRecord +import org.radarbase.huawei.converter.FieldValues +import org.radarbase.huawei.converter.HuaweiDailyPolymerizeConverter +import org.radarbase.huawei.converter.HuaweiDataConverter +import org.radarbase.huawei.request.RestRequest +import org.radarbase.huawei.user.User +import org.radarbase.huawei.user.UserRepository +import java.time.Duration +import java.time.Instant +import java.time.ZoneOffset +import java.time.format.DateTimeFormatter + +/** + * Route backed by `POST /healthkit/v2/sampleSet:dailyPolymerize`, used for every Huawei + * `.statistics` data type. + * + * Huawei's `sampleSet:polymerize` endpoint (see [HuaweiSampleSetRoute]) does not accept a + * `groupByTime`-aggregated query for every data type - some (confirmed live: `resting_calories`) + * reject it with `"does not support the query mode, please use dailyPolymerize API"`. This route + * calls that dedicated day-granularity statistics endpoint instead, which takes a day-string range + * (`startDay`/`endDay`, format `yyyyMMdd`, at most 31 days apart) rather than epoch timestamps. + * + * @author yatharthranjan + */ +open class HuaweiDailyPolymerizeRoute( + userRepository: UserRepository, + private val dataTypeName: String, + private val topic: String, + maxIntervalPerRequest: Duration = Duration.ofDays(30L), + buildRecord: ( + fields: FieldValues, + startTime: Instant, + endTime: Instant?, + timeReceived: Instant, + ) -> SpecificRecord, +) : HuaweiRoute(userRepository, maxIntervalPerRequest) { + + override val converters: List = + listOf(HuaweiDailyPolymerizeConverter(topic, buildRecord)) + + override fun toString(): String = "huawei_" + topic.removePrefix("connect_huawei_") + + override fun generateRequests( + user: User, + start: Instant, + end: Instant, + max: Int, + ): Sequence = chunkedRanges(start, end, max).map { (rangeStart, rangeEnd) -> + RestRequest( + request = createPostRequest( + user, + "sampleSet:dailyPolymerize", + buildRequestBody(rangeStart, rangeEnd), + baseUrl = HUAWEI_API_BASE_URL_V2, + ), + user = user, + route = this, + startDate = rangeStart, + endDate = rangeEnd, + ) + } + + private fun buildRequestBody(start: Instant, end: Instant): String { + val root = MAPPER.createObjectNode() + root.putArray("dataTypes").add(dataTypeName) + root.put("startDay", DAY_FORMATTER.format(start)) + root.put("endDay", DAY_FORMATTER.format(end)) + root.put("timeZone", "+0000") + return MAPPER.writeValueAsString(root) + } + + companion object { + private val MAPPER = ObjectMapper() + private val DAY_FORMATTER = DateTimeFormatter.ofPattern("yyyyMMdd").withZone(ZoneOffset.UTC) + } +} diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt index 81f94a32..9c7cd11e 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRoute.kt @@ -59,10 +59,15 @@ abstract class HuaweiRoute( .build() } - protected fun createPostRequest(user: User, path: String, jsonBody: String): Request { + protected fun createPostRequest( + user: User, + path: String, + jsonBody: String, + baseUrl: String = HUAWEI_API_BASE_URL, + ): Request { val accessToken = userRepository.getAccessToken(user) return Request.Builder() - .url("$HUAWEI_API_BASE_URL/$path".toHttpUrl()) + .url("$baseUrl/$path".toHttpUrl()) .header("Authorization", "Bearer $accessToken") .post(jsonBody.toRequestBody(JSON_MEDIA_TYPE)) .build() diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index d98c7fa9..f9f0b85e 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -984,17 +984,28 @@ object HuaweiRouteFactory { timeReceived: Instant, ) -> SpecificRecord, ): HuaweiRouteDefinition = HuaweiRouteDefinition(key, defaultTopic) { repo, topic -> - HuaweiSampleSetRoute( - userRepository = repo, - // Huawei's polymerize API has no dataCollector for a literal "*.statistics" data - // type - ".statistics" is only this connector's/RADAR-Schemas' label for "the - // groupByTime-aggregated variant of the underlying raw data type", so it must be - // stripped from the dataTypeName actually sent on the wire. - dataTypeName = VENDOR_PREFIX + dataTypeSuffix.removeSuffix(".statistics"), - topic = topic, - groupByTimeUnit = if (dataTypeSuffix.endsWith(".statistics")) "day" else null, - buildRecord = buildRecord, - ) + // Huawei's polymerize API rejects a groupByTime-aggregated query for at least some data + // types (confirmed live: "com.huawei.resting_calories does not support the query mode, + // please use dailyPolymerize API") - the day-aggregated ("*.statistics") variant of every + // data type must go through the dedicated sampleSet:dailyPolymerize endpoint instead, using + // the underlying raw data type name (the ".statistics" suffix is only this + // connector's/RADAR-Schemas' label and is never sent on the wire). + val rawDataTypeName = VENDOR_PREFIX + dataTypeSuffix.removeSuffix(".statistics") + if (dataTypeSuffix.endsWith(".statistics")) { + HuaweiDailyPolymerizeRoute( + userRepository = repo, + dataTypeName = rawDataTypeName, + topic = topic, + buildRecord = buildRecord, + ) + } else { + HuaweiSampleSetRoute( + userRepository = repo, + dataTypeName = rawDataTypeName, + topic = topic, + buildRecord = buildRecord, + ) + } } private fun healthRecordDefinition( diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt index 664e80a7..3e15485e 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiSampleSetRoute.kt @@ -30,13 +30,13 @@ import java.time.Instant /** * Route backed by `POST /healthkit/v1/sampleSet:polymerize`, which covers the large majority of - * Huawei Health Kit data types (all `continuous.*`, `instantaneous.*`, `cgm_blood_glucose`, - * `active_hours`, `daily_activity_summary`, `emotion`, `heart_rate_variability`, `vo2max`, - * `resting_calories.statistics`, `sleep.on_off_bed_record`, and `sleep_respiratory_*` types). + * raw (non-`.statistics`) Huawei Health Kit data types (all `continuous.*`, `instantaneous.*`, + * `cgm_blood_glucose`, `active_hours`, `daily_activity_summary`, `emotion`, + * `heart_rate_variability`, `vo2max`, `sleep.on_off_bed_record`, and `sleep_respiratory_*` types). + * Returns raw, un-aggregated sample points for [dataTypeName] over the requested time range. * - * When [groupByTimeUnit] is set, the request aggregates sample points into buckets of that size — - * this is how Huawei's `.statistics` data types are queried. When it is `null`, the endpoint - * returns raw, un-aggregated sample points for [dataTypeName] over the requested time range. + * The day-aggregated `.statistics` variant of a data type is not queried through this route - + * see [HuaweiDailyPolymerizeRoute]. * * @author yatharthranjan */ @@ -44,7 +44,6 @@ open class HuaweiSampleSetRoute( userRepository: UserRepository, private val dataTypeName: String, private val topic: String, - private val groupByTimeUnit: String? = null, maxIntervalPerRequest: Duration = Duration.ofDays(30L), buildRecord: ( fields: FieldValues, @@ -83,12 +82,6 @@ open class HuaweiSampleSetRoute( root.putArray("polymerizeWith").addObject().put("dataTypeName", dataTypeName) root.put("startTime", start.toEpochMilli()) root.put("endTime", end.toEpochMilli()) - if (groupByTimeUnit != null) { - val groupPeriod = root.putObject("groupByTime").putObject("groupPeriod") - groupPeriod.put("unit", groupByTimeUnit) - groupPeriod.put("value", 1) - groupPeriod.put("timeZone", "+0000") - } return MAPPER.writeValueAsString(root) } diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt index dd7aa8e3..48e05947 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -95,6 +95,7 @@ class HuaweiRouteFactoryTest { private fun fixtureFor(route: HuaweiRoute) = when (route) { is HuaweiActivityRecordRoute -> activityRecordFixture() is HuaweiHealthRecordRoute -> healthRecordFixture() + is HuaweiDailyPolymerizeRoute -> dailyPolymerizeFixture() is HuaweiSampleSetRoute -> sampleSetFixture() else -> error("Unknown route type: ${route::class}") } @@ -111,6 +112,22 @@ class HuaweiRouteFactoryTest { return root } + private fun dailyPolymerizeFixture(): ObjectNode { + val root = mapper.createObjectNode() + val groups = root.putArray("group") + val group = groups.addObject() + group.put("startTime", START_MILLIS) + group.put("endTime", END_MILLIS) + val sampleSet = group.putArray("sampleSet") + val collector = sampleSet.addObject() + val samplePoints = collector.putArray("samplePoints") + val point = samplePoints.addObject() + point.put("startTime", START_NANOS) + point.put("endTime", END_NANOS) + point.set("value", genericValueArray()) + return root + } + private fun healthRecordFixture(): ObjectNode { val root = mapper.createObjectNode() val records = root.putArray("healthRecords") From da4ddfd45bae969a9167d44aaf973d08dbd14f0d Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Thu, 6 Aug 2026 10:48:37 +0000 Subject: [PATCH 23/27] Fix activityRecords route against the official REST API reference Per Huawei's "Querying Created Exercise Records" spec: - The endpoint is on API version v2, not v1. - The response's array of records is under the key "activityRecord" (singular), not "activityRecords" - we were reading the wrong key, so every activity_record request was silently producing zero records regardless of what the API actually returned, with no error logged. - Each record's description field is "desc", not "description". Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../huawei/converter/HuaweiActivityRecordConverter.kt | 9 ++++++--- .../radarbase/huawei/route/HuaweiActivityRecordRoute.kt | 3 ++- .../org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt | 4 ++-- 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt index 26248540..e70c6b63 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/converter/HuaweiActivityRecordConverter.kt @@ -23,7 +23,7 @@ import org.radarcns.connector.huawei.HuaweiActivityRecord import java.time.Instant /** - * Converts `GET /healthkit/v1/activityRecords` responses into [HuaweiActivityRecord]s. + * Converts `GET /healthkit/v2/activityRecords` responses into [HuaweiActivityRecord]s. * * Field names below follow the Huawei Health Kit `ActivityRecord`/`Device`/`ActivitySummary` * model (activity record id, name, description, time zone, activity type, device manufacturer and @@ -39,7 +39,10 @@ class HuaweiActivityRecordConverter( override fun processRecords(root: JsonNode, user: User): Sequence> { val timeReceived = Instant.now() - val records = root.get("activityRecords") ?: root.get("records") ?: return emptySequence() + val records = root.get("activityRecord") + ?: root.get("activityRecords") + ?: root.get("records") + ?: return emptySequence() return records.asSequence() .mapCatching { record -> val startTime = record.epochInstant("startTime") @@ -65,7 +68,7 @@ class HuaweiActivityRecordConverter( endTime = epochInstant("endTime")?.toEpoch() activityRecordId = textOrNull("id") ?: textOrNull("activityRecordId") name = textOrNull("name") - description = textOrNull("description") + description = textOrNull("desc") ?: textOrNull("description") timeZone = textOrNull("timeZone") activityTypeId = textOrNull("activityType") ?: textOrNull("activityTypeId") activeTimeMillis = longOrNull("activeTime") ?: longOrNull("activeTimeMillis") diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt index ebd75def..c8c8b347 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiActivityRecordRoute.kt @@ -26,7 +26,7 @@ import java.time.Duration import java.time.Instant /** - * Route backed by `GET /healthkit/v1/activityRecords`, covering the Huawei Health Kit Activity + * Route backed by `GET /healthkit/v2/activityRecords`, covering the Huawei Health Kit Activity * Records API (workout / physical-activity sessions). * * @author yatharthranjan @@ -56,6 +56,7 @@ class HuaweiActivityRecordRoute( "startTime" to rangeStart.toEpochMilli().toString(), "endTime" to rangeEnd.toEpochMilli().toString(), ), + baseUrl = HUAWEI_API_BASE_URL_V2, ), user = user, route = this, diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt index 48e05947..d7483909 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -140,13 +140,13 @@ class HuaweiRouteFactoryTest { private fun activityRecordFixture(): ObjectNode { val root = mapper.createObjectNode() - val records = root.putArray("activityRecords") + val records = root.putArray("activityRecord") val record = records.addObject() record.put("startTime", START_MILLIS) record.put("endTime", END_MILLIS) record.put("id", "activity-1") record.put("name", "Run") - record.put("description", "Morning run") + record.put("desc", "Morning run") record.put("timeZone", "Europe/London") record.put("activityType", "1") record.put("activeTime", 1000L) From 9c90c4ed96c2b9161b957bab02cad437f5af7f81 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Thu, 6 Aug 2026 11:01:23 +0000 Subject: [PATCH 24/27] Disable daily_activity_summary by default pending a redesign "com.huawei.daily_activity_summary" is not a real sampleSet dataTypeName (confirmed live: "no default dataCollector found"). The goal fields this route maps to actually belong to a separate endpoint (GET /healthkit/v2/sampleConfigs?type=9002&id=<...>, "Querying Activity Goals" - one call per goal type), while the achieved-value fields would need to come from the existing continuous/statistics routes. That needs a route that issues multiple requests and merges them, which is a real design decision rather than an endpoint/field-name fix like the other routes touched this session, so disable it by default (huawei.daily_activity_summary.enabled=false) rather than leave it erroring or guess at a composite implementation. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../huawei/route/HuaweiRouteFactory.kt | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index f9f0b85e..661b2d9e 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -222,10 +222,20 @@ object HuaweiRouteFactory { ) add( + // Disabled by default: "com.huawei.daily_activity_summary" is not a real Huawei + // sampleSet dataTypeName (confirmed live: "no default dataCollector found"). The + // goal fields this route maps (stepsGoal/activeCaloriesGoal/exerciseTimeGoal/ + // activeHoursGoal) actually belong to a completely different endpoint + // (GET /healthkit/v2/sampleConfigs?type=9002&id=<900200006..900200009>, "Querying + // Activity Goals"), and the achieved-value fields would need to come from the + // existing continuous/statistics routes instead. Needs a real redesign (a route that + // issues multiple requests and merges them) before this can work - not a simple + // endpoint/field-name fix like the other routes here. sampleSetDefinition( "daily_activity_summary", "daily_activity_summary", "connect_huawei_daily_activity_summary", + enabledByDefault = false, ) { f, start, end, received -> HuaweiDailyActivitySummary.newBuilder().apply { time = start.toEpoch() @@ -977,13 +987,18 @@ object HuaweiRouteFactory { key: String, dataTypeSuffix: String, defaultTopic: String, + enabledByDefault: Boolean = true, buildRecord: ( fields: FieldValues, startTime: Instant, endTime: Instant?, timeReceived: Instant, ) -> SpecificRecord, - ): HuaweiRouteDefinition = HuaweiRouteDefinition(key, defaultTopic) { repo, topic -> + ): HuaweiRouteDefinition = HuaweiRouteDefinition( + key, + defaultTopic, + enabledByDefault, + ) { repo, topic -> // Huawei's polymerize API rejects a groupByTime-aggregated query for at least some data // types (confirmed live: "com.huawei.resting_calories does not support the query mode, // please use dailyPolymerize API") - the day-aggregated ("*.statistics") variant of every From 89ea098b9ae55ee0f437763f615da76100e94532 Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Thu, 6 Aug 2026 11:15:57 +0000 Subject: [PATCH 25/27] Query *.total routes by their raw/delta data type via dailyPolymerize "*.total" is never a valid Huawei request dataTypeName (confirmed live: "no default dataCollector found for: com.huawei.continuous.steps.total"), matching the official dailyPolymerize doc's own example: it requests "com.huawei.continuous.steps.delta" and gets back a response labelled "com.huawei.continuous.steps.total". Adds queryDataTypeSuffix/ useDailyPolymerize overrides to sampleSetDefinition() and points continuous_steps_total, continuous_distance_total, and continuous_calories_burnt_total at their sibling raw/delta data type through the dailyPolymerize endpoint instead of requesting the ".total" name directly via plain polymerize. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../huawei/route/HuaweiRouteFactory.kt | 27 +++++++++++++------ 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index 661b2d9e..c9c98aed 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -422,6 +422,8 @@ object HuaweiRouteFactory { "continuous_calories_burnt_total", "continuous.calories.burnt.total", "connect_huawei_continuous_calories_burnt_total", + queryDataTypeSuffix = "continuous.calories.burnt", + useDailyPolymerize = true, ) { f, start, end, received -> HuaweiContinuousCaloriesBurntTotal.newBuilder().apply { time = start.toEpoch() @@ -451,6 +453,8 @@ object HuaweiRouteFactory { "continuous_distance_total", "continuous.distance.total", "connect_huawei_continuous_distance_total", + queryDataTypeSuffix = "continuous.distance.delta", + useDailyPolymerize = true, ) { f, start, end, received -> HuaweiContinuousDistanceTotal.newBuilder().apply { time = start.toEpoch() @@ -586,6 +590,8 @@ object HuaweiRouteFactory { "continuous_steps_total", "continuous.steps.total", "connect_huawei_continuous_steps_total", + queryDataTypeSuffix = "continuous.steps.delta", + useDailyPolymerize = true, ) { f, start, end, received -> HuaweiContinuousStepsTotal.newBuilder().apply { time = start.toEpoch() @@ -988,6 +994,17 @@ object HuaweiRouteFactory { dataTypeSuffix: String, defaultTopic: String, enabledByDefault: Boolean = true, + // Huawei's polymerize API rejects a groupByTime-aggregated query for at least some data + // types (confirmed live: "com.huawei.resting_calories does not support the query mode, + // please use dailyPolymerize API") - the day-aggregated ("*.statistics") variant of every + // data type must go through the dedicated sampleSet:dailyPolymerize endpoint instead. + // Likewise, "*.total" data types are never valid *request* dataTypeNames (confirmed live: + // "no default dataCollector found for: com.huawei.continuous.steps.total") - Huawei's own + // dailyPolymerize example queries the "*.delta" data type and gets a "*.total"-labelled + // response back, so a "*.total" route must override [queryDataTypeSuffix] to name its + // sibling raw/delta data type instead. + queryDataTypeSuffix: String = dataTypeSuffix.removeSuffix(".statistics"), + useDailyPolymerize: Boolean = dataTypeSuffix.endsWith(".statistics"), buildRecord: ( fields: FieldValues, startTime: Instant, @@ -999,14 +1016,8 @@ object HuaweiRouteFactory { defaultTopic, enabledByDefault, ) { repo, topic -> - // Huawei's polymerize API rejects a groupByTime-aggregated query for at least some data - // types (confirmed live: "com.huawei.resting_calories does not support the query mode, - // please use dailyPolymerize API") - the day-aggregated ("*.statistics") variant of every - // data type must go through the dedicated sampleSet:dailyPolymerize endpoint instead, using - // the underlying raw data type name (the ".statistics" suffix is only this - // connector's/RADAR-Schemas' label and is never sent on the wire). - val rawDataTypeName = VENDOR_PREFIX + dataTypeSuffix.removeSuffix(".statistics") - if (dataTypeSuffix.endsWith(".statistics")) { + val rawDataTypeName = VENDOR_PREFIX + queryDataTypeSuffix + if (useDailyPolymerize) { HuaweiDailyPolymerizeRoute( userRepository = repo, dataTypeName = rawDataTypeName, From 309b8074946fc6f1d2ef437321f3d8fd38bb1b8c Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Thu, 6 Aug 2026 11:23:27 +0000 Subject: [PATCH 26/27] Fix continuous_spo2_statistics data type name and field keys Per the official "SpO2" data type reference: the statistics data type is documented under the "continuous." namespace (com.huawei.continuous.spo2.statistics), but its underlying raw detailed data type is under a different namespace entirely (com.huawei.instantaneous.spo2, not com.huawei.continuous.spo2, which doesn't exist - matching the live "Invalid dataTypeName." error). Also corrects the field-value keys read from the response (saturation_avg/max/min/last, not the generic avg/max/min/last this route had guessed). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../radarbase/huawei/route/HuaweiRouteFactory.kt | 13 +++++++++---- .../huawei/route/HuaweiRouteFactoryTest.kt | 1 + 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index c9c98aed..07098e65 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -558,15 +558,20 @@ object HuaweiRouteFactory { "continuous_spo2_statistics", "continuous.spo2.statistics", "connect_huawei_continuous_spo2_statistics", + // Statistics variant is documented under "continuous.", but its underlying raw + // detailed data type is "com.huawei.instantaneous.spo2" - a different namespace, + // per the official "SpO2" data type reference. + queryDataTypeSuffix = "instantaneous.spo2", + useDailyPolymerize = true, ) { f, start, end, received -> HuaweiContinuousSpo2Statistics.newBuilder().apply { time = start.toEpoch() timeReceived = received.toEpoch() endTime = end?.toEpoch() - saturationAvg = f.getDouble("avg") - saturationMax = f.getDouble("max") - saturationMin = f.getDouble("min") - saturationLast = f.getDouble("last") + saturationAvg = f.getDouble("saturation_avg") + saturationMax = f.getDouble("saturation_max") + saturationMin = f.getDouble("saturation_min") + saturationLast = f.getDouble("saturation_last") }.build() }, ) diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt index d7483909..eb25e897 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -209,6 +209,7 @@ class HuaweiRouteFactoryTest { "min_breathe_rate", "min_breathrate_baseline", "min_spo2", "off_bed_time", "on_off_bed_state", "predicted_calories", "prepare_sleep_time", "record_day", "record_id", "remarks", "sample_source", "sampling_frequency", "sleep_efficiency", + "saturation_avg", "saturation_last", "saturation_max", "saturation_min", "sleep_latency", "sleep_score", "sleep_state", "sleep_type", "sphygmus_avg", "sphygmus_last", "sphygmus_max", "sphygmus_min", "status", "steps", "steps_delta", "steps_target", "sub_status", "systolic_pressure_avg", "systolic_pressure_max", From 1bf0b9e0e6d1e40ac6ace5a2ae8d1b6ebad1ac1e Mon Sep 17 00:00:00 2001 From: yatharthranjan Date: Thu, 6 Aug 2026 11:39:32 +0000 Subject: [PATCH 27/27] Fix daily_activity_summary, distance_total, altitude_statistics, active_hours Per the official "Daily Activity" data type reference pages: - daily_activity_summary is itself a real "Atomic Sampling Statistical Data Type" queried by day via dailyPolymerize - it's not a derived label needing a different raw type, and not the separate sampleConfigs-based "Workout Goals" endpoint as previously assumed. Re-enabled by default and switched to dailyPolymerize, with field keys corrected to the documented camelCase names (steps, activeCalories, exerciseTime, activeHours, stepsGoal, activeCaloriesGoal, exerciseTimeGoal, activeHoursGoal), matching the Avro schema's own field names. - continuous_distance_total's single field is "distance", not "distance_total". - continuous_altitude_statistics's underlying raw data type is "com.huawei.instantaneous.altitude" (a different namespace than its "continuous."-labelled statistics name), the same namespace mismatch already fixed for SpO2. - active_hours (raw) and active_hours_statistics were incorrectly sharing one builder function reading the same field keys, but they have genuinely different response shapes: the raw type's only field is "isActive", while the statistics type's only field is "activeHours" (with no moderate/high intensity minute breakdown on either, unlike what the shared builder assumed). Split into two builder functions with the correct field keys for each. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011w6G3vGzKQ4ogcLgtWwzun --- .../huawei/route/HuaweiRouteFactory.kt | 66 ++++++++++++------- .../huawei/route/HuaweiRouteFactoryTest.kt | 35 +++++----- 2 files changed, 60 insertions(+), 41 deletions(-) diff --git a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt index 07098e65..abff6274 100644 --- a/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt +++ b/huawei-library/src/main/kotlin/org/radarbase/huawei/route/HuaweiRouteFactory.kt @@ -222,33 +222,30 @@ object HuaweiRouteFactory { ) add( - // Disabled by default: "com.huawei.daily_activity_summary" is not a real Huawei - // sampleSet dataTypeName (confirmed live: "no default dataCollector found"). The - // goal fields this route maps (stepsGoal/activeCaloriesGoal/exerciseTimeGoal/ - // activeHoursGoal) actually belong to a completely different endpoint - // (GET /healthkit/v2/sampleConfigs?type=9002&id=<900200006..900200009>, "Querying - // Activity Goals"), and the achieved-value fields would need to come from the - // existing continuous/statistics routes instead. Needs a real redesign (a route that - // issues multiple requests and merges them) before this can work - not a simple - // endpoint/field-name fix like the other routes here. + // "com.huawei.daily_activity_summary" is itself a documented "Atomic Sampling + // Statistical Data Type" (per the official "Daily Activity Data" reference) queried by + // day via dailyPolymerize - it is not derived from a separate raw type, and it is NOT + // the separate sampleConfigs-based "Workout Goals" endpoint. Its field names are + // camelCase (matching the Avro schema field names directly), unlike most other Huawei + // data types' snake_case field names. sampleSetDefinition( "daily_activity_summary", "daily_activity_summary", "connect_huawei_daily_activity_summary", - enabledByDefault = false, + useDailyPolymerize = true, ) { f, start, end, received -> HuaweiDailyActivitySummary.newBuilder().apply { time = start.toEpoch() timeReceived = received.toEpoch() endTime = end?.toEpoch() steps = f.getInt("steps") - activeCalories = f.getInt("calories") - exerciseTime = f.getInt("exercise_time") - activeHours = f.getInt("active_hours") - stepsGoal = f.getInt("steps_target") - activeCaloriesGoal = f.getInt("calories_target") - exerciseTimeGoal = f.getInt("exercise_time_target") - activeHoursGoal = f.getInt("active_hours_target") + activeCalories = f.getInt("activeCalories") + exerciseTime = f.getInt("exerciseTime") + activeHours = f.getInt("activeHours") + stepsGoal = f.getInt("stepsGoal") + activeCaloriesGoal = f.getInt("activeCaloriesGoal") + exerciseTimeGoal = f.getInt("exerciseTimeGoal") + activeHoursGoal = f.getInt("activeHoursGoal") }.build() }, ) @@ -259,7 +256,7 @@ object HuaweiRouteFactory { "active_hours", "connect_huawei_active_hours", ) { f, start, end, received -> - f.toActiveHours(start, end, received) + f.toRawActiveHours(start, end, received) }, ) add( @@ -268,7 +265,7 @@ object HuaweiRouteFactory { "active_hours.statistics", "connect_huawei_active_hours_statistics", ) { f, start, end, received -> - f.toActiveHours(start, end, received) + f.toActiveHoursStatistics(start, end, received) }, ) @@ -296,6 +293,11 @@ object HuaweiRouteFactory { "continuous_altitude_statistics", "continuous.altitude.statistics", "connect_huawei_continuous_altitude_statistics", + // Statistics variant is documented under "continuous.", but its underlying raw + // detailed data type is "com.huawei.instantaneous.altitude" - a different + // namespace, per the official "Altitude" data type reference. + queryDataTypeSuffix = "instantaneous.altitude", + useDailyPolymerize = true, ) { f, start, end, received -> HuaweiContinuousAltitudeStatistics.newBuilder().apply { time = start.toEpoch() @@ -460,7 +462,7 @@ object HuaweiRouteFactory { time = start.toEpoch() timeReceived = received.toEpoch() endTime = end?.toEpoch() - distance = f.getDouble("distance_total") + distance = f.getDouble("distance") }.build() }, ) @@ -816,7 +818,25 @@ object HuaweiRouteFactory { ) } - private fun FieldValues.toActiveHours( + /** + * The raw `com.huawei.active_hours` data type's only documented field is `isActive` (whether + * that hour had at least moderate-intensity activity) - it has no moderate/high intensity + * minute breakdown, unlike what [toActiveHoursStatistics] reads. + */ + private fun FieldValues.toRawActiveHours( + start: Instant, + end: Instant?, + received: Instant, + ): HuaweiActiveHours = HuaweiActiveHours.newBuilder().apply { + time = start.toEpoch() + timeReceived = received.toEpoch() + endTime = end?.toEpoch() + activeHours = getInt("isActive") + }.build() + + /** The `com.huawei.active_hours.statistics` data type's only documented field is `activeHours` + * (the number of active hours in the statistical period). */ + private fun FieldValues.toActiveHoursStatistics( start: Instant, end: Instant?, received: Instant, @@ -824,9 +844,7 @@ object HuaweiRouteFactory { time = start.toEpoch() timeReceived = received.toEpoch() endTime = end?.toEpoch() - activeHours = getInt("active_hours") - moderateIntensityMinutes = getInt("moderate_intensity_minutes") - highIntensityMinutes = getInt("high_intensity_minutes") + activeHours = getInt("activeHours") }.build() private fun FieldValues.toContinuousActivityStatistics( diff --git a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt index eb25e897..53355c7b 100644 --- a/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt +++ b/huawei-library/src/test/kotlin/org/radarbase/huawei/route/HuaweiRouteFactoryTest.kt @@ -196,27 +196,28 @@ class HuaweiRouteFactoryTest { private const val END_NANOS = END_MILLIS * 1_000_000L private val LITERAL_FIELD_KEYS = listOf( - "active_hours", "active_hours_target", "all_sleep_time", "arrhythmia_result", + "active_hours", "active_hours_target", "activeCalories", "activeHours", + "activeCaloriesGoal", "activeHoursGoal", "all_sleep_time", "arrhythmia_result", "arrhythmia_type", "ascent_total", "avg", "avg_breathe_rate", "avg_heart_rate", "awake_time", "calories", "calories_target", "calories_total", "correlate_mealtime", "correlate_sleep", "count", "deep_sleep_part", "deep_sleep_time", "descent_total", "diastolic_pressure_avg", "diastolic_pressure_max", "diastolic_pressure_min", - "distance_delta", "distance_total", "dream_time", "duration", "emotion", "event_name", - "exercise_time", "exercise_time_target", "exercise_type", "extend_data", - "fall_asleep_time", "go_bed_time", "heart_rate_variability_rmssd", - "high_body_temperature_alarm", "last", "level", "light_sleep_time", "max", - "max_breathe_rate", "max_breathrate_baseline", "max_spo2", "meal", "min", - "min_breathe_rate", "min_breathrate_baseline", "min_spo2", "off_bed_time", - "on_off_bed_state", "predicted_calories", "prepare_sleep_time", "record_day", - "record_id", "remarks", "sample_source", "sampling_frequency", "sleep_efficiency", - "saturation_avg", "saturation_last", "saturation_max", "saturation_min", - "sleep_latency", "sleep_score", "sleep_state", "sleep_type", "sphygmus_avg", - "sphygmus_last", "sphygmus_max", "sphygmus_min", "status", "steps", "steps_delta", - "steps_target", "sub_status", "systolic_pressure_avg", "systolic_pressure_max", - "systolic_pressure_min", "threshold", "timezone", "total_calories", "type", - "user_symptom", "value", "vo2max", "voltage_data", "wakeup_count", "wakeup_time", - "zone1_duration", "zone2_duration", "zone3_duration", "zone4_duration", - "zone5_duration", + "distance", "distance_delta", "distance_total", "dream_time", "duration", "emotion", + "event_name", "exercise_time", "exercise_time_target", "exerciseTime", + "exerciseTimeGoal", "exercise_type", "extend_data", "fall_asleep_time", "go_bed_time", + "heart_rate_variability_rmssd", "high_body_temperature_alarm", "isActive", "last", + "level", "light_sleep_time", "max", "max_breathe_rate", "max_breathrate_baseline", + "max_spo2", "meal", "min", "min_breathe_rate", "min_breathrate_baseline", "min_spo2", + "off_bed_time", "on_off_bed_state", "predicted_calories", "prepare_sleep_time", + "record_day", "record_id", "remarks", "sample_source", "sampling_frequency", + "sleep_efficiency", "saturation_avg", "saturation_last", "saturation_max", + "saturation_min", "sleep_latency", "sleep_score", "sleep_state", "sleep_type", + "sphygmus_avg", "sphygmus_last", "sphygmus_max", "sphygmus_min", "status", "steps", + "steps_delta", "steps_target", "stepsGoal", "sub_status", "systolic_pressure_avg", + "systolic_pressure_max", "systolic_pressure_min", "threshold", "timezone", + "total_calories", "type", "user_symptom", "value", "vo2max", "voltage_data", + "wakeup_count", "wakeup_time", "zone1_duration", "zone2_duration", "zone3_duration", + "zone4_duration", "zone5_duration", ) } }