Skip to content

feat: AJDA-2445 add command to migrate data-apps orchestrator/flow tasks to data-app-control#101

Open
sykora-ji wants to merge 2 commits into
mainfrom
sykorajiri-AJDA-2445
Open

feat: AJDA-2445 add command to migrate data-apps orchestrator/flow tasks to data-app-control#101
sykora-ji wants to merge 2 commits into
mainfrom
sykorajiri-AJDA-2445

Conversation

@sykora-ji

Copy link
Copy Markdown

Link to issue

Description

Adds a new manage:migrate-data-apps-orchestrator-tasks command to cli-utils that rewrites orchestration/flow tasks referencing the legacy keboola.data-apps component so they use keboola.data-app-control instead.

  • DataAppOrchestratorTaskMigrator (new): given a Storage API Components client, lists all configurations of both keboola.orchestrator (legacy orchestrations) and keboola.flow (conditional flows — verified on a live stack to be a distinct component, not a variant of keboola.orchestrator), scans each configuration's tasks[] for tasks pointing at keboola.data-apps, resolves the app ID from either inline configData or a referenced configId, and rewrites the task's componentId to keboola.data-app-control with parameters.appId set — while preserving every other field on the task object (type, mode, delay, retry, variableOverrides) unchanged, since the keboola.flow schema requires type alongside componentId/mode. Tasks whose keboola.data-apps parameters carry an unsupported task value (anything other than the implicit/app-start/start variants used to launch an app) are left untouched and reported separately, so unrelated task types (create/delete/terminate/restore/...) are never silently mistransformed. Already-migrated tasks are skipped, making re-runs idempotent.
  • MigrateDataAppsOrchestratorTasks (new command): accepts a Manage API token, stack URL, and either a comma-separated list of project IDs or all. For each target project it mints a short-lived, scoped Storage API token via createProjectStorageToken() and runs the migrator against it; failures on one project are caught and logged without aborting the run. Defaults to dry-run (only reports what would change); --force/-f performs the actual updateConfiguration calls with a changeDescription. Prints a summary of projects/configurations/tasks scanned, migrated, and skipped.
  • Registered in cli.php and documented in README.md under "Project manipulation".

Manually verified end-to-end against a test flow on the canary-orion stack (dry-run, force run, idempotent re-run, and schema validity after the rewrite) before opening this PR.

Release Notes

  • Justification
    • The platform is moving data-app orchestration/flow tasks from the legacy keboola.data-apps component to the newer keboola.data-app-control component. Orchestrations and conditional flows created before this change still reference the legacy component, so a one-off bulk-migration tool is needed to move them over before the legacy component/UI path can be retired.
    • No behavior change for end users of Keboola Connection — this is an internal operations tool used by the AJDA team to update stored configurations; the new component performs the same "start app" action as the old one.
  • Plans for Customer Communication
    • No customer communication needed. This is an internal maintenance tool, not a customer-facing feature or behavioral change.
  • Impact Analysis
    • The tool only mutates orchestrator/flow configurations that already reference keboola.data-apps; everything else is left untouched. Tasks with an unrecognized keboola.data-apps parameter shape are skipped rather than guessed at, to avoid silently breaking a flow.
    • Not behind a feature flag — this is a manually-triggered CLI command run against a chosen project or stack, not part of the platform's request path, so there is no partial rollout state. Single-tenant setups are affected the same way as any other project: only if they have an orchestration/flow with a keboola.data-apps task.
  • Deployment Plan
    • No special rollout — merging deploys the updated cli-utils Docker image via the existing CI pipeline (tag push). The command itself is then run manually by an engineer, per project or per stack, as needed; it is not invoked automatically.
  • Rollback Plan
    • The tool change itself can be reverted/redeployed like any other cli-utils change. The configuration rewrites it performs are not automatically reversible, but each affected orchestrator/flow configuration keeps its Storage API version history (with the changeDescription set by this command), so any specific config can be rolled back to a prior version via Storage API if needed.
  • Post-Release Support Plan
    • No automated monitoring needed since the command is run on demand. [TODO: confirm whether Support needs to be notified before/after running this against production projects, and whether there is a tracked list of affected projects.]

…sks to data-app-control

Adds manage:migrate-data-apps-orchestrator-tasks, which rewrites orchestration
and conditional-flow tasks pointing at the legacy keboola.data-apps component
so they use keboola.data-app-control instead. Supports a single project or an
entire stack via a Manage token, defaults to dry-run, and is idempotent.
@linear-code

linear-code Bot commented Jul 9, 2026

Copy link
Copy Markdown

AJDA-2445

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an operational CLI command to bulk-migrate orchestrator/flow tasks from the legacy keboola.data-apps component to keboola.data-app-control, with a dry-run default and an opt-in force mode to persist changes via Storage API.

Changes:

  • Introduces manage:migrate-data-apps-orchestrator-tasks command to iterate selected or all projects, mint temporary Storage tokens, and run the migration (dry-run by default; --force updates configs).
  • Adds DataAppOrchestratorTaskMigrator to scan keboola.orchestrator and keboola.flow configurations, rewrite eligible tasks, and summarize migrated vs skipped tasks.
  • Registers the command in cli.php and documents usage/behavior in README.md.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
src/Keboola/Console/Command/MigrateDataAppsOrchestratorTasks.php New Symfony Console command wiring Manage API + Storage API access, project selection, dry-run/force execution, and summary output.
src/Keboola/Console/Command/DataAppOrchestratorTaskMigrator.php Implements the config/task scanning and rewrite logic for orchestrator/flow configurations.
README.md Documents the new command under “Project manipulation”, including args/options and behavior.
cli.php Registers the new command in the CLI application bootstrap.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/Keboola/Console/Command/MigrateDataAppsOrchestratorTasks.php
Comment thread src/Keboola/Console/Command/DataAppOrchestratorTaskMigrator.php
Comment thread src/Keboola/Console/Command/MigrateDataAppsOrchestratorTasks.php Outdated
Validate the projects argument strictly instead of silently dropping
invalid IDs, catch Storage API errors when resolving a configId reference
so one bad reference doesn't abort the whole project, cache resolved
configIds to avoid repeated lookups, and fix the "Checked N projects"
summary to include disabled/errored projects.
@sykora-ji sykora-ji requested a review from MiroCillik July 9, 2026 11:11
@sykora-ji sykora-ji marked this pull request as ready for review July 9, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants