Skip to content

MINIFICPP-2859 Extend Conan usage and integrate it in Windows and MacOS CI jobs#2212

Open
lordgamez wants to merge 1 commit into
apache:mainfrom
lordgamez:MINIFICPP-2859
Open

MINIFICPP-2859 Extend Conan usage and integrate it in Windows and MacOS CI jobs#2212
lordgamez wants to merge 1 commit into
apache:mainfrom
lordgamez:MINIFICPP-2859

Conversation

@lordgamez

Copy link
Copy Markdown
Contributor
  • Add conan support to bootstrap script
  • Add conan support for the following libraries: argparse, jemalloc, libsodium, Libssh2, benchmark, gsl-lite, json-schema-validator, mimalloc, pugixml, yaml-cpp, prometheus, range-v3, liblzma, paho-mqtt, asio, couchbase, librdkafka, magic_enum, open62541
  • Remove unused uvc library
  • Remove mbedtls dependency and use openssl instead in open62541
  • Disable with_tools option for rocksdb
  • Add conan CI integration to Windows and MacOS jobs

https://issues.apache.org/jira/browse/MINIFICPP-2859


Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

  • Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically main)?

  • Is your initial contribution a single, squashed commit?

For code changes:

  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file?
  • If applicable, have you updated the NOTICE file?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends MiNiFi C++’s Conan-based dependency workflow and integrates it into macOS and Windows CI, shifting more third-party dependency handling toward Conan recipes and consistent CMake target usage.

Changes:

  • Add/extend Conan recipes for multiple third-party dependencies (e.g., RocksDB, open62541, magic_enum, couchbase) and wire them into the build.
  • Refactor CMake integration to prefer Get* modules and namespaced imported targets (e.g., yaml-cpp::yaml-cpp, gsl-lite::gsl-lite, etc.).
  • Update bootstrap and CI workflows to run conan install and incorporate Conan-specific compatibility handling.

Reviewed changes

Copilot reviewed 90 out of 93 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
thirdparty/rocksdb/config.yml Updates RocksDB thirdparty version mapping and adds license header.
thirdparty/rocksdb/all/conanfile.py Adjusts RocksDB Conan build options (e.g., disables tools, skips thirdparty).
thirdparty/rocksdb/all/conandata.yml Adds license header and updates source metadata for RocksDB.
thirdparty/open62541/open62541.patch Removes old patch location (moved under recipe patches).
thirdparty/open62541/config.yml Adds thirdparty version mapping for open62541.
thirdparty/open62541/all/patches/open62541.patch Centralizes open62541 patch used by Conan recipe/build.
thirdparty/open62541/all/conanfile.py Adds open62541 Conan recipe (now using OpenSSL).
thirdparty/open62541/all/conandata.yml Adds open62541 sources + patch metadata for Conan recipe.
thirdparty/magic-enum/config.yml Adds thirdparty version mapping for magic_enum.
thirdparty/magic-enum/all/conanfile.py Adds magic_enum Conan recipe.
thirdparty/magic-enum/all/conandata.yml Adds magic_enum source metadata for Conan recipe.
thirdparty/couchbase/config.yml Adds thirdparty version mapping for couchbase SDK.
thirdparty/couchbase/all/patches/use_fmt_instead_of_spdlog_fmt.patch Patch to prefer fmt headers instead of spdlog-bundled fmt.
thirdparty/couchbase/all/patches/remove-thirdparty.patch Patch to remove couchbase vendored spdlog/asio usage in favor of provided deps.
thirdparty/couchbase/all/patches/c++23_fixes.patch Patch to address C++23 incomplete-type/unique_ptr include strictness.
thirdparty/couchbase/all/ConanThirdPartyDependencies.cmake Adds a Conan-oriented dependency discovery module for couchbase build.
thirdparty/couchbase/all/conanfile.py Adds couchbase C++ client Conan recipe, wiring deps and CMake options.
thirdparty/couchbase/all/conandata.yml Adds couchbase sources + patch metadata for Conan recipe.
minifi-api/CMakeLists.txt Switches gsl-lite linkage to namespaced target.
minifi_main/CMakeLists.txt Updates linked dependency target names and argparse acquisition.
LICENSE Removes libuvc and mbedTLS notices consistent with dependency changes.
libminifi/test/schema-tests/CMakeLists.txt Switches JSON schema validator include to Get* module.
libminifi/CMakeLists.txt Updates core library link targets to namespaced targets and removes direct includes.
extensions/windows-event-log/CMakeLists.txt Updates pugixml target usage to namespaced target.
extensions/standard-processors/controllers/XMLRecordSetWriter.cpp Fixes pugixml text setting to use c_str() for string values.
extensions/standard-processors/CMakeLists.txt Updates range-v3/asio/pugixml/jsoncons integration to Get* and namespaced targets.
extensions/sftp/CMakeLists.txt Switches to GetLibSSH2 and namespaced libssh2 target.
extensions/prometheus/CMakeLists.txt Switches to GetPrometheus and links via unified target list variable.
extensions/opc/CMakeLists.txt Removes mbedTLS path and switches to GetOpen62541.
extensions/mqtt/CMakeLists.txt Switches to GetPahoMqttC and updates target name for linking.
extensions/libarchive/CMakeLists.txt Switches LZMA acquisition to GetLibLZMA.
extensions/kafka/rdkafka_utils.h Updates librdkafka header include path to include subdir form.
extensions/kafka/PublishKafka.h Updates librdkafka header include path.
extensions/kafka/KafkaConnection.h Updates librdkafka header include path.
extensions/kafka/ConsumeKafka.h Updates librdkafka header include path.
extensions/kafka/CMakeLists.txt Switches librdkafka acquisition to Getlibrdkafka and links via RdKafka::rdkafka++.
extensions/couchbase/CMakeLists.txt Switches to GetCouchbase and uses unified couchbase target name.
extension-framework/CMakeLists.txt Updates core dependency link list to namespaced targets.
encrypt-config/CMakeLists.txt Switches argparse acquisition to GetArgparse and namespaced targets.
core-framework/common/CMakeLists.txt Updates core-framework common deps to namespaced targets.
core-framework/CMakeLists.txt Updates core-framework deps to namespaced targets.
controller/CMakeLists.txt Switches argparse acquisition to GetArgparse and namespaced targets.
conanfile.py Expands Conan requirements/options and exports Conan-based CMake source selectors.
CONAN.md Updates Conan build documentation (bootstrap usage, export vs create, remote add).
CMakeLists.txt Refactors ccache launcher configuration, adds CMP0091 default, and adopts Get* modules + namespaced targets.
cmake/PugiXml.cmake Updates pugixml version/hash used by FetchContent build.
cmake/PahoMqttC.cmake Simplifies patching and removes old alias target for paho-mqtt-c.
cmake/Open62541.cmake Switches encryption to OpenSSL, updates patch path, and updates dependencies.
cmake/MiNiFiOptions.cmake Adds many new thirdparty “source” multi-options and removes SYSTEM option.
cmake/MbedTLS.cmake Removes bundled mbedTLS module.
cmake/LibLZMA.cmake Makes LibLZMA alias creation idempotent and adjusts module path handling.
cmake/Jsoncons.cmake Adds jsoncons::jsoncons alias if missing.
cmake/GetYamlCpp.cmake Adds Conan-vs-build selector wrapper for yaml-cpp.
cmake/GetSpdlog.cmake Makes spdlog alias creation idempotent in Conan mode.
cmake/GetRocksDB.cmake Makes RocksDB alias creation idempotent in Conan mode.
cmake/GetRangeV3.cmake Adds Conan-vs-build selector wrapper for range-v3.
cmake/GetPugiXml.cmake Adds Conan-vs-build selector wrapper for pugixml.
cmake/GetPrometheus.cmake Adds Conan-vs-build selector wrapper and consolidates targets list.
cmake/GetPahoMqttC.cmake Adds Conan-vs-build selector wrapper for paho-mqtt-c.
cmake/GetOpen62541.cmake Adds Conan-vs-build selector wrapper for open62541.
cmake/GetMiMalloc.cmake Adds Conan-vs-build selector wrapper for mimalloc.
cmake/GetMagicEnum.cmake Adds Conan-vs-build selector wrapper for magic_enum.
cmake/GetLibSSH2.cmake Adds Conan-vs-build selector wrapper for libssh2.
cmake/GetLibSodium.cmake Adds Conan-vs-build selector wrapper for libsodium.
cmake/Getlibrdkafka.cmake Adds Conan-vs-build selector wrapper for librdkafka.
cmake/GetLibLZMA.cmake Adds Conan-vs-build selector wrapper for liblzma.
cmake/GetJsonSchemaValidator.cmake Adds Conan-vs-build selector wrapper for json-schema-validator.
cmake/GetJsoncons.cmake Adds Conan-vs-build selector wrapper for jsoncons.
cmake/GetJemalloc.cmake Adds Conan-vs-build selector wrapper for jemalloc.
cmake/GetGslLite.cmake Adds Conan-vs-build selector wrapper for gsl-lite.
cmake/GetCouchbase.cmake Adds Conan-vs-build selector wrapper for couchbase C++ client.
cmake/GetCatch2.cmake Makes Catch2 alias creation idempotent in Conan mode.
cmake/GetBenchmark.cmake Adds Conan-vs-build selector wrapper for benchmark.
cmake/GetAsio.cmake Replaces mbedTLS export handling with Conan-vs-build selector for Asio.
cmake/GetArgparse.cmake Replaces unrelated FetchContent content with argparse Conan-vs-build selector.
cmake/FetchLibSSH2.cmake Adds Libssh2::libssh2 alias for consistent linking.
cmake/Fetchlibrdkafka.cmake Updates librdkafka version/hash and adds header install shim + alias.
cmake/Crc32c.cmake Makes crc32c alias creation idempotent.
cmake/Couchbase.cmake Updates Asio handling and patch paths; adds canonical target alias.
cmake/Couchbase_1_0_2.cmake Removes obsolete couchbase 1.0.2 build module.
cmake/CivetWeb.cmake Makes civetweb alias creation idempotent.
cmake/BundledRocksDB.cmake Disables RocksDB tools in bundled build.
cmake/BundledLibSodium.cmake Renames imported libsodium target to namespaced libsodium::libsodium.
cmake/BundledJemalloc.cmake Updates jemalloc source/version and renames imported target to jemalloc::jemalloc.
cmake/BundledAwsSdkCpp.cmake Normalizes CMake arg quoting/formatting for AWS SDK build.
cmake/BuildTests.cmake Switches yaml-cpp linkage in test targets to namespaced target.
cmake/Asio.cmake Adds asio::asio alias target.
bootstrap/requirements.txt Adds Conan and PyYAML to bootstrap Python dependencies.
bootstrap/package_manager.py Simplifies venv activation call in Windows environment setup.
bootstrap/minifi_option.py Adds USE_CONAN and custom malloc handling to bootstrap option flow.
bootstrap/cli.py Adds Conan install/export flow to bootstrap build orchestration.
.github/workflows/ci.yml Enables Conan in macOS/Windows CI, adds compatibility plugin, and uploads packages.
.github/conan/compatibility.py Adds a Conan compatibility extension for broader package reuse across settings.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread CMakeLists.txt
Comment thread cmake/GetMagicEnum.cmake
Comment thread cmake/GetRangeV3.cmake
Comment thread cmake/Fetchlibrdkafka.cmake Outdated
Comment thread bootstrap/cli.py
Comment thread .github/workflows/ci.yml
Comment thread .github/workflows/ci.yml
Comment thread thirdparty/open62541/all/conanfile.py
Comment thread cmake/PahoMqttC.cmake
@lordgamez lordgamez marked this pull request as ready for review July 8, 2026 07:48
…OS CI jobs

- Add conan support to bootstrap script
- Add conan support for the following libraries: argparse, jemalloc,
  libsodium, Libssh2, benchmark, gsl-lite, json-schema-validator,
  mimalloc, pugixml, yaml-cpp, prometheus, range-v3, liblzma, paho-mqtt,
  asio, couchbase, librdkafka, magic_enum, open62541
- Remove unused uvc library
- Remove mbedtls dependency and use openssl instead in open62541
- Disable with_tools option for rocksdb
- Add conan CI integration to Windows and MacOS jobs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants