Use vcpkg for dependency management#1964
Conversation
Replace the custom scripts/update_deps.py dependency fetcher with vcpkg in manifest mode for Vulkan-Headers, googletest, and (on Windows) detours. BUILD.md is updated to document the vcpkg-based setup. Also fixes scripts/gn/gn.py to use subprocess.run(check=True) instead of subprocess.call, a drive-by fix unrelated to vcpkg: the previous calls silently ignored non-zero exit codes. Caught this during my work on vcpkg to make sure I didn't accidentally break the chromium build.
|
Author jpr42 not on autobuild list. Waiting for curator authorization before starting CI build. |
1 similar comment
|
Author jpr42 not on autobuild list. Waiting for curator authorization before starting CI build. |
|
we are about to release a SDK in 2-ish weeks, I am curious if we want to get these all ready and merge it in after... not sure on the implication it will have (cc @richard-lunarg) |
|
For the loader the impact should be negligible. The SDK doesn't use update_deps.py IIRC. |
|
Having had a quick look through the changes, the big issue blocking this will be downstream consumers of Vulkan-Loader. Because this yeet's the update_deps.py script, any consumer of Vulkan-Loader that uses |
Correct, however it does use a script that knows the dependencies between each repo then propagates each repo's dependencies using |
For context those various |
| # the builtin-baseline in vcpkg.json. | ||
| # | ||
| # IMHO this is less work moving forward than trying to maintain a separate overlay for vulkan-headers, | ||
| # and it will also make it easier to use the vcpkg port of vulkan-headers in other projects. |
There was a problem hiding this comment.
- @mikes-lunarg since this will (potentially) impact how vulkan-headers is updated in the future.
Here is an example PR up updating upstream vulkan-headers in vcpkg itself:
microsoft/vcpkg#29066
Technically you can just keep on maintaining this custom portfile.
I'm just pointing out that it might be easier to update upstream the builtin-baseline see vcpkg.json.
That way other dependencies like google-test are periodically getting updated as well.
It would also be a nice to have for the entire open source community.
| # vcpkg manifest-mode install directory (when not nested under a build/ dir) | ||
| /vcpkg_installed/ | ||
|
|
There was a problem hiding this comment.
See the codegen CI job for why this was added.
| -D BUILD_TESTS=ON \ | ||
| -D UPDATE_DEPS=ON \ | ||
| -D VCPKG_MANIFEST_FEATURES=tests \ | ||
| -D "CMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_ROOT }}/scripts/buildsystems/vcpkg.cmake" \ |
There was a problem hiding this comment.
I think we can reduce some code duplication by setting CMAKE_TOOLCHAIN_FILE as an environment variable instead.
There was a problem hiding this comment.
I tested to make sure binary caching was working well on my fork. The benefits should become clear over time. Especially for more complex repos like VVL.
| else() | ||
| message(FATAL_ERROR "Could not find googletest directory. See BUILD.md") | ||
| endif() | ||
| find_package(GTest CONFIG REQUIRED) |
| target_compile_definitions(detours PUBLIC "_CRT_SECURE_NO_WARNINGS=1") | ||
| set_target_properties(detours PROPERTIES COMPILE_FLAGS /EHsc) | ||
| endif() | ||
| # vcpkg's detours port ships a prebuilt static lib + headers, not a CMake package config. |
There was a problem hiding this comment.
New detours code is much simpler.
| # value supplied this way, and the Makefile's comparisons are all uppercase. | ||
| string(TOUPPER "${VCPKG_TARGET_ARCHITECTURE}" DETOURS_TARGET_PROCESSOR) | ||
|
|
||
| vcpkg_build_nmake( |
There was a problem hiding this comment.
vcpkg has the ability to interact with build systems aside from CMake.
Making it easier than ever to just what you want.
There was a problem hiding this comment.
See official vcpkg documentation:
https://learn.microsoft.com/en-us/vcpkg/get_started/get-started?pivots=shell-powershell#1---set-up-vcpkg
Configuring the VCPKG_ROOT environment variable isn't just a random convention I made up.
|
Furthermore, if every package eventually gets updated into vcpkg proper. It would trivialize creating the vulkan-sdk. You can get a sense for this by looking at the codegen job: The idea being once everything is updated upstream in vcpkg proper you just run vcpkg to build everything easily. That's looking ahead in the future but I think it's a great idea. |
|
Also vcpkg will be much friendlier to other contributors compared to update_deps.py which is a highly custom workflow with various gotchas. |
|
And while I am more than happy to look after things for a while since life is going great right now. vcpkg is open source with good documentation. So there is plenty of resources on how to use it properly. Instead of hidden wisdom in python scripts that don't make sense anymore. |
Replace the custom scripts/update_deps.py dependency fetcher with vcpkg in manifest mode for Vulkan-Headers, googletest, and (on Windows) detours.
BUILD.md is updated to document the vcpkg-based setup.
Also fixes scripts/gn/gn.py to use subprocess.run(check=True) instead of subprocess.call, a drive-by fix unrelated to vcpkg: the previous calls silently ignored non-zero exit codes. Caught this during my work on vcpkg to make sure I didn't accidentally break the chromium build.