feat(pypi): support importing uv.lock file by aignas · Pull Request #3785 · bazel-contrib/rules_python

aignas · 2026-05-16T17:07:35Z

Part of this is vibe coded, but I thought that the approach might have been rigorous
enough to submit a PR.

The strategy was:

First add a way for us to create a uv.lock file from the lock rule.
Then add a uv.lock file to JSON converter.
Then add a way to read the uv.lock file together with the requirements file
and verify things are OK.
Reuse most of the code.

Extra things that we could do:

Full test suite for various uv.lock scenarios and ensure parity with
requirements.txt files.
Call the PyPI index to understand if the packages are yanked or not - lock
file does not have that information.
Read the pyproject.toml file to get the index values for each package.

Summary:

feat(pypi): add uv.lock parsing support to parse_requirements
test(pypi): add tests for uv.lock parsing in parse_requirements
test(uv): add lock rule integration tests for uv.lock format
test: add uv_pypi end-to-end integration test
docs: add uv.lock documentation and sample files

Closes #3557
Work towards #2787

gemini-code-assist

Code Review

This pull request adds support for uv.lock files in rules_python, introducing a toml2json conversion utility and updating the lock rule and requirement parsing logic. The review identifies critical compatibility issues, specifically the toml2json tool's reliance on Python 3.11's tomllib and missing serialization for date/time objects. Furthermore, the feedback highlights logic errors in how package extras are handled—which could lead to dependency bloat or failed consistency checks—and suggests improvements for platform resolution and path handling in shell scripts.

gemini-code-assist · 2026-05-16T17:12:54Z

+    """Parse requirements using uv.lock as the primary source."""
+    ret = _parse_uv_lock_json(
+        uv_lock_json = uv_lock_json,
+        all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],


If requirements_by_platform is empty (which is common when using uv.lock as the primary source), all_platforms will be an empty list. This results in all packages having empty target_platforms, which will likely cause issues in downstream rules that expect platform information for wheel selection.

Suggested change

all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],

all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else sorted(platforms.keys()),

gemini-code-assist · 2026-05-16T17:12:55Z

+        for extra in pkg.get("provides-extras", pkg.get("extras", [])):
+            if extra not in entry["extras"]:
+                entry["extras"][extra] = None


Including all provides-extras in the requirement line for every package is likely incorrect. provides-extras lists all extras a package defines, not necessarily what was resolved or requested. Including all of them will force the installation of all optional dependencies for every package in the lock file, leading to significant dependency bloat. It might be better to omit extras from the requirement line if the lock file already provides the specific version and URL, or only include the extras that were part of the resolution.

- tomllib: try/except fallback to tomli for Python <3.11 - json_serializer: add datetime.date and datetime.time support - all_platforms: use sorted(platforms.keys()) fallback - $PWD/ path: check if python_path is absolute - extra_pip_args: pass through to _parse_uv_lock_json - Add uv_lock tests: multiple packages, extra_pip_args, multi-os - Update plan.md with review cycle instructions

aignas · 2026-05-16T18:33:12Z

Addressed all review comments. Please re-review when you get a chance. @gemini-code-assist

gemini-code-assist

Code Review

This pull request introduces support for uv.lock files in rules_python, adding a new lock_format attribute to the lock rule and a toml2json utility for parsing. It also implements a consistency check between uv.lock and requirements files. The reviewer identified a potential runtime error in the uv.lock parser due to unsafe dictionary access for optional hash keys and suggested simplifying redundant logic during hash extraction.

- Add python_3_14 toolchain in MODULE.bazel - Always use Python 3.14 interpreter for uv.lock JSON conversion - Remove tomli fallback from toml2json.py (Python 3.14 has tomllib) - Simplify convert_uv_lock_to_json to use fixed interpreter

- Add bazel-in-bazel integration test under tests/integration/uv_lock/ - Test converts uv.lock TOML to JSON using toml2json tool - Remove dead uv_pypi test data (placeholder only) - Restore tomli fallback in toml2json.py for backward compatibility

aignas · 2026-05-16T18:43:40Z

Addressed all review comments and added Python 3.14 toolchain + integration test. Please re-review. @gemini-code-assist

gemini-code-assist

Code Review

This pull request introduces support for uv.lock files as a primary metadata source in rules_python. Key additions include a toml2json tool for parsing, a new uv_lock format for the lock rule, and consistency checks within parse_requirements. Reviewer feedback suggests adopting more idiomatic Starlark dictionary access, implementing safer hash prefix stripping, and improving the portability of shell commands by replacing hardcoded environment variables with Bazel-provided paths.

jvolkman · 2026-05-17T14:39:43Z

The latest release of toml.bzl makes bazel_lib a dev dependency. It now uses skylib for the bzl_library rules.

Remove the old toml2json Python tool and uv_lock.bzl in favor of the pure Starlark toml.bzl decoder. Update tests to pass toml_decode mock, remove is_rules_python_root references, and fix virtual package test expectations. Clean up BUILD.bazel files that referenced deleted targets.

aignas · 2026-05-19T03:54:01Z

            ),
            logger = logger,
        ),
+        uv_lock = pip_attr.uv_lock,


Add a test to ensure this is tested.

… lock support Remove the lock_format attribute and detect whether to use uv lock or uv pip compile from the output file extension (.lock = uv lock, else requirements). Add a Windows bat template for uv lock support. The python interpreter is passed through --python flag consistently.

rickeylev

Overall LGTM. Some minor questions and nits. But overall, this fit in rather nicely with the pipstar code you've written, nice!

rickeylev · 2026-06-07T07:04:26Z

(agent comment) Addressed the second review cycle comments (enforcing uv.lock suffix, renaming _template attribute, restoring requirement parsing comments, and registering missing Stardoc TOML dependencies). Please re-review @gemini-code-assist @rickeylev

gemini-code-assist

Code Review

This pull request implements support for uv.lock files in rules_python, allowing them to be used as the primary source for package metadata and introducing a rule to generate them. The feedback identifies several issues: a potential path resolution failure in lock.bzl when the Python interpreter path is absolute, an undefined repository error caused by exposing virtual packages with no sources in the hub, and a version mismatch for the toml.bzl dependency between MODULE.bazel and py_repositories.bzl.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-06-07T07:07:02Z

+        item = struct(
+            name = norm_name,
+            is_exposed = True,
+            is_multiple_versions = len(versions) > 1,
+            # TODO @aignas 2026-05-17: use the default index that is used in parsing the
+            # requirements if it is not known in the uv.lock file. We need to get this from the
+            # pyproject.toml file uv.tool configuration.
+            index_url = "",
+            srcs = pkg_srcs,
+        )


Setting is_exposed = True for virtual packages (which have no sources, i.e., srcs = []) causes the hub repository to expose them as targets. However, since they have no sources, no corresponding whl_library repository is created for them. This leads to undefined repository errors (e.g., @hub_virtual_pkg not found) when Bazel attempts to resolve or query the hub's targets. We should set is_exposed to len(pkg_srcs) > 0 so that virtual packages without sources are not exposed in the hub.

Suggested change

item = struct(

name = norm_name,

is_exposed = True,

is_multiple_versions = len(versions) > 1,

# TODO @aignas 2026-05-17: use the default index that is used in parsing the

# requirements if it is not known in the uv.lock file. We need to get this from the

# pyproject.toml file uv.tool configuration.

index_url = "",

srcs = pkg_srcs,

)

item = struct(

name = norm_name,

is_exposed = len(pkg_srcs) > 0,

is_multiple_versions = len(versions) > 1,

# TODO @aignas 2026-05-17: use the default index that is used in parsing the

# requirements if it is not known in the uv.lock file. We need to get this from the

# pyproject.toml file uv.tool configuration.

index_url = "",

srcs = pkg_srcs,

)

gemini-code-assist · 2026-06-07T07:07:02Z

+        struct(
+            name = "virtual_pkg",
+            index_url = "",
+            is_exposed = True,
+            is_multiple_versions = False,
+            srcs = [],
+        ),


Update the unit test expectation to match the corrected is_exposed behavior for virtual packages (which should be False since they have no sources).

Suggested change

struct(

name = "virtual_pkg",

index_url = "",

is_exposed = True,

is_multiple_versions = False,

srcs = [],

),

struct(

name = "virtual_pkg",

index_url = "",

is_exposed = False,

is_multiple_versions = False,

srcs = [],

)

rickeylev · 2026-06-07T07:50:35Z

Addressed the remaining unresolved review threads (adopting str.removeprefix(), syncing toml.bzl to 0.4.1 in py_repositories.bzl, adding absolute path check in lock.bzl, formatting uv_lock docstrings with VERSION_NEXT_FEATURE, and removing temporary Python 3.14 toolchains). Please re-review @gemini-code-assist @rickeylev

gemini-code-assist

Code Review

This pull request introduces support for using uv.lock files as the primary source of package metadata in rules_python. It integrates a pure Starlark TOML decoder to parse uv.lock and extract packages, versions, extras, and sources (wheels, sdists, VCS, and direct URLs). Additionally, it updates the lock rule to support generating uv.lock files and adds comprehensive unit tests. The code review identified critical issues where the shell and batch scripts fail when copying uv.lock onto itself, a correctness bug where assigning all platforms to multiple versions of a package causes repository conflicts during hub building, and an issue where git revisions are not appended to VCS URLs, leading to unpinned dependencies.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-06-07T07:52:35Z

+                distribution = info["distribution"],
+                extra_pip_args = extra_pip_args or [],
+                requirement_line = requirement_line,
+                target_platforms = list(all_platforms),


Setting target_platforms = list(all_platforms) for all package versions in _parse_uv_lock introduces a correctness bug when a package has multiple versions in uv.lock (e.g., due to different Python versions or environment markers).

Because all versions of the package are assigned the exact same target_platforms, they will all generate the same config settings in _add_whl_library. This causes a conflict during the hub building phase, resulting in a build failure: attempting to override an existing repo '...' for config setting '...' with a new repo '...'.

To fix this, we should parse and evaluate the resolution-markers (or package-level markers) for each package version against the configured platforms to determine the correct subset of target_platforms for each version. If no markers are present, we can default to all_platforms.

rickeylev

Fixed a variety of comments

rickeylev · 2026-06-09T06:50:22Z

@aignas PTAL -- CI is happy! But the unresolved comments from Gemini I think might be valid?

- Fix the condition in lock() macro to match '.lock' suffix instead of 'uv.lock', so that uv.lock format files with custom names are handled correctly. - Add missing load for native_test in lock_tests.bzl - Rename uv_lock_expected.txt to uv_lock_expected.lock to match the output filename used by the uv_lock_test test.

aignas · 2026-06-20T04:43:18Z

test this please

aignas · 2026-06-20T09:37:15Z

It seems that the extras are defined in a different way than requirements.txt files and it means that we should handle it slightly differently from what we have today. The good news is that it will benefit us in many other ways, but the drawback is that it makes the uv.lock implementation a little bit more work.

The gist is that each package in the lock file has its dependencies encoded per extra. The good news is that we can just feed this dependency tree to the hub repo, then do the cycles resolution and then we don't need the requirements_cycles definition. #1975 had code how we could deal with this sort of stuff.

Since the dep graph is coming from uv.lock itself, we have the following options:

Generate the requirements lines based on what extras we are using in the uv.lock. This would allow us to easily plug into what we already have. However, implementing that is almost the same as export a uv.lock file to requirements.txt file. Which I assume may have some challenges. We could just invoke the uv in the repo phase to do this.
Generate one whl_library per Python dependency. Then we would need to do some smart stuff in order to handle cycles in the hub repository, but the main idea would be to actually have the dependency graph in the hub repository so that we can correctly select the minimum set of whl_library targets that have to be present here.

We could already do this if we read the whl_metadata from the SimpleAPI and pulled the RequirementsDist values for each package. That way we would not need to depend on the uv.lock format (which could change many times in the future). That would also mean that we could work the same way if py.lock file was used with pip.parse.

So this uv.lock file consumption only works if we:

Don't have extras usage in the lock file
convert the uv.lock file into requirements.txt in starlark or execute uv itself.

aignas added 9 commits May 17, 2026 01:56

feat(pypi): add toml2json tool for converting uv.lock TOML to JSON

a58cf50

test(pypi): add tests for toml2json tool

363c141

feat(pypi): add uv_lock utility for uv.lock to JSON conversion

6f4e925

feat(uv): update lock rule with uv.lock JSON conversion support

29a7993

feat(pypi): add uv.lock parsing support to parse_requirements

a063d53

test(pypi): add tests for uv.lock parsing in parse_requirements

9fea66c

test(uv): add lock rule integration tests for uv.lock format

aa19ceb

test: add uv_pypi end-to-end integration test

ee069e4

docs: add uv.lock documentation and sample files

dd53e29

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

aignas added 2 commits May 17, 2026 02:18

remove dead code and add plan.md for iterating

d5709ea

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

Comment thread python/private/pypi/parse_requirements.bzl Outdated

Comment thread python/private/pypi/parse_requirements.bzl Outdated

aignas added 3 commits May 17, 2026 03:38

docs: update plan.md with completed stages

6db3345

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

aignas added 2 commits May 18, 2026 22:11

wip

1f7b695

aignas commented May 19, 2026

View reviewed changes

aignas added 5 commits May 19, 2026 20:51

add deps for workspace

b46e0fd

fix the lock rule

be3012a

avoid eager loading of the hub repo

e3646a5

toml bump

9c1e30d

rickeylev approved these changes May 27, 2026

View reviewed changes

wip: merge main

41427dc

gemini-code-assist Bot reviewed Jun 7, 2026

View reviewed changes

rickeylev reviewed Jun 7, 2026

View reviewed changes

Comment thread python/private/py_repositories.bzl

Comment thread python/uv/private/lock.bzl Outdated

Comment thread python/private/pypi/extension.bzl

Comment thread python/private/pypi/parse_requirements.bzl

rickeylev force-pushed the aignas.feat.uv-lock branch 2 times, most recently from 7c98eb2 to 67c6b23 Compare June 8, 2026 07:45

rickeylev marked this pull request as ready for review June 9, 2026 06:48

This was referenced Jun 17, 2026

ci: Python dependency CVE scanning + Renovate pep621 + auto-export workflow Syndic/unnatural_designs#139

Merged

ci(renovate): stop Renovate from text-editing requirements_lock.txt Syndic/unnatural_designs#147

Merged

aignas added 5 commits June 20, 2026 12:02

wip

a165d8e

use artisanal intelligence to craft a better structure for the code

e3f6e9e

fixup

e2c0de2

fixup

7c7417d

move the templates to a separate dir

fad0083

aignas force-pushed the aignas.feat.uv-lock branch from a19879b to fad0083 Compare June 20, 2026 04:05

aignas added 2 commits June 20, 2026 13:13

fix(uv): align Windows .bat templates with Linux .sh behavior

0b57ff3

wip

f6be465

aignas marked this pull request as draft June 20, 2026 09:12

aignas added 3 commits June 20, 2026 18:40

fixup

0e0f871

add a converter of uv.lock to requirements.txt

8e18213

make the extras work

bb3054f

	all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],
	all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else sorted(platforms.keys()),

Uh oh!

Conversation

aignas commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aignas commented May 16, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

aignas commented May 16, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jvolkman commented May 17, 2026

Uh oh!

Uh oh!

aignas May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rickeylev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rickeylev commented Jun 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist Bot Jun 7, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rickeylev commented Jun 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

aignas commented May 16, 2026 •

edited

Loading

rickeylev commented Jun 7, 2026 •

edited

Loading