refactor(client)!: apply ruff style checks, raise minimum Python to 3.9#83
refactor(client)!: apply ruff style checks, raise minimum Python to 3.9#83d3flex wants to merge 11 commits into
Conversation
d63e45d to
8ed4e4d
Compare
|
Meh. Why? We already have pylint and black enforced via CI. I'm not really a fan of expending the substantial amount of time and effort it would take to review this purely to switch to a different one. What is the benefit to anybody? Sure, this is under the os-autoinst umbrella, but I wrote it initially as an external project and was never asked if I'm OK with it being subject to OpenQA Style Guides or whatever. This would take me like a day to review and cause a lot of that annoying thing where you do I'd be fine with a smaller PR that fixes concrete issues. |
okurz
left a comment
There was a problem hiding this comment.
Meh. Why?
We already have pylint and black enforced via CI. I'm not really a fan of expending the substantial amount of time and effort it would take to review this purely to switch to a different one. What is the benefit to anybody?
ruff is getting adopted in multiple projects and provides benefits of much quicker and more extensive checks as well as allowing to automatically fix most style violations automatically.
This would take me like a day to review
You are surely exaggerating. I am sure you wouldn't need that long :)
and cause a lot of that annoying thing where you do
git blameand run into a useless style commit like these ones, and I'm not sure I see the benefit.
The benefit is that you would see this one commit and after that there should be less or no style adjustement commits.
I also don't like the bump to Python 3.9 as a minimum without any clear improvement.
As older versions go EOL also tooling drops support for those older versions hence I assume this bump is necessary.
I'd be fine with a smaller PR that fixes concrete issues.
that of course is possible to keep the delta that needs to be reviewed low. However, I still approve the PR as a whole as it looks sound to me.
|
|
||
| if not scheme: | ||
| if server in ("localhost", "127.0.0.1", "::1"): | ||
| if server in {"localhost", "127.0.0.1", "::1"}: |
There was a problem hiding this comment.
that is, why change this from a tuple to a set?
But if we just keep the current black / pylint combination, I also wouldn't see any. Because we're already adjusted to black/pylint style and have been since day 1. I'm already wasting more time than I'd like reviewing this, but for a start:
Sigh. I really wish I hadn't had to waste all that time on this. And now you get to waste more. |
AdamWill
left a comment
There was a problem hiding this comment.
see review in previous comment. it made more sense to review this commit-by-commit.
The `method` parameter in `RequestError.__init__` was annotated as `str`, but `ty` flags it as possibly `None` at call sites. Using `Optional[str]` from `typing` corrects the annotation without any behavior change. This fix was surfaced by an initial `ruff`/`ty` run as part of aligning style tooling with other openQA-related projects. No logic or API behavior changes. Issue: https://progress.opensuse.org/issues/198026 Signed-off-by: Ioannis Bonatakis <ybonatakis@suse.com>
The CI matrix tests only Python 3.11+. RHEL 8 (the last reason to keep the 3.6 floor) has reached EOL, and the Fedora maintainer confirmed that breaking 3.6 compatibility is acceptable. No downstream consumer requires Python older than 3.9. Bumping requires-python here, before the ruff/ty configuration, lets those tools use py39 as the single baseline from the start without needing target-version workarounds. A follow-up commit removes the now-dead sys.version_info guard and moves Callable/MutableMapping/Sequence to collections.abc.
Configure ruff as the project linter with a broad rule set (E, F, D, PL, B, SIM, UP, FBT, TRY, FA, PTH, TC, and more), ty for type checking, and flake8-copyright with "Red Hat" author to match this project's copyright notices. Sets line-length = 120, preview mode, per-file ignores for tests, and docstring code formatting. Issue: https://progress.opensuse.org/issues/198026
Fix all violations surfaced by an initial ruff run before any further manual changes: - D100: missing module docstring in setup.py - D107: missing __init__ docstrings - D205: missing blank line between summary and description - D400/D415: first line of docstring must end with punctuation No logic or API behavior changes. Issue: https://progress.opensuse.org/issues/198026
Fix PTH (flake8-use-pathlib) violations surfaced by the initial ruff run.
Replace all os.path and os.makedirs calls with pathlib.Path equivalents
across setup.py, client.py, and conftest.py.
- setup.py: Path(__file__).resolve().parent replaces
path.abspath(path.dirname(__file__)); / operator replaces path.join;
Path.read_text() replaces open()+read().
- client.py: Path.home() replaces os.path.expanduser('~'); the paths
tuple now holds Path objects so config.read() receives them directly.
- conftest.py: confpath.mkdir(parents=True) replaces os.makedirs;
Path(__file__).resolve().parent replaces the os.path chain for
locating the test data directory; import os removed.
- Test mocks updated from os.path.expanduser to pathlib.Path.home
because Python 3.12 reimplemented Path.expanduser() without calling
os.path.expanduser internally, which silently broke the old mock.
No logic or API behavior changes.
Issue: https://progress.opensuse.org/issues/198026
The `parse: bool = True` parameter in `do_request` conflated two unrelated concerns. Callers had to know the flag existed and pass `parse=False` just to get a `requests.Response` back — adding complexity with no real benefit. For type checkers the situation was worse: the return type changes between `Any` and `requests.Response` depending on the runtime value of `parse`, which required a pair of `@overload` stubs with `Literal[True]`/`Literal[False]` discrimination to make it expressible at all. There was also a latent bug: `do_request(parse=True)` (the default) would call `resp.json()` on every successful response, including HTTP 204 No Content. A direct caller hitting a 204 endpoint would get a `JSONDecodeError` instead of the response. The 204 guard only existed in `openqa_request`, which always called `do_request(parse=False)` — making the `parse=True` default effectively dead and misleading. The fix makes `do_request` a pure transport/retry layer that always returns a raw `requests.Response`. A new `_parse_response` private method handles JSON/YAML content negotiation. `openqa_request` calls `do_request`, checks for 204 No Content (a server constraint, not a caller choice), then delegates to `_parse_response`. The `@overload` stubs and the `assert False` dead-code guard are removed.
…type errors Internal callers (find_clones, get_jobs, get_latest_build) subscript the result of openqa_request like a dict, but openqa_request returns OpenQAResponse = Union[Dict[str, Any], requests.Response] because of the 204 No Content path. ty flags requests.Response as non-subscriptable, making these call sites type errors. The fix introduces two helpers: - _build_request holds the shared logic (settings conversion, path prefixing, do_request delegation) that was previously inlined in openqa_request. Both public and internal entry points call it. - _openqa_request_json is an internal method that always calls _parse_response and returns Dict[str, Any]. It is for callers that know the endpoint will never return 204 and need a concrete dict type. openqa_request keeps its existing OpenQAResponse return type and handles the 204 case. Internal callers are switched to _openqa_request_json. Tests updated to mock the new internal method.
Add the future import to client.py and exceptions.py to allow PEP 604 union syntax (X | Y) in annotations without a Python 3.10 minimum. Modernise the Dict[str, str] annotation in the Job TypedDict. conftest.py is left without type annotations; they are not needed for test fixtures and the reviewer noted they are unnecessary. FA100/FA102 are suppressed in the ruff config so the rule is not enforced going forward. Issue: https://progress.opensuse.org/issues/198026
Raise max-positional-args and max-args to 7 instead of restructuring the request methods. Alternatives (TypedDict with Unpack, positional- only / keyword-only split) would break downstream callers like os-autoinst-scripts for no functional gain — the signatures are stable public API with all-defaulted parameters. Use HTTPStatus.NO_CONTENT instead of bare 204 so the meaning is clear without knowing HTTP codes. Set scheme = "https" unconditionally, then override to "http" for localhost; avoids a ternary assignment that ruff SIM108 would flag, and reads more clearly than an if/else with two branches. The extra branch raises mccabe complexity of __init__ from 9 to 10, so bump max-complexity accordingly. Issue: https://progress.opensuse.org/issues/198026
The old name shadows the Python builtin ConnectionError. Renaming avoids confusion and fixes ruff A001. This is a breaking change for downstream code that imports ConnectionError from this package. Issue: https://progress.opensuse.org/issues/198026
8ed4e4d to
6a3af88
Compare
Many suggestions make sense. As such I have reverted some changes, in order to come to an alignment with your arguments. I also tried to squash all it was asked and even more wherever I thought it is sensible. So it might be a bit different PR to review. I tested it and the changes look ok and you should not expect any serious disruptions. The main goal I think is the adaption of the tooling which brings significant benefits over black/pylint — ruff replaces both with a single, faster tool with more rules, already adopted across the os-autoinst ecosystem — and keeps sanity between all the projects. Here a summary of the accepted changes on your requests:
I keep the I hope this is in the direction that you want. otherwise what would be the minumum changes which you would see as acceptable |
|
Thanks for revising, will take another look as I have time. |
Ruff already handles linting; its formatter covers the same role as black, making black a redundant dependency. Issue: https://progress.opensuse.org/issues/198026 Signed-off-by: Ioannis Bonatakis <ybonatakis@suse.com>
6a3af88 to
f6b30b9
Compare
|
The claim that "RHEL 8 has reached EOL" is not correct. It won't be officially EOL until 2029. Please adjust that commit message. |
| where = ["src"] | ||
|
|
||
| [tool.ruff] | ||
| line-length = 120 |
There was a problem hiding this comment.
Our line length for black was 100, this is the line length I generally use. Why change it?
| # D: pydocstyle (for D100, D101, etc.) | ||
| # Q: flake8-quotes (for Q000) | ||
| # PL: Pylint | ||
| extend-select =[ |
There was a problem hiding this comment.
why a space before the = but not after?
| "PLR", # https://docs.astral.sh/ruff/rules/#refactor-plr | ||
| "PLW", # https://docs.astral.sh/ruff/rules/#warning-plw | ||
| "FA", # https://docs.astral.sh/ruff/rules/#flake8-future-annotations-fa | ||
| "UP", # https://docs.astral.sh/ruff/rules/#pyupgrade-up |
There was a problem hiding this comment.
I don't really love this massive laundry list...how is it to be maintained / reasoned about? Do we have to add new rules to it regularly? Does ruff not support some sort of profile concept or something?
| "DOC201", # docstring-missing-returns | ||
| "DOC501", # docstring-missing-exception | ||
| "COM812", # missing-trailing-comma | ||
| # Retry logic: intentional raise-inside-try pattern; TypeError("msg") is idiomatic |
There was a problem hiding this comment.
can we not ignore the specific cases of these that we are OK with, instead of disabling the rules for the entire codebase? other linters/formatters support this, it seems weird ruff doesn't, if it doesn't.
| [tool.ruff.lint.mccabe] | ||
| max-complexity = 9 | ||
|
|
||
| [tool.ruff.lint.per-file-ignores] |
There was a problem hiding this comment.
so we can at least ignore per file...maybe we should move the TRY ignores commented on above down here so they're at least just per-file, not codebase-wide?
| docstring-code-format = true | ||
|
|
||
| [tool.ty.analysis] | ||
| allowed-unresolved-imports = ["netsnmp"] |
There was a problem hiding this comment.
huh? we don't import netsnmp anywhere. what's this for?
| def __init__( | ||
| self, server: str = "", scheme: str = "", retries: int = 5, wait: int = 10 | ||
| ) -> None: | ||
| """Initialize the client, reading config and setting up the session.""" |
There was a problem hiding this comment.
requiring docstrings for __init__ seems weird. we all know what __init__ is for, this is just wasted text. I usually only add docstrings for __init__ functions if there's some actual reason to document something in the __init__ that isn't already covered in the class docstring.
| do whatever it likes with). You can use this directly instead | ||
| """Prepare and submit a requests.Request, returning the parsed output. | ||
|
|
||
| Unless parse is False, in which case return the response for the caller to |
There was a problem hiding this comment.
This reads weirdly. You've cut up a single sentence into two sentences to try and adhere to the 'summary and description' docstring layout (which is just a choice, btw, not a rule written down anywhere, but apparently ruff requires it?), but that breaks the meaning and makes this read weirdly; I would never start a paragraph with "Unless".
I'd suggest changing the summary line to "Prepare and submit a Request, returning the parsed output or response" or similar.
| cloned as 4 and 4 was cloned as 5, you'll wind up with 5. If both | ||
| a job and its clone are already in the iterable, the original will | ||
| be removed. | ||
| """Replace cloned jobs with their clones in an iterable of job dicts. |
There was a problem hiding this comment.
This reads like it mutates the sequence it is passed, but it does not.
| """ | ||
|
|
||
| def __init__(self, method: Optional[str], url: str, status_code: int, text: str) -> None: | ||
| """Store the method, URL, status code, and response text of the failed request.""" |
There was a problem hiding this comment.
again, pointless wastes of texts to satisfy a bizarre "rule".
|
|
||
| """Main client functionality.""" | ||
|
|
||
| import configparser |
There was a problem hiding this comment.
the changes to import order here aren't related to the main purpose of the commit they're in, nor are they mentioned in the commit message. they're correct, but I think they should either be in a separate commit or at least acknowledged as an 'along-the-way' change in the commit message.
| from requests.exceptions import ConnectionError as RConnectionError | ||
| from typing import Optional | ||
|
|
||
| from requests.exceptions import ConnectionError as RConnectionError |
There was a problem hiding this comment.
again, import order fix unrelated to the commit it's in. this case is worse because it's in a file the commit does not otherwise touch.
| import openqa_client.exceptions as oqe | ||
|
|
||
|
|
||
| class TestClient: |
There was a problem hiding this comment.
why are we removing the class wrapper here? I like to keep all tests in classes, and there's no justification for the change. It also makes it more or less impossible to see what, if anything, this PR really changes in the tests.
|
I think "refactor(client): add _build_request and _openqa_request_json to fix type errors" should be squashed into "refactor: split do_request into transport and parse layers" because it's resolving issues that only exist because of the split. |
| # Retry logic: intentional raise-inside-try pattern; TypeError("msg") is idiomatic | ||
| "TRY201", "TRY300", "TRY301", "TRY003", | ||
| # Don't enforce future-annotations import or pyupgrade annotation style | ||
| "FA100", "FA102", |
There was a problem hiding this comment.
"style: add from __future__ import annotations to source modules" notes that FA100 and FA102 are ignored in order to allow us to not bother with annotations in conftest.py , so maybe these should be per-file ignores?
| """ | ||
| resp = self._build_request(method, path, params, retries, wait, data, json) | ||
| if resp.status_code == 204: | ||
| if resp.status_code == HTTPStatus.NO_CONTENT: |
There was a problem hiding this comment.
this should be merged into "refactor: split do_request into transport and parse layers", because that commit introduced this code.
|
why is there an exclamation point in the commit message "fix(exceptions)!: rename ConnectionError to OpenQAConnectionError (A001)"? |
| """ | ||
| if not build and not jobs: | ||
| raise TypeError("iterate_jobs: either 'jobs' or 'build' must be specified") | ||
| raise openqa_client.exceptions.MissingArgumentError |
There was a problem hiding this comment.
this change is not to do with OpenQAConnectionError and is not documented in the commit message. I think introducing MissingArgumentError and raising it here should be a separate commit from the OpenQAConnectionError rename.
|
"style(client): fix PLR0913, PLR0917, PLR2004" adds four lines to the imports of |
|
|
||
| [tool.ruff.lint.per-file-ignores] | ||
| "tests/*" = ["PLR2004", "ANN401", "PLR0913", "S101", "S105", "S106", "PLR0917", "D103"] | ||
| "tests/*" = ["PLR2004", "ANN401", "ANN001", "ANN201", "ANN202", "PLR0913", "S101", "S105", "S106", "PLR0917", "D103"] |
There was a problem hiding this comment.
these additional ignores are not explained in the commit message. please explain them.
Aligning with other openQA projects' ruff configuration. The rule set exposed style debt, a naming conflict with
Python's builtin ConnectionError, and a latent JSONDecodeError on 204 responses. Tracked in
https://progress.opensuse.org/issues/198026.