Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ concurrency:
jobs:
scan:
name: Full-history scan
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Checkout complete history
Expand Down
7 changes: 7 additions & 0 deletions docs/0. Overview/0.5. Assistants.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,10 @@ Access to the assistant is available for a monthly fee of $10. To subscribe, ple
While our assistant is highly accurate—with an estimated 90% precision rate—it is crucial to approach its output with a critical mindset. Like any AI, it can occasionally make mistakes or generate suggestions that are not optimal for your specific context. Always verify critical information and use its output as a guide, not an absolute directive.

For situations requiring guaranteed expert oversight, we highly recommend our [human mentoring](./0.4. Mentoring.md) service, which provides a perfect blend of AI-driven efficiency and human expertise.

### What about the coding agent working inside my repository?

The MLOps Coding Assistant answers questions *about* the course. It is a different tool from the coding agent that edits files in your own project, and that agent needs its own guidance to follow the practices taught here:

- **`AGENTS.md`**: the open, tool-agnostic file that tells any coding agent your project's exact commands, definition of done, conventions, and layout. [6.2. Readme](../6. Sharing/6.2. Readme.md#what-is-agentsmd) explains what belongs in it, what it deliberately cannot enforce, and how to keep it from drifting away from your code.
- **[MLOps Coding Skills](https://github.com/MLOps-Courses/mlops-coding-skills)**: the methodology of this course packaged as [Agent Skills](https://agentskills.io/home), one skill per chapter, that your agent loads on demand.
41 changes: 37 additions & 4 deletions docs/1. Initializing/1.3. uv (project).md
Original file line number Diff line number Diff line change
Expand Up @@ -77,21 +77,24 @@ Your `pyproject.toml` becomes the single source of truth for your project's conf

[project]
name = "bikes"
version = "4.1.0"
version = "6.0.0"
description = "Predict the number of bikes available."
authors = [{ name = "Médéric HURIER", email = "github@fmind.dev" }]
readme = "README.md"
license = { file = "LICENSE.txt" }
license = "MIT"
license-files = ["LICENSE.txt"]
keywords = ["mlops", "python", "package"]
requires-python = ">=3.14"
dependencies = [
"loguru>=0.7.3",
"matplotlib>=3.11.0",
"mlflow>=3.14.0",
# numba/numpy/pyarrow/shap are ABI-coupled: keep permissive floors so uv's universal
# (all-platform) resolution stays satisfiable; the lockfile pins the latest tested versions.
"numba>=0.61.0",
"numpy>=2.1.3",
"omegaconf>=2.3.1",
"pandas>=2.3.3", # MLflow 3.x pins pandas<3
"pandas>=2.3.3", # MLflow 3.x pins pandas<3; pandas 3.0 is held back by MLflow compat
"pandera>=0.32.1",
"plotly>=6.8.0",
"plyer>=2.1.0",
Expand Down Expand Up @@ -122,12 +125,16 @@ bikes = 'bikes.scripts:main'
# SYSTEMS

[build-system]
requires = ["uv_build>=0.9.0"]
# Keep the upper bound at least one minor ahead of the pinned `uv` tool: without it
# `uv build` warns, and a future breaking `uv_build` would silently break the sdist.
requires = ["uv_build>=0.9,<0.13"]
build-backend = "uv_build"
```

The `[build-system]` table tells packaging tools how to turn your source tree into a distributable artifact. This course uses [`uv_build`](https://docs.astral.sh/uv/concepts/build-backend/), uv's own fast, built-in build backend, so a single tool manages your dependencies, environment, and packaging. With it in place, `uv build` produces the wheel and source distribution ready to publish.

Note the **upper bound** on the requirement: `uv_build>=0.9,<0.13` rather than a bare `uv_build>=0.9.0`. Semantic versioning only promises a stable interface from 1.0 onwards; below it, a minor release is allowed to break things. `uv_build` is still pre-1.0, so an unbounded requirement means a future `0.13` could change how your source tree is packaged the next time anyone builds your project — and you would find out from a broken wheel, not from a failed resolution. `uv build` itself warns when the bound is missing. Keep the ceiling at least one minor ahead of the `uv` version you have pinned, and raise it deliberately after you have tested the new backend.

## How do you manage project dependencies with `uv`?

`uv` simplifies adding, removing, and updating dependencies. It automatically updates your `pyproject.toml` file and re-syncs your environment.
Expand Down Expand Up @@ -171,6 +178,32 @@ The `uv.lock` file is a lockfile that records the exact versions of every packag

While `pyproject.toml` might specify a version range (e.g., `pandas>=2.2`), `uv.lock` pins a specific version (e.g., `pandas==2.2.3`). When you run `uv sync`, `uv` will use the lockfile if it exists, ensuring that every developer on your team and every CI/CD run uses the exact same set of package versions. This prevents the "it works on my machine" problem and ensures that your builds are deterministic and stable over time.

## What do you do when a dependency blocks a security fix?

Sooner or later a package you depend on will declare an upper bound that keeps you on a vulnerable version of one of *its* dependencies. This is not hypothetical: at the time of writing, MLflow 3.15.1 still declares `cryptography<50`, while the fix for `PYSEC-2026-3552` — a PKCS#7 Bleichenbacher oracle — only ships in `cryptography` 50.0.0. Resolve without intervention and you get 49.0.0, and your vulnerability scan fails.

You have two ways out, and they are not equivalent:

- **Suppress the finding**: add the advisory to an ignore list so the scanner stops reporting it. The vulnerable code is still installed and still running. You have removed the alarm, not the risk.
- **Override the constraint**: tell the resolver to ignore the stale upper bound and install the patched library anyway, then let your test suite prove the combination still works. The vulnerability is genuinely gone; the risk you have taken on is an untested version pairing, and tests are exactly the tool for that.

`uv` supports the second option with [`override-dependencies`](https://docs.astral.sh/uv/concepts/resolution/#dependency-overrides), which replaces a declared constraint during resolution:

```toml
[tool.uv]
# MLflow 3.15 still declares `cryptography<50`, but the fix for PYSEC-2026-3552
# (PKCS#7 Bleichenbacher oracle) only ships in 50.0.0. Overriding the stale upper
# bound installs the patched library; the test suite is what proves MLflow still
# works with it. Drop this override once MLflow relaxes the constraint upstream.
override-dependencies = ["cryptography>=50"]
```

Three habits make an override safe rather than reckless:

1. **Document why it exists**, including the advisory identifier, directly above the entry.
1. **Prove it with tests**: an override is only defensible if your suite exercises the code paths that use the overridden package.
1. **Write down its exit condition**: state what has to happen upstream for the override to be deleted, and remove it when that happens. An undated override quietly becomes a permanent fork of someone else's dependency graph.

## How do you run commands in the `uv` managed environment?

`uv` provides the `uv run` command to execute scripts within the context of your project's virtual environment, so you don't need to manually activate it (e.g., `source .venv/bin/activate`).
Expand Down
18 changes: 12 additions & 6 deletions docs/3. Productionizing/3.0. Package.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,14 @@ Here is an example with explanations for each section:
# Core project metadata used by PyPI and installation tools.
[project]
name = "bikes"
version = "3.0.0"
version = "6.0.0"
description = "Predict the number of bikes available."
authors = [{ name = "Médéric HURIER", email = "github@fmind.dev" }]
readme = "README.md"
requires-python = ">=3.14"
dependencies = [] # List your production dependencies here
license = { file = "LICENSE.txt" }
license = "MIT" # an SPDX expression, per PEP 639
license-files = ["LICENSE.txt"]
keywords = ["mlops", "python", "package"]

# URLs that appear on your package's PyPI page.
Expand All @@ -77,16 +78,21 @@ Changelog = "https://github.com/fmind/bikes/blob/main/CHANGELOG.md"
[project.scripts]
bikes = 'bikes.scripts:main'

# Configures uv to install optional dependency groups by default during development.
[tool.uv]
default-groups = ["checks", "commits", "dev", "docs", "notebooks"]
# Development-only dependencies, kept out of the production install.
[dependency-groups]
dev = ["pytest>=9.1.1", "ruff>=0.16.2", "ty>=0.0.69,<0.1"]
notebook = ["ipykernel>=6.29.5", "nbformat>=5.10.4"]

# Specifies the build backend (uv_build, in this case) to create the package.
[build-system]
requires = ["uv_build>=0.9.0"]
# Keep the upper bound at least one minor ahead of the pinned `uv` tool: without it
# `uv build` warns, and a future breaking `uv_build` would silently break the sdist.
requires = ["uv_build>=0.9,<0.13"]
build-backend = "uv_build"
```

The upper bound on `uv_build` is not decoration. A pre-1.0 project makes no compatibility promise across minor releases, so an unbounded `uv_build>=0.9.0` lets a future `0.13` change how your source tree is packaged behind your back; `uv build` warns about exactly this. Bound it, and raise the ceiling deliberately once you have built and tested against the new backend. The same reasoning explains the `ty>=0.0.69,<0.1` range above.

## Where should you structure the source code for your package?

Always place your package's source code inside a `src` directory. This is known as the [**`src` layout**](https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/) and is a best practice for several reasons:
Expand Down
28 changes: 21 additions & 7 deletions docs/4. Validating/4.0. Typing.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,16 +133,27 @@ class GridCVSearcher(pdt.BaseModel):
The recommended tool for static type checking is [ty](https://docs.astral.sh/ty/), a new type checker from [Astral](https://astral.sh/) (the makers of `uv` and `Ruff`). Like its siblings, `ty` is written in Rust and is exceptionally fast, so it can run on every save and on every commit without slowing you down. It can be run from the command line or integrated directly into your [editor](https://docs.astral.sh/ty/editors/) through its language server.

```bash
# Install ty into your "check" dependency group
uv add --group check ty
# Install ty into your "dev" dependency group
uv add --group dev ty

# Type-check your source and test code
uv run ty check
```

By default, `ty check` checks the whole project; you can also pass explicit paths (e.g. `uv run ty check src/ tests/`).
By default, `ty check` checks the whole project; you can also pass explicit paths (e.g. `uv run ty check src/ tests/`). The MLOps Python Package wires this command to `mise run check:types`.

**Note**: `ty` is still **pre-1.0** and moving fast. Pin it to a compatible range (e.g. `ty>=0.0.56,<0.1`) rather than an open-ended version, and treat it as a local and CI check that you upgrade deliberately, since new releases can add or refine diagnostics.
**Note**: `ty` is still **pre-1.0** and moving fast, so both reference repositories declare it as a bounded range rather than a simple floor:

```toml
[dependency-groups]
dev = [
"ty>=0.0.69,<0.1", # pre-1.0: pin a compatible range until it stabilizes
]
```

The reason is the versioning contract. Under [semantic versioning](https://semver.org/), a pre-1.0 project makes no compatibility promise at all: the `0.0.x` segment is free to add diagnostics, rename rules, or change what counts as an error. A plain `ty>=0.0.69` therefore lets `uv sync` pick up a release that reports brand-new errors on code you never touched, and your build turns red for a reason unrelated to your commit. The `<0.1` upper bound keeps that upgrade a deliberate act: you raise the floor, run `uv run ty check`, and fix or silence the new diagnostics in a dedicated change.

Apply the same reasoning to any pre-1.0 tool you put in a gate. Once a project reaches 1.0 and promises backward compatibility within a major, a simple floor (`>=`) is enough again.

Several other type checkers exist, and you will encounter them in older projects:

Expand All @@ -167,9 +178,12 @@ python-version = "3.14"
# ty (pre-1.0) does not model every dynamic ML library yet; relax the
# categories they trigger so ty still gates real errors.
[tool.ty.rules]
invalid-argument-type = "ignore" # dynamic ML/MLflow call sites
invalid-type-form = "ignore" # pandera typed Series / dynamic type forms
unresolved-attribute = "ignore" # dynamic attributes on MLflow/pandera objects
invalid-argument-type = "ignore" # dynamic ML/MLflow call sites
invalid-frozen-dataclass-subclass = "ignore" # ty reads frozen Pydantic models as frozen dataclasses
invalid-type-form = "ignore" # pandera typed Series / dynamic type forms
not-subscriptable = "ignore" # optional dynamic MLflow attributes
possibly-missing-submodule = "ignore" # lazily-loaded mlflow.* submodules
unresolved-attribute = "ignore" # dynamic attributes on MLflow/pandera objects
```

If you need to bypass a diagnostic for a specific line, add a `# ty: ignore` comment with the rule name:
Expand Down
74 changes: 64 additions & 10 deletions docs/4. Validating/4.1. Linting.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ Key advantages of Ruff include:
- **VS Code Extension**: The official [Ruff VS Code extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) integrates these features directly into your editor.

```bash
# Install Ruff into your "check" dependency group
uv add --group check ruff
# Install Ruff into your "dev" dependency group
uv add --group dev ruff

# Run Ruff to lint your codebase
uv run ruff check src/ tests/
Expand All @@ -43,22 +43,43 @@ To keep your repository clean, remember to add the `.ruff_cache/` directory to y

Linter configurations are typically placed in the `pyproject.toml` file. This allows you to define project-wide rules, customize behavior, and ensure every developer uses the same settings.

Here is a sample configuration for Ruff:
Here is the configuration used by the MLOps Python Package, abridged to show its shape:

```toml
[tool.ruff]
# automatic fix when possible
fix = true
# define the default indent width
indent-width = 4
# define the default line length
line-length = 100
line-length = 120
# define the default python version
target-version = "py314"

[tool.ruff.lint]
# an explicit, reviewed selection instead of Ruff's evolving defaults
select = [
"B", # flake8-bugbear
"D", # pydocstyle (documented public API)
"E", # pycodestyle errors
"F", # pyflakes
"I", # isort (import sorting)
"N", # pep8-naming
"PTH", # flake8-use-pathlib
"RUF", # ruff-specific rules
"S", # flake8-bandit (security; replaces bandit)
"SIM", # flake8-simplify
"T20", # flake8-print
"UP", # pyupgrade
"W", # pycodestyle warnings
]
ignore = [
"E501", # line length handled by the formatter
]

[tool.ruff.lint.pydocstyle]
# set the expected docstring style
convention = "google"

[tool.ruff.lint.per-file-ignores]
# exceptions for docstrings in tests
"tests/*.py" = ["D100", "D103"]
# exceptions for docstrings and asserts in tests
"tests/**" = ["D100", "D103", "S101"]
```

If you need to ignore a specific rule for a single line, you can use an inline `noqa` (no quality assurance) comment:
Expand All @@ -68,6 +89,39 @@ If you need to ignore a specific rule for a single line, you can use an inline `
from project.module import specific_import # noqa: F401
```

## Why should you write an explicit rule selection?

Ruff ships with a default rule set, and that default set is not stable across releases. [Ruff 0.16.0](https://github.com/astral-sh/ruff/releases) (released 2026-07-23) expanded it from **59 rules to 413**, pulling in whole families that had previously been opt-in. You can verify the current number yourself:

```bash
# Print the settings Ruff would use with no configuration at all
uv run ruff check --show-settings --isolated .
```

The consequence depends entirely on how your project is configured:

- **With an explicit `select` list** (the reference repositories), nothing changed on upgrade. You opted into thirty-one named families, Ruff enforces exactly those, and a release that broadens the defaults is invisible to you. New rules arrive when you decide to add a family, not when a dependency resolves.
- **Without a `select` list**, the upgrade silently multiplied your enforced rules by seven. A `uv sync` that pulled Ruff 0.16 could turn a green repository red with hundreds of violations in code nobody had touched, and the diff that "caused" it would be a lockfile bump.

This is the practical argument for writing `select` explicitly, even if your initial selection matches today's defaults: the list becomes a reviewed decision recorded in your repository, and upgrading the linter stops being a source of surprise failures.

## Why does your Ruff version floor matter?

Linting and formatting must agree across every machine that runs them: your editor, your teammates' checkouts, your git hooks, and CI. Ruff 0.16 changed formatter behavior too (it now formats Python inside Markdown by default, see [4.5. Formatting](./4.5.%20Formatting.md)), which means an **older Ruff disagrees with a repository formatted by 0.16**. A contributor whose environment resolved 0.15 would see `ruff format --check` fail on files they never opened.

The fix is to raise the dependency floor whenever you adopt a behavior change, and to say why in a comment:

```toml
[dependency-groups]
dev = [
# Ruff 0.16 formats Python inside Markdown and rewrote the default rule set:
# an older Ruff would disagree with this repository's formatting, so floor it.
"ruff>=0.16.2",
]
```

Pair that floor with a committed `uv.lock` (see [1.3. uv (project)](../1.%20Initializing/1.3.%20uv%20(project).md)) so everyone resolves the same version, and with `uv lock --check` in your `check:format` task so a stale lockfile fails the gate.

## How does linting differ from formatting?

While often used together, linting and formatting have distinct purposes:
Expand Down
Loading