Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
c8eccca
github actions workflow wip
righ Feb 19, 2023
73f3000
typo
righ Feb 19, 2023
63e17b5
update readme
righ Feb 19, 2023
8a39e5d
username
righ Feb 19, 2023
a9b000a
delete badge
righ Feb 19, 2023
a86be1b
publish
righ Feb 19, 2023
85bd7e0
token
righ Feb 19, 2023
678eb30
delete
righ Feb 19, 2023
2a01cd4
make async and rename
righ Feb 19, 2023
665eeb4
version 2.1.0 for python
righ Feb 19, 2023
036026e
use action
righ Feb 19, 2023
04ab0ac
update version
righ Feb 19, 2023
b8bcfd9
update
righ Feb 19, 2023
6a915a8
update
righ Feb 19, 2023
11e5560
stringify
righ Feb 19, 2023
2624af2
specify dist
righ Feb 19, 2023
691b237
debug
righ Feb 19, 2023
ecc97e1
upload by twine
righ Feb 19, 2023
1748029
do not use pypa/gh-action-pypi-publish@release/v1
righ Feb 19, 2023
737ae17
codecov
righ Feb 19, 2023
6f99e85
use action
righ Feb 19, 2023
2e66202
fix path
righ Feb 19, 2023
f0ea342
pict constraints
righ Aug 4, 2024
7d37688
v2.3.0 alpha
righ Aug 6, 2024
5551b9a
Merge pull request #37 from walkframe/condition
righ Aug 6, 2024
b3ddf83
exports
righ Aug 6, 2024
19a8856
version
righ Aug 6, 2024
d754477
export
righ Aug 6, 2024
197e86b
alpha2
righ Aug 6, 2024
fde798d
v2.3.0
righ Aug 6, 2024
5f8434d
fix: Remove covertable from dependencies
righ Aug 7, 2024
47c59da
v2.3.1
righ Aug 7, 2024
b566e8d
2.3.2-alpha.0
righ Aug 7, 2024
9da9250
2.3.2-alpha.1
righ Aug 7, 2024
f60e1e8
webpack
righ Aug 7, 2024
2bdab40
2.3.2-alpha.4
righ Aug 21, 2024
7cf71ac
fix: set default true
righ Aug 21, 2024
84f670d
2.3.2-alpha.5
righ Aug 21, 2024
014aa00
fix: conditions for application of prefilter
righ Aug 24, 2024
b104389
Merge pull request #38 from walkframe/fix/prefilter
righ Aug 26, 2024
4a3ec74
2.4.0-alpha.0
righ Aug 26, 2024
8faaedf
test: add tests
righ Aug 27, 2024
0299911
2.4.0-alpha.1
righ Aug 28, 2024
2184f4b
update README and types
righ Aug 28, 2024
1151c43
2.4.0-alpha.2
righ Aug 28, 2024
9bdb036
update
righ Aug 28, 2024
c0d0684
2.4.0
righ Aug 28, 2024
ababb7f
update README
righ Aug 29, 2024
89b1b7e
update
righ Aug 29, 2024
3dca423
update
righ Aug 29, 2024
3288393
update link
righ Aug 31, 2024
6b06f52
2.4.1
righ Aug 31, 2024
636eae7
feat: Controller and progress
righ Mar 17, 2025
268793a
v2.5.0
righ Mar 17, 2025
09c6755
fix: 0div
righ Mar 17, 2025
abcbf43
v2.5.1
righ Mar 17, 2025
90611d0
fix: consume row pairs when the prefilter does not match.
righ Mar 28, 2025
32c4d24
feat: use fnv1a32 for hash
righ Apr 8, 2026
57bb947
feat: add options for pict
righ Apr 10, 2026
2390d3c
fix: workflows
righ Apr 16, 2026
5550c55
Merge pull request #43 from walkframe/feature/use-fnv1a32
righ Apr 16, 2026
53caa09
3.0.0-rc.0
righ Apr 16, 2026
aacece6
fix: workflows
righ Apr 16, 2026
071e25d
fix: run exec jest
righ Apr 16, 2026
f984787
fix: workflow
righ Apr 16, 2026
227b8d8
fix: add heavy.pict
righ Apr 16, 2026
4849de7
docs
righ Apr 17, 2026
55ab64f
codecov
righ Apr 17, 2026
676e27f
update docs
righ Apr 17, 2026
c6d1b53
3.0.0-rc.1
righ Apr 17, 2026
ec40f2c
fix: docs
righ Apr 17, 2026
4f442a3
feat: support arithmetic expressions
righ Apr 18, 2026
51fa10c
3.0.0-rc.2
righ Apr 18, 2026
1e836f4
feat: use set for in-condition automatically
righ Apr 18, 2026
ba20b66
update docs
righ Apr 18, 2026
3870df0
fix: parameter color for demo
righ Apr 18, 2026
9ce6c9e
add a doc
righ Apr 18, 2026
b61fca3
v3.0.0
righ Apr 18, 2026
9ec0ed5
fix: README
righ Apr 18, 2026
d54bebf
update doc
righ Apr 18, 2026
80b9a2f
feat: about, privacy policy
righ Apr 18, 2026
29230ec
fix: doc
righ Apr 18, 2026
80c78e9
fix(pict): align constraint & sub-model parsing with PICT spec
righ Aug 2, 2026
ab4d2cb
feat(pict): classify `#` comments and expose them on the model
righ Aug 2, 2026
354d1a5
test(pict): cover numeric/float IN sets and LIKE literal/wildcard sem…
righ Aug 2, 2026
bd37571
docs(pict): document numeric IN sets and optional sub-model order
righ Aug 2, 2026
fdad87d
Merge pull request #51 from walkframe/fix/pict-spec-conformance
righ Aug 2, 2026
39aaed7
chore: bump version to 3.1.0
righ Aug 2, 2026
29fd6ff
feat(vscode): add PICT VS Code extension with publish workflow
righ Aug 16, 2026
5365dce
ci(vscode): pin pnpm via packageManager for the publish workflow
righ Aug 16, 2026
d208dec
docs: document the PICT VS Code extension
righ Aug 17, 2026
1c77364
docs: update site favicon to the current CoverTable logo
righ Aug 18, 2026
217d0bd
ci: disable Dependabot for the docs site
righ Aug 18, 2026
76fbad0
feat(optimize): SA post-processor with cooperative island model + end…
righ Aug 19, 2026
d7861d3
feat(python): port the SA optimizer (optimize / optimize_parallel)
righ Aug 19, 2026
8b1e25a
chore: add covertable-usage skill for Claude Code
righ Aug 19, 2026
34c2c6d
chore: address code-quality review comments
righ Aug 19, 2026
dcace79
Merge pull request #53 from walkframe/optimize
righ Aug 19, 2026
a76bf7c
chore: bump covertable to 3.2.0 and vscode extension to 0.2.0
righ Aug 19, 2026
454de57
fix(docs): unbreak Pages build and add optimize toggle to PICT tool
righ Aug 19, 2026
3ceee5a
docs(readme): link Performance to evidence/VERIFICATION.md
righ Aug 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 0 additions & 143 deletions .circleci/config.yml

This file was deleted.

8 changes: 8 additions & 0 deletions .claude/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"permissions": {
"allow": [
"Bash(pip3 index:*)",
"Bash(python3 -m pip install covertable==999)"
]
}
}
155 changes: 155 additions & 0 deletions .claude/skills/covertable-usage/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
---
name: covertable-usage
description: How to use the CoverTable library (this repo) to generate pairwise / N-wise covering arrays in TypeScript or Python — the make() API and its options, declarative constraints and the Constraint builder, PICT-format models via PictModel, weights/presets/subModels, and the SA post-processor (Controller.optimize / optimizeParallel). Load when writing or reviewing code that calls covertable, builds PICT models, generates test-case tables, or tunes covering-array size.
---

# CoverTable usage

CoverTable generates **pairwise (N-wise) covering arrays** with a one-test-at-a-time greedy (AETG-style) algorithm, plus an optional simulated-annealing post-processor that shrinks the array further. TypeScript is the primary implementation; Python mirrors it.

This skill is the fast path to writing *correct* CoverTable code. For prose docs see `docs/contents/` (rendered at https://covertable.walkframe.com ) and the package READMEs.

## Entry points (TypeScript)

The npm package (`covertable`, currently v3.x) has three export paths — pick the smallest one you need:

```ts
import { make, makeAsync, Controller, sorters, criteria, NeverMatch } from "covertable";
import { PictModel } from "covertable/pict"; // PICT-format model parser
import { Constraint } from "covertable/shortcuts"; // concise constraint builder
```

Inside this repo the sources live under `typescript/src/` (`index.ts`, `controller.ts`, `types.ts`, `pict/`, `shortcuts/`). Python lives under `python/`.

## `make(factors, options)`

`factors` is either an **array of value-lists** (rows come back as arrays, same order) or an **object** keyed by factor name (rows come back as objects). Prefer the object form when constraints are involved — constraints reference factors by key.

```ts
// array form
make([["iPhone","Pixel"], ["iOS","Android"], ["Chrome","Safari"]]);

// object form
make({
machine: ["iPhone", "Pixel", "XPERIA"],
os: ["iOS", "Android"],
browser: ["Chrome", "Safari"],
});
```

`make` throws `NeverMatch` (with `.uncoveredPairs`) if constraints make some required tuple impossible to cover. `makeAsync` is the generator form (yields rows as they are produced); it does **not** throw NeverMatch — check `ctrl.stats.uncoveredPairs` yourself.

### Options (`OptionsType`, see `typescript/src/types.ts`)

| Option | Type | Default | Notes |
|---|---|---|---|
| `strength` | `number` | `2` | N-wise. Cost grows **exponentially** with strength — use >2 only when required. (Renamed from `length` pre-v3.) |
| `subModels` | `SubModelType[]` | — | `{ fields, strength? }`. Apply a different strength to a group of factors; cross-model pairs stay at the global `strength`. |
| `weights` | `WeightsType` | — | **Index-keyed**: `{ Browser: { 0: 10 } }` weights value index 0. Only biases the completion phase — never changes the minimum row count. Use `weightsByValue` from `covertable/pict` to key by value. |
| `presets` | `PresetRowType[]` | — | Rows that must appear (PICT "seeding"). Partial rows are completed; rows that violate constraints or contain unknown values are **silently dropped**. |
| `constraints` | `Expression[]` | — | Declarative; top-level array is an implicit AND. See below. |
| `comparer` | `Comparer` | — | Custom `eq/ne/gt/lt/gte/lte/in` functions for constraint evaluation. Disables parallel optimize (functions can't cross the worker boundary). |
| `sorter` | `sorters.hash \| sorters.random` | `hash` | `hash` is reproducible (honors `salt`); `random` differs each run and is fastest. |
| `criterion` | `criteria.greedy \| criteria.simple` | `greedy` | `greedy` minimizes rows (slower); `simple` is fast but yields more rows (needed for very wide cases like `2^100`). |
| `salt` | `string \| number` | `""` | Mixed into `hash` ordering. Same factors + same salt ⇒ identical output. (Renamed from `seed` pre-v3.) |
| `tolerance` | `number` | `0` | `greedy` only. Higher = faster but more rows. |

## Constraints

Constraints are evaluated under **Kleene three-valued logic**: while a referenced factor is not yet set in the row, the condition is `null` (deferred), not `false` — so the generator prunes early without discarding viable rows. Model "IF A THEN B" as `A → B` ≡ `¬A ∨ B` (an `or`).

Raw object form:

```ts
make(factors, {
constraints: [
// IF machine = iPhone THEN os = iOS
{ operator: "or", conditions: [
{ operator: "ne", left: "machine", value: "iPhone" },
{ operator: "eq", left: "os", value: "iOS" },
]},
],
});
```

Prefer the **`Constraint` builder** (`covertable/shortcuts`) — pass `typeof factors` for `$`-field autocomplete. `$name` = field reference, anything else = literal:

```ts
import { Constraint } from "covertable/shortcuts";
const c = new Constraint<typeof factors>();

make(factors, {
constraints: [
c.or(c.ne("$machine", "iPhone"), c.eq("$os", "iOS")), // IF iPhone THEN iOS
c.lte(c.mul("$Price", "$Qty"), 5000), // arithmetic operands
c.not(c.eq("$OS", "Linux")),
],
});
```

- Comparisons: `eq ne gt lt gte lte in`. Logical: `and or not`. Arithmetic (as operands): `add sub mul div mod pow`, plus variadic `sum`/`product`.
- **`fn` escape hatch** for logic that can't be expressed declaratively. You MUST list the fields it depends on so three-valued logic knows when to evaluate — a missing dependency makes the condition `null`:
```ts
c.fn(["OS", "Browser"], (row) => row.OS !== "Linux" || row.Browser !== "Safari");
```
- Python: `and_() or_() not_()` (trailing underscore); `fn` takes a `lambda row: ...`.

## PICT models (`covertable/pict`)

CoverTable reads **PICT-format** model text (a *superset* of Microsoft PICT — adds arithmetic in constraints, `#` comments, and the `~` negative-value prefix; those extensions won't run in the original PICT tool).

```ts
import { PictModel } from "covertable/pict";

const model = new PictModel(`
Type: Single, Span, Stripe, Mirror, RAID-5
Size: 10, 100, 500, 1000, 5000, 10000
File system: FAT, FAT32, NTFS

IF [File system] = "FAT" THEN [Size] <= 4096;
IF [File system] = "FAT32" THEN [Size] <= 32000;
`, { caseInsensitive: true, strict: false });

const rows = model.make(); // options can be passed and merge with the model's
model.issues; // parse issues; with strict:true the ctor throws PictModelError on errors
```

- Sections: **Parameters** (`Name: v1, v2`), **Sub-models** (`{ A, B } @ N`), **Constraints** (`IF … THEN …;`). Also supports weights `(N)`, negatives `~value`, and aliases.
- `strict: true` throws `PictModelError` if any error-severity issue is found; otherwise inspect `model.issues`.
- Negative (`~`) values are re-prefixed with `~` in the output rows for display.
- `model.make(options)` merges the model's own constraints/subModels/weights with any you pass.

## Shrinking the array: `Controller.optimize` (SA post-process)

The greedy `make` result can be shrunk further with simulated annealing. Build a `Controller` (so `strength`/`constraints`/`comparer` are shared and can't drift), `make`, then `optimize`:

```ts
import { Controller } from "covertable";

const ctrl = new Controller(factors, { strength: 2, constraints });
const rows = ctrl.make();
const smaller = ctrl.optimize(rows, { budgetMs: 60_000 }); // single-thread
// const smaller = await ctrl.optimizeParallel(rows, { budgetMs: 60_000, workers: 8 });
```

`PictModel` has the same `optimize()` / `optimizeParallel()` (call `make()`/`makeAsync()` first).

Key facts about optimize (`OptimizeTuning` in `types.ts`):

- **Anytime**: returns the smallest array found within `budgetMs` (default 1000). Aborting via `signal` still returns a valid covering array. `onProgress` fires when a smaller array is accepted.
- Every result is **re-verified** to still cover all required tuples (see `evidence/repro/` for independent verification).
- The main knob is `budgetMs`. The annealing knobs (`startTemperature`, `endTemperature`, `targetedMoveRate`, `minCollateralSamples`, `initialIterations`, `iterationGrowth`) have sensible defaults — usually leave them.
- `seed` makes a run reproducible.

### `optimizeParallel({ workers: N })`

Cooperative **island model**: N workers with distinct seeds and move strategies share a global-best array; laggards adopt it, a couple of scouts keep exploring. The win is **variance/robustness** — a faster, more reproducible path to a given size — **not a smaller array** (it does not push past the combinatorial wall). Falls back to single-thread when workers/`SharedArrayBuffer` are unavailable or the run uses a custom `comparer` or `fn`-constraint. In **bundled** environments (e.g. the VS Code extension) set `workerUrl` to a standalone module that re-exports `__workerReduce`.

## Gotchas checklist

- `weights` keys are **value indices**, not values — use `weightsByValue` to key by value.
- `presets` that violate constraints or use unknown values are **dropped silently**, not errored.
- `make` throws `NeverMatch`; `makeAsync` does not — read `ctrl.stats.uncoveredPairs`.
- High `strength` and wide factor sets blow up combinatorially. For very wide low-strength cases, `criteria.simple` may be required.
- `optimizeParallel` won't beat single-thread on final *size*; it buys speed/reproducibility. Don't promise a smaller array from more workers.
- Reproducibility comes from `sorters.hash` + fixed `salt` (generation) and a fixed `seed` (optimize).
12 changes: 12 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
version: 2
updates:
# Disable Dependabot for the docs (Docusaurus) site, including security updates.
# The docs site produces many PRs for transitive devDependencies that we don't need.
# open-pull-requests-limit: 0 stops version updates; ignore "*" stops security updates.
- package-ecosystem: "npm"
directory: "/docs"
schedule:
interval: "weekly"
open-pull-requests-limit: 0
ignore:
- dependency-name: "*"
39 changes: 39 additions & 0 deletions .github/workflows/docs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Deploy Docs to Cloudflare Pages

on:
push:
branches:
- master
- feature/docs

jobs:
deploy:
runs-on: ubuntu-latest
defaults:
run:
working-directory: docs

steps:
- uses: actions/checkout@v6

- uses: pnpm/action-setup@v5
with:
package_json_file: docs/package.json

- uses: actions/setup-node@v6
with:
node-version: "24"

- name: Install dependencies
run: pnpm install

- name: Build Docusaurus
run: pnpm build

- name: Deploy to Cloudflare Pages
uses: cloudflare/wrangler-action@v3
with:
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
command: pages deploy docs/build --project-name=covertable --branch=${{ github.ref_name }}
packageManager: pnpm
Loading