Skip to content

feat: add optional viewport field to plan files + --viewport CLI flag for responsive/mobile testing #306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
waterWang wants to merge 1 commit into
base: main
Choose a base branch
from
+92 −1
Open

feat: add optional viewport field to plan files + --viewport CLI flag for responsive/mobile testing #306

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions DOCUMENTATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,7 @@ Get this exact skeleton without hand-copying it from this file: `testsprite test
| `name` | yes | string | An assertable behavior statement (subject + verb + outcome), not a noun fragment. |
| `description` | no | string | One-sentence elaboration of `name` — the condition plus the expected outcome. |
| `priority` | no | `"p0"` \| `"p1"` \| `"p2"` \| `"p3"` | p0 = must-pass, p1 = important paths, p2 = edge cases, p3 = cosmetic. |
| `viewport` | no | string | Browser viewport for the frontend runner, in `<width>x<height>` form (e.g. `"390x844"` for a mobile device). Forwarded to the backend so the browser-use runner can size the viewport before executing plan steps. Absent means the runner's desktop default. |
| `planSteps` | yes | `Array<{ type: "action" \| "assertion", description: string }>` | **1–200 steps**, describing user intent in plain language, not selectors. |

**Size cap:** the whole file must be **≤ 256 KB** (`test create-batch` caps the aggregate batch at 5 MB / 50 specs). Both caps are enforced client-side before any network call.
Expand Down
5 changes: 5 additions & 0 deletions schemas/plan.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@
"enum": ["p0", "p1", "p2", "p3"],
"description": "Optional. p0 = must-pass, p1 = important paths, p2 = edge cases, p3 = cosmetic."
},
"viewport": {
"type": "string",
"pattern": "^[1-9]\\d*x[1-9]\\d*$",
"description": "Optional. Desktop viewport for the frontend browser run, in `<width>x<height>` form (e.g. \"390x844\" for a mobile device). Forwarded to the backend so the browser-use runner can size the viewport before executing plan steps — the only way responsive/mobile-only UI (e.g. an `md:hidden` bottom nav) can be exercised. Absent means the runner's desktop default."
},
"planSteps": {
"type": "array",
"minItems": 1,
Expand Down
60 changes: 59 additions & 1 deletion src/commands/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1895,6 +1895,15 @@ export interface CliPlanInput {
name: string;
description?: string;
priority?: CliCreatePriority;
/**
* Optional desktop viewport for the frontend browser run, in
* `<width>x<height>` form (e.g. `"390x844"` for a mobile device).
* When present it is forwarded to the backend so the browser-use
* runner can size the viewport before executing plan steps — the
* only way responsive/mobile-only UI (e.g. `md:hidden` bottom nav)
* can be exercised. Absent means the runner's desktop default.
*/
viewport?: string;
planSteps: CliPlanStep[];
}

Expand Down Expand Up @@ -2336,6 +2345,13 @@ interface CreateFromPlanOptions extends CommonOptions {
timeoutIsDefault?: boolean;
/** Reserved for the M3.3 chain. Per-run target URL override. */
targetUrl?: string;
/**
* Optional browser viewport override for the frontend runner, in
* `<width>x<height>` form (e.g. "390x844"). When set alongside
* `--plan-from`, overrides any viewport in the plan JSON file.
* Validated client-side before the POST.
*/
viewport?: string;
/**
* Names of `test create` flags the caller supplied that `--plan-from`
* ignores (identity lives in the JSON). Surfaced as a stderr advisory
Expand Down Expand Up @@ -2425,6 +2441,23 @@ export async function runCreateFromPlan(

const plan = readPlanFromGuarded(opts.planFrom, { ignoredFlags: opts.ignoredFlags });

// `--viewport` is a CLI-level override of the viewport in the plan JSON —
// the one `--plan-from` field that is legitimately overridable from the
// command line (the plan file pins projectId/type/name/planSteps, but the
// viewport is a run-environment concern the caller may want to vary
// without editing the file, e.g. a mobile smoke pass on a desktop plan).
if (opts.viewport !== undefined) {
if (!/^\d+x\d+$/.test(opts.viewport)) {
throw localValidationError(
'viewport',
'must be a string in `<width>x<height>` format (e.g. "390x844")',
undefined,
'flag',
);
}
plan.viewport = opts.viewport;
Comment on lines +2449 to +2458

@coderabbitai coderabbitai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Use one positive-dimension validator for every viewport input.

schemas/plan.schema.json requires each dimension to match [1-9]\d*, but these runtime checks use \d+. Therefore 0x844, 390x0, and 0390x0844 pass --viewport, assertPlanShape, and test lint, even though the schema rejects them. test create --plan-from can send a value that local validation should reject.

Define one VIEWPORT_PATTERN and use it in the CLI override, assertPlanShape, and collectPlanIssues.

Proposed fix 
+const VIEWPORT_PATTERN = /^[1-9]\d*x[1-9]\d*$/;

-    if (!/^\d+x\d+$/.test(opts.viewport)) {
+    if (!VIEWPORT_PATTERN.test(opts.viewport)) {

-    if (typeof obj.viewport !== 'string' || !/^\d+x\d+$/.test(obj.viewport)) {
+    if (typeof obj.viewport !== 'string' || !VIEWPORT_PATTERN.test(obj.viewport)) {

As per path instructions, plan validation and test lint are local/offline and should reject invalid plans before network requests.

Also applies to: 2763-2772, 2854-2858

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/commands/test.ts` around lines 2449 - 2458, Define a shared
VIEWPORT_PATTERN matching positive, non-zero dimensions without leading zeroes,
then replace the duplicated viewport regex checks in the CLI override,
assertPlanShape, and collectPlanIssues. Ensure all local validation paths,
including test create --plan-from and test lint, reject values such as 0x844,
390x0, and 0390x0844 consistently with schemas/plan.schema.json.

Source: Path instructions

}

const stderrFn = deps.stderr ?? ((line: string) => process.stderr.write(`${line}\n`));

// Non-fatal advisory for `{{...}}`-style placeholders in step
Expand Down Expand Up @@ -2472,6 +2505,7 @@ export async function runCreateFromPlan(
name: plan.name,
description: plan.description,
priority: plan.priority,
viewport: plan.viewport,
planSteps: plan.planSteps,
};

Expand Down Expand Up @@ -2726,6 +2760,17 @@ function assertPlanShape(
requireEnum(`${prefix}priority`, obj.priority, CLI_CREATE_PRIORITIES);
}

if (obj.viewport !== undefined) {
if (typeof obj.viewport !== 'string' || !/^\d+x\d+$/.test(obj.viewport)) {
throw localValidationError(
`${prefix}viewport`,
'must be a string in `<width>x<height>` format when present (e.g. "390x844")',
undefined,
'field',
);
}
}

// `planSteps` missing is the single most common agent
// hallucination: LLMs (Copilot included) reliably nest steps under
// `plan.steps` or a bare top-level `steps`. Point directly at the fix
Expand Down Expand Up @@ -2806,6 +2851,11 @@ function collectPlanIssues(
if (obj.priority !== undefined) {
check(() => requireEnum(`${prefix}priority`, obj.priority, CLI_CREATE_PRIORITIES));
}
if (obj.viewport !== undefined) {
if (typeof obj.viewport !== 'string' || !/^\d+x\d+$/.test(obj.viewport)) {
issues.push({ field: `${prefix}viewport`, reason: 'must be a string in `<width>x<height>` format' });
}
}
check(() =>
requireArrayLength(`${prefix}planSteps`, obj.planSteps, {
min: 1,
Expand Down Expand Up @@ -9548,10 +9598,16 @@ export function createTestCommand(deps: TestDeps = {}): Command {
.option('--name <name>', 'human-readable test name (becomes `title` in storage)')
.option('--description <text>', 'optional human description (≤ 2000 chars)')
.option('--priority <prio>', 'optional priority — one of: p0, p1, p2, p3')
.option(
'--viewport <WxH>',
'optional browser viewport for the frontend runner (e.g. "390x844" for mobile). ' +
'With --plan-from, overrides the viewport in the plan JSON.',
)
Comment on lines +9601 to +9605

@coderabbitai coderabbitai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Do not silently ignore --viewport outside --plan-from.

The option is registered on the shared test create command. The action forwards it only in the --plan-from branch at Line [9708]. The regular runCreate call at Lines [9722-9748] does not receive it. A valid code-file invocation can therefore succeed without applying the requested viewport.

If viewport is plan-only, reject the flag before runCreate with localValidationError so the command returns VALIDATION_ERROR with exit code 5. Otherwise, thread viewport through runCreate and its request body.

As per path instructions, this thin client must preserve correctness and clear error handling.

Also applies to: 9708-9708

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/commands/test.ts` around lines 9601 - 9605, Handle the shared test create
command’s viewport option in both execution paths: if viewport is supported only
with --plan-from, reject its use before runCreate via localValidationError so
the command returns VALIDATION_ERROR with exit code 5; otherwise pass viewport
through runCreate and include it in the request body. Update the --plan-from
forwarding at the existing action branch while preserving the thin client’s
existing error handling.

Source: Path instructions

.option('--code-file <path>', 'file containing the test code (≤ 350 KB)')
.option(
'--plan-from <path>',
'JSON file with the full FE test definition — projectId, type, name, planSteps[] all live in the file ' +
'JSON file with the full FE test definition — projectId, type, name, planSteps[], ' +
'and optional viewport/description/priority all live in the file ' +
'(≤ 256 KB; mutually exclusive with --code-file). In this mode --project/--type/--name/--description/--priority are ignored.',
)
.option(
Expand Down Expand Up @@ -9649,6 +9705,7 @@ export function createTestCommand(deps: TestDeps = {}): Command {
{
...resolveCommonOptions(command),
planFrom: cmdOpts.planFrom,
viewport: cmdOpts.viewport,
run: cmdOpts.run === true,
wait: cmdOpts.wait === true,
timeout: parseTimeoutFlag(cmdOpts.timeout, 'timeout'),
Expand Down Expand Up @@ -10774,6 +10831,7 @@ interface CreateFlagOpts {
planFrom?: string;
/** Print the canonical plan-file skeleton and exit. */
planTemplate?: boolean;
viewport?: string;
run?: boolean;
wait?: boolean;
timeout?: string;
Expand Down
27 changes: 27 additions & 0 deletions src/lib/plan-schema.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,33 @@ describe('schemas/plan.schema.json', () => {
expect(await passesRealValidator(dir, plan)).toBe(true);
});

it('accepts a valid viewport string (e.g. "390x844") and rejects an invalid one', async () => {
const valid = { ...PLAN_TEMPLATE_WITH_SCHEMA, viewport: '390x844' };
expect(validate(valid)).toBe(true);
expect(await passesRealValidator(dir, valid)).toBe(true);

const invalid = { ...PLAN_TEMPLATE_WITH_SCHEMA, viewport: 'abc' };
expect(validate(invalid)).toBe(false);
expect(await passesRealValidator(dir, invalid)).toBe(false);
});

it('accepts a plan with viewport + all other optional fields', async () => {
const plan = {
projectId: 'prj_abc123',
type: 'frontend',
name: 'Mobile test plan',
description: 'Exercises mobile layout.',
priority: 'p1',
viewport: '390x844',
planSteps: [
{ type: 'action', description: 'tap the bottom nav' },
{ type: 'assertion', description: 'verify the mobile sidebar is visible' },
],
};
expect(validate(plan)).toBe(true);
expect(await passesRealValidator(dir, plan)).toBe(true);
});

it('rejects type: "backend" — schema is the ground truth for the --plan-from COMMAND, which rejects backend end-to-end (both sides must agree)', async () => {
const plan = { ...PLAN_TEMPLATE_WITH_SCHEMA, type: 'backend' };
expect(validate(plan)).toBe(false);
Expand Down
Loading