Skip to content

feat(lapis): add sequence position fields syntax to aggregated endpoint#1768

Open
fhennig wants to merge 8 commits into
mainfrom
feat/sequence-position-fields
Open

feat(lapis): add sequence position fields syntax to aggregated endpoint#1768
fhennig wants to merge 8 commits into
mainfrom
feat/sequence-position-fields

Conversation

@fhennig

@fhennig fhennig commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds Name[N] syntax (e.g. S[501]) to the fields parameter of /aggregated, allowing users to request specific sequence positions as group-by columns
  • Shorthand [N] is supported for single-segmented genomes and resolves to the single nucleotide sequence
  • Sequence names are validated case-insensitively against both genes and nucleotide segments from the reference genome
  • Generates a .map({"S[501]":="S".at(501)}) SaneQL step before .groupBy(...) when position fields are present — the user-facing name is used directly as the SaneQL alias, so no post-processing is needed
  • Response column keys mirror the input exactly: S[501] in → S[501] out, [501] in → [501] out

Example

POST /sample/aggregated
{"fields": ["country", "S[501]"]}

Generates:

default.filter(true).map({"S[501]":="S".at(501)}).groupBy({"count":=count()}, {"country", "S[501]"})

Test plan

  • Unit tests for Name[N] parsing in SequenceFiltersRequestWithFieldsTest
  • Error cases: unknown sequence name, position ≤ 0, shorthand on multi-segmented genome
  • SaneQL generation tests in SiloQueryToSaneQlTest for position-only and mixed fields
  • Full test suite passes

🤖 Generated with Claude Code

Allow users to request specific sequence positions as group-by fields
using `Name[N]` syntax (e.g. `S[501]`). For genomes with a sequence
named `main`, the shorthand `[N]` is also accepted. Position fields
generate a `.map(...)` step in SaneQL before `.groupBy(...)`, and
appear in the response under the alias `Name_N` (e.g. `S_501`).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lapis Ready Ready Preview, Comment Jul 15, 2026 12:24pm

Request Review

Echo S[501] back as the response column key instead of the internal
SaneQL alias S_501, so input and output are symmetric for callers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ack in response

Shorthand [N] now validates via isSingleSegmented() (consistent with
how the rest of the codebase handles single-segment-only features)
rather than checking for a hardcoded 'main' name.
SequencePositionField gains isSingleSegment flag so the response key
mirrors the input: [501] in → [501] out, S[501] in → S[501] out.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…field aliasing

SaneQlAssignment now quotes its name (consistent with SaneQlIdentifier,
safer against injection). Position fields use their user-facing name
(e.g. S[501]) directly as the SaneQL alias, so SILO returns the column
under that name already — no post-processing remapping needed in the
controller.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for requesting specific sequence positions (e.g. S[501], and [501] for single-segment genomes) as group-by columns in the /aggregated endpoint by parsing them from the fields request parameter and generating the appropriate SaneQL .map(...).groupBy(...) pipeline.

Changes:

  • Introduces RequestField with a new SequencePositionField type and updates request parsing to recognize Name[N] / [N] syntax.
  • Extends aggregated SaneQL generation to add a .map(...) step that computes requested positions and includes them in .groupBy(...).
  • Updates SaneQL AST rendering/tests to quote assignment keys (e.g. {"count":=count()}) and adds unit coverage for new parsing/generation.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
lapis/src/main/kotlin/org/genspectrum/lapis/request/Field.kt Adds RequestField + SequencePositionField, and extends field conversion to parse/validate position syntax against the reference genome schema.
lapis/src/main/kotlin/org/genspectrum/lapis/request/SequenceFiltersRequestWithFields.kt Updates request model to store fields as List<RequestField> and keeps deserialization via the field converter.
lapis/src/main/kotlin/org/genspectrum/lapis/silo/SiloQuery.kt Adds sequencePositionFields to aggregated actions and emits .map(...) before .groupBy(...) when present.
lapis/src/main/kotlin/org/genspectrum/lapis/silo/SaneQlAst.kt Changes assignment rendering to quote/escape keys (e.g. "name":=value) to align with SaneQL expectations and harden output.
lapis/src/main/kotlin/org/genspectrum/lapis/model/SiloQueryModel.kt Splits RequestField values into metadata fields vs. sequence-position fields for aggregated query construction.
lapis/src/main/kotlin/org/genspectrum/lapis/controller/LapisController.kt Ensures response headers/field lists use outputColumnName (aggregated) and filters non-metadata fields out of details fields handling.
lapis/src/main/kotlin/org/genspectrum/lapis/controller/ControllerDescriptions.kt Updates aggregated endpoint parameter description to document the new position-field syntax.
lapis/src/main/kotlin/org/genspectrum/lapis/request/PhyloTreeSequenceFiltersRequest.kt Adjusts phylo-tree parsing to work with the generalized FieldConverter<RequestField> while enforcing a Field result.
lapis/src/test/kotlin/org/genspectrum/lapis/request/SequenceFiltersRequestWithFieldsTest.kt Adds unit tests for parsing valid/invalid Name[N] / [N] field forms.
lapis/src/test/kotlin/org/genspectrum/lapis/silo/SiloQueryToSaneQlTest.kt Updates expected SaneQL strings and adds coverage for aggregated SaneQL with sequence position fields.
lapis/src/test/kotlin/org/genspectrum/lapis/silo/SaneQlAstTest.kt Updates AST rendering assertions to match quoted assignment output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lapis/src/main/kotlin/org/genspectrum/lapis/controller/ControllerDescriptions.kt Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@fhennig

fhennig commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Ok, something weird with the test? But still reviewable I think. I'll look into it.

…ns.kt

The AGGREGATED_GROUP_BY_FIELDS_DESCRIPTION const was missing its closing
triple-quote, which caused the compiler to swallow the following
declaration as part of the string and fail with unresolved reference
and syntax errors.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Comment thread lapis/src/main/kotlin/org/genspectrum/lapis/request/Field.kt
Previously SequencePositionField entries were silently dropped, causing
/details to return a 200 with all metadata fields instead of erroring.
Now it returns a 400 with a clear message, per Copilot review feedback.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add aggregation by sequence positions

2 participants