Skip to content

Add Rowid data type and YQL key generators - #39

Draft
zinal wants to merge 10 commits into
mainfrom
cursor/rowid-type-support-906d
Draft

Add Rowid data type and YQL key generators#39
zinal wants to merge 10 commits into
mainfrom
cursor/rowid-type-support-906d

Conversation

@zinal

@zinal zinal commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Changelog entry

Add new Rowid primitive type (14-byte opaque key) and Rowid::* YQL generators for row/column table primary keys.

Spec: https://gist.github.com/zinal/13445422ef0c5a7cdad0c3ae9d657767
Reference Uuid generators PR: ydb-platform#45923

Changelog category

  • Experimental feature

Description for reviewers

This PR proposes end-to-end support for the new Rowid data type from the pk_generation design, plus YQL generators mirroring the Uuid UDF API.

Type design

Property Choice
Type id NYql.NProto.Rowid = 0x1205, public Ydb.Type.ROWID
Size Fixed 14 bytes
Byte order Internal == external (no mixed-endian, unlike Uuid)
Text form Base64 without padding19 chars (spec said 18; corrected: ceil(14*8/6)=19)
Public wire bytes_value with length 14 (not low_128/high_128)
Storage win Fits TCell::MaxInlineSize() and TUnboxedValuePod embedded buffer (14)

YQL API (Rowid UDF module)

Rowid::newRowKey([T1, ...]) -> Rowid
Rowid::newColumnKey([T1, ...]) -> Rowid
Rowid::newRowGroup(PFX, COUNT, [T1, ...]) -> List<Rowid>  -- PFX: Uint64 | Rowid

Optional dependency args work like RandomUuid() / Uuid PR generators.

Bit layouts (per spec):

  • Row key: [12 prefix][31 ts sec][5 rand][64 rand]
  • Column key: [31 ts sec][1 rand][80 rand]

What this PR wires

  • Type IDs: YQL proto, public API proto, scheme NTypeIds, SDK EPrimitiveType
  • Scheme type TRowid : char[14], registry, cell compare/hash
  • YQL/MiniKQL: parse/format (Base64), cast table, hash/compare/equate, pack/presort, serialization, Min/Max, SQL name Rowid
  • Convert paths: ydb_convert, mkql_proto, JSON/YSON/value formatting, SHOW CREATE, KQP literals
  • C++ SDK: TRowidValue, parser/builder, Base64 ToString()
  • UDF module + C++ sort-order UT + YQL canon tests
  • Linked into ydbd

Intentionally deferred / follow-ups

  • Arrow FixedSizeBinary(14) path (currently variable binary(), like DyNumber)
  • Full OLAP/columnshard/export/CDC/CLI coverage parity with Uuid
  • Non-C++ SDKs (Go/Python/Java) generated/hand-written bindings
  • Documentation PR for user-facing docs
  • Feature-flag / experimental gating if required by release process

Test plan

  • ./ya make --build relwithdebinfo -tA ydb/library/yql/udfs/common/rowid/ut
  • ./ya make --build relwithdebinfo -tA ydb/library/yql/udfs/common/rowid/test
  • ./ya make --build relwithdebinfo -tA yql/essentials/types/rowid
  • Builds for ydb_convert, mkql_proto, SDK value, JSON/YSON value paths
Open in Web Open in Cursor 

cursoragent and others added 10 commits August 4, 2026 16:22
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Co-authored-by: Maksim Zinal <zinal@ydb.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants