Support vector value type

[lolski]

Product change and motivation

Adds support for declaring vector. This lets users work with embeddings, and later express vector-similarity queries.

1. Declaring a vector embedding attribute

define
attribute embedding,
    value vector(64, "float32");

2. Inserting a vector embedding attribute: using declarative syntax

insert
$x isa document,
    has embedding vector([1.0, 2.0, 3.0], "float32");

3. Inserting a vector embedding attribute: using given syntax

given $e: vector(64, "float8");
insert
$x isa document,
    has embedding $e;

Implementation

• Updated Pest grammar to allow array declarations
• AST & parser: BuiltinValueTypeArray struct and visitors
• Updated tests: declaring double array, inserting an array, and calling a function using an array

[krishnangovindraj]

If not function, why function shaped :(

[flyingsilverfin]

@krishnangovindraj rust has the same:

// 1. Definition
struct Color(i32, i32, i32);
struct Point(f64, f64);

[lolski]

as a reader i don't find it confusing as it's easy to differentiate them based on the context. in schema declaration where a type is expected, seeing vector(64, "float32") immediately tells me that 64 and "float32" is a type parameter of vector rather than arguments to a function in a function call.

[krishnangovindraj]

But this is the context for an expression rather than a type-declaration. It's not crazy to overload syntax, but this looks like it is just a built-in function named vector? And maybe it should be, because:

• It's not grouped with the other value_literals, so it presumably can't be used in contexts where they can.
• The first argument is an expression_list, so it's going to be sent to the ExpressionExecutor all the time anyway.
• It's technically not a "literal" if it has to be evaluated.

[dmitrii-ubskii]

A side effect of this is that nonsense like vector([2024-12-12], "supercalifragilisticexpialidocious"); is not rejected by this grammar. It will fail to parse as a vector_literal, fall back to an expression_function, and the user gets an error saying Could not resolve function with name 'vector'. instead of a more useful error message like Built-in function 'vector' cannot be applied to ....

If the vector literal constructor is function-like, I don't think it belongs in the grammar.

[flyingsilverfin]

This will return a vector of expressions to the server yes? Do you want to do the validation there? I don't see any we wouldn't allow it, but just to be certain - you'll allow things like vector([1+1, 5.0], "float32") yes?

[dmitrii-ubskii]

"float32" (etc. down the line) should be separate rules in the pest grammar, and vector_precision should be compound atomic ${ }.
We already have the parser, we shouldn't need to trim its outputs.

[dmitrii-ubskii]

A side effect of this is that nonsense like vector([2024-12-12], "supercalifragilisticexpialidocious"); is not rejected by this grammar. It will fail to parse as a vector_literal, fall back to an expression_function, and the user gets an error saying Could not resolve function with name 'vector'. instead of a more useful error message like Built-in function 'vector' cannot be applied to ....

If the vector literal constructor is function-like, I don't think it belongs in the grammar.