diff --git a/CHANGELOG.md b/CHANGELOG.md index dc29d64..f4362ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,10 +3,26 @@ v0.3.1 (Unreleased) ### Query translation * **`toStartOf*` date-time functions** via `EF.Functions`: `ToStartOfYear`, `ToStartOfQuarter`, `ToStartOfMonth`, `ToStartOfWeek` (with optional week `mode`), `ToStartOfDay`, `ToStartOfHour`, `ToStartOfMinute`, `ToStartOfSecond`, the fixed buckets `ToStartOfFiveMinutes` / `ToStartOfTenMinutes` / `ToStartOfFifteenMinutes`, and the general `ToStartOfInterval(source, value, unit)`. Each maps to the matching ClickHouse function and works in `GROUP BY`. Return types follow ClickHouse: the calendar buckets (`Year`/`Quarter`/`Month`/`Week`) return `Date`, the day/hour/minute buckets return `DateTime`, and `ToStartOfSecond` returns `DateTime64`. All accept `DateTime`/`DateTime64` columns, and the plain truncation functions also accept `DateOnly`; `ToStartOfInterval` requires a `DateTime`/`DateTime64` column on older ClickHouse, which rejects a `DateOnly` (Date/Date32) source with `Illegal type Date32 of 1st argument` for every unit; recent versions accept it. `ToStartOfInterval` takes a `ClickHouseInterval` unit (`Second`…`Year`) and emits `toStartOfInterval(source, toInterval(value))`; the unit must be a constant so it can be translated. The default `Date`/`DateTime` result types only span 1970–2149/2106, so ClickHouse narrows out-of-range values — enable `enable_extended_results_for_datetime_functions` (e.g. `set_enable_extended_results_for_datetime_functions=1` in the connection string) for range-preserving `Date32`/`DateTime64` results. +* **Standard `DateTime` members and methods now translate to SQL.** Previously the provider registered no date/time member translator, so only a direct comparison worked and every member threw `The LINQ expression ... could not be translated`. One shared translator serves `DateTime`, `DateTimeOffset` and `DateOnly`, because the ClickHouse function is the same for each. ([#55](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/55)) + * **Components** — `.Year` → `toYear`, `.Month` → `toMonth`, `.Day` → `toDayOfMonth`, `.Hour` → `toHour`, `.Minute` → `toMinute`, `.Second` → `toSecond`, `.Millisecond` → `toMillisecond`, `.DayOfYear` → `toDayOfYear`. These ClickHouse functions return `UInt8`/`UInt16`, which the provider's integer mappings widen to `int` on read. `DateOnly` gets the date components only, matching the members it declares. + * **`.DayOfWeek`** → `toDayOfWeek(x, 2)`. Week mode 2 agrees with `System.DayOfWeek` exactly (Sunday 0 … Saturday 6), so no arithmetic correction is applied — the default mode 0 starts the week on Monday, which is why the mode argument is always sent. The result carries a number-backed enum mapping, because this provider maps a C# `enum` to a ClickHouse string and that mapping would otherwise render `x.DayOfWeek == DayOfWeek.Sunday` as a comparison against `'Sunday'`. + * **`.Date`** → `toStartOfDay`, which keeps the timezone of the source. Note that `toStartOfDay` returns a `DateTime`, whose range is 1970–2106, and ClickHouse **wraps** a value outside that window rather than reporting it — so `.Date` on a `DateTime64` column holding a pre-1970 date reads back wrong. Enable `enable_extended_results_for_datetime_functions` (for example `set_enable_extended_results_for_datetime_functions=1` in the connection string) to get a range-preserving `DateTime64` result. This is the same caveat that already applies to `EF.Functions.ToStartOfDay`. + * **`.TimeOfDay`** → `toTime64(x, 7)`; precision 7 is one .NET tick, so no part of the value is lost (`toTime` would drop the fraction). This is the one member here that needs a recent server: `toTime64` arrived with the `Time64` type in **ClickHouse 25.6**, and an earlier server answers `Function with name 'toTime64' does not exist`. Every other function in this entry works on 24.8 LTS. + * **`DateTime.UtcNow`** → `now64(7, 'UTC')`, **`DateTime.Now`** → `now64(7)` and **`DateTime.Today`** → `toStartOfDay(now())`. `today()` is not used for `.Today` because it returns a `Date`, whereas the member's type is `DateTime`. + * **`.AddYears(n)`** → `addYears` and **`.AddMonths(n)`** → `addMonths`. Both take an `int` in .NET, and ClickHouse clamps the day of month the same way .NET does, so `2026-01-31` plus one month gives `2026-02-28` in both. A constant beyond the bound .NET applies to the argument itself — 10 000 years or 120 000 months — is not translated, because ClickHouse saturates at the year 9999 where .NET raises `ArgumentOutOfRangeException`. Only the argument can be checked while translating; whether the *result* fits depends on the column value, and a result past 9999 still saturates on the server. + * **`.AddDays`/`.AddHours`/`.AddMinutes`/`.AddSeconds`/`.AddMilliseconds`** take a `double` in .NET. .NET splits the integral and fractional parts, scales each to whole **ticks** (100 ns), and truncates any fractional tick toward zero — so `AddSeconds(0.1234567)` adds exactly 1 234 567 ticks, while `AddMilliseconds(0.99995)` adds 9 999 ticks rather than one millisecond. The matching ClickHouse function takes a whole number of its own unit and discards the rest, so `addDays(x, 1.5)` would add only one day. A constant argument is therefore folded with the .NET algorithm during translation and then expressed in the coarsest unit that holds it exactly: a whole number of the unit emits the natural function (`addDays(x, 1)`), and otherwise `addMilliseconds` carries the exact count (`AddDays(1.5)` → `addMilliseconds(x, 129600000)`). Preferring the natural function keeps the store type of the source and keeps `Date`/`Date32` columns working, since `addMilliseconds` rejects those outright. + * A **sub-millisecond** offset, a **non-constant** offset, and a value **outside the `DateTime` range** are deliberately left untranslated rather than rounded to fit. Milliseconds are as fine as the translation goes, because `addNanoseconds` would express a tick exactly but promotes the result to `DateTime64(9)`, whose Int64 nanosecond count cannot span the `DateTime64` range — that would trade a rounding error for a silently wrong date. An untranslated call still gives the correct .NET value through client evaluation in a projection, and reports a clear reason in a predicate. `DateOnly.AddDays` takes an `int`, so it always emits `addDays`. + * A previously-unsupported Northwind query, `GroupJoin_aggregate_anonymous_key_selectors2`, now passes as a result of these translations; its provider-specific "not translatable" override is removed. + * For a `DateTimeOffset` property, component results use the timezone the column declares. The default store type pins that timezone to UTC; explicitly configured named and fixed-offset zones are preserved. + * **`Add*` translates only for a column whose timezone has one offset for every instant.** ClickHouse arithmetic follows the declared timezone, and a named zone with daylight saving disagrees with .NET in two ways. The calendar functions (`addDays`, `addMonths`, `addYears`) keep the wall clock but cannot produce the hour the clocks skip — on a `DateTime64(7, 'Europe/London')` column holding `2026-03-28 01:30`, `addDays(x, 1)` answers `00:30` where .NET answers `01:30`. The absolute functions (`addHours` … `addMilliseconds`, including the `addMilliseconds` fallback that a fractional `AddDays` uses) move the instant, so an interval crossing a transition shifts the wall clock by an hour. For `DateTimeOffset` there is a third difference: .NET preserves the instance offset, which the column's zone can change. So `Add*` is translated for `'UTC'` and `Fixed/UTC±HH:MM:SS` store types, and — for `DateTime`/`DateOnly` only — for a store type that declares no timezone, which the driver reads as a UTC wall clock. Everything else is client-evaluated in a projection and reports the limitation in a predicate. **Known limit:** a timezone-less `DateTime` column still has its calendar arithmetic evaluated in the server's `session_timezone`, which is invisible during translation; declare the timezone in the store type when that setting observes daylight saving. + * Not yet translated: `.Ticks`, `.AddTicks`, the `.Microsecond`/`.Nanosecond` members, and `DateTimeOffset.Now`. The ClickHouse-specific functions such as `dateDiff` and `dateTrunc` are tracked in [#58](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/58). + * **Behaviour change:** `DateTime.Now` and `DateTime.Today` in a *projection* used to be evaluated on the client; they now read the **server** clock. The value therefore follows the server's timezone rather than the client's, and comes back with `DateTimeKind.Unspecified` instead of `Local`. Use `DateTime.UtcNow` for an instant that does not depend on server configuration. `DateTimeOffset.UtcNow` also reads a UTC-pinned server clock; `DateTimeOffset.Now` stays on the client so its local offset is preserved. In a predicate the `DateTime` clock members were untranslatable before, so nothing changes there. + ### Types * **`DateTimeOffset` is now a supported property type**, and maps to `DateTime64(7, 'UTC')`. The provider previously had no mapping for it, so EF Core fell back to `DateTimeOffsetToStringConverter` and made a `String` column with no warning — which broke queries against a real `DateTime64` column with `TYPE_MISMATCH`, and stopped `SaveChanges` from writing the value at all. The store type pins the timezone to `'UTC'` so the instant does not depend on the server's `session_timezone`, and precision 7 is one .NET tick (100 ns), so the round trip is exact. `HasPrecision(n)` is honoured for a property with no `HasColumnType`, and keeps the UTC pin. Note that ClickHouse has no type that stores a UTC offset: `DateTime64` holds an instant, and a declared timezone only decides how that instant is rendered. The instant is kept and the offset is not, so a value read back carries the offset of the column's declared timezone. A non-UTC column such as `DateTime64(6, 'Asia/Tokyo')` reads correctly, as does a fixed-offset column such as `DateTime64(7, 'Fixed/UTC+05:30:00')`, and `DateTimeOffset` composes into `Array`, `Map` and `Tuple` columns. Where an instant cannot be reproduced exactly the read throws rather than returning a value that is quietly wrong: the repeated hour when clocks go back in a zone whose candidate offsets are both non-zero (`Europe/Paris`), and a value before 1900 in a named zone, where the offset is Local Mean Time to the second. Writing a value the store type cannot hold throws for the same reason — ClickHouse wraps it rather than reporting it, which matters for precision 8 and 9, whose ranges are narrower than `DateTimeOffset`. **Behaviour change:** a `DateTimeOffset` property that relied on the old `String` column now resolves to `DateTime64(7, 'UTC')` — add `HasConversion()` to keep the previous shape. See the [DateTimeOffset section of the README](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore#datetimeoffset) for the daylight-saving and `MinValue`/`MaxValue` limits. ([#53](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/53)) ### Bug fixes +* **Subtracting one date/time value from another no longer fails with an internal error.** `dt1 - dt2` gives a `TimeSpan`, which ClickHouse has no operator for — `dateDiff` returns a count of whole units instead. The expression used to reach type-mapping inference and fail with an `InvalidCastException` or a bare `No coercion operator is defined between types ...`, both of which name CLR types the user never wrote. The subtraction is now reported as not translatable, with the reason attached. In a projection EF Core can therefore fall back to the client and return the correct `TimeSpan`; in a predicate, where no fallback exists, the message explains why and what to do instead. Shifting a date by a `TimeSpan` gets its own message, because it does have a translatable equivalent: `x.AddDays(-7)` works where `x - TimeSpan.FromDays(7)` does not. ([#55](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/55)) * `ToStartOfWeek` now rejects row-dependent week modes during query translation, and `ToStartOfInterval` likewise rejects row-dependent interval sizes. ClickHouse requires these operands to be constant for the query; literals and captured query parameters remain supported. * **Composite columns now convert their components on read.** `Array(T)`, `Map(K, V)` and `Tuple(...)` read the whole column through `GetValue`, so a component mapping's own read pipeline never ran, and any component whose CLR type differs from the driver's type threw `InvalidCastException`. This is not new with `DateTimeOffset` — `DateOnly[]`, `Dictionary` and `Tuple` were already affected, because `DateOnly` also arrives from the driver as a `DateTime`. The composite is now rebuilt component by component, with the same two steps EF Core applies to a scalar column: the mapping's data-reader conversion, then its `ValueConverter`. An `enum` component, a `List` component and a nested composite therefore all read correctly, and a component that needs no conversion keeps the direct cast. **Known limit:** *writing* a component that needs a `ValueConverter` still does not work, because the bulk insert path passes model values to the driver without applying converters ([#54](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/54)) — an `enum` inside a composite is written as its raw ordinal. * **`Array(Nullable(T))` DDL is no longer double-wrapped.** For a value-type element the store type came out as `Array(Nullable(Nullable(T)))`, which ClickHouse rejects with `Nested type Nullable(T) cannot be inside Nullable type`, so `EnsureCreated` and migrations both failed. The component mapping is resolved from a store type that already carries the wrapper, and `HasColumnType(...)` text is kept verbatim, so the nullable-element wrapper added a second one. It now adds the wrapper only when the inner store type does not already have one, including through `LowCardinality(Nullable(T))`. Reference-type elements were never affected. diff --git a/README.md b/README.md index 5371e84..fa6ca90 100644 --- a/README.md +++ b/README.md @@ -176,8 +176,9 @@ ambiguous, and it does not change before 1900. Two points of its own do: `List`, `Dictionary` and `Tuple` all round trip. -`DateTimeOffset` members such as `.Year` and `.UtcDateTime` do not translate to SQL yet. This -applies to `DateTime` as well — see [#55](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/55). +The standard members and methods — `.Year`, `.DayOfWeek`, `.AddDays(n)` and the rest — translate to +SQL; see [Date/Time Functions](#datetime-functions). `.UtcDateTime`, `.LocalDateTime` and `.Offset` +do not. ## Current Status @@ -221,6 +222,77 @@ ClickHouse returns `NULL` from a scalar subquery that matches no rows, where sta ### Date/Time Functions +#### Standard members and methods + +The standard .NET date/time members translate to ClickHouse functions, for `DateTime`, `DateTimeOffset` and `DateOnly` alike: + +| .NET | ClickHouse | +| --- | --- | +| `.Year` `.Month` `.Day` | `toYear` `toMonth` `toDayOfMonth` | +| `.Hour` `.Minute` `.Second` `.Millisecond` | `toHour` `toMinute` `toSecond` `toMillisecond` | +| `.DayOfYear` | `toDayOfYear` | +| `.DayOfWeek` | `toDayOfWeek(x, 2)` | +| `.Date` | `toStartOfDay` | +| `.TimeOfDay` | `toTime64(x, 7)` (needs ClickHouse 25.6 or later) | +| `.AddYears(n)` `.AddMonths(n)` | `addYears` `addMonths` | +| `.AddDays(n)` `.AddHours(n)` `.AddMinutes(n)` `.AddSeconds(n)` `.AddMilliseconds(n)` | `addDays` `addHours` … (see below) | +| `DateTime.UtcNow` | `now64(7, 'UTC')` | +| `DateTime.Now` | `now64(7)` | +| `DateTime.Today` | `toStartOfDay(now())` | +| `DateTimeOffset.UtcNow` | `now64(7, 'UTC')` | + +```csharp +// Runs entirely on the server +var busyHours = await ctx.Events + .Where(e => e.Timestamp.Year == 2026 && e.Timestamp.DayOfWeek == DayOfWeek.Sunday) + .GroupBy(e => e.Timestamp.Hour) + .Select(g => new { Hour = g.Key, Count = g.Count() }) + .ToListAsync(); + +var recent = await ctx.Events + .Where(e => e.Timestamp > DateTime.UtcNow.AddDays(-7)) + .ToListAsync(); +``` + +`DateOnly` gets the date components only, which are the members it declares. For a `DateTimeOffset` property the result is in the timezone the column declares. The default mapping pins that timezone to UTC, while an explicit store type can select a named or fixed-offset timezone; the value read by .NET carries that same declared-zone offset. + +Points worth knowing: + +**`.DayOfWeek` needs no correction.** ClickHouse week mode 2 agrees with `System.DayOfWeek` exactly — Sunday is 0 through to Saturday 6 — so the value is used as it comes back. The mode argument is always sent, because the default mode starts the week on Monday. + +**`DateTime.Now` and `DateTime.Today` read the server clock**, so they follow the *server's* timezone, not the client's, and they come back with `DateTimeKind.Unspecified`. Use `DateTime.UtcNow` when you need an instant that does not depend on server configuration. `DateTimeOffset.UtcNow` also reads a UTC-pinned server clock. `DateTimeOffset.Now` remains client-evaluated in a projection, because its observable local offset cannot be reconstructed from a UTC-pinned server value. + +**`.Date` narrows outside 1970–2106.** `toStartOfDay` returns a `DateTime`, and ClickHouse *wraps* a value outside that window instead of reporting it — so `.Date` on a `DateTime64` column holding a pre-1970 date reads back wrong. Enable [`enable_extended_results_for_datetime_functions`](https://clickhouse.com/docs/operations/settings/settings#enable_extended_results_for_datetime_functions) — for example `set_enable_extended_results_for_datetime_functions=1` in the connection string — to get a range-preserving `DateTime64` result. + +**A fractional `Add*` argument is exact or is not translated.** `AddDays` and the other time-based methods take a `double`. .NET splits the integral and fractional parts, scales each to *ticks* (100 ns), and truncates any fractional tick toward zero, so `AddSeconds(0.1234567)` adds exactly 1 234 567 ticks. The ClickHouse `addDays` function takes a whole number of days and discards the rest, so it cannot be used directly. A constant argument is folded with the .NET algorithm and then expressed in the coarsest unit that holds it exactly: + +```csharp +e.Timestamp.AddDays(1) // addDays(ts, 1) +e.Timestamp.AddDays(1.5) // addMilliseconds(ts, 129600000) +e.Timestamp.AddMilliseconds(0.5) // not translated — 5 000 ticks is below millisecond resolution +e.Timestamp.AddDays(offsetVariable) // not translated — cannot be checked for exactness +``` + +The natural function keeps the column's store type, and it is the only form that works on a `Date`/`Date32` column — ClickHouse rejects `addMilliseconds` on those. Anything the provider cannot express exactly is left untranslated rather than rounded to fit, so a projection still gives the correct .NET value through client evaluation, while a predicate reports why. `DateOnly.AddDays` takes an `int`, so it always emits `addDays`. + +**`.TimeOfDay` needs ClickHouse 25.6 or later.** It maps to `toTime64`, which arrived with the `Time64` type in 25.6. On an earlier server the query fails with `Function with name 'toTime64' does not exist`. Every other function on this page works on 24.8 LTS. + +**An integral `Add*` argument outside the .NET bound is not translated.** .NET rejects more than 10 000 years or 120 000 months whatever the instance holds, while ClickHouse saturates at the end of its own range — `addYears(x, 20000)` answers the year 9999. Such a call is therefore left on the client, so the `ArgumentOutOfRangeException` still happens. Note that only the argument can be checked during translation: whether the *result* also fits depends on the column value, and a result past the year 9999 still saturates on the server. + +**`Add*` requires a column whose timezone has one offset.** ClickHouse arithmetic follows the timezone the column declares, and a named zone with daylight saving disagrees with .NET in two ways. The calendar functions (`addDays`, `addMonths`, `addYears`) keep the wall clock, but cannot produce the hour the clocks skip: on a `DateTime64(7, 'Europe/London')` column holding `2026-03-28 01:30`, `addDays(x, 1)` answers `00:30` where .NET answers `01:30`. The absolute functions (`addHours` … `addMilliseconds`) move the instant, so any interval that crosses a transition shifts the wall clock by an hour. For a `DateTimeOffset` there is a third difference: .NET preserves the instance's offset, which the column's zone can change. + +The provider therefore translates `Add*` only when the store type declares `'UTC'` or a `Fixed/UTC±HH:MM:SS` offset — plus, for `DateTime` and `DateOnly`, when it declares no timezone at all, because the driver reads such a column as a UTC wall clock. Anything else stays on the client in a projection and reports the limitation in a predicate. + +> **One residual limit.** A `DateTime` column that declares no timezone still has its *calendar* arithmetic (`AddDays`, `AddMonths`, `AddYears`) evaluated in the server's `session_timezone`, which the provider cannot see while translating. If that setting names a zone with daylight saving, those results can differ from .NET by an hour. Declare the timezone in the store type — for example `HasColumnType("DateTime64(3, 'UTC')")` — to remove the ambiguity. `DateTimeOffset` is not affected, because its default store type already pins `'UTC'`. + +**Arithmetic on two date/time values is not translated.** `dt1 - dt2` and `time1 - time2` give a `TimeSpan`, and `date + timeSpan` mixes types ClickHouse rejects; `dateDiff` returns a count of whole units, and `Time64` subtraction returns a decimal number of seconds. In a projection EF Core reads the columns and does the arithmetic on the client, which gives the correct result. In a predicate there is no client fallback, so the query fails with an explanation. + +For a rolling window, use the `Add*` methods rather than a `TimeSpan`. `x > DateTime.UtcNow.AddDays(-7)` translates; `x > DateTime.UtcNow - TimeSpan.FromDays(7)` does not, and says so. + +Not yet translated: `.Ticks`, `.AddTicks`, the `.Microsecond`/`.Nanosecond` members, and `DateTimeOffset.Now`. + +#### `toStartOf*` bucketing + The ClickHouse `toStartOf*` family is exposed through `EF.Functions`, so you can bucket and truncate timestamps directly in queries, including in `GROUP BY`: `ToStartOfYear`, `ToStartOfQuarter`, `ToStartOfMonth`, `ToStartOfWeek` (with an optional ClickHouse week `mode`), `ToStartOfDay`, `ToStartOfHour`, `ToStartOfMinute`, `ToStartOfSecond`, `ToStartOfFiveMinutes`, `ToStartOfTenMinutes`, `ToStartOfFifteenMinutes`, and the general `ToStartOfInterval(source, value, unit)`. diff --git a/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseDateTimeMemberTranslator.cs b/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseDateTimeMemberTranslator.cs new file mode 100644 index 0000000..221afbd --- /dev/null +++ b/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseDateTimeMemberTranslator.cs @@ -0,0 +1,252 @@ +using System.Reflection; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Diagnostics; +using Microsoft.EntityFrameworkCore.Query; +using Microsoft.EntityFrameworkCore.Query.SqlExpressions; +using Microsoft.EntityFrameworkCore.Storage; +using Microsoft.EntityFrameworkCore.Storage.ValueConversion; + +namespace ClickHouse.EntityFrameworkCore.Query.ExpressionTranslators.Internal; + +/// +/// Translates the standard date/time members of , +/// and to ClickHouse functions. +/// +/// +/// +/// One class serves all three CLR types, because the ClickHouse function is the same for each: the +/// to* extraction functions accept Date, Date32, DateTime and +/// DateTime64 alike. registers the members that the +/// given type declares, so gets the date components only. +/// +/// +/// For the result is in the timezone the column declares. The default +/// mapping pins that timezone to UTC; an explicitly configured named or fixed-offset timezone is +/// preserved instead. In either case, a materialized value carries the same declared-zone offset, so +/// its components agree with the server result. +/// +/// +public class ClickHouseDateTimeMemberTranslator : IMemberTranslator +{ + /// + /// Week mode 2 makes toDayOfWeek agree with exactly: Sunday is 0 + /// through to Saturday is 6. The default mode 0 starts the week on Monday, so the argument is + /// required and no arithmetic correction is needed. + /// + private const byte SundayFirstWeekMode = 2; + + /// + /// One tick is 100 ns, which is Time64 precision 7, and one .NET + /// tick is also the resolution of now64(7). Asking for that precision keeps + /// exact, whereas toTime drops the fraction. + /// + private const int TickPrecision = 7; + + /// Members that map to a ClickHouse function taking the source value alone. + private static readonly Dictionary ComponentFunctions = []; + + private static readonly HashSet DayOfWeekMembers = []; + private static readonly HashSet DateMembers = []; + private static readonly HashSet TimeOfDayMembers = []; + + /// Static members that read the server clock. + private static readonly Dictionary ServerClockMembers = []; + + private readonly ISqlExpressionFactory _sqlExpressionFactory; + private readonly IRelationalTypeMappingSource _typeMappingSource; + + /// How a server-clock member is built: a function name, and whether it pins UTC. + private enum ServerClock + { + /// now64(7, 'UTC') — an exact instant, independent of server settings. + UtcNow, + + /// now64(7) — the server's local clock, in its configured timezone. + LocalNow, + + /// toStartOfDay(now()) — midnight today on the server's local clock. + LocalToday + } + + /// + /// The mapping given to a result. Built once, because it composes a + /// converter over the Int32 mapping and nothing about it varies per translation. + /// + private readonly RelationalTypeMapping? _dayOfWeekMapping; + + static ClickHouseDateTimeMemberTranslator() + { + RegisterInstanceMembers(typeof(DateTime), hasTimeComponents: true); + RegisterInstanceMembers(typeof(DateTimeOffset), hasTimeComponents: true); + RegisterInstanceMembers(typeof(DateOnly), hasTimeComponents: false); + + ServerClockMembers.Add(Property(typeof(DateTime), nameof(DateTime.UtcNow)), ServerClock.UtcNow); + ServerClockMembers.Add(Property(typeof(DateTime), nameof(DateTime.Now)), ServerClock.LocalNow); + ServerClockMembers.Add(Property(typeof(DateTime), nameof(DateTime.Today)), ServerClock.LocalToday); + + // UtcNow is an instant and the default DateTimeOffset store type is UTC-pinned. + // DateTimeOffset.Now deliberately stays untranslated: unlike UtcNow, it exposes the client's + // current local offset, which a UTC-pinned server value cannot preserve. + ServerClockMembers.Add(Property(typeof(DateTimeOffset), nameof(DateTimeOffset.UtcNow)), ServerClock.UtcNow); + } + + public ClickHouseDateTimeMemberTranslator( + ISqlExpressionFactory sqlExpressionFactory, + IRelationalTypeMappingSource typeMappingSource) + { + _sqlExpressionFactory = sqlExpressionFactory; + _typeMappingSource = typeMappingSource; + + // DayOfWeek is an enum, and this provider maps a C# enum to a ClickHouse string. That mapping + // would render the other side of a comparison as 'Sunday' against a number, so the result + // carries a number-backed enum mapping instead. + _dayOfWeekMapping = typeMappingSource.FindMapping(typeof(int)) is { } intMapping + ? (RelationalTypeMapping)intMapping.WithComposedConverter(new EnumToNumberConverter()) + : null; + } + + public SqlExpression? Translate( + SqlExpression? instance, + MemberInfo member, + Type returnType, + IDiagnosticsLogger logger) + { + if (instance is null) + { + return TranslateServerClock(member, returnType); + } + + // toYear and friends return UInt8/UInt16, which the provider's integer mappings widen on read. + if (ComponentFunctions.TryGetValue(member, out var function)) + { + return _sqlExpressionFactory.Function( + name: function, + arguments: [instance], + nullable: true, + argumentsPropagateNullability: [true], + returnType: returnType, + typeMapping: _typeMappingSource.FindMapping(returnType)); + } + + if (DayOfWeekMembers.Contains(member)) + { + return _sqlExpressionFactory.Function( + name: "toDayOfWeek", + arguments: [instance, _sqlExpressionFactory.Constant(SundayFirstWeekMode)], + nullable: true, + // Only the source propagates nullability; the week mode is a constant. + argumentsPropagateNullability: [true, false], + returnType: returnType, + typeMapping: _dayOfWeekMapping); + } + + // toStartOfDay keeps the timezone of the source, which is what DateTime.Date means for a + // column: midnight on the same calendar day that the column renders. + // + // Note that toStartOfDay returns a DateTime, whose range is 1970-2106. ClickHouse wraps a value + // outside that window rather than reporting it, so a DateTime64 column holding a date before + // 1970 reads back wrong unless the session enables + // enable_extended_results_for_datetime_functions, which widens the result to DateTime64. This + // matches EF.Functions.ToStartOfDay and is documented alongside it. + if (DateMembers.Contains(member)) + { + return _sqlExpressionFactory.Function( + name: "toStartOfDay", + arguments: [instance], + nullable: true, + argumentsPropagateNullability: [true], + returnType: returnType, + // Reuse the source's mapping only when it describes the member's own CLR type. A mapping + // for a different type cannot be coerced during materialization. + typeMapping: instance.TypeMapping?.ClrType == returnType + ? instance.TypeMapping + : _typeMappingSource.FindMapping(returnType)); + } + + if (TimeOfDayMembers.Contains(member)) + { + return _sqlExpressionFactory.Function( + name: "toTime64", + arguments: [instance, _sqlExpressionFactory.Constant(TickPrecision)], + nullable: true, + argumentsPropagateNullability: [true, false], + returnType: returnType, + typeMapping: _typeMappingSource.FindMapping($"Time64({TickPrecision})")); + } + + return null; + } + + private SqlExpression? TranslateServerClock(MemberInfo member, Type returnType) + { + if (!ServerClockMembers.TryGetValue(member, out var clock)) + { + return null; + } + + var mapping = _typeMappingSource.FindMapping(returnType); + + // DateTime.Today is midnight today. today() returns a Date, whereas the member's type is + // DateTime, so truncate the clock value instead and keep a DateTime store type. + if (clock == ServerClock.LocalToday) + { + return _sqlExpressionFactory.Function( + name: "toStartOfDay", + arguments: [Niladic("now", returnType, mapping)], + nullable: false, + argumentsPropagateNullability: [false], + returnType: returnType, + typeMapping: mapping); + } + + List arguments = [_sqlExpressionFactory.Constant(TickPrecision)]; + if (clock == ServerClock.UtcNow) + { + arguments.Add(_sqlExpressionFactory.Constant("UTC")); + } + + return _sqlExpressionFactory.Function( + name: "now64", + arguments: arguments, + nullable: false, + argumentsPropagateNullability: arguments.Select(_ => false), + returnType: returnType, + typeMapping: mapping); + } + + private SqlExpression Niladic(string name, Type returnType, RelationalTypeMapping? typeMapping) + => _sqlExpressionFactory.Function( + name: name, + arguments: [], + nullable: false, + argumentsPropagateNullability: [], + returnType: returnType, + typeMapping: typeMapping); + + private static void RegisterInstanceMembers(Type type, bool hasTimeComponents) + { + ComponentFunctions.Add(Property(type, nameof(DateTime.Year)), "toYear"); + ComponentFunctions.Add(Property(type, nameof(DateTime.Month)), "toMonth"); + ComponentFunctions.Add(Property(type, nameof(DateTime.Day)), "toDayOfMonth"); + ComponentFunctions.Add(Property(type, nameof(DateTime.DayOfYear)), "toDayOfYear"); + + DayOfWeekMembers.Add(Property(type, nameof(DateTime.DayOfWeek))); + + if (!hasTimeComponents) + { + return; + } + + ComponentFunctions.Add(Property(type, nameof(DateTime.Hour)), "toHour"); + ComponentFunctions.Add(Property(type, nameof(DateTime.Minute)), "toMinute"); + ComponentFunctions.Add(Property(type, nameof(DateTime.Second)), "toSecond"); + ComponentFunctions.Add(Property(type, nameof(DateTime.Millisecond)), "toMillisecond"); + + DateMembers.Add(Property(type, nameof(DateTime.Date))); + TimeOfDayMembers.Add(Property(type, nameof(DateTime.TimeOfDay))); + } + + private static MemberInfo Property(Type type, string name) + => type.GetProperty(name) + ?? throw new InvalidOperationException($"Property {type.Name}.{name} was not found."); +} diff --git a/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseDateTimeMethodTranslator.cs b/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseDateTimeMethodTranslator.cs index fa31725..743a230 100644 --- a/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseDateTimeMethodTranslator.cs +++ b/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseDateTimeMethodTranslator.cs @@ -1,5 +1,6 @@ using System.Reflection; using ClickHouse.EntityFrameworkCore.Metadata; +using ClickHouse.EntityFrameworkCore.Storage.Internal.Mapping; using Microsoft.EntityFrameworkCore; using Microsoft.EntityFrameworkCore.Diagnostics; using Microsoft.EntityFrameworkCore.Query; @@ -9,13 +10,47 @@ namespace ClickHouse.EntityFrameworkCore.Query.ExpressionTranslators.Internal; /// -/// Translates the EF.Functions.ToStartOf* extension methods -/// () to their ClickHouse SQL functions. +/// Translates date/time method calls to ClickHouse SQL functions: the +/// EF.Functions.ToStartOf* extension methods +/// (), and the standard Add* methods of +/// , and . Addition is +/// limited to source columns whose declared timezone has one offset for every instant, so +/// daylight-saving rules cannot make the server disagree with .NET — see +/// . /// public class ClickHouseDateTimeMethodTranslator : IMethodCallTranslator { private readonly ISqlExpressionFactory _sqlExpressionFactory; + /// The bound .NET puts on the argument of AddYears, independently of the instance. + private const int MaxAddYears = 10_000; + + /// The bound .NET puts on the argument of AddMonths, independently of the instance. + private const int MaxAddMonths = 120_000; + + /// + /// Add* methods that take an , keyed to their ClickHouse function and the + /// largest count .NET accepts. An integer count needs no precision check, but it still needs the + /// range check: ClickHouse saturates at the end of its own range where .NET throws, so + /// AddYears(20000) would return the year 9999 instead of raising + /// . + /// + private static readonly Dictionary IntegralAddMethods = []; + + /// + /// Add* methods that take a , keyed to their ClickHouse function and + /// the tick length of the method's own unit. Keeping the two families in separate dictionaries + /// makes a unit with no fixed tick length (a month, a year) unrepresentable here. + /// + private static readonly Dictionary FractionalAddMethods = []; + + private enum TickConversionResult + { + Success, + NonConstant, + OutOfRange + } + /// /// Maps the generic method definitions that take only the source value (and, for /// ToStartOfWeek(source, mode), an extra scalar argument) directly to a ClickHouse function name. @@ -105,8 +140,54 @@ void RegisterSourceOnly(string methodName, string sqlFunction) && parameters[2].ParameterType == typeof(int) && parameters[3].ParameterType == typeof(ClickHouseInterval); }) ?? throw new InvalidOperationException("Method ToStartOfInterval with strict signature not found."); + + RegisterAddMethods(typeof(DateTime), hasTimeComponents: true); + RegisterAddMethods(typeof(DateTimeOffset), hasTimeComponents: true); + + // DateOnly declares no time-based Add* method, and its AddDays takes an int. + RegisterAddMethods(typeof(DateOnly), hasTimeComponents: false); } + /// + /// Registers the Add* methods that declares. + /// + /// + /// AddYears and AddMonths take an on every supported type, so they + /// map straight onto addYears/addMonths. The time-based methods take a + /// on , which needs the exactness check that + /// applies. On , AddDays takes an + /// instead, so it is registered as integral; its only bound is the end of the + /// range. + /// + private static void RegisterAddMethods(Type type, bool hasTimeComponents) + { + IntegralAddMethods.Add(Method(type, nameof(DateTime.AddYears), typeof(int)), ("addYears", MaxAddYears)); + IntegralAddMethods.Add(Method(type, nameof(DateTime.AddMonths), typeof(int)), ("addMonths", MaxAddMonths)); + + if (!hasTimeComponents) + { + IntegralAddMethods.Add( + Method(type, nameof(DateOnly.AddDays), typeof(int)), + ("addDays", DateOnly.MaxValue.DayNumber)); + return; + } + + RegisterFractionalAdd(type, nameof(DateTime.AddDays), "addDays", TimeSpan.TicksPerDay); + RegisterFractionalAdd(type, nameof(DateTime.AddHours), "addHours", TimeSpan.TicksPerHour); + RegisterFractionalAdd(type, nameof(DateTime.AddMinutes), "addMinutes", TimeSpan.TicksPerMinute); + RegisterFractionalAdd(type, nameof(DateTime.AddSeconds), "addSeconds", TimeSpan.TicksPerSecond); + RegisterFractionalAdd(type, nameof(DateTime.AddMilliseconds), "addMilliseconds", TimeSpan.TicksPerMillisecond); + } + + private static void RegisterFractionalAdd(Type type, string methodName, string function, long ticksPerUnit) + => FractionalAddMethods.Add( + Method(type, methodName, typeof(double)), + (function, ticksPerUnit, DateTime.MaxValue.Ticks / ticksPerUnit)); + + private static MethodInfo Method(Type type, string name, Type argumentType) + => type.GetRuntimeMethod(name, [argumentType]) + ?? throw new InvalidOperationException($"Method {type.Name}.{name}({argumentType.Name}) was not found."); + public ClickHouseDateTimeMethodTranslator(ISqlExpressionFactory sqlExpressionFactory) { _sqlExpressionFactory = sqlExpressionFactory; @@ -118,6 +199,32 @@ public ClickHouseDateTimeMethodTranslator(ISqlExpressionFactory sqlExpressionFac IReadOnlyList arguments, IDiagnosticsLogger logger) { + if (instance is not null) + { + if (IsAddMethod(method) && !CanTranslateAdd(method, instance)) + { + return null; + } + + if (IntegralAddMethods.TryGetValue(method, out var integral)) + { + return IsWithinIntegralBound(arguments[0], integral.MaxUnitCount) + ? AddFunction(integral.Function, instance, arguments[0], method.ReturnType) + : null; + } + + if (FractionalAddMethods.TryGetValue(method, out var fractional)) + { + return TranslateFractionalAdd( + instance, + fractional.Function, + fractional.TicksPerUnit, + fractional.MaxUnitCount, + arguments[0], + method.ReturnType); + } + } + var genericMethod = method.IsGenericMethod ? method.GetGenericMethodDefinition() : method; if (SupportedMethods.TryGetValue(genericMethod, out var function)) @@ -180,6 +287,216 @@ public ClickHouseDateTimeMethodTranslator(ISqlExpressionFactory sqlExpressionFac return null; } + /// + /// Translates one Add* call whose .NET argument is a , or returns + /// when no exact translation exists. + /// + /// + /// + /// .NET separates the integral and fractional parts, scales each to ticks, and truncates any + /// fractional tick toward zero. Consequently, + /// AddSeconds(0.1234567) adds exactly 1 234 567 ticks. The matching ClickHouse function takes + /// a whole number of its own unit and discards the rest, so addDays(x, 1.5) would add one day. + /// The two agree only when the tick count divides exactly into the function's unit. + /// + /// + /// A constant is therefore folded to ticks here and then expressed in the coarsest unit that holds + /// it exactly. The natural unit is preferred, and not only for readability: it keeps the store type + /// of the source, and addMilliseconds rejects a Date or Date32 source outright + /// (ILLEGAL_TYPE_OF_ARGUMENT). + /// + /// + /// Everything else is left untranslated on purpose, rather than rounded to fit. This covers a + /// sub-millisecond offset and any value that is not a constant. Milliseconds are as fine as this + /// goes: addNanoseconds would express a tick exactly but promotes the result to + /// DateTime64(9), whose Int64 nanosecond count cannot span the DateTime64 range, which + /// would trade a rounding error for a silently wrong date. An untranslated call still gives the + /// correct .NET value through client evaluation in a projection, and reports a clear reason in a + /// predicate. + /// + /// + private SqlExpression? TranslateFractionalAdd( + SqlExpression instance, + string function, + long ticksPerUnit, + long maxUnitCount, + SqlExpression value, + Type returnType) + { + if (TryGetTicks(value, ticksPerUnit, maxUnitCount, out var ticks) != TickConversionResult.Success) + { + return null; + } + + if (ticks % ticksPerUnit == 0) + { + return AddFunction(function, instance, _sqlExpressionFactory.Constant(ticks / ticksPerUnit), returnType); + } + + if (ticks % TimeSpan.TicksPerMillisecond != 0) + { + return null; + } + + return AddFunction( + "addMilliseconds", + instance, + _sqlExpressionFactory.Constant(ticks / TimeSpan.TicksPerMillisecond), + returnType); + } + + /// + /// Computes the tick count that .NET would add and reports why it cannot be folded when necessary. + /// + /// + /// Mirrors .NET 10's DateTime.AddUnits implementation: reject values outside that method's + /// per-unit bound, split the integral and fractional parts, and truncate fractional ticks toward + /// zero. A value that cannot represent is rejected, so + /// the comes from .NET during client evaluation instead of + /// from a wrapped Int64 on the server, which ClickHouse reports as a decimal overflow — or, for the + /// larger magnitudes, does not report at all. + /// + private static TickConversionResult TryGetTicks( + SqlExpression value, + long ticksPerUnit, + long maxUnitCount, + out long ticks) + { + ticks = default; + + if (value is not SqlConstantExpression { Value: double constantValue }) + { + return TickConversionResult.NonConstant; + } + + if (!double.IsFinite(constantValue) || Math.Abs(constantValue) > maxUnitCount) + { + return TickConversionResult.OutOfRange; + } + + var integralPart = Math.Truncate(constantValue); + var fractionalPart = constantValue - integralPart; + ticks = (long)integralPart * ticksPerUnit; + ticks += (long)(fractionalPart * ticksPerUnit); + + return TickConversionResult.Success; + } + + internal static bool IsAddMethod(MethodInfo method) + => IntegralAddMethods.ContainsKey(method) || FractionalAddMethods.ContainsKey(method); + + /// + /// Whether an integral Add* argument is inside the bound .NET applies to it regardless of the + /// instance. A non-constant argument cannot be checked, and is translated because an + /// needs no exactness check. + /// + private static bool IsWithinIntegralBound(SqlExpression value, long maxUnitCount) + => value is not SqlConstantExpression { Value: int constantValue } + || Math.Abs((long)constantValue) <= maxUnitCount; + + /// + /// Returns a provider-specific explanation when a recognized Add* method was deliberately + /// left untranslated. + /// + internal static string? GetUnsupportedAddTranslationErrorDetails( + MethodInfo method, + SqlExpression instance, + SqlExpression value) + { + if (!IsAddMethod(method)) + { + return null; + } + + var displayName = $"{method.DeclaringType?.Name}.{method.Name}"; + + if (!CanTranslateAdd(method, instance)) + { + var timezone = (instance.TypeMapping as IClickHouseTimezoneTypeMapping)?.Timezone; + var timezoneDescription = timezone is null ? "no declared timezone" : $"timezone '{timezone}'"; + + return method.DeclaringType == typeof(DateTimeOffset) + ? $"The '{displayName}' method cannot be translated for a DateTimeOffset column with " + + $"{timezoneDescription}. .NET preserves the instance offset, while ClickHouse applies " + + "the column timezone's calendar rules and may change the offset across a daylight-saving " + + "transition. Use a UTC or Fixed/UTC offset store type, or perform the addition on the client." + : $"The '{displayName}' method cannot be translated for a column with {timezoneDescription}, " + + "because that timezone changes offset. ClickHouse calendar arithmetic keeps the wall clock " + + "but cannot produce the hour the clocks skip, and its absolute arithmetic shifts the wall " + + "clock by an hour across a transition; .NET does neither. Use a UTC or Fixed/UTC offset " + + "store type, or perform the addition on the client."; + } + + if (IntegralAddMethods.TryGetValue(method, out var integral) + && !IsWithinIntegralBound(value, integral.MaxUnitCount)) + { + return $"The '{displayName}' argument is outside the range that .NET accepts for that unit, so " + + "it cannot be translated safely. ClickHouse saturates at the end of its own range where " + + ".NET raises ArgumentOutOfRangeException. Let .NET evaluate the call to preserve it."; + } + + if (!FractionalAddMethods.TryGetValue(method, out var fractional)) + { + return null; + } + + var tickResult = TryGetTicks(value, fractional.TicksPerUnit, fractional.MaxUnitCount, out var ticks); + if (tickResult == TickConversionResult.NonConstant) + { + return $"The '{displayName}' argument must be a constant so its exact .NET tick count can be " + + "checked before translating it to ClickHouse. Perform the addition on the client when " + + "the offset is row-dependent or parameterized."; + } + + if (tickResult == TickConversionResult.OutOfRange) + { + return $"The '{displayName}' argument is outside the range that .NET accepts for that unit, so " + + "it cannot be translated safely. Let .NET evaluate the call to preserve its " + + "ArgumentOutOfRangeException."; + } + + if (ticks % fractional.TicksPerUnit != 0 + && ticks % TimeSpan.TicksPerMillisecond != 0) + { + return $"The '{displayName}' argument produces a sub-millisecond tick offset that ClickHouse " + + "cannot represent exactly without narrowing the supported DateTime64 range. Only exact " + + "whole-unit or whole-millisecond offsets are translated."; + } + + return null; + } + + /// + /// Whether the source column's declared timezone lets an Add* method keep .NET semantics. + /// + /// + /// + /// A source must declare a fixed offset. .NET preserves the instance + /// offset, which a named zone's calendar rules can change, and a store type that declares no + /// timezone leaves the rendering to the server. + /// + /// + /// A or source is read as a wall clock, so it only has + /// to avoid a zone that changes offset — see + /// for the two ways that breaks. A store + /// type with no declared timezone is read as a UTC wall clock and stays translatable. + /// + /// + private static bool CanTranslateAdd(MethodInfo method, SqlExpression instance) + => method.DeclaringType == typeof(DateTimeOffset) + ? instance.TypeMapping is ClickHouseDateTimeOffsetTypeMapping { HasFixedOffset: true } + : !ClickHouseTimezones.MayObserveDaylightSaving( + (instance.TypeMapping as IClickHouseTimezoneTypeMapping)?.Timezone); + + private SqlExpression AddFunction(string function, SqlExpression instance, SqlExpression value, Type returnType) + => _sqlExpressionFactory.Function( + name: function, + arguments: [instance, value], + nullable: true, + argumentsPropagateNullability: [true, true], + returnType: returnType, + typeMapping: instance.TypeMapping); + private static bool IsQueryConstant(SqlExpression expression) => expression is SqlConstantExpression or SqlParameterExpression; diff --git a/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseMemberTranslatorProvider.cs b/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseMemberTranslatorProvider.cs index c97a393..aab320b 100644 --- a/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseMemberTranslatorProvider.cs +++ b/src/EFCore.ClickHouse/Query/ExpressionTranslators/Internal/ClickHouseMemberTranslatorProvider.cs @@ -16,6 +16,7 @@ public ClickHouseMemberTranslatorProvider( [ new ClickHouseArrayMethodTranslator(sqlExpressionFactory, typeMappingSource), new ClickHouseStringMethodTranslator(sqlExpressionFactory), + new ClickHouseDateTimeMemberTranslator(sqlExpressionFactory, typeMappingSource), ]); } } diff --git a/src/EFCore.ClickHouse/Query/Internal/ClickHouseSqlTranslatingExpressionVisitor.cs b/src/EFCore.ClickHouse/Query/Internal/ClickHouseSqlTranslatingExpressionVisitor.cs index dc66ad3..f31eb74 100644 --- a/src/EFCore.ClickHouse/Query/Internal/ClickHouseSqlTranslatingExpressionVisitor.cs +++ b/src/EFCore.ClickHouse/Query/Internal/ClickHouseSqlTranslatingExpressionVisitor.cs @@ -1,4 +1,5 @@ using System.Linq.Expressions; +using ClickHouse.EntityFrameworkCore.Query.ExpressionTranslators.Internal; using Microsoft.EntityFrameworkCore.Query; using Microsoft.EntityFrameworkCore.Query.SqlExpressions; @@ -8,6 +9,13 @@ public class ClickHouseSqlTranslatingExpressionVisitor : RelationalSqlTranslatin { private readonly ClickHouseArrayLinqTranslator _arrayLinqTranslator; + /// + /// Reasons already reported for this translation. A single unsupported call can be reached more + /// than once — the same expression may appear twice in a predicate, and building the reason for an + /// Add* call re-visits its operands — so the set keeps the message from repeating. + /// + private readonly HashSet _reportedTranslationErrors = new(StringComparer.Ordinal); + public ClickHouseSqlTranslatingExpressionVisitor( RelationalSqlTranslatingExpressionVisitorDependencies dependencies, QueryCompilationContext queryCompilationContext, @@ -41,7 +49,100 @@ public override SqlExpression GenerateGreatest(IReadOnlyList expr /// Select(...).Contains(...) lambda patterns. /// protected override Expression VisitMethodCall(MethodCallExpression methodCallExpression) - => _arrayLinqTranslator.TryTranslate(methodCallExpression, out var translated) - ? translated - : base.VisitMethodCall(methodCallExpression); + { + if (_arrayLinqTranslator.TryTranslate(methodCallExpression, out var arrayTranslation)) + { + return arrayTranslation; + } + + var translated = base.VisitMethodCall(methodCallExpression); + + if (translated == QueryCompilationContext.NotTranslatedExpression + && ClickHouseDateTimeMethodTranslator.IsAddMethod(methodCallExpression.Method) + && methodCallExpression.Object is { } methodInstance + && methodCallExpression.Arguments is [var methodArgument] + && Visit(methodInstance) is SqlExpression sqlInstance + && Visit(methodArgument) is SqlExpression sqlArgument + && ClickHouseDateTimeMethodTranslator.GetUnsupportedAddTranslationErrorDetails( + methodCallExpression.Method, sqlInstance, sqlArgument) is { } errorDetails) + { + ReportTranslationError(errorDetails); + } + + return translated; + } + + /// Attaches a reason to the translation failure, at most once per distinct reason. + private void ReportTranslationError(string details) + { + if (_reportedTranslationErrors.Add(details)) + { + AddTranslationErrorDetails(details); + } + } + + /// + /// Reports a clear reason when two date/time values are added or subtracted. + /// + /// + /// + /// ClickHouse has no operator for any of these shapes, and each one fails differently: + /// + /// + /// + /// One date minus another gives a in .NET, whereas dateDiff + /// returns a count of whole units. + /// + /// + /// One time of day minus another gives a in .NET, whereas ClickHouse + /// Time64 subtraction gives a Decimal number of seconds. + /// + /// + /// A date plus or minus a keeps the date type in .NET, whereas ClickHouse + /// rejects the mixed operands outright (Illegal types ... of arguments of function plus). + /// + /// + /// + /// Left alone, each of these reaches type-mapping inference or the server and fails with an internal + /// cast error or raw SQL error that names types the user never wrote. Reporting the reason here turns + /// that into EF Core's normal "could not be translated" message with an explanation attached — which + /// also restores client evaluation in a projection, where the .NET result is correct. + /// + /// + protected override Expression VisitBinary(BinaryExpression binaryExpression) + { + if (binaryExpression.NodeType is ExpressionType.Add or ExpressionType.Subtract + && IsDateOrTimeType(binaryExpression.Left.Type) + && IsDateOrTimeType(binaryExpression.Right.Type)) + { + // Shifting a date by a span has a translatable equivalent, so point at it rather than + // sending the reader to the client. Subtracting two dates has none. + ReportTranslationError( + IsSpanType(binaryExpression.Right.Type) && !IsSpanType(binaryExpression.Left.Type) + ? "Adding or subtracting a TimeSpan is not supported, because ClickHouse rejects the " + + "mixed operands. Use the Add* methods instead — 'x.AddDays(-7)' translates where " + + "'x - TimeSpan.FromDays(7)' does not." + : "Arithmetic on two date or time values is not supported, because ClickHouse has no " + + "operator that matches the .NET result. Compare the two values directly, or project " + + "them and do the arithmetic on the client."); + + return QueryCompilationContext.NotTranslatedExpression; + } + + return base.VisitBinary(binaryExpression); + } + + /// Whether the type is a length of time rather than a point in time. + private static bool IsSpanType(Type type) + => (Nullable.GetUnderlyingType(type) ?? type) == typeof(TimeSpan); + + private static bool IsDateOrTimeType(Type type) + { + var unwrapped = Nullable.GetUnderlyingType(type) ?? type; + return unwrapped == typeof(DateTime) + || unwrapped == typeof(DateTimeOffset) + || unwrapped == typeof(DateOnly) + || unwrapped == typeof(TimeSpan) + || unwrapped == typeof(TimeOnly); + } } diff --git a/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTime64TypeMapping.cs b/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTime64TypeMapping.cs index f94811b..90beff9 100644 --- a/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTime64TypeMapping.cs +++ b/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTime64TypeMapping.cs @@ -3,7 +3,7 @@ namespace ClickHouse.EntityFrameworkCore.Storage.Internal.Mapping; -public class ClickHouseDateTime64TypeMapping : RelationalTypeMapping +public class ClickHouseDateTime64TypeMapping : RelationalTypeMapping, IClickHouseTimezoneTypeMapping { private const int DefaultPrecision = 3; diff --git a/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTimeOffsetTypeMapping.cs b/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTimeOffsetTypeMapping.cs index 945ef02..2114e29 100644 --- a/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTimeOffsetTypeMapping.cs +++ b/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTimeOffsetTypeMapping.cs @@ -46,7 +46,8 @@ namespace ClickHouse.EntityFrameworkCore.Storage.Internal.Mapping; /// On write, refuses a value the store type cannot hold. ClickHouse /// wraps such a value rather than reporting it. /// -public class ClickHouseDateTimeOffsetTypeMapping : RelationalTypeMapping, IClickHouseWriteValidatingTypeMapping +public class ClickHouseDateTimeOffsetTypeMapping + : RelationalTypeMapping, IClickHouseWriteValidatingTypeMapping, IClickHouseTimezoneTypeMapping { /// /// One .NET tick is 100 ns, which is precision 7. This makes the round trip exact, so a @@ -54,7 +55,7 @@ public class ClickHouseDateTimeOffsetTypeMapping : RelationalTypeMapping, IClick /// public const int DefaultPrecision = 7; - public const string DefaultTimezone = "UTC"; + public const string DefaultTimezone = ClickHouseTimezones.Utc; /// .NET cannot render more than 7 fractional digits, because a tick is its smallest unit. private const int MaxFractionalDigits = 7; @@ -68,11 +69,10 @@ public class ClickHouseDateTimeOffsetTypeMapping : RelationalTypeMapping, IClick /// /// ClickHouse spells a fixed-offset timezone Fixed/UTC±HH:MM:SS. See - /// for why the pattern is this strict. + /// for why the pattern is this strict. It is shared with the + /// query translators, which ask the same question about a source column. /// - private static readonly Regex FixedOffsetRegex = new( - @"^Fixed/UTC([+-])(\d{2}):(\d{2}):(\d{2})$", - RegexOptions.Compiled | RegexOptions.CultureInvariant); + private static readonly Regex FixedOffsetRegex = ClickHouseTimezones.FixedOffsetRegex; // DateTimeOffset holds an offset only within ±14 hours, and only in whole minutes. ClickHouse // accepts both a larger magnitude and a finer granularity, for example 'Fixed/UTC+00:00:42'. @@ -96,6 +96,13 @@ public class ClickHouseDateTimeOffsetTypeMapping : RelationalTypeMapping, IClick /// public string? Timezone { get; } + /// + /// Whether the declared timezone has one offset for every instant. Date/time addition can only + /// preserve semantics for these mappings: named zones with daylight + /// saving may change offset while .NET deliberately keeps the instance offset. + /// + internal bool HasFixedOffset => ClickHouseTimezones.IsFixedOffset(Timezone); + public ClickHouseDateTimeOffsetTypeMapping() : this(DefaultPrecision, DefaultTimezone) { diff --git a/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTimeTypeMapping.cs b/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTimeTypeMapping.cs index 6a2aa85..52c7d31 100644 --- a/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTimeTypeMapping.cs +++ b/src/EFCore.ClickHouse/Storage/Internal/Mapping/ClickHouseDateTimeTypeMapping.cs @@ -3,7 +3,7 @@ namespace ClickHouse.EntityFrameworkCore.Storage.Internal.Mapping; -public class ClickHouseDateTimeTypeMapping : RelationalTypeMapping +public class ClickHouseDateTimeTypeMapping : RelationalTypeMapping, IClickHouseTimezoneTypeMapping { public string? Timezone { get; } diff --git a/src/EFCore.ClickHouse/Storage/Internal/Mapping/IClickHouseTimezoneTypeMapping.cs b/src/EFCore.ClickHouse/Storage/Internal/Mapping/IClickHouseTimezoneTypeMapping.cs new file mode 100644 index 0000000..6a91ee3 --- /dev/null +++ b/src/EFCore.ClickHouse/Storage/Internal/Mapping/IClickHouseTimezoneTypeMapping.cs @@ -0,0 +1,68 @@ +using System.Text.RegularExpressions; + +namespace ClickHouse.EntityFrameworkCore.Storage.Internal.Mapping; + +/// +/// A date/time mapping whose ClickHouse store type can declare a timezone, such as +/// DateTime64(3, 'Europe/London'). +/// +/// +/// The declared timezone decides how ClickHouse renders an instant, and therefore how its date/time +/// functions behave. classifies the name. +/// +public interface IClickHouseTimezoneTypeMapping +{ + /// + /// The timezone the store type declares, or when it declares none. + /// + string? Timezone { get; } +} + +/// +/// Classifies the timezone name in a ClickHouse date/time store type. +/// +public static class ClickHouseTimezones +{ + /// The one named timezone that is known to have no daylight saving. + public const string Utc = "UTC"; + + /// + /// ClickHouse spells a fixed-offset timezone Fixed/UTC±HH:MM:SS. The pattern is strict + /// because the offset is read back from the captured groups, and .NET has no timezone of that name. + /// + public static readonly Regex FixedOffsetRegex = new( + @"^Fixed/UTC([+-])(\d{2}):(\d{2}):(\d{2})$", + RegexOptions.Compiled | RegexOptions.CultureInvariant); + + /// + /// Whether the timezone has one offset for every instant, so that adding a calendar unit and adding + /// the matching span of absolute time always agree. + /// + public static bool IsFixedOffset(string? timezone) + => timezone == Utc || (timezone is not null && FixedOffsetRegex.IsMatch(timezone)); + + /// + /// Whether the store type names a timezone that may change offset, which makes ClickHouse date/time + /// arithmetic disagree with .NET. + /// + /// + /// + /// A named zone with daylight saving breaks both halves of the ClickHouse add* family, in + /// different ways. The calendar functions (addDays, addMonths, addYears) keep + /// the wall clock, which normally matches .NET, but a result that lands in the hour the clocks skip + /// does not exist: measured on ClickHouse 26.7, + /// addDays(toDateTime64('2024-03-30 01:30:00', 3, 'Europe/London'), 1) gives + /// 2024-03-31 00:30, whereas .NET gives 01:30. The absolute functions + /// (addHours … addMilliseconds) move the instant, so any interval that crosses a + /// transition shifts the wall clock by an hour against .NET. + /// + /// + /// A store type that declares no timezone is not one of these. The driver reads such a column as a + /// UTC wall clock, so absolute arithmetic agrees with .NET. Its calendar arithmetic still follows the + /// server's session_timezone, which cannot be seen from here — that residual limit is + /// documented rather than translated around, because refusing it would give up the default mapping. + /// + /// + public static bool MayObserveDaylightSaving(string? timezone) + => timezone is not null && !IsFixedOffset(timezone); +} diff --git a/test/EFCore.ClickHouse.FunctionalTests/Query/NorthwindJoinQueryClickHouseTest.cs b/test/EFCore.ClickHouse.FunctionalTests/Query/NorthwindJoinQueryClickHouseTest.cs index 6aaffeb..7c26242 100644 --- a/test/EFCore.ClickHouse.FunctionalTests/Query/NorthwindJoinQueryClickHouseTest.cs +++ b/test/EFCore.ClickHouse.FunctionalTests/Query/NorthwindJoinQueryClickHouseTest.cs @@ -36,11 +36,6 @@ public override Task Take_in_collection_projection_with_FirstOrDefault_on_top_le public override Task SelectMany_with_client_eval_with_constructor(bool async) => AssertUnsupported(() => base.SelectMany_with_client_eval_with_constructor(async)); - // Complex LINQ pattern not translatable - public override Task GroupJoin_aggregate_anonymous_key_selectors2(bool async) - => Assert.ThrowsAsync( - () => base.GroupJoin_aggregate_anonymous_key_selectors2(async)); - private static async Task AssertUnsupported(Func test) => await Assert.ThrowsAsync(test); } diff --git a/test/EFCore.ClickHouse.Tests/DateTimeMemberTranslationTests.cs b/test/EFCore.ClickHouse.Tests/DateTimeMemberTranslationTests.cs new file mode 100644 index 0000000..e79357c --- /dev/null +++ b/test/EFCore.ClickHouse.Tests/DateTimeMemberTranslationTests.cs @@ -0,0 +1,1143 @@ +using System.Linq.Expressions; +using Microsoft.EntityFrameworkCore; +using Xunit; + +namespace EFCore.ClickHouse.Tests; + +public class DateTimeMemberEntity +{ + public long Id { get; set; } + + /// Mapped to ClickHouse DateTime, which holds whole seconds only. + public DateTime Timestamp { get; set; } + + /// Mapped to ClickHouse DateTime64(7). Precision 7 is one .NET tick. + public DateTime Timestamp64 { get; set; } + + /// Mapped to ClickHouse Date32. + public DateOnly Date { get; set; } + + /// + /// A on a named timezone with daylight-saving transitions. The driver reads + /// this column as a wall clock in that zone, which is what makes ClickHouse arithmetic on it + /// disagree with .NET. + /// + public DateTime TimestampLondon { get; set; } + + /// A on a timezone that declares one offset for every instant. + public DateTime TimestampUtc { get; set; } + + /// Mapped to ClickHouse DateTime64(7, 'UTC') by the DateTimeOffset mapping. + public DateTimeOffset Offset { get; set; } + + /// Mapped to a named timezone with daylight-saving transitions. + public DateTimeOffset OffsetLondon { get; set; } + + /// Mapped to a fixed-offset ClickHouse timezone. + public DateTimeOffset OffsetFixed { get; set; } + + /// Mapped to a timezone-less ClickHouse date/time type. + public DateTimeOffset OffsetNaive { get; set; } +} + +public class DateTimeMemberDbContext : DbContext +{ + public DbSet Events => Set(); + + private readonly string _connectionString; + + public DateTimeMemberDbContext(string connectionString) + { + _connectionString = connectionString; + } + + protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) + { + optionsBuilder.UseClickHouse(_connectionString); + } + + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + modelBuilder.Entity(entity => + { + entity.ToTable("datetime_member_test"); + entity.HasKey(e => e.Id); + entity.Property(e => e.Id).HasColumnName("id"); + entity.Property(e => e.Timestamp).HasColumnName("ts"); + entity.Property(e => e.Timestamp64).HasColumnName("ts64").HasColumnType("DateTime64(7)"); + entity.Property(e => e.Date).HasColumnName("d"); + entity.Property(e => e.TimestampLondon).HasColumnName("ts_london") + .HasColumnType("DateTime64(7, 'Europe/London')"); + entity.Property(e => e.TimestampUtc).HasColumnName("ts_utc") + .HasColumnType("DateTime64(7, 'UTC')"); + entity.Property(e => e.Offset).HasColumnName("off"); + entity.Property(e => e.OffsetLondon).HasColumnName("off_london") + .HasColumnType("DateTime64(7, 'Europe/London')"); + entity.Property(e => e.OffsetFixed).HasColumnName("off_fixed") + .HasColumnType("DateTime64(7, 'Fixed/UTC+05:30:00')"); + entity.Property(e => e.OffsetNaive).HasColumnName("off_naive") + .HasColumnType("DateTime64(7)"); + }); + } +} + +public class DateTimeMemberFixture : IAsyncLifetime +{ + public string ConnectionString { get; private set; } = string.Empty; + + /// + /// Row 1's instant: Sunday 2026-08-16 13:47:32.1234567. A Sunday on purpose — ClickHouse + /// toDayOfWeek gives Sunday 7 in its default mode and 0 in mode 2, so a Sunday is the day + /// that proves the mode argument reaches the server. + /// + public static readonly DateTime Instant = new DateTime(2026, 8, 16, 13, 47, 32).AddTicks(1_234_567); + + /// Row 1's time of day, to one tick. + public static readonly TimeSpan InstantTimeOfDay = TimeSpan.FromTicks(496_521_234_567); + + /// + /// Row 3's wall clock in Europe/London. The UK moves from +00:00 to +01:00 at 01:00 UTC on + /// 2026-03-29, so 01:30 on the following day does not exist. Both halves of the ClickHouse + /// add* family therefore disagree with .NET here: addDays(x, 1) keeps the wall clock + /// but cannot produce 01:30, and addHours(x, 24) moves the instant and lands on 02:30. + /// + public static readonly DateTime LondonBeforeTransition = new(2026, 3, 28, 1, 30, 0); + + public async Task InitializeAsync() + { + ConnectionString = await SharedContainer.GetConnectionStringAsync(); + + using var connection = new global::ClickHouse.Driver.ADO.ClickHouseConnection(ConnectionString); + await connection.OpenAsync(); + + using var createCmd = connection.CreateCommand(); + createCmd.CommandText = """ + CREATE TABLE datetime_member_test ( + id Int64, + ts DateTime, + ts64 DateTime64(7), + d Date32, + ts_london DateTime64(7, 'Europe/London'), + ts_utc DateTime64(7, 'UTC'), + off DateTime64(7, 'UTC'), + off_london DateTime64(7, 'Europe/London'), + off_fixed DateTime64(7, 'Fixed/UTC+05:30:00'), + off_naive DateTime64(7) + ) ENGINE = MergeTree() + ORDER BY id + """; + await createCmd.ExecuteNonQueryAsync(); + + using var insertCmd = connection.CreateCommand(); + // Row 2 is the last day of a month, so AddMonths and AddYears have a day to clamp. + // Row 3 sits just before a daylight-saving transition, so its Add* results land on a wall clock + // that ClickHouse and .NET disagree about. See DateTimeMemberFixture.LondonBeforeTransition. + insertCmd.CommandText = """ + INSERT INTO datetime_member_test + (id, ts, ts64, d, ts_london, ts_utc, + off, off_london, off_fixed, off_naive) VALUES + (1, '2026-08-16 13:47:32', '2026-08-16 13:47:32.1234567', '2026-08-16', + '2026-08-16 13:47:32.1234567', '2026-08-16 13:47:32.1234567', + '2026-08-16 13:47:32.1234567', '2026-08-16 13:47:32.1234567+00:00', + '2026-08-16 13:47:32.1234567+00:00', '2026-08-16 13:47:32.1234567'), + (2, '2026-01-31 00:00:00', '2026-01-31 00:00:00.0000000', '2026-01-31', + '2026-01-31 00:00:00.0000000', '2026-01-31 00:00:00.0000000', + '2026-01-31 00:00:00.0000000', '2026-03-28 12:00:00.0000000+00:00', + '2026-01-31 00:00:00.0000000+00:00', '2026-01-31 00:00:00.0000000'), + (3, '2026-03-28 01:30:00', '2026-03-28 01:30:00.0000000', '2026-03-28', + '2026-03-28 01:30:00.0000000', '2026-03-28 01:30:00.0000000', + '2026-03-28 01:30:00.0000000', '2026-03-28 01:30:00.0000000+00:00', + '2026-03-28 01:30:00.0000000+00:00', '2026-03-28 01:30:00.0000000') + """; + await insertCmd.ExecuteNonQueryAsync(); + } + + public Task DisposeAsync() => Task.CompletedTask; +} + +public class DateTimeMemberTranslationTest : IClassFixture +{ + private readonly DateTimeMemberFixture _fixture; + + public DateTimeMemberTranslationTest(DateTimeMemberFixture fixture) + { + _fixture = fixture; + } + + private async Task SelectSingleAsync( + Func, IQueryable> selector, + long id = 1) + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + return await selector(context.Events.AsNoTracking().Where(e => e.Id == id)).SingleAsync(); + } + + private async Task> WhereIdsAsync( + System.Linq.Expressions.Expression> predicate) + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + return await context.Events.AsNoTracking().Where(predicate) + .OrderBy(e => e.Id).Select(e => e.Id).ToListAsync(); + } + + // ---------------------------------------------------------------- components + + [Fact] + public async Task Year_translates_to_toYear() + => Assert.Equal(2026, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.Year))); + + [Fact] + public async Task Month_translates_to_toMonth() + => Assert.Equal(8, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.Month))); + + [Fact] + public async Task Day_translates_to_toDayOfMonth() + => Assert.Equal(16, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.Day))); + + [Fact] + public async Task Hour_translates_to_toHour() + => Assert.Equal(13, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.Hour))); + + [Fact] + public async Task Minute_translates_to_toMinute() + => Assert.Equal(47, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.Minute))); + + [Fact] + public async Task Second_translates_to_toSecond() + => Assert.Equal(32, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.Second))); + + [Fact] + public async Task Millisecond_translates_to_toMillisecond() + => Assert.Equal(123, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.Millisecond))); + + [Fact] + public async Task DayOfYear_translates_to_toDayOfYear() + => Assert.Equal(228, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.DayOfYear))); + + [Fact] + public async Task Components_agree_with_dotnet_on_a_second_precision_column() + { + var result = await SelectSingleAsync(q => q.Select(e => new { e.Timestamp.Year, e.Timestamp.Hour, e.Timestamp.Second })); + + Assert.Equal(DateTimeMemberFixture.Instant.Year, result.Year); + Assert.Equal(DateTimeMemberFixture.Instant.Hour, result.Hour); + Assert.Equal(DateTimeMemberFixture.Instant.Second, result.Second); + } + + // ---------------------------------------------------------------- DayOfWeek + + [Fact] + public async Task DayOfWeek_projects_the_dotnet_value() + { + var result = await SelectSingleAsync(q => q.Select(e => e.Timestamp64.DayOfWeek)); + + // .NET DayOfWeek.Sunday is 0. ClickHouse mode 2 agrees; the default mode would give 7. + Assert.Equal(DayOfWeek.Sunday, result); + } + + [Fact] + public async Task DayOfWeek_compares_against_a_dotnet_constant() + { + // The provider maps a C# enum to a ClickHouse string, so this is the test that proves the + // constant renders as a number rather than as 'Sunday'. + Assert.Equal([1L], await WhereIdsAsync(e => e.Timestamp64.DayOfWeek == DayOfWeek.Sunday)); + } + + [Fact] + public async Task DayOfWeek_of_a_Saturday_is_six() + { + // Row 2 is 2026-01-31, a Saturday. + Assert.Equal(DayOfWeek.Saturday, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.DayOfWeek), id: 2)); + } + + // ---------------------------------------------------------------- Date / TimeOfDay + + [Fact] + public async Task Date_translates_to_toStartOfDay() + => Assert.Equal(new DateTime(2026, 8, 16), await SelectSingleAsync(q => q.Select(e => e.Timestamp64.Date))); + + [Fact] + public async Task TimeOfDay_keeps_tick_precision() + { + var result = await SelectSingleAsync(q => q.Select(e => e.Timestamp64.TimeOfDay)); + + // toTime would drop the fraction; toTime64(x, 7) keeps every tick. + Assert.Equal(DateTimeMemberFixture.InstantTimeOfDay, result); + } + + // ---------------------------------------------------------------- DateOnly + + [Fact] + public async Task DateOnly_components_translate() + { + var result = await SelectSingleAsync(q => q.Select(e => new + { + e.Date.Year, + e.Date.Month, + e.Date.Day, + e.Date.DayOfYear, + e.Date.DayOfWeek + })); + + Assert.Equal(2026, result.Year); + Assert.Equal(8, result.Month); + Assert.Equal(16, result.Day); + Assert.Equal(228, result.DayOfYear); + Assert.Equal(DayOfWeek.Sunday, result.DayOfWeek); + } + + // ---------------------------------------------------------------- DateTimeOffset + + [Fact] + public async Task DateTimeOffset_components_translate() + { + var result = await SelectSingleAsync(q => q.Select(e => new + { + e.Offset.Year, + e.Offset.Month, + e.Offset.Day, + e.Offset.Hour, + e.Offset.Minute, + e.Offset.Second, + e.Offset.DayOfYear, + e.Offset.DayOfWeek + })); + + // The store type is UTC-pinned, and a value read back carries +00:00, so every component + // describes the same instant on both sides. + Assert.Equal(2026, result.Year); + Assert.Equal(8, result.Month); + Assert.Equal(16, result.Day); + Assert.Equal(13, result.Hour); + Assert.Equal(47, result.Minute); + Assert.Equal(32, result.Second); + Assert.Equal(228, result.DayOfYear); + Assert.Equal(DayOfWeek.Sunday, result.DayOfWeek); + } + + [Fact] + public async Task DateTimeOffset_components_agree_with_dotnet() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + var expected = await context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Offset).SingleAsync(); + var actual = await SelectSingleAsync(q => q.Select(e => new { e.Offset.Year, e.Offset.Hour, e.Offset.Second })); + + Assert.Equal(expected.Year, actual.Year); + Assert.Equal(expected.Hour, actual.Hour); + Assert.Equal(expected.Second, actual.Second); + } + + [Fact] + public async Task DateTimeOffset_Date_translates() + => Assert.Equal( + new DateTime(2026, 8, 16), + await SelectSingleAsync(q => q.Select(e => e.Offset.Date))); + + [Fact] + public async Task DateTimeOffset_TimeOfDay_keeps_tick_precision() + => Assert.Equal( + DateTimeMemberFixture.InstantTimeOfDay, + await SelectSingleAsync(q => q.Select(e => e.Offset.TimeOfDay))); + + [Fact] + public async Task DateTimeOffset_AddDays_translates() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Offset.AddDays(1)); + + Assert.Contains("addDays", query.ToQueryString()); + var result = await query.SingleAsync(); + Assert.Equal(new DateTimeOffset(2026, 8, 17, 13, 47, 32, TimeSpan.Zero).AddTicks(1_234_567), result); + } + + [Fact] + public async Task DateTimeOffset_AddDays_on_a_fixed_offset_mapping_translates() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var source = await context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.OffsetFixed).SingleAsync(); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.OffsetFixed.AddDays(1)); + + Assert.Contains("addDays", query.ToQueryString()); + Assert.Equal(source.AddDays(1), await query.SingleAsync()); + } + + [Fact] + public async Task DateTimeOffset_AddDays_on_a_dst_mapping_uses_client_semantics_across_the_transition() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var source = await context.Events.AsNoTracking().Where(e => e.Id == 2) + .Select(e => e.OffsetLondon).SingleAsync(); + var query = context.Events.AsNoTracking().Where(e => e.Id == 2) + .Select(e => e.OffsetLondon.AddDays(1)); + + // The UK moves from +00:00 to +01:00 on 2026-03-29. DateTimeOffset.AddDays preserves + // the source's +00:00 offset; ClickHouse addDays would instead apply London's calendar + // rules and return a value one hour earlier as an instant. + Assert.Equal(new DateTimeOffset(2026, 3, 28, 12, 0, 0, TimeSpan.Zero), source); + Assert.DoesNotContain("addDays", query.ToQueryString()); + Assert.Equal(source.AddDays(1), await query.SingleAsync()); + } + + [Fact] + public async Task DateTimeOffset_AddDays_on_a_timezone_less_mapping_uses_client_evaluation() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var source = await context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.OffsetNaive).SingleAsync(); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.OffsetNaive.AddDays(1)); + + Assert.DoesNotContain("addDays", query.ToQueryString()); + Assert.Equal(source.AddDays(1), await query.SingleAsync()); + } + + [Fact] + public async Task DateTimeOffset_AddMonths_on_a_dst_mapping_in_a_predicate_reports_the_reason() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking() + .Where(e => e.OffsetLondon.AddMonths(1) > e.OffsetLondon); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("timezone 'Europe/London'", exception.Message); + Assert.Contains("daylight-saving transition", exception.Message); + } + + [Fact] + public async Task DateTimeOffset_AddDays_on_a_timezone_less_mapping_in_a_predicate_reports_the_reason() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking() + .Where(e => e.OffsetNaive.AddDays(1) > e.OffsetNaive); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("no declared timezone", exception.Message); + } + + [Fact] + public async Task DateTimeOffset_AddMonths_clamps_like_dotnet() + => Assert.Equal( + new DateTimeOffset(2026, 2, 28, 0, 0, 0, TimeSpan.Zero), + await SelectSingleAsync(q => q.Select(e => e.Offset.AddMonths(1)), id: 2)); + + [Fact] + public async Task DateTimeOffset_DayOfWeek_compares_against_a_dotnet_constant() + => Assert.Equal([1L], await WhereIdsAsync(e => e.Offset.DayOfWeek == DayOfWeek.Sunday)); + + [Fact] + public async Task DateTimeOffset_UtcNow_runs_on_the_server() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking() + .Where(e => e.Offset < DateTimeOffset.UtcNow.AddYears(100)) + .Select(e => e.Id); + + Assert.Contains("now64", query.ToQueryString()); + Assert.Equal([1L, 2L, 3L], await query.OrderBy(id => id).ToListAsync()); + } + + // ---------------------------------------------------------------- Add* + + [Fact] + public async Task AddYears_translates_to_addYears() + => Assert.Equal( + DateTimeMemberFixture.Instant.AddYears(1), + await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddYears(1)))); + + [Fact] + public async Task AddMonths_translates_to_addMonths() + => Assert.Equal( + DateTimeMemberFixture.Instant.AddMonths(1), + await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddMonths(1)))); + + [Fact] + public async Task AddMonths_clamps_the_day_like_dotnet() + { + // Row 2 is 2026-01-31, so one month lands on 2026-02-28 in both systems. + var result = await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddMonths(1)), id: 2); + + Assert.Equal(new DateTime(2026, 2, 28), result); + Assert.Equal(new DateTime(2026, 1, 31).AddMonths(1), result); + } + + [Fact] + public async Task AddDays_with_a_whole_number_translates_to_addDays() + => Assert.Equal( + DateTimeMemberFixture.Instant.AddDays(1), + await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddDays(1)))); + + [Fact] + public async Task AddDays_with_a_negative_whole_number_translates_to_addDays() + => Assert.Equal( + DateTimeMemberFixture.Instant.AddDays(-1), + await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddDays(-1)))); + + [Fact] + public async Task AddDays_with_a_fraction_keeps_dotnet_semantics() + { + // addDays(x, 1.5) would discard the fraction and add one day. The translation must not. + var result = await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddDays(1.5))); + + Assert.Equal(DateTimeMemberFixture.Instant.AddDays(1.5), result); + Assert.Equal(new DateTime(2026, 8, 18, 1, 47, 32).AddTicks(1_234_567), result); + } + + [Fact] + public async Task AddHours_translates() + => Assert.Equal( + DateTimeMemberFixture.Instant.AddHours(2), + await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddHours(2)))); + + [Fact] + public async Task AddMinutes_with_a_fraction_keeps_dotnet_semantics() + => Assert.Equal( + DateTimeMemberFixture.Instant.AddMinutes(0.5), + await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddMinutes(0.5)))); + + [Fact] + public async Task AddSeconds_with_a_fraction_keeps_dotnet_semantics() + => Assert.Equal( + DateTimeMemberFixture.Instant.AddSeconds(1.5), + await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddSeconds(1.5)))); + + [Fact] + public async Task AddMilliseconds_translates() + => Assert.Equal( + DateTimeMemberFixture.Instant.AddMilliseconds(1), + await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddMilliseconds(1)))); + + [Fact] + public async Task AddMilliseconds_truncates_positive_fractional_ticks_like_dotnet() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddMilliseconds(0.99995)); + + // .NET 10 truncates 9 999.5 fractional ticks toward zero. Rounding would incorrectly + // turn this into one whole millisecond and make it look translatable. + Assert.DoesNotContain("addMilliseconds", query.ToQueryString()); + Assert.Equal(DateTimeMemberFixture.Instant.AddTicks(9_999), await query.SingleAsync()); + } + + [Fact] + public async Task AddMilliseconds_truncates_negative_fractional_ticks_like_dotnet() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddMilliseconds(-0.99995)); + + Assert.DoesNotContain("addMilliseconds", query.ToQueryString()); + Assert.Equal(DateTimeMemberFixture.Instant.AddTicks(-9_999), await query.SingleAsync()); + } + + [Fact] + public async Task AddDays_on_a_Date32_column_keeps_the_date_store_type() + { + // DateOnly.AddDays takes an int, so this must use addDays — addMilliseconds rejects a Date32. + var result = await SelectSingleAsync(q => q.Select(e => e.Date.AddDays(1))); + + Assert.Equal(new DateOnly(2026, 8, 17), result); + } + + [Fact] + public async Task AddMonths_on_a_Date32_column_clamps_like_dotnet() + => Assert.Equal( + new DateOnly(2026, 2, 28), + await SelectSingleAsync(q => q.Select(e => e.Date.AddMonths(1)), id: 2)); + + [Fact] + public async Task AddSeconds_below_millisecond_resolution_keeps_dotnet_semantics() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddSeconds(0.1234567)); + + // .NET adds exactly 1 234 567 ticks. No ClickHouse unit holds that without promoting the result + // to DateTime64(9), so the call is left untranslated and the client supplies the exact value. + Assert.DoesNotContain("addSeconds", query.ToQueryString()); + Assert.DoesNotContain("addMilliseconds", query.ToQueryString()); + + var result = await query.SingleAsync(); + Assert.Equal(DateTimeMemberFixture.Instant.AddSeconds(0.1234567), result); + Assert.Equal(DateTimeMemberFixture.Instant.AddTicks(1_234_567), result); + } + + [Fact] + public async Task AddMilliseconds_below_millisecond_resolution_keeps_dotnet_semantics() + { + // This is exactly 5 000 ticks — not 0 ms and not 1 ms. + var result = await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddMilliseconds(0.5))); + + Assert.Equal(DateTimeMemberFixture.Instant.AddMilliseconds(0.5), result); + Assert.Equal(DateTimeMemberFixture.Instant.AddTicks(5_000), result); + } + + [Fact] + public async Task AddDays_with_a_parameter_keeps_dotnet_semantics() + { + var days = 1.5; + + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddDays(days)); + + // A parameter cannot be checked for exactness, so it is not translated. Rounding it on the server + // would disagree with .NET, because ClickHouse round() is banker's rounding. + Assert.DoesNotContain("addDays", query.ToQueryString()); + Assert.Equal(DateTimeMemberFixture.Instant.AddDays(1.5), await query.SingleAsync()); + } + + [Fact] + public async Task AddDays_with_a_parameter_in_a_predicate_reports_a_reason() + { + var days = 1.5; + + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Timestamp64.AddDays(days) > e.Timestamp); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("argument must be a constant", exception.Message); + } + + [Fact] + public async Task AddMilliseconds_below_millisecond_resolution_in_a_predicate_reports_the_reason() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking() + .Where(e => e.Timestamp64.AddMilliseconds(0.99995) > e.Timestamp64); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("sub-millisecond tick offset", exception.Message); + } + + [Fact] + public async Task AddSeconds_above_the_positive_dotnet_unit_bound_is_not_translated() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddSeconds(315_537_897_599.5)); + + Assert.DoesNotContain("addSeconds", query.ToQueryString()); + Assert.DoesNotContain("addMilliseconds", query.ToQueryString()); + await Assert.ThrowsAsync(() => query.SingleAsync()); + } + + [Fact] + public async Task AddSeconds_below_the_negative_dotnet_unit_bound_is_not_translated() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddSeconds(-315_537_897_599.5)); + + Assert.DoesNotContain("addSeconds", query.ToQueryString()); + Assert.DoesNotContain("addMilliseconds", query.ToQueryString()); + await Assert.ThrowsAsync(() => query.SingleAsync()); + } + + [Fact] + public async Task AddSeconds_outside_the_dotnet_unit_bound_in_a_predicate_reports_the_reason() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking() + .Where(e => e.Timestamp64.AddSeconds(315_537_897_599.5) > e.Timestamp64); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("outside the range that .NET accepts", exception.Message); + } + + [Fact] + public async Task AddDays_beyond_the_dotnet_range_throws_the_dotnet_exception() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddDays(1e30)); + + // Folding this would wrap to Int64.MaxValue and give a server decimal-overflow error, or worse a + // silently wrong date. Left untranslated, .NET raises its own exception on the client. + await Assert.ThrowsAsync(() => query.SingleAsync()); + } + + // ------------------------------------------------- Add* on a daylight-saving DateTime column + + [Fact] + public async Task AddDays_on_a_dst_mapping_uses_client_semantics_through_the_skipped_hour() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var source = await context.Events.AsNoTracking().Where(e => e.Id == 3) + .Select(e => e.TimestampLondon).SingleAsync(); + var query = context.Events.AsNoTracking().Where(e => e.Id == 3) + .Select(e => e.TimestampLondon.AddDays(1)); + + // 2026-03-29 01:30 does not exist in London: the clocks go straight from 01:00 to 02:00. + // ClickHouse addDays keeps the wall clock but cannot land there, and answers 00:30 instead. + Assert.Equal(DateTimeMemberFixture.LondonBeforeTransition, source); + Assert.DoesNotContain("addDays", query.ToQueryString()); + Assert.Equal(new DateTime(2026, 3, 29, 1, 30, 0), await query.SingleAsync()); + } + + [Fact] + public async Task AddHours_on_a_dst_mapping_uses_client_semantics_across_the_transition() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 3) + .Select(e => e.TimestampLondon.AddHours(24)); + + // addHours moves the instant, so the server would render 02:30 where .NET keeps the wall + // clock and gives 01:30. + Assert.DoesNotContain("addHours", query.ToQueryString()); + Assert.Equal(new DateTime(2026, 3, 29, 1, 30, 0), await query.SingleAsync()); + } + + [Fact] + public async Task AddDays_with_a_fraction_on_a_dst_mapping_uses_client_semantics() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 3) + .Select(e => e.TimestampLondon.AddDays(1.5)); + + // The addMilliseconds fallback is absolute too, so the whole family stays on the client. + Assert.DoesNotContain("addMilliseconds", query.ToQueryString()); + Assert.Equal( + DateTimeMemberFixture.LondonBeforeTransition.AddDays(1.5), await query.SingleAsync()); + } + + [Fact] + public async Task AddDays_on_a_dst_mapping_in_a_predicate_reports_the_reason() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking() + .Where(e => e.TimestampLondon.AddDays(1) > e.TimestampLondon); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("timezone 'Europe/London'", exception.Message); + Assert.Contains("changes offset", exception.Message); + } + + [Fact] + public async Task AddHours_on_a_utc_mapping_still_translates() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 3) + .Select(e => e.TimestampUtc.AddHours(24)); + + // A declared UTC zone has one offset for every instant, so the gate must not catch it. + Assert.Contains("addHours", query.ToQueryString()); + Assert.Equal(new DateTime(2026, 3, 29, 1, 30, 0), await query.SingleAsync()); + } + + [Fact] + public async Task AddDays_on_a_timezone_less_mapping_still_translates() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 3) + .Select(e => e.Timestamp64.AddDays(1)); + + // The driver reads a timezone-less column as a UTC wall clock, so this stays translatable. + Assert.Contains("addDays", query.ToQueryString()); + Assert.Equal(new DateTime(2026, 3, 29, 1, 30, 0), await query.SingleAsync()); + } + + // ------------------------------------------------- integral Add* range + + [Fact] + public async Task AddYears_above_the_dotnet_bound_is_not_translated() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddYears(20_000)); + + // ClickHouse saturates at the year 9999; .NET raises instead, and that is what must survive. + Assert.DoesNotContain("addYears", query.ToQueryString()); + await Assert.ThrowsAsync(() => query.SingleAsync()); + } + + [Fact] + public async Task AddMonths_above_the_dotnet_bound_is_not_translated() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddMonths(500_000)); + + Assert.DoesNotContain("addMonths", query.ToQueryString()); + await Assert.ThrowsAsync(() => query.SingleAsync()); + } + + [Fact] + public async Task DateOnly_AddDays_above_the_dotnet_bound_is_not_translated() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Date.AddDays(4_000_000)); + + Assert.DoesNotContain("addDays", query.ToQueryString()); + await Assert.ThrowsAsync(() => query.SingleAsync()); + } + + [Fact] + public async Task AddYears_above_the_dotnet_bound_in_a_predicate_reports_the_reason() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking() + .Where(e => e.Timestamp64.AddYears(20_000) > e.Timestamp64); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("outside the range that .NET accepts", exception.Message); + } + + [Fact] + public async Task AddYears_at_the_dotnet_bound_still_translates() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + // The bound is inclusive, and only the argument is checked here — the instance decides whether + // the result also fits, and that cannot be known during translation. + Assert.Contains( + "addYears", + context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddYears(10_000)).ToQueryString()); + } + + [Fact] + public async Task AddYears_with_a_parameter_still_translates() + { + var years = 1; + + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.AddYears(years)); + + // An int needs no exactness check, so a parameter is translated even though its magnitude + // cannot be checked. + Assert.Contains("addYears", query.ToQueryString()); + Assert.Equal(DateTimeMemberFixture.Instant.AddYears(1), await query.SingleAsync()); + } + + [Fact] + public async Task Add_composes_with_a_component_member() + => Assert.Equal(2027, await SelectSingleAsync(q => q.Select(e => e.Timestamp64.AddYears(1).Year))); + + // ---------------------------------------------------------------- server clock + + [Fact] + public async Task UtcNow_runs_on_the_server() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + // Both bounds are offset by a century so the outcome does not depend on the day the suite runs. + var query = context.Events.AsNoTracking().Where(e => e.Timestamp64 < DateTime.UtcNow.AddYears(100)) + .Select(e => e.Id); + + // If EF Core evaluated DateTime.UtcNow on the client, the SQL would carry a literal instead. + Assert.Contains("now64", query.ToQueryString()); + Assert.Equal([1L, 2L, 3L], await query.OrderBy(id => id).ToListAsync()); + + var none = await context.Events.AsNoTracking() + .Where(e => e.Timestamp64 < DateTime.UtcNow.AddYears(-100)) + .Select(e => e.Id).ToListAsync(); + + Assert.Empty(none); + } + + [Fact] + public async Task Today_runs_on_the_server() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking().Where(e => e.Timestamp64 < DateTime.Today) + .Select(e => e.Id); + + Assert.Contains("toStartOfDay", query.ToQueryString()); + Assert.Contains("now()", query.ToQueryString()); + + // Execute it too: a SQL-shape assertion alone would not catch the server rejecting the call. + var beforeToday = await query.ToListAsync(); + + // Row 2 is 2026-01-31, which is before today on any day this suite can run. + Assert.Contains(2L, beforeToday); + } + + // ---------------------------------------------------------------- subtraction + + [Fact] + public async Task Subtracting_two_date_times_in_a_projection_now_works() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + // Previously this failed with an opaque cast/coercion error from type-mapping inference. The + // subtraction is now reported as not translatable, so EF Core reads both columns and subtracts + // on the client, which is the correct .NET result. + var result = await context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64 - e.Timestamp).SingleAsync(); + + Assert.Equal(TimeSpan.FromTicks(1_234_567), result); + } + + [Fact] + public async Task Subtracting_two_date_times_in_a_predicate_reports_a_clear_reason() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + // A predicate cannot fall back to the client, so this is where the reason must surface. + var query = context.Events.AsNoTracking() + .Where(e => e.Timestamp64 - e.Timestamp > TimeSpan.Zero); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("Arithmetic on two date or time values", exception.Message); + } + + [Fact] + public async Task Subtracting_a_TimeSpan_in_a_predicate_points_at_the_Add_methods() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + // 'x - TimeSpan.FromDays(7)' is a common way to write a rolling window, and the advice for + // subtracting two dates does not fit it: AddDays(-7) translates. + var query = context.Events.AsNoTracking() + .Where(e => e.Timestamp64 > DateTime.UtcNow - TimeSpan.FromDays(7)); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + Assert.Contains("Use the Add* methods instead", exception.Message); + Assert.Contains("AddDays(-7)", exception.Message); + } + + [Fact] + public async Task A_repeated_untranslatable_call_reports_its_reason_once() + { + var days = 1.5; + + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + var query = context.Events.AsNoTracking() + .Where(e => e.Timestamp64.AddDays(days) > e.Timestamp + && e.Timestamp64.AddDays(days) < e.Timestamp); + + var exception = await Assert.ThrowsAsync(() => query.ToListAsync()); + + const string reason = "argument must be a constant"; + Assert.Equal(1, CountOccurrences(exception.Message, reason)); + } + + private static int CountOccurrences(string haystack, string needle) + { + var count = 0; + for (var i = haystack.IndexOf(needle, StringComparison.Ordinal); + i >= 0; + i = haystack.IndexOf(needle, i + needle.Length, StringComparison.Ordinal)) + { + count++; + } + + return count; + } + + [Fact] + public async Task Subtracting_two_times_of_day_in_a_projection_works() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + // ClickHouse Time64 subtraction gives a Decimal of seconds, not a TimeSpan, so this must stay + // on the client rather than emit SQL that materializes into the wrong type. + var result = await context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.TimeOfDay - e.Timestamp.TimeOfDay).SingleAsync(); + + Assert.Equal(TimeSpan.FromTicks(1_234_567), result); + } + + [Fact] + public async Task Adding_a_time_of_day_to_a_date_in_a_projection_works() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + // ClickHouse rejects DateTime + Time64 outright, so this must stay on the client too. + var result = await context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64.Date + e.Timestamp64.TimeOfDay).SingleAsync(); + + Assert.Equal(DateTimeMemberFixture.Instant, result); + } + + [Fact] + public async Task Adding_a_TimeSpan_to_a_date_in_a_projection_works() + { + await using var context = new DateTimeMemberDbContext(_fixture.ConnectionString); + + var result = await context.Events.AsNoTracking().Where(e => e.Id == 1) + .Select(e => e.Timestamp64 + TimeSpan.FromHours(1)).SingleAsync(); + + Assert.Equal(DateTimeMemberFixture.Instant.AddHours(1), result); + } +} + +public class DateTimeMemberTranslationOfflineTest +{ + private sealed class OfflineContext : DbContext + { + public DbSet Events => Set(); + + protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) + { + optionsBuilder.UseClickHouse("Host=localhost;Protocol=http;Port=8123;Database=test"); + } + + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + modelBuilder.Entity(entity => + { + entity.ToTable("datetime_member_test"); + entity.HasKey(e => e.Id); + entity.Property(e => e.Timestamp64).HasColumnType("DateTime64(7)"); + entity.Property(e => e.OffsetLondon).HasColumnType("DateTime64(7, 'Europe/London')"); + entity.Property(e => e.OffsetFixed).HasColumnType("DateTime64(7, 'Fixed/UTC+05:30:00')"); + entity.Property(e => e.OffsetNaive).HasColumnType("DateTime64(7)"); + }); + } + } + + private static string Sql(Func, IQueryable> selector) + { + using var context = new OfflineContext(); + return selector(context.Events).ToQueryString(); + } + + private static void AssertNonFiniteAddReportsOutOfRange( + Expression> predicate) + { + using var context = new OfflineContext(); + var query = context.Events.Where(predicate); + + var exception = Assert.Throws(() => query.ToQueryString()); + + Assert.Contains("outside the range that .NET accepts", exception.Message); + } + + [Fact] + public void Year_emits_toYear() + => Assert.Contains("toYear(", Sql(q => q.Select(e => e.Timestamp64.Year))); + + [Fact] + public void Day_emits_toDayOfMonth() + => Assert.Contains("toDayOfMonth(", Sql(q => q.Select(e => e.Timestamp64.Day))); + + [Fact] + public void DayOfWeek_emits_week_mode_two() + { + var sql = Sql(q => q.Select(e => e.Timestamp64.DayOfWeek)); + + // Assert the mode argument itself — a bare "2" would also match toDayOfWeek(x, 12). + Assert.Contains(", 2)", sql); + Assert.Contains("toDayOfWeek(", sql); + } + + [Fact] + public void DayOfWeek_comparison_emits_a_number_not_a_string() + { + var sql = Sql(q => q.Where(e => e.Timestamp64.DayOfWeek == DayOfWeek.Sunday).Select(e => e.Id)); + + Assert.DoesNotContain("'Sunday'", sql); + Assert.Contains("= 0", sql); + } + + [Fact] + public void TimeOfDay_emits_toTime64_with_tick_precision() + => Assert.Contains("toTime64(", Sql(q => q.Select(e => e.Timestamp64.TimeOfDay))); + + [Fact] + public void AddDays_with_a_whole_number_emits_addDays() + { + var sql = Sql(q => q.Select(e => e.Timestamp64.AddDays(1))); + + Assert.Contains("addDays(", sql); + Assert.DoesNotContain("addMilliseconds", sql); + } + + [Fact] + public void AddDays_with_a_fraction_emits_addMilliseconds() + { + var sql = Sql(q => q.Select(e => e.Timestamp64.AddDays(1.5))); + + // 1.5 days is exactly 129 600 000 ms, folded at translation time. + Assert.Contains("addMilliseconds(", sql); + Assert.Contains("129600000", sql); + } + + [Fact] + public void AddMilliseconds_below_millisecond_resolution_emits_no_add_function() + { + // 0.5 ms is 5 000 ticks, which no ClickHouse unit holds exactly without promoting to + // DateTime64(9), so the call must not be translated. + var sql = Sql(q => q.Select(e => e.Timestamp64.AddMilliseconds(0.5))); + + Assert.DoesNotContain("addMilliseconds", sql); + Assert.DoesNotContain("addNanoseconds", sql); + } + + [Fact] + public void AddSeconds_with_a_whole_number_of_milliseconds_emits_addMilliseconds() + { + // 1.5 s is 1 500 ms exactly, so it is still translatable — just not in seconds. + var sql = Sql(q => q.Select(e => e.Timestamp64.AddSeconds(1.5))); + + Assert.Contains("addMilliseconds(", sql); + Assert.Contains("1500", sql); + } + + [Fact] + public void AddHours_with_a_whole_number_emits_addHours() + => Assert.Contains("addHours(", Sql(q => q.Select(e => e.Timestamp64.AddHours(3)))); + + [Fact] + public void DateTimeOffset_AddDays_on_a_fixed_offset_mapping_emits_addDays() + => Assert.Contains("addDays(", Sql(q => q.Select(e => e.OffsetFixed.AddDays(1)))); + + [Fact] + public void DateTimeOffset_AddDays_on_a_dst_mapping_emits_no_add_function() + => Assert.DoesNotContain("addDays(", Sql(q => q.Select(e => e.OffsetLondon.AddDays(1)))); + + [Fact] + public void AddSeconds_with_nan_reports_out_of_range() + => AssertNonFiniteAddReportsOutOfRange(e => e.Timestamp64.AddSeconds(double.NaN) > e.Timestamp64); + + [Fact] + public void AddSeconds_with_positive_infinity_reports_out_of_range() + => AssertNonFiniteAddReportsOutOfRange( + e => e.Timestamp64.AddSeconds(double.PositiveInfinity) > e.Timestamp64); + + [Fact] + public void AddSeconds_with_negative_infinity_reports_out_of_range() + => AssertNonFiniteAddReportsOutOfRange( + e => e.Timestamp64.AddSeconds(double.NegativeInfinity) > e.Timestamp64); + + [Fact] + public void UtcNow_emits_a_utc_pinned_now64() + => Assert.Contains("now64(7, 'UTC')", Sql(q => q.Where(e => e.Timestamp64 < DateTime.UtcNow).Select(e => e.Id))); + + [Fact] + public void Now_emits_a_timezone_less_now64() + { + var sql = Sql(q => q.Where(e => e.Timestamp64 < DateTime.Now).Select(e => e.Id)); + + Assert.Contains("now64(7)", sql); + Assert.DoesNotContain("'UTC'", sql); + } + + [Fact] + public void DateTimeOffset_UtcNow_emits_a_utc_pinned_now64() + => Assert.Contains( + "now64(7, 'UTC')", + Sql(q => q.Where(e => e.Offset < DateTimeOffset.UtcNow).Select(e => e.Id))); + + [Fact] + public void DateTimeOffset_Now_is_left_for_client_evaluation() + => Assert.DoesNotContain("now64", Sql(q => q.Select(_ => DateTimeOffset.Now))); +}