Add entityType discriminator and isolate tables from views - #683
Open
ruolin59 wants to merge 12 commits into
Open
Add entityType discriminator and isolate tables from views#683ruolin59 wants to merge 12 commits into
ruolin59 wants to merge 12 commits into
Conversation
Tables and views share one (databaseId, objectId) pointer key space, so a name must resolve to exactly one catalog object. This adds a nullable entityType discriminator end-to-end and makes every table path aware of it. Semantics: NULL and any case spelling of TABLE mean table; any case spelling of VIEW means view; any other non-null value fails closed. The column is nullable with no backfill, so existing rows and existing table writes are untouched -- ordinary commits still write no discriminator. Read paths filter in the query, never by post-filtering a returned Page. A fetch-then-filter implementation returns short pages and inflated totals; the predicate and its countQuery are the same shared String constant, so content and count cannot diverge. Applied to both /hts query families, the internal catalog listings, listHouseTables, searchTables, and database enumeration. Write paths separate typed load from name occupancy. findById and findTableRefById answer "can this be loaded as a table?" and hide non-table rows; the new findOccupyingEntityTypeById answers "is this name taken, and by what?" without parsing metadata. CREATE and rename-destination consult occupancy before authorization, storage allocation, metadata writes, and pointer saves, so a collision is an accurate 409 rather than a misleading concurrency error. HTS errors propagate rather than reading as a free name. The drop guard lives in findTableRefById and OpenHouseInternalCatalog rather than doRefresh, because deleteTable deliberately bypasses loadTable so drops survive corrupted metadata; a doRefresh-only guard would be inert there. Wrong-type read and drop return 404; collisions return 409. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ruolin59
force-pushed
the
views-entity-type-discriminator
branch
from
August 13, 2026 23:54
dddcf42 to
76c72fb
Compare
The @query annotations added to HouseTableRepository were inert in production and broke a universal convention in this repo, so this reverts that interface to its pre-change state and drops the tests that only exercised them. Why they were inert: TablesSpringApplication excludes DataSourceAutoConfiguration, so the tables service has no DataSource bean and the only @EnableJpaRepositories scan is HTS-scoped. No Spring Data proxy of HouseTableRepository can ever be created there. The sole bean behind that interface is the hand-written HouseTableRepositoryImpl, which ignores @query entirely and talks to HTS over HTTP. HTS in turn already applies the same table-only predicate in SQL inside UserTableHtsJdbcRepository, so production filtering is complete without these annotations. Why they were wrong stylistically: only a handful of files in this repo carry @query, and every one of them executes against a real database. The established precedent for exactly this shape is HtsRepository, an empty interface whose JPA semantics live entirely on its impl/jdbc class. Production interfaces declare the contract; implementations own behavior. Restoring the interface puts HouseTableRepository back in line with that, and leaves internalcatalog's main sources with no spring-data-jpa usage at all. Why the removed tests go with them: the eleven deleted listing tests in RepositoryTest, DatabasesControllerTest and TablesControllerTest ran against the H2 Spring Data double, where the annotations did take effect. The production methods they covered (listTables, listHouseTables, searchTables, findAllIds) are byte-for-byte unchanged by this change set, so those tests were verifying a test double rather than production code. The genuine coverage for the same acceptance criteria lives in services/housetables, where the predicate actually runs in SQL. Every view isolation guard test that exercises real production logic is kept. Adding the same filtering to the H2 doubles is deliberately left out; it belongs with the view-commit work, since nothing in main sources writes a VIEW discriminator yet, which would make the filter unreachable and untestable today. Verified: housetables 153, internalcatalog 124, tables 519 (was 530, exactly the 11 removed), tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, plus spotlessCheck. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reverts the table predicate on both findAllDistinctDatabaseIds overloads in UserTableHtsJdbcRepository to their pre-change form, and drops the two tests that only asserted the reverted behavior. The four table-row filters are untouched: findAllByDatabaseIdIgnoreCase, the tableId-pattern variant, their paginated forms, and the findAllByFilters entity-type clause remain exactly as they are. Those are the genuine production filtering for this ticket. These two methods return a projection of database-ID strings, not rows, so no view can appear in their output under any implementation. The filter did not hide a view; it only changed which database names get listed. That is outside the scope this change set set for itself. The design enumerates the queries that need the table predicate and this is not among them, the stated harm is that SHOW TABLES would return views, and the acceptance criterion is that no view appears in a table listing. A database listing is not a table listing. Filtering here also contradicts three other design statements taken together: a namespace maps to an already-existing database and is never created implicitly, the server never auto-creates databases, and HTS infers databases from object rows and has no way to represent an empty database. With the filter, a database holding only views becomes non-existent by the only existence mechanism OpenHouse has - while views may only be created in databases that already exist. Concretely this path is Spark's SHOW DATABASES via OpenHouseCatalog.listNamespaces(). With the filter, a view-only namespace would be missing from SHOW DATABASES while still being addressable at /v2/databases/foo/views/v1. The rule this restores: queries that enumerate objects must be type-scoped; queries that enumerate containers must not. Removed with it, as they asserted only the reverted behavior: HtsRepositoryTest#testFindDistinctDatabasesExcludesViewOnlyDatabases and HtsControllerTest#testDatabaseQueriesExcludeViewOnlyDatabases. No fixture, helper or import became unused. The pre-existing testFindDistinctDatabases and the entity-type case/garbage matrix are unaffected and stay. Verified: housetables 151 (was 153, exactly the 2 removed), internalcatalog 124, tables 519, tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, plus spotlessCheck. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ionUtils Restores the private rootMetadataFileLocation in OpenHouseInternalTableOperations to its pre-change form, doing the naming work inline, and deletes MetadataLocationUtils along with its test. The stated goal was to move this into a shared helper so the table and view paths use one implementation. The view path is not part of this change, so the helper has exactly one production caller: the very method it was extracted from. That is indirection rather than sharing. The caller now hops through a private wrapper into a public util, and OpenHouseInternalTableOperations picked up an import and a delegation without getting any simpler. The codecName parameter exists only to serve a future view caller, since Iceberg's table and view compression defaults differ, and the helper's test covered a gzip path that no production caller passes today. An extraction is a refactor that a second caller justifies. The view commit work will have that second caller and can do the extraction then, with the real shape of both callers in hand. This is the same reasoning that deferred the HouseTableMapper ViewMetadata overload out of this change. Behavior is unchanged, as it was when the code was extracted: identical path format, five-digit zero-padded version, random UUID, and extension resolved from the same codec property. Every OpenHouseInternalTableOperations metadata-location test passes untouched. The plain-text javadoc reference to this method in InternalRepositoryUtils#getSchemeLessPath again describes the inline implementation it was written against. The doRefresh non-table guard in this file is untouched; that is real view isolation logic and stays. Verified: internalcatalog 121 (was 124, exactly the 3 MetadataLocationUtilsTest cases), housetables 151, tables 519, tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, plus spotlessCheck. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…nstead Restores OpenHouseInternalCatalog#resolveFileIO to its pre-change form and gives the raw-pointer test fixtures the storage type they were missing. The guard was compensating for a malformed fixture, not for a production condition. seedRawPointer built a HouseTable with databaseId, tableId, clusterId, tableUri, tableUUID, tableLocation, tableVersion and entityType but no storageType, so storageType.fromString(null) threw. A row seeded that way would have thrown just the same with entityType TABLE; the discriminator was incidental to the failure. The HTS schema settles it: storage_type is VARCHAR(128) DEFAULT 'hdfs' NOT NULL, so a null storage type cannot exist in production, whereas entity_type is DEFAULT NULL and is null on every pre-existing row. The guard was also wrong on its own terms. A real view row carries a valid storage type, so the original code returns the view's actual storage; skipping the row instead consults storageSelector, which can resolve to a different storage than the one the object is really on. And it is unreachable for the purpose it claimed: dropTable rejects a view before reaching this line, and on the newTableOps path doRefresh already treats a view as absent while create-over-view is stopped by the occupancy check. So the fix belongs in the fixture. Both seedRawPointer helpers now set storageType from storageManager.getDefaultStorage(), the same value a real table gets through HouseTableMapper. That makes the seeded row well-formed rather than merely tolerated. Every view-isolation guard test still passes, and now passes because the pointer is realistic rather than because production skips it: drop-VIEW, rename source and destination, CREATE-over-VIEW occupancy, findTableRefById, and the 404/409 status assertions, including all four case and garbage parameterizations of each. The dropTable and renameTable entity-type guards in this file are untouched, and so is the stripOhNamespace null-safety in the mapper - entity_type is DEFAULT NULL, so MapStruct's implicit String conversion would NPE on the real production mapping path without it. Verified: internalcatalog 121, tables 519, housetables 151, tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green with no count change from this commit, plus spotlessCheck. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…e pattern queries Deletes both findAllByDatabaseIdIgnoreCase overloads and routes listTables through findAllByFilters, and gives the two findAllByDatabaseIdAndTableIdLikeAllIgnoreCase overloads an entityType parameter. The paginated listTables already called findAllByFilters(databaseId, null, null, null, null, null, pageable) before this change set; it was switched to findAllByDatabaseIdIgnoreCase along the way. Consolidating restores that shape with entityType added. The non-paginated overload now matches it. The two plain methods were redundant with the parameterized family. Compared clause by clause: databaseId uses the same lower() comparison, tableId is exact equality rather than LIKE so an unset value adds no constraint, every other filter is guarded by an IS NULL check, DISTINCT over a single PK'd root is a no-op, and a null entityType takes the same predicate branch that the old hard-coded table predicate expressed. Identical results, one query family instead of two. The pattern overloads keep their own query because folding pattern matching into findAllByFilters would mean either a second tableId parameter or turning its exact match into a LIKE - and OpenHouse identifiers routinely contain underscores, so a LIKE there would silently treat them as wildcards. They now take entityType instead, reusing the same predicate constant. No listing method has a type baked into its name any more, and the call sites pass the request's own entityType rather than a hard-coded value, so the view path needs no new query methods - only entityType=VIEW at a call site. Verified: housetables 151 and tables 519, both unchanged and green, as expected for a refactor with identical semantics. HtsControllerTest 26, HtsRepositoryTest 17 and UserTablesServiceTest 21 all pass, which covers the rerouted list and pattern paths. Plus spotlessCheck. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… guards Removes every Java-side entity-type check in the tables service and the internal catalog, along with the tests that exercised them. What remains is the discriminator itself and the SQL that filters on it. Point-read type filtering is deferred to the view-commit ticket, where it will be done at the query level in HTS - a table-scoped getUserTable plus a neutral entity endpoint - rather than as Java guards layered on top of a type-blind read. Shipping the guards here would mean writing them twice and migrating callers off them a ticket later. The epic's acceptance criteria are evaluated across all six tickets rather than per ticket. Nothing deploys until the whole epic ships, and substantial client work is still required before a view can be created at all, so there is no window in which views exist unprotected by this deferral. Removed: the doRefresh non-table guard; the dropTable guard; the renameTable source guard and occupied-destination preflight; the findTableRefById type filter; findOccupyingEntityTypeById and its interface declaration and shared raw-pointer helper; and rejectNonTableNameOccupancy with both call sites. The five production files affected are now byte-identical to their pre-change state. Newly dead with them: HouseTableSerdeUtils.isTableEntityType, isViewEntityType, TABLE_ENTITY_TYPE and VIEW_ENTITY_TYPE, which had no remaining main-source caller. ENTITY_TYPE_FIELD_NAME stays - it is @VisibleForTesting like its neighbours in that class and backs the serde registration test, which is substrate. Write validation keeps its own ENTITY_TYPE_REGEX in ValidatorConstants and never depended on the removed constants. Kept as substrate: the schema column; UserTableRow, UserTable, UserTableDto and UserTablesMapper plumbing; HouseTable.entityType with its serde registration and mapper handling; the entity-type SQL predicate and its four query users in HTS; write validation; the stripOhNamespace null-safety; and every HTS-layer test for the list predicates and the round trip. Verified: housetables 151, internalcatalog 87, tables 475, tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, no surviving test failed. Plus spotlessCheck and checkstyle. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…lers
Adds a table-scoped point read to HTS and wires getUserTable to it, so a view
at a table's key is invisible to the table path because of the query rather
than because every caller checks.
getUserTable is the single HTS endpoint behind every table point read in the
tables service, so filtering it there makes four call sites correct with no
Java guard at all:
doRefresh findById -> getUserTable -> 404 -> HouseTableNotFound,
already caught, leaves Optional.empty, refreshes from a
null location exactly as for an absent row
findTableRefById findHouseTable catches the same exception and returns
empty
dropTable findHouseTable returns empty, so the existing
orElseThrow raises NoSuchTableException
rename source loadTable(from) -> doRefresh -> no metadata -> the same
NoSuchTableException
findByDatabaseIdIgnoreCaseAndTableIdIgnoreCase stays neutral on purpose.
HtsRepository.findById and existsById delegate to it and back putUserTable,
deleteUserTable, restoreUserTable and renameUserTable inside HTS, which must
see a row of any type to detect a collision at a shared key. Only the read
serving getUserTable changed.
TABLE_ROW_PREDICATE returns as the single statement of "null or TABLE", with
ENTITY_TYPE_FILTER_PREDICATE now composed from it, so the row test is written
once. No view-only method is added: nothing in this change reads views, and
the list queries already reach them through the entityType parameter.
Still deferred to the view-commit ticket, because they need the neutral
fetcher: occupancy, the rename destination preflight, and reading a view back
over HTTP.
The tables-service guard tests could not follow this filter - those tests run
the H2 double, which never goes through HTS - so the coverage moves to
services/housetables where the query actually executes: the case and garbage
matrix on the new point read, the neutral read still seeing every type, the
service-level getUserTable behavior, and the HTTP 404. Replicating the
predicate into the doubles was deliberately not done; that is the
testing-the-fake pattern already reverted for the list queries.
testEntityTypePutAndGetRoundTrip now asserts the view PUT is readable through
the PUT response and the persisted row, and that the table-scoped GET returns
404. That is the deferred neutral read, not a regression.
Verified: housetables 177, internalcatalog 87, tables 475,
tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, plus
spotlessCheck and checkstyle.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Applies the agreed query-level contract: a method whose name says "table" filters to tables, everything else stays neutral or takes entityType as a parameter. Renamed and filtered, because every caller assumes tables: findAllByDatabaseIdIgnoreCase -> findAllTablesByDatabaseIdIgnoreCase findAllByDatabaseIdIgnoreCase(Pageable) -> findAllTablesByDatabaseIdIgnoreCase(Pageable) findAllByDatabaseIdAndTableIdLikeAllIgnoreCase -> findAllTablesBy... findAllByDatabaseIdAndTableIdLikeAllIgnoreCase(Pageable) -> findAllTablesBy...(Pageable) "TableId" in those names is the column table_id, which under a shared key space holds a view's name too, so the old names were column-scoped and type-ambiguous rather than already table-scoped. Both findAllByDatabaseIdIgnoreCase overloads were removed earlier in this branch when listTables was consolidated onto findAllByFilters; they are restored under the new names and listTables routes back to them. The paged overload was declared but never called before this branch, so adopting it for paged listTables costs nothing. Added findTableByDatabaseIdIgnoreCaseAndTableIdIgnoreCase, which getUserTable now calls. That is the single HTS endpoint behind every table point read in the tables service, so the guards removed earlier are correct by construction: findById maps a 404 to HouseTableNotFoundException, which doRefresh already catches to leave an empty Optional and refresh from a null location, and which findHouseTable already catches to return empty - so dropTable's existing orElseThrow raises NoSuchTableException, findTableRefById returns empty, and a rename whose source is a view fails in loadTable. findByDatabaseIdIgnoreCaseAndTableIdIgnoreCase stays neutral and untouched. findById delegates to it and backs putUserTable, deleteUserTable and restoreUserTable, which must see a row of any type to detect a collision at a shared key. existsBy, deleteBy, renameTableId and both findAllDistinctDatabaseIds overloads are unchanged; findAllByFilters keeps entityType as a parameter because general search is caller-parameterized by design. No view-only method is added: nothing here reads views. TABLE_ROW_PREDICATE is the single statement of "null or TABLE" and is reused verbatim in every filtered query including the paged countQuery. With the list and pattern queries hard-coding the table predicate again, the entityType entry in isNonKeyFieldsNullForUserTable is load-bearing once more: it routes a databaseId + entityType=VIEW request to findAllByFilters instead of to a table-only listing. Tests live in services/housetables, where the query actually runs; the predicate was deliberately not replicated into the services/tables H2 doubles. Verified: housetables 177, internalcatalog 87, tables 475, tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, plus spotlessCheck and checkstyle. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… parameter /hts/tables and /hts/tables/query are table endpoints, so the queries behind them hard-code the table predicate and entityType is no longer a query parameter anywhere. Views get mirror endpoints in the view-commit ticket. That removes the parameterized type clause entirely: ENTITY_TYPE_FILTER_PREDICATE is deleted and TABLE_ROW_PREDICATE is the single statement of "null or TABLE", appended to every table-named query and repeated verbatim in each paged countQuery through the same constant. No :entityType parameter remains in the repository. Table-scoped reads, all filtered, none parameterized: findTableByDatabaseIdIgnoreCaseAndTableIdIgnoreCase new; getUserTable calls it findAllTablesByDatabaseIdIgnoreCase restored, renamed, filtered findAllTablesByDatabaseIdIgnoreCase(Pageable) restored, renamed, filtered findAllTablesByDatabaseIdAndTableIdLikeAllIgnoreCase renamed, filtered ...(Pageable) renamed, filtered findAllTablesByFilters renamed, filtered, entityType param dropped ...(Pageable) renamed, filtered, entityType param dropped "TableId" in the pattern names is the column table_id, which under a shared key space holds a view's name too, so those names were column-scoped rather than already table-scoped. Neutral and untouched: findByDatabaseIdIgnoreCaseAndTableIdIgnoreCase, which findById delegates to and which putUserTable, deleteUserTable and restoreUserTable need in order to see a row of any type at a shared key; plus existsBy, deleteBy, renameTableId and both findAllDistinctDatabaseIds overloads. No view-only method is added; nothing here reads views. With entityType gone from the query surface, isNonKeyFieldsNullForUserTable and the query branch of OpenHouseUserTableHtsApiValidator are restored to their pre-change form, so listDatabases, listTables, listTablesWithPattern and searchTables route exactly as at base. The transport-model @pattern stays: entityType is still a valid PUT payload field. Because getUserTable is the one HTS endpoint behind every table point read in the tables service, the guards removed earlier are correct by construction. A 404 becomes HouseTableNotFoundException, which doRefresh already catches to leave an empty Optional and refresh from a null location, and which findHouseTable already catches to return empty - so dropTable's existing orElseThrow raises NoSuchTableException, findTableRefById returns empty, and a rename whose source is a view fails inside loadTable. Tests follow the surface: the type-selection tests are replaced by ones asserting the table-scoped families never return a view, and the entityType query parameter is now pinned as bound-but-ignored at the mapper, service and HTTP layers. The predicate was deliberately not replicated into the services/tables H2 doubles. Verified: housetables 175, internalcatalog 87, tables 475, tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, plus spotlessCheck and checkstyle. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Restores findAllByFilters and findAllByDatabaseIdAndTableIdLikeAllIgnoreCase
to general methods that take entityType, and adds table-scoped default methods
that delegate to them. Nothing is renamed, and the general forms stay available
for the view and neutral work.
One shared ENTITY_TYPE_PREDICATE now spells all three branches out:
null matches any type - genuinely general, not a table default
TABLE matches TABLE and a stored null, because an absent discriminator
means a table on a column that is nullable with no backfill
VIEW matches VIEW
An unrecognized request value matches no branch, so garbage fails closed. Note
this changes what a null entityType means: it used to be a disguised table
default, and it now returns both types, which is why every table caller pins
TABLE explicitly.
Added, all default and owning no JPQL:
findAllTablesByFilters x2
findAllTablesByDatabaseIdAndTableIdLikeAllIgnoreCase x2
findTableByDatabaseIdIgnoreCaseAndTableIdIgnoreCase
The pattern family keeps its own @query because findAllByFilters matches
tableId exactly; folding a LIKE into it would make _ a wildcard and OpenHouse
identifiers routinely contain underscores. It shares the same predicate
constant.
The point read delegates rather than carrying its own query. The alternative
was a dedicated three-clause @query, which would read slightly more directly
but would restate the table branch of a predicate that already exists. Since
the key is the primary key, at most one row can match, so unwrapping the first
element is exact. The tradeoff is that the hottest read in HTS now runs the
general select DISTINCT; the key predicate is still exact, but say the word if
you would rather pay a duplicated clause to avoid the DISTINCT.
Untouched: findByDatabaseIdIgnoreCaseAndTableIdIgnoreCase, which backs findById
for putUserTable, deleteUserTable and restoreUserTable and must see a row of
any type at a shared key; existsBy; deleteBy; renameTableId; and both
findAllDistinctDatabaseIds overloads. No view-only method is added.
Call sites: listTables and searchTables use findAllTablesByFilters,
listTablesWithPattern uses the table-scoped pattern wrapper, and getUserTable
uses the table-scoped point read. entityType is not read from the wire, so
isNonKeyFieldsNullForUserTable and the validator's query branch stay at their
pre-change form and all four routes behave as at base.
Because getUserTable is the one HTS endpoint behind every table point read in
the tables service, the guards removed earlier remain correct by construction:
a 404 becomes HouseTableNotFoundException, which doRefresh already catches to
leave an empty Optional and which findHouseTable already catches to return
empty, so dropTable throws NoSuchTableException, findTableRefById returns
empty, and a rename off a view source fails inside loadTable.
Verified: housetables 177, internalcatalog 87, tables 475,
tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, plus
spotlessCheck and checkstyle.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
entityType is no longer a parameter anywhere in the query layer. A caller picks a type by picking a method: findAllByFilters returns both types, findAllTablesByFilters returns tables, and findAllViewsByFilters arrives with the view ticket. That drops the delegating-default idea: a typed wrapper cannot tell a parameterless general method what to filter, so each typed method carries its own @query. To avoid restating the filter body, the six general clauses are extracted once into COMMON_FILTER_CLAUSES and the typed sibling composes that constant with TABLE_ROW_PREDICATE. The pattern family is split the same way through PATTERN_KEY_CLAUSES. The extraction is provably behavior-preserving: both findAllByFilters overloads now read "select DISTINCT u from UserTableRow u where " + COMMON_FILTER_CLAUSES, which expands byte-for-byte to the ba400b3 string. The pattern overloads are restored to their ba400b3 form exactly - derived queries with no @query at all. Added, table-scoped, each with its own query composed from the shared constants: findTableByDatabaseIdIgnoreCaseAndTableIdIgnoreCase findAllTablesByFilters x2 findAllTablesByDatabaseIdAndTableIdLikeAllIgnoreCase x2 Nothing is renamed and no view method is added. Unchanged from ba400b3: findByDatabaseIdIgnoreCaseAndTableIdIgnoreCase, which backs findById for putUserTable, deleteUserTable and restoreUserTable and must see a row of any type at a shared key; existsBy; deleteBy; renameTableId; and both findAllDistinctDatabaseIds overloads. The two findAllByDatabaseIdIgnoreCase overloads stay deleted, since findAllTablesByFilters(db, null, ...) covers them, which is what paged listTables already did at base. Call sites: listTables and searchTables use findAllTablesByFilters, listTablesWithPattern uses the table pattern methods, getUserTable uses the table point read. entityType is not read from the wire, so isNonKeyFieldsNullForUserTable and the validator's query branch remain at their pre-change form and all four routes behave as at base. Because getUserTable is the one HTS endpoint behind every table point read in the tables service, the guards removed earlier stay correct by construction: a 404 becomes HouseTableNotFoundException, which doRefresh already catches to leave an empty Optional and which findHouseTable already catches to return empty, so dropTable throws NoSuchTableException, findTableRefById returns empty, and a rename off a view source fails inside loadTable. Verified: housetables 177, internalcatalog 87, tables 475, tables-test-fixtures 8, spark-3.5 catalogTest 66 - all green, plus spotlessCheck and checkstyle. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tables and views share a single
(databaseId, objectId)pointer key space, so aname must resolve to exactly one catalog object. This adds a nullable
entityTypediscriminator end-to-end — MySQL/H2 → HTS → generated client →internal
HouseTablepointer — and makes every table path aware of it.Semantics:
NULLand any case spelling ofTABLEmean table; any casespelling of
VIEWmeans view; any other non-null value fails closed (neithertable nor view). The column is nullable and deliberately not backfilled, so
existing rows stay valid and ordinary table commits continue to write no
discriminator at all — no change to existing
metadata.jsoncontent.Resolves BDP-108403 (consolidating BDP-108404, -108405, -108406, -108409, -108422).
Read paths
Views are filtered in the repository query, including the count query — never by
post-filtering a
Page, which would return short pages and inflated totals.IS NULLis required because the backfill is deferred; plain equality would hideevery existing table.
upper(...)makes the check independent of provider collation.The predicate is a shared constant, so each
countQueryis the same literal as itsvalueand the two cannot diverge.Applied to both
/htsquery families,listTablesand its paginated overload,listHouseTables, allsearchTablesoverloads, and database enumeration.Write paths
Two different questions, deliberately separated:
VIEW?findById/findTableRefById/loadTablefindOccupyingEntityTypeById(new)CREATE and rename-destination check occupancy before authorization, allocation,
metadata writes, and pointer saves — otherwise a CREATE at a view's name looks free
and fails only at the HTS publish boundary, after writing an orphaned
metadata.json. HTS 4xx/5xx propagate rather than reading as a free name.The drop guard cannot live in
doRefresh—deleteTablebypassesloadTablesodrops survive corrupted metadata, so the guard sits in
findTableRefByIdandOpenHouseInternalCatalog.dropTable.Wrong-type read/drop returns 404, collisions 409, never 400 — the Java/Spark
client treats 400 and 404 identically.
Beyond the ticket's literal scope — please review deliberately
(a)
entityTypeAPI validation touchesservices/common.ENTITY_TYPE_REGEXis added to
ValidatorConstantsand enforced inOpenHouseUserTableHtsApiValidator. The ticket did not ask for validation, but anew API field with an enumerated domain should not accept arbitrary strings, and
the constant belongs beside its sibling definitions. Append-only; no existing
constant or validator branch changed.
(b) Catalog
renameTablenow rejects any occupied destination. Previouslythe catalog performed no destination check at all and relied on the HTS primary
key. It now preflights occupancy, so an occupied destination is an explicit
collision before the transaction opens rather than a constraint violation after
a metadata write.
Deferred:
HouseTableMapper.toHouseTable(ViewMetadata, FileIO)Named in the ticket's scope, but moved to the view-commit ticket.
org.apache.iceberg.view.ViewMetadatadoes not exist iniceberg-core-1.2.0.20(verified: 0 classes, vs 1 in
1.5.2.17), andtables-test-fixtures-iceberg-1.2loadsinternalcatalogclasses againstIceberg 1.2. A 1.5-only type on a Spring-managed bean therefore fails during
lookup-method introspection with
NoClassDefFoundError— 7 fixture failures —even though nothing calls it. The overload has zero callers in this change, and
the 1.2/1.5 bean boundary is an architectural decision better made where the
view path is actually built.
Incidental fix:
stripOhNamespacenull-safetyMapStruct adopts any
String -> Stringmethod on a mapper as an implicit conversionand applies it to every String property.
stripOhNamespaceassumed a non-null key,which held only because
HouseTablehad no nullable String field.entityTypeis thefirst, so the mapper NPE'd. Now null-safe — this was a latent pre-existing bug; a null
tableVersionwould have triggered it identically.Test coverage
Written test-first: the full suite landed red and went through review rounds
before any production code, specifically to confirm each test can fail against
the wrong implementation it targets.
asserting content and page size and
totalElementsandtotalPages—totalPagesis what kills a post-filter that rebuilds aPageImplwith filtered content but the original total.AlreadyExistsExceptionand HTTP 409 as provingnothing (both the OpenHouse and Iceberg variants map to 409, and the DB
primary-key fallback throws the same type). They lean on exact message
identity, byte-identical pointer state, an unchanged
*.metadata.jsonfileset, and
verify(authorizationHandler, never()).checkAccessDecision(...).NULL/TABLE/table/TaBlE/VIEW/view/ViEw/UNKNOWN/"") at both the SQL layer and the Java guard layer. The SQL testsare explicitly scoped: they prove the
upper(...)normalization, not providercollation.
NULLrows visible and fully operablethroughout;
normalTableCommitDoesNotStampEntityTypeasserts ordinary commitsadd no
openhouse.entityTypeproperty.Results — all green
services:housetablesiceberg:openhouse:internalcatalogservices:tablestables-test-fixtures_2.12(Iceberg 1.2)tables-test-fixtures-iceberg-1.5_2.12openhouse-spark-3.5-itestBranchJavaTestspotlessCheck+checkstyleBoth published fixture variants were compiled and the 1.2 fixture's local-server
tests run, because
HouseTableRepositoryis a shared interface whose predicatesare inherited by Spring Data proxies in published
src/mainfixture code.Rollout
schema.sqlusesCREATE TABLE IF NOT EXISTS, so it does not migrate anexisting table. Production needs a separately owned
ALTER TABLE user_table_row ADD COLUMN entity_type VARCHAR(128) DEFAULT NULLapplied before any code that writes
VIEW. No data backfill is required.HouseTableRepositoryImplintentionally trusts HTS page content and totals rather than filtering
locally, so a tables service running against an older, unfiltered HTS could
leak a view into a listing.
collation behavior. Insert
NULL/TABLE/table/TaBlE/VIEW/view/ViEw/UNKNOWNrows and exercise both deployed services before views becomewritable.
Follow-ups tracked elsewhere
restoreTableis the one write to the shared key space without an occupancypreflight, and
soft_deleted_user_table_rowhas noentity_typecolumn. Notreachable today (views cannot be created yet), but drop table → create view →
restore table would clobber the view once views are writable. Tracked on the
view-commit ticket.