Skip to content

Prevent stale symbol table children from replacing newer root tables - #1655

Open
Dongnyoung wants to merge 6 commits into
FasterXML:3.xfrom
Dongnyoung:investigate/symbol-canonicalizer-lifecycle
Open

Prevent stale symbol table children from replacing newer root tables#1655
Dongnyoung wants to merge 6 commits into
FasterXML:3.xfrom
Dongnyoung:investigate/symbol-canonicalizer-lifecycle

Conversation

@Dongnyoung

Copy link
Copy Markdown
Contributor

Fixes #1654

Summary

Prevent stale child symbol tables from replacing newer, larger root symbol tables when parsers are closed out of creation order.

Previously, mergeChild() compared the child state against the root state observed at release time. This made the update atomic, but did not detect that the child itself could have been created from an older parent state. As a result, an older, smaller child could replace a root table that had already been updated by another child.

Child symbol tables now retain the parent table state they were created from. During release, a stale child no longer replaces the root if another child has already advanced it to a newer, larger state.

This applies to both CharsToNameCanonicalizer and ByteQuadsCanonicalizer. The existing reset behavior for oversized symbol tables is preserved.

Tests

Added coverage to SymbolTableMergingTest for out-of-order stale child release with:

  • byte-backed parsers
  • char-backed parsers

Ran:

  • .\mvnw.cmd -Dtest=SymbolTableMergingTest '-Dsurefire.useModulePath=false' test
  • .\mvnw.cmd '-Dtest=SymbolTableMergingTest,TestSymbolTables,TestByteBasedSymbols,TestHashCollisionChars,PlaceholderSymbolTableTest,TestSymbolsWithMediaItem' '-Dsurefire.useModulePath=false' test

@cowtowncoder

Copy link
Copy Markdown
Member

I don't know about this one -- is there an actual real problem resolved here?

@Dongnyoung

Copy link
Copy Markdown
Contributor Author

@cowtowncoder
Fair question. I measured this as a symbol-table cache/performance regression rather than a parsing correctness issue.

The regression is reproducible through normal JsonParser usage: parser A reads one field name, parser B created from the same JsonFactory reads a larger set and closes first, and then A closes. Before the fix, A's stale child can replace the newer root table.

With a synthetic document containing 1,024 unique field names, I observed:

pre-fix:
bytes: rootSize 1024 -> 1
chars: rootSize 1024 -> 1

fixed:
bytes: rootSize remains 1024
chars: rootSize remains 1024

I also ran a small JMH benchmark with -prof gc, measuring the next parse of the same document after restoring either the baseline or regressed root state before each parse:

pre-fix:
bytes baseline: 18.5 us/op,     664 B/op
bytes stale:    51.7 us/op, 211,681 B/op

chars baseline: 24.1 us/op,     568 B/op
chars stale:    35.4 us/op,  87,072 B/op

fixed:
bytes baseline: 18.6 us/op, 664 B/op
bytes stale:    18.0 us/op, 664 B/op

chars baseline: 24.4 us/op, 576 B/op
chars stale:    24.4 us/op, 576 B/op

So in this synthetic workload, the stale root causes the following parse to re-canonicalize field names that had already been learned, with a clear allocation and parsing-time cost. The fix prevents that backwards transition.

I wouldn't characterize this as a parsing correctness bug, but the cache regression is measurable and can result from normal parser lifecycle.

@cowtowncoder

Copy link
Copy Markdown
Member

@Dongnyoung Isn't there concern that TableInfo _parentTableInfo might lead to unintentional memory retention via parent chain?

@github-actions

Copy link
Copy Markdown
Contributor

📈 Overall Code Coverage

Metric Coverage Change
Instructions coverage 83.90% 📈 +0.000%
Branches branches 76.88% 📈 +0.000%

Overall project coverage from JaCoCo test results. Change values compare against the latest base branch build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stale symbol table child can replace a newer root table

2 participants