Benchmarking optimizations - #84
Merged
Merged
Conversation
Arshdeep54
approved these changes
Jun 12, 2026
4adex
added a commit
that referenced
this pull request
Jul 23, 2026
* feat: add batch insert and search (#74) Signed-off-by: Arshdeep54 <balarsh535@gmail.com> * Inmemory storage implementation (#82) * Implement in-memory storage engine * Wire in-memory storage through API snapshots * Add HTTP smoke coverage for in-memory storage * Handle in-memory storage errors in gRPC * Add in-memory checkpoint header * Add async Python client (#83) * Add async Python client * Deduplicate Python gRPC connection helpers * Benchmarking optimizations (#84) * Reduce in-memory checkpoint lock scope * Optimize HNSW distance calculations * Expose HNSW tuning parameters * Add Python batch vector client methods * Expose KD-tree rebuild tuning parameters * Update tui with new name and details (#79) * Support query-time HNSW ef * Async batch support (#85) * Update Python client to support batch insert and search * Add batch support to the async client * Add master test script for all usage example files Run linter on client/python * Add test for api parity between sync and async classes --------- Signed-off-by: Arshdeep54 <balarsh535@gmail.com> Co-authored-by: Arshdeep Singh <110114582+Arshdeep54@users.noreply.github.com> Co-authored-by: Khushal Agrawal <117133859+SK1PPR@users.noreply.github.com> Co-authored-by: Khushal Agrawal <dev@khushal.net> Co-authored-by: Prachet Patel <patelprachet@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR prepares VortexDB for ANN-Benchmarks integration by improving index configurability, reducing HNSW overhead, and adding Python batch client APIs for faster benchmark data loading.
Changes
Optimized HNSW distance calculation paths
sqrtfor Euclidean comparisons1.0 - dotscoringExposed HNSW tuning parameters
HNSW_MHNSW_M0HNSW_MAX_LAYERHNSW_EF_CONSTRUCTIONHNSW_EFExposed KD-tree rebuild tuning parameters
KD_TREE_BALANCE_THRESHOLDKD_TREE_DELETE_REBUILD_RATIOAdded Python batch client APIs
insert_batch(...)search_batch(...)Regenerated Python gRPC stubs to include existing batch RPCs
Kept
FlatIndexunchanged since it has no meaningful index-level tunables beyond query-timekandsimilarityTesting