VectorTemplate, IntVector and PermutationVector - #1273
Conversation
90ee200 to
847d0e7
Compare
e9c38f2 to
16fbca5
Compare
There was a problem hiding this comment.
Pull request overview
This PR removes the Python-side typed vector wrappers (notably IntVector and related templates) and continues migrating C++/pybind interfaces away from int_vector_type / bool_vector_type toward standard containers (std::vector, std::set) and NumPy arrays, with corresponding updates to tests and bindings.
Changes:
- Deleted the Python
IntVector/PermutationVectorimplementation and its tests, and removed exports fromresdata.util.util. - Updated multiple C++ APIs and pybind bindings to use
std::vector<int>/std::vector<bool>and NumPy arrays instead of ERT typed vectors. - Updated Python and C++ tests to construct ACTNUM/index lists using built-in lists,
std::vector, and returned containers.
Reviewed changes
Copilot reviewed 59 out of 59 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/util_tests/test_vectors.py | Removes IntVector-focused unit tests (file deleted). |
| tests/rd_tests/test_region.py | Stops using IntVector for actnum in region tests. |
| tests/rd_tests/test_rd_3dkw.py | Replaces IntVector ACTNUM setup with Python lists. |
| tests/rd_tests/test_layer.py | Replaces IntVector ACTNUM setup with Python lists. |
| tests/rd_tests/test_kw_function.py | Replaces IntVector ACTNUM setup with Python lists. |
| tests/rd_tests/test_grid.py | Replaces IntVector ACTNUM setup; adjusts type-error parametrization. |
| tests/rd_tests/test_grid_equinor.py | Replaces IntVector ACTNUM setup with Python lists. |
| python/resdata/util/util/vector_template.py | Removes Python vector template implementation (file deleted). |
| python/resdata/util/util/permutation_vector.py | Removes Python permutation vector wrapper (file deleted). |
| python/resdata/util/util/int_vector.py | Removes Python IntVector wrapper (file deleted). |
| python/resdata/util/util/init.py | Stops exporting IntVector/PermutationVector/VectorTemplate. |
| python/resdata/summary/rd_sum.py | Removes IntVector import from summary module. |
| python/resdata/grid/rd_region.py | Changes index-list return types to list[int]. |
| python/resdata/grid/rd_grid.py | Changes exportACTNUM() to return a NumPy array type. |
| python/resdata/grid/faults/fault_block.pyi | Updates stubs to return list[int] instead of IntVector. |
| python/resdata/geometry/geo_region.py | Changes getActiveList() to return list[int]. |
| lib/util/vector_template.cpp.in | Removes bool_vector usage from documentation snippet. |
| lib/util/type_vector_functions.cpp | Removes bool/int vector conversion helpers (file deleted). |
| lib/util/tests/ert_util_vector_test.cpp | Drops include of removed int_vector header. |
| lib/util/tests/ert_util_type_vector_test.cpp | Removes typed-vector C tests (file deleted). |
| lib/util/tests/ert_util_type_vector_functions.cpp | Removes type-vector-functions tests (file deleted). |
| lib/util/tests/ert_util_string_util.cpp | Removes string_util tests relying on int/bool vectors (file deleted). |
| lib/util/tests/ert_util_copy_file.cpp | Drops unused string_util include. |
| lib/util/string_util.cpp | Removes string_util implementation (file deleted). |
| lib/tests/test_rd_region.cpp | Migrates region list assertions to std::vector APIs. |
| lib/tests/test_rd_kw.cpp | Migrates indexed set usage to std::vector<int>. |
| lib/tests/test_layer.cpp | Drops include of removed int_vector header. |
| lib/tests/test_geometry.cpp | Migrates geo_region index list assertions to std::vector. |
| lib/resdata/tests/rd_region.cpp | Migrates region list checks to std::vector. |
| lib/resdata/tests/rd_nnc_vector.cpp | Drops include of removed int_vector header. |
| lib/resdata/tests/rd_nnc_test.cpp | Drops include of removed int_vector header. |
| lib/resdata/tests/rd_nnc_info_test.cpp | Drops include of removed int_vector header. |
| lib/resdata/tests/rd_kw_init.cpp | Drops include of removed bool_vector header. |
| lib/resdata/rd_sum.cpp | Replaces bool/int typed vectors with std::vector<bool> and std::vector<int>. |
| lib/resdata/rd_region.cpp | Replaces internal typed vectors with std::vector<int>; updates indexed kw operations. |
| lib/resdata/rd_region_pybind.cpp | Returns STL containers directly via pybind11 instead of IntVector wrappers. |
| lib/resdata/rd_kw.cpp | Updates indexed operations to accept const std::vector<int>&. |
| lib/resdata/rd_kw_pybind.cpp | Updates _fix_uninitialized binding to take a NumPy array. |
| lib/resdata/rd_grid.cpp | Drops include of removed int_vector header. |
| lib/resdata/rd_grid_pybind.cpp | Changes _init_actnum to return a NumPy array. |
| lib/resdata/rd_grav_common.cpp | Iterates region active indices via STL container. |
| lib/resdata/rd_file_view.cpp | Drops include of removed int_vector header. |
| lib/resdata/fault_block.cpp | Migrates internal lists to std::vector; regions to std::set. |
| lib/resdata/fault_block_pybind.cpp | Returns STL containers directly; drops IntVector wrapping. |
| lib/resdata/fault_block_layer.cpp | Drops include of removed int_vector header. |
| lib/resdata/cwrap_pybind.cpp | Removes IntVector helper and int_vector_type caster. |
| lib/private-include/detail/resdata/cwrap_pybind.hpp | Removes IntVector forward declaration. |
| lib/include/resdata/rd_region.hpp | Changes region list getters to return const std::vector<int>&. |
| lib/include/resdata/rd_kw.hpp | Updates indexed kw APIs to use std::vector<int> (but still exposes int_vector_type elsewhere). |
| lib/include/resdata/rd_file_view.hpp | Drops include of removed int_vector header. |
| lib/include/resdata/nnc_vector.hpp | Drops include of removed int_vector header. |
| lib/include/resdata/nnc_info.hpp | Drops include of removed int_vector header. |
| lib/include/resdata/fault_block.hpp | Migrates members to STL containers and updates signatures. |
| lib/include/ert/util/type_vector_functions.hpp | Removes type-vector-functions header (file deleted). |
| lib/include/ert/util/string_util.hpp | Removes string_util header (file deleted). |
| lib/include/ert/geometry/geo_region.hpp | Changes geo_region index list getter to return std::vector<int>. |
| lib/geometry/geo_region.cpp | Replaces cached int_vector index list with on-demand std::vector<int> construction. |
| lib/geometry/geo_region_pybind.cpp | Returns STL container directly from binding. |
| lib/CMakeLists.txt | Stops generating/building bool vector and string/type-vector utility code and tests. |
Suppressed comments (1)
lib/include/resdata/rd_kw.hpp:12
- This header still depends on
ert/util/int_vector.hppand exposesint_vector_typein the public API (rd_kw_fread_indexed_data(..., const int_vector_type *index_map, ...)), while the newer indexed operations in the same header have been migrated tostd::vector<int>. If the goal is to remove the typed vector utilities entirely,rd_kw_fread_indexed_data(and any callers/impl) should be migrated tostd::vector<int>/std::span<const int>as well so theint_vectordependency can be dropped.
#include <ert/util/type_macros.hpp>
#include <ert/util/int_vector.hpp>
#include <ert/util/util.hpp>
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
6777ad9 to
8dd681b
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 67 out of 67 changed files in this pull request and generated 1 comment.
Suppressed comments (5)
lib/resdata/rd_grav_common.cpp:139
- Same issue as above:
const autocopies the region’s active-index vector; keep it as a reference to avoid copying before iteration.
const auto index_vector = rd_region_get_active_list(region);
lib/resdata/rd_region.cpp:1169
rd_region_get_kw_index_list()returns aconst std::vector<int>&, but this code copies it intotarget_index. That adds unnecessary allocation/copying on every call (potentially for large regions).
const std::vector<int> target_index =
rd_region_get_kw_index_list(rd_region, rd_kw, force_active);
rd_kw_copy_indexed(rd_kw, target_index, src_kw);
lib/resdata/rd_grav_common.cpp:69
rd_region_get_active_list()returns aconst std::vector<int>&, but usingconst autohere drops the reference and copies the whole index list before iterating. That is unnecessary work and could be expensive for large regions.
This issue also appears on line 139 of the same file.
const auto index_vector = rd_region_get_active_list(region);
lib/include/ert/geometry/geo_region.hpp:19
- Returning
const std::vector<int>by value is counterproductive: it can inhibit moves (the returned rvalue is const) and adds no safety. Prefer returningstd::vector<int>(non-const) by value.
void geo_region_reset(geo_region_type *region);
const std::vector<int> geo_region_get_index_list(geo_region_type *region);
lib/geometry/geo_region.cpp:139
- Same issue as the header: this function returns
const std::vector<int>by value. Dropping the top-levelconstavoids inhibiting move semantics and matches typical C++ practice.
const std::vector<int> geo_region_get_index_list(geo_region_type *region) {
55463b0 to
8a15fae
Compare
8a15fae to
b59f80a
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 48 out of 48 changed files in this pull request and generated 1 comment.
Suppressed comments (5)
lib/resdata/rd_grav_common.cpp:140
- Same as above:
const auto index_vector = rd_region_get_active_list(region);copies the vector. Prefer iterating the returned reference directly to avoid per-call copies.
const auto index_vector = rd_region_get_active_list(region);
for (const auto index : index_vector) {
lib/resdata/rd_region.cpp:1169
rd_region_get_kw_index_list()returns aconst std::vector<int>&, buttarget_indexis currently copied. This creates unnecessary work for potentially large regions; bind to a const reference instead.
const std::vector<int> target_index =
rd_region_get_kw_index_list(rd_region, rd_kw, force_active);
rd_kw_copy_indexed(rd_kw, target_index, src_kw);
lib/include/resdata/rd_kw.hpp:12
- This header still includes and publicly exposes
int_vector_typeviard_kw_fread_indexed_data(..., const int_vector_type *index_map, ...), which keeps theert/util/int_vector.hppdependency even though most indexed APIs were migrated tostd::vector<int>. If the goal is to removeint_vectorentirely, consider migratingrd_kw_fread_indexed_data(and its call sites) to takestd::vector<int>/std::span<const int>and then drop theint_vectorinclude here.
#include <vector>
#include <ert/util/type_macros.hpp>
#include <ert/util/int_vector.hpp>
#include <ert/util/util.hpp>
lib/resdata/rd_grav_common.cpp:70
rd_region_get_active_list()now returns aconst std::vector<int>&, but storing it inconst auto index_vector = ...makes an unnecessary copy of (potentially large) index lists. Iterate the returned container by reference to avoid extra allocations/copies on every call.
This issue also appears on line 139 of the same file.
const auto index_vector = rd_region_get_active_list(region);
for (const auto index : index_vector) {
lib/include/ert/geometry/geo_region.hpp:18
geo_region_get_index_list()now returns astd::vector<int>by value, which allocates and copies on every call. If this function is called repeatedly (e.g., from Python), this can be a noticeable regression compared to the previous cached list. Consider restoring caching ingeo_region_structand returning aconst std::vector<int>&(or another non-owning view) to avoid repeated allocations.
geo_region_type *geo_region_alloc(const geo_pointset_type *pointset,
bool preselect);
void geo_region_free(geo_region_type *region);
void geo_region_reset(geo_region_type *region);
const std::vector<int> geo_region_get_index_list(geo_region_type *region);
5d01d9a to
f8fe76d
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 48 out of 48 changed files in this pull request and generated no new comments.
Suppressed comments (1)
lib/resdata/rd_region.cpp:172
- The module-level API example above still documents these getters as returning
const int *and assignsrd_region_get_global_list(...)to a pointer (lines 41–74). That example no longer compiles with the newconst std::vector<int>&return type; update it to use a vector reference and.size()/indexing.
const std::vector<int> &rd_region_get_global_list(rd_region_type *region) {
rd_region_assert_global_index_list(region);
return region->global_index_list;
f8fe76d to
f45f31e
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 48 out of 48 changed files in this pull request and generated no new comments.
Suppressed comments (2)
python/resdata/grid/rd_region.py:929
active_size()andglobal_size()immediately above still computelen(self.get_*_list()). These bindings now convert the full C++ vectors into Python lists, so a size query has changed from O(1) to an O(n) allocation/copy; this is costly for million-cell grids and repeated size checks. Please expose dedicated size bindings (or otherwise query the C++ vector sizes directly) and use those from the size methods.
lib/geometry/geo_region.cpp:143- This removes the cached index list and now scans the entire point set on every call. In particular, Python
GeoRegion.__len__calls this getter, and__repr__calls it twice, turning formerly cached/O(1) queries into repeated O(point-count) scans and list allocations. Preserve a cachedstd::vector<int>with invalidation, or add a direct selected-count path for size queries.
const std::vector<int> geo_region_get_index_list(geo_region_type *region) {
std::vector<int> index_list;
index_list.reserve(region->size());
for (size_t i = 0; i < region->size(); i++)
if (region->active_mask[i])
aaf23ed to
b0aea1b
Compare
This changes the behavior from being cached to simply building a new index_list when required. Therefore, we need to keep track of the amount of selected indices to avoid calls to len being linear in time.
b0aea1b to
614e9ad
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 49 out of 49 changed files in this pull request and generated no new comments.
Suppressed comments (2)
python/resdata/grid/rd_region.py:929
- Returning a Python list here makes the existing
__nonzero__implementation at lines 137-139 copy every selected global index merely to determine whether the region is empty. This regressesbool(region)from a constant-size wrapper/length check to an O(n) allocation; update__nonzero__to use the newglobal_size()helper instead.
lib/geometry/geo_region.cpp:161 - Reserve only the number of selected entries. Reserving
region->size()allocates space for every point even when the active list is empty or sparse, sogetActiveList()can transiently consume O(total point count) memory for an O(selected count) result.
index_list.reserve(region->size());
614e9ad to
3ba7c83
Compare
3ba7c83 to
f87b097
Compare
f87b097 to
40c025d
Compare
ajaust
left a comment
There was a problem hiding this comment.
What is the purpose of this PR? The subject mentions the removal of IntVector and bool_vector_type, but the body says
This removes VectorTemplate, IntVector and PermutationVector which is deprecated in version 6.3.
which does not mention bool_vector_type at all. I also still find the bool_vector_type:
$ git branch
fix_build
knowledge
main
python-trusted-publishing
* remove_bool_vector_type
remove_double_vector
remove_time_t_vector
use-uv-with-lockfile
$ rg bool_vector_type
lib/util/type_vector_functions.cpp
7:int_vector_type *bool_vector_alloc_active_list(const bool_vector_type *mask) {
18:bool_vector_type *int_vector_alloc_mask(const int_vector_type *active_list) {
19: bool_vector_type *mask = bool_vector_alloc(0, false);
lib/util/string_util.cpp
84: bool_vector_type *mask = int_vector_alloc_mask(active_list);
116: bool_vector_type *active_mask) {
131:bool_vector_type *string_util_alloc_active_mask(const char *range_string) {
132: bool_vector_type *mask = bool_vector_alloc(0, false);
lib/util/vector_template.cpp.in
1185: bool_vector_type * v2;
lib/util/tests/ert_util_string_util.cpp
29:static void test2(const bool_vector_type *active_mask) {
46: bool_vector_type *active_mask =
lib/util/tests/ert_util_type_vector_functions.cpp
14: bool_vector_type *mask = int_vector_alloc_mask(index_list);
29: bool_vector_type *mask = bool_vector_alloc(0, false);
lib/include/ert/util/type_vector_functions.hpp
11:int_vector_type *bool_vector_alloc_active_list(const bool_vector_type *mask);
12:bool_vector_type *int_vector_alloc_mask(const int_vector_type *active_list);
lib/include/ert/util/string_util.hpp
13: bool_vector_type *active_mask);
14:bool_vector_type *string_util_alloc_active_mask(const char *range_string);
| for (size_t index = 0; index < i_list.size(); index++) { | ||
| int i = i_list[index]; | ||
| int j = j_list[index]; |
There was a problem hiding this comment.
I am unsure if we are at risk of an out-of-bounds access due to how add_cell works.
In add_cell we have the following:
i_list.push_back(i);
j_list.push_back(j);
global_index_list.push_back(rd_grid_get_global_index3(grid, i, j, k));If i_list.push_back(i) succeeds, but j_list.push_back(j) fails, j_list will hold one fewer element than i_list and we have an out-of-bounds access.
Do you think this is possible or will the application always (safely) abort if a memory allocation in push_back fails?
There was a problem hiding this comment.
You are right, I think the right course of action is to switch to a list of tuple. I added a commit that I will squash with the fault_block commit which addresses this.
I appologize for the confusion the bool_vector_type removal was moved to the "Remove unused utilities" PR in the stack, and I didn't notice the title was out of date. |
This removes VectorTemplate, IntVector and PermutationVector which is deprecated in version 6.3.