init: use MPIR_CVAR_FINALIZE_ATEXIT to bypass re-init - #7933
Conversation
|
test:mpich/ch4/ofi |
sonjahapp
left a comment
sonjahapp
left a comment
There was a problem hiding this comment.
Thanks for this PR!
Just found a few minor typo fixes for the testlist. 😉
Thanks for the catch. I will fix them in the next push. By the way, do you think we can remove the PMI atexit hook completely now we have the CVAR option? |
Hmm... I would keep it. It leaves freedom to the device to enable more "fine grained" solutions for re-init. The new CVAR is quite the "hammer" since it postpones almost the entire MPI cleanup to an atexit handler. For example, in the psp device we offer the ability to finalize most parts in |
Add this cvar as a work-around for sessions re-init. There is no good solution to synchronize shared memory over re-init now that we switched to total asynchronous initialization. Consider the following case: * Process A and B both initialized shm and both re-initialize. * B re-init shm while A delays. * B send a message to A via shm. * A (still in the previous epoch) either discards the message or error. * A re-init. * A never receives the message B send. Delay the final finalize via atexit avoids the re-init altogether.
Putting MPII_Finalize in atexit obviates the need for pmi atexit hook.
|
test:mpich/ch4/ofi |
2 participants
Pull Request Description
As cc91750#r3758517489 show, we can't rely on local init_counter to collectively resolve conflicts between session re-init epochs. In fact, after a full-day's effort, I concluded there is no way to create a solution for
We just can't have a collective synchronization when we can't establish collective assumptions.
This PR reverts the previous attempts and add
MPIR_CVAR_FINALIZE_ATEXITto workaround. When the cvar is set, we delay the finalMPII_Finalizetoatexit, avoiding the necessity of re-initialize both shm and netmod.Author Checklist
Particularly focus on why, not what. Reference background, issues, test failures, xfail entries, etc.
Commits are self-contained and do not do two things at once.
Commit message is of the form:
module: short descriptionCommit message explains what's in the commit.
Whitespace checker. Warnings test. Additional tests via comments.
For non-Argonne authors, check contribution agreement.
If necessary, request an explicit comment from your companies PR approval manager.