Redfs ubuntu hwe writeback switch to dio#186
Closed
hbirth wants to merge 2 commits into
Closed
Conversation
Add fuse_cache_wr_exclusive_lock() to take the lock shared for normal writes; direct, appending and non-writeback-cache (suid-drop) writes stay exclusive. Extending writes grow i_size under fi->lock before the write and reconcile short writes afterwards. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
A FUSE_NOTIFY_INVAL_INODE data invalidation means another (remote) entity
is modifying the file.
Rather than react to a single notify, keep a per-inode moving average of
how fast data invalidations arrive for the whole file: an EWMA of the
inter-arrival interval, updated under fi->lock on every notify
(fuse_notify_inval_hot()). The inode is latched only once the average
spacing drops below an internal threshold (FUSE_NOTIFY_DIO_INTERVAL) while
a local writer is open; a lone or occasional notify keeps the average high
and does not trip the switch. The heuristic has no external knob -- its
parameters (EWMA weight, threshold, seed) are source-level constants.
Introduce the forced-direct-IO latch (FUSE_I_FORCE_DIO):
- fuse_reverse_inval_inode() folds each data invalidation into the moving
average and sets the latch when it trips with a local writer present;
- fuse_file_{read,write}_iter() and fuse_cache_write_iter() route to the
direct path while latched; fuse_dio_{wr_exclusive_lock,lock,unlock}()
use the shared parallel-dio path and bypass the cached/uncached
accounting;
- fuse_file_io_open() opens new files uncached so they do not re-enter
caching mode;
- fuse_prepare_release() clears the latch once the last writer is gone
and fuse_file_release() drops any clean folios a racing read
repopulated; fuse_file_mmap() reverts to caching mode (a mapping needs
the page cache).
Latching to direct IO is only coherent if no buffered write can deposit
dirty folios into the page cache after it has been dropped. Add a
per-inode rw_semaphore, wb_inval_rwsem, to serialise the buffered-write
page-cache dirtying against the latch transition. The writeback path
holds it for read around the dirtying and re-checks the latch under it;
fuse_reverse_inval_inode() holds it for write around its invalidate +
latch set. The notification may be delivered by the same server thread
that still owes a reply to an in-flight write holding the inode lock, so
it takes the rwsem with a trylock and never blocks: if the writer has gone
it skips the latch and only invalidates the notified range. The writer's
read-side section stays free of server round-trips because under fc->dlm
the partial-write RMW read is skipped.
Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR will hold the lock shared in the writeback path and switch to DIO on a fuse notify storm (which indicates that we have DLM lock contention from multiple nodes)