forked from atom/node-pathwatcher
-
-
Notifications
You must be signed in to change notification settings - Fork 2
Introduce an alternative Linux implementation using inotify…
#7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
savetheclocktower
wants to merge
1
commit into
master
Choose a base branch
from
custom-inotify-implementation
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,268 @@ | ||
|
|
||
| #include "InotifyFileWatcher.hpp" | ||
|
|
||
| #include <errno.h> | ||
| #include <limits.h> | ||
| #include <poll.h> | ||
| #include <string.h> | ||
| #include <sys/inotify.h> | ||
| #include <unistd.h> | ||
|
|
||
| #ifdef DEBUG | ||
| #include <iostream> | ||
| #endif | ||
|
|
||
| namespace { | ||
|
|
||
| // IN_CREATE/IN_DELETE/IN_MOVED_*: children appearing, disappearing, or being | ||
| // renamed within the watched directory. | ||
| // IN_MODIFY/IN_CLOSE_WRITE: content changes to children. | ||
| // IN_DELETE_SELF/IN_MOVE_SELF: the watched directory itself goes away. | ||
| constexpr uint32_t kWatchMask = IN_CREATE | IN_DELETE | IN_DELETE_SELF | | ||
| IN_MOVE_SELF | IN_MOVED_FROM | IN_MOVED_TO | | ||
| IN_MODIFY | IN_CLOSE_WRITE; | ||
|
|
||
| constexpr size_t kEventBufLen = | ||
| 64 * (sizeof(struct inotify_event) + NAME_MAX + 1); | ||
|
|
||
| } // namespace | ||
|
|
||
| InotifyFileWatcher::InotifyFileWatcher() { | ||
| inotifyFd = inotify_init1(IN_CLOEXEC | IN_NONBLOCK); | ||
| if (inotifyFd == -1) { | ||
| isValid = false; | ||
| return; | ||
| } | ||
|
|
||
| // A pipe lets the destructor unblock poll() cleanly without signals. | ||
| if (pipe(wakeupPipe) == -1) { | ||
| close(inotifyFd); | ||
| inotifyFd = -1; | ||
| isValid = false; | ||
| return; | ||
| } | ||
|
|
||
| eventThread = std::thread(&InotifyFileWatcher::eventLoop, this); | ||
| } | ||
|
|
||
| InotifyFileWatcher::~InotifyFileWatcher() { | ||
| isValid = false; | ||
| stopping = true; | ||
|
|
||
| // Unblock the event loop thread. | ||
| char byte = 0; | ||
| write(wakeupPipe[1], &byte, 1); | ||
|
|
||
| if (eventThread.joinable()) | ||
| eventThread.join(); | ||
|
|
||
| close(wakeupPipe[0]); | ||
| close(wakeupPipe[1]); | ||
| // Closing the inotify fd automatically removes all of its watches. | ||
| if (inotifyFd >= 0) | ||
| close(inotifyFd); | ||
| } | ||
|
|
||
| efsw::WatchID InotifyFileWatcher::addWatch(const std::string &path, | ||
| efsw::FileWatchListener *listener, | ||
| bool /* _useRecursion */) { | ||
| #ifdef DEBUG | ||
| std::cout << "InotifyFileWatcher::addWatch: " << path << std::endl; | ||
| #endif | ||
| if (!isValid) | ||
| return efsw::Errors::WatcherFailed; | ||
|
|
||
| std::string dir = path; | ||
| if (dir.empty() || dir.back() != '/') | ||
| dir += '/'; | ||
|
|
||
| int wd = inotify_add_watch(inotifyFd, dir.c_str(), kWatchMask); | ||
| if (wd < 0) { | ||
| switch (errno) { | ||
| case ENOENT: | ||
| return efsw::Errors::FileNotFound; | ||
| case EACCES: | ||
| return efsw::Errors::FileNotReadable; | ||
| default: | ||
| return efsw::Errors::WatcherFailed; | ||
| } | ||
| } | ||
|
|
||
| std::lock_guard<std::mutex> lock(mapMutex); | ||
| efsw::WatchID handle = nextHandleID++; | ||
| handlesToWatches[handle] = {dir, listener, wd}; | ||
| wdToHandles.insert({wd, handle}); | ||
| return handle; | ||
| } | ||
|
|
||
| void InotifyFileWatcher::removeWatch(efsw::WatchID handle) { | ||
| int wd = -1; | ||
| bool wasLastHandleForWd = false; | ||
| { | ||
| std::lock_guard<std::mutex> lock(mapMutex); | ||
| auto it = handlesToWatches.find(handle); | ||
| if (it == handlesToWatches.end()) | ||
| return; | ||
| wd = it->second.wd; | ||
| handlesToWatches.erase(it); | ||
|
|
||
| auto range = wdToHandles.equal_range(wd); | ||
| for (auto wit = range.first; wit != range.second; ++wit) { | ||
| if (wit->second == handle) { | ||
| wdToHandles.erase(wit); | ||
| break; | ||
| } | ||
| } | ||
| wasLastHandleForWd = (wdToHandles.find(wd) == wdToHandles.end()); | ||
| } | ||
|
|
||
| if (wasLastHandleForWd) { | ||
| // EINVAL here just means the kernel already removed this watch (e.g. the | ||
| // directory was deleted); safe to ignore. | ||
| inotify_rm_watch(inotifyFd, wd); | ||
| } | ||
| } | ||
|
|
||
| void InotifyFileWatcher::forgetWatch(int wd) { | ||
| std::lock_guard<std::mutex> lock(mapMutex); | ||
| auto range = wdToHandles.equal_range(wd); | ||
| for (auto it = range.first; it != range.second;) { | ||
| handlesToWatches.erase(it->second); | ||
| it = wdToHandles.erase(it); | ||
| } | ||
| } | ||
|
|
||
| void InotifyFileWatcher::sendFileAction(int wd, const std::string &filename, | ||
| efsw::Action action, | ||
| const std::string &oldFilename) { | ||
| // Copy out (handle, dir, listener) under the lock, then call into listener | ||
| // code without holding it. | ||
| std::vector<std::tuple<efsw::WatchID, std::string, efsw::FileWatchListener *>> | ||
| targets; | ||
| { | ||
| std::lock_guard<std::mutex> lock(mapMutex); | ||
| auto range = wdToHandles.equal_range(wd); | ||
| for (auto it = range.first; it != range.second; ++it) { | ||
| auto hit = handlesToWatches.find(it->second); | ||
| if (hit != handlesToWatches.end()) | ||
| targets.emplace_back(it->second, hit->second.dir, hit->second.listener); | ||
| } | ||
| } | ||
|
|
||
| for (auto &[handle, dir, listener] : targets) | ||
| listener->handleFileAction(handle, dir, filename, action, oldFilename); | ||
| } | ||
|
|
||
| void InotifyFileWatcher::eventLoop() { | ||
| char buf[kEventBufLen]; | ||
|
|
||
| // State for pairing IN_MOVED_FROM with a following IN_MOVED_TO that shares | ||
| // its cookie and watch — i.e. a rename within the same directory. | ||
| bool hasPendingMove = false; | ||
| int pendingWd = -1; | ||
| uint32_t pendingCookie = 0; | ||
| std::string pendingFilename; | ||
|
|
||
| auto flushPendingMove = [&]() { | ||
| if (!hasPendingMove) | ||
| return; | ||
| // No matching IN_MOVED_TO arrived: the file was moved somewhere we're not | ||
| // watching (or out of the filesystem entirely). | ||
| sendFileAction(pendingWd, pendingFilename, efsw::Actions::Delete); | ||
| hasPendingMove = false; | ||
| }; | ||
|
|
||
| while (!stopping) { | ||
| struct pollfd fds[2]; | ||
| fds[0] = {inotifyFd, POLLIN, 0}; | ||
| fds[1] = {wakeupPipe[0], POLLIN, 0}; | ||
|
|
||
| // If we're sitting on an unpaired IN_MOVED_FROM, don't block forever — | ||
| // give its IN_MOVED_TO a brief window to show up, then resolve it. | ||
| int timeoutMs = hasPendingMove ? 10 : -1; | ||
|
|
||
| int r = poll(fds, 2, timeoutMs); | ||
| if (r < 0) { | ||
| if (errno == EINTR) | ||
| continue; | ||
| break; | ||
| } | ||
|
|
||
| if (stopping || (fds[1].revents & POLLIN)) | ||
| break; | ||
|
|
||
| if (r == 0) { | ||
| flushPendingMove(); | ||
| continue; | ||
| } | ||
|
|
||
| if (!(fds[0].revents & POLLIN)) | ||
| continue; | ||
|
|
||
| ssize_t len = read(inotifyFd, buf, sizeof(buf)); | ||
| if (len <= 0) | ||
| continue; | ||
|
|
||
| ssize_t i = 0; | ||
| while (i < len) { | ||
| auto *event = reinterpret_cast<struct inotify_event *>(&buf[i]); | ||
| i += sizeof(struct inotify_event) + event->len; | ||
|
|
||
| if (event->mask & IN_Q_OVERFLOW) | ||
| continue; | ||
|
|
||
| std::string filename = event->len > 0 ? std::string(event->name) : ""; | ||
|
|
||
| if (hasPendingMove) { | ||
| if ((event->mask & IN_MOVED_TO) && event->wd == pendingWd && | ||
| event->cookie == pendingCookie) { | ||
| // Same-directory rename — almost always an atomic save. | ||
| sendFileAction(event->wd, filename, efsw::Actions::Moved, | ||
| pendingFilename); | ||
| hasPendingMove = false; | ||
| continue; | ||
| } | ||
| flushPendingMove(); | ||
| // fall through and process this event normally | ||
| } | ||
|
|
||
| if (event->mask & IN_IGNORED) { | ||
| // The kernel already removed this watch (explicit removeWatch, | ||
| // directory deleted, or filesystem unmounted). | ||
| forgetWatch(event->wd); | ||
| continue; | ||
| } | ||
|
|
||
| if (event->mask & IN_MOVED_FROM) { | ||
| hasPendingMove = true; | ||
| pendingWd = event->wd; | ||
| pendingCookie = event->cookie; | ||
| pendingFilename = filename; | ||
| continue; | ||
| } | ||
|
|
||
| if (event->mask & (IN_DELETE_SELF | IN_MOVE_SELF)) { | ||
| // The watched directory itself is gone or has moved. Report it as a | ||
| // deletion of the directory; the caller can re-addWatch if it cares | ||
| // to keep following it (we have no way to learn its new path). | ||
| sendFileAction(event->wd, "", efsw::Actions::Delete); | ||
| continue; | ||
| } | ||
|
|
||
| if (event->mask & IN_CREATE) { | ||
| sendFileAction(event->wd, filename, efsw::Actions::Add); | ||
| } else if (event->mask & IN_DELETE) { | ||
| sendFileAction(event->wd, filename, efsw::Actions::Delete); | ||
| } else if (event->mask & IN_MOVED_TO) { | ||
| // Arrived from outside this directory (or its IN_MOVED_FROM partner | ||
| // already timed out): treat it as a brand-new file. | ||
| sendFileAction(event->wd, filename, efsw::Actions::Add); | ||
| sendFileAction(event->wd, filename, efsw::Actions::Modified); | ||
| } else if (event->mask & (IN_MODIFY | IN_CLOSE_WRITE)) { | ||
| sendFileAction(event->wd, filename, efsw::Actions::Modified); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| flushPendingMove(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| #pragma once | ||
|
|
||
| #include "../../vendor/efsw/include/efsw/efsw.hpp" | ||
| #include <atomic> | ||
| #include <mutex> | ||
| #include <string> | ||
| #include <thread> | ||
| #include <unordered_map> | ||
|
|
||
| // An API-compatible replacement for `efsw::FileWatcher` that talks to inotify | ||
| // directly. Plays the same role on Linux that `KqueueFileWatcher` and | ||
| // `FSEventsFileWatcher` play on macOS. | ||
| // | ||
| // Key differences from `FileWatcherInotify`: | ||
| // | ||
| // * A single `inotify` instance backs every watch; `addWatch()` just calls | ||
| // `inotify_add_watch()` against it and gets back a watch descriptor (wd). No | ||
| // per-path file descriptors, and no fd-limit juggling. | ||
| // * No recursion: the `_useRecursion` flag is ignored, and only the directory | ||
| // passed to addWatch() is watched. The existing JS layer only ever calls | ||
| // `addWatch()` with a directory path — either the directory being watched | ||
| // directly, or the parent of a watched file — so this matches how the old | ||
| // inotify backend was used in practice anyway. | ||
| // * Renames are reconstructed from `IN_MOVED_FROM`/`IN_MOVED_TO` pairs that | ||
| // share a cookie and land on the same watch. This is how editors' atomic | ||
| // saves (write tmp file, rename over target) show up, and it's reported as | ||
| // `Moved`, which the existing JS rename-handling collapses into a `change` | ||
| // event for the watched file. Cross-directory moves are reported as a Delete | ||
| // (on the source watch) plus an Add+Modified (on the destination watch), | ||
| // matching the old inotify-based behavior. | ||
| // * If the watched directory itself is removed or renamed (`IN_DELETE_SELF` / | ||
| // `IN_MOVE_SELF`), we report a single Delete for the directory. `inotify` | ||
| // gives us no way to recover the new path of a moved directory, so the | ||
| // caller must re-`addWatch` if it wants to keep following it. | ||
| // | ||
| class InotifyFileWatcher { | ||
| public: | ||
| InotifyFileWatcher(); | ||
| ~InotifyFileWatcher(); | ||
|
|
||
| efsw::WatchID addWatch(const std::string &path, | ||
| efsw::FileWatchListener *listener, | ||
| bool _useRecursion = false); | ||
|
|
||
| void removeWatch(efsw::WatchID handle); | ||
|
|
||
| bool isValid = true; | ||
|
|
||
| private: | ||
| struct Watch { | ||
| std::string dir; // always ends with '/' | ||
| efsw::FileWatchListener *listener; | ||
| int wd; | ||
| }; | ||
|
|
||
| void eventLoop(); | ||
|
|
||
| // Looks up every handle registered for `wd` and dispatches `action` to each | ||
| // of their listeners. | ||
| void sendFileAction(int wd, const std::string &filename, efsw::Action action, | ||
| const std::string &oldFilename = ""); | ||
|
|
||
| // Drops all bookkeeping for `wd` without calling `inotify_rm_watch()`; used | ||
| // when the kernel tells us (via IN_IGNORED) that it already dropped the | ||
| // watch on its own (e.g. the directory was deleted or unmounted). | ||
| void forgetWatch(int wd); | ||
|
|
||
| long nextHandleID = 1; | ||
| int inotifyFd = -1; | ||
| int wakeupPipe[2] = {-1, -1}; | ||
| std::atomic<bool> stopping{false}; | ||
| std::mutex mapMutex; | ||
| std::thread eventThread; | ||
|
|
||
| std::unordered_map<efsw::WatchID, Watch> handlesToWatches; | ||
| std::unordered_multimap<int, efsw::WatchID> wdToHandles; | ||
| }; | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Humour my ignorance here, if this is an API compatible replacement then why do we still need to include this? Or is it only a particular part being replaced?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
.hppfile being referenced includes type definitions. So we need it in order to be able to refer to things below likeefsw::FileWatchListenerandefsw::WatchID.But, yes, in another sense only a particular part is being replaced. In
core.hwe conditionally define theFileWatchertype as referring to either anefsw::FileWatcher(on Windows) or an API-compatible replacement (on other platforms). In order to prove it's an API-compatible replacement, we have to referenceefsw‘s types.I imagine that we'll eventually use our own implementation on Windows, at which point we can remove
efswfrom the repo entirely and just use types entirely of our own definition.