DFS is an experimental, quota-aware distributed filesystem for Linux and macOS. It combines Git's shared namespace and history with git-annex's content-addressed storage and exposes the result as a regular FUSE drive.
Every peer sees the complete file and directory tree. File content is downloaded only when opened, explicitly fetched, or pinned. Moves and renames therefore work even when the content is not stored locally.
Warning
This is an MVP. Use a separate test dataset and keep an independent backup. Linux and macOS are exercised with native FUSE integration tests, but broader application-compatibility testing is still needed.
- A complete Git-backed namespace on every peer.
- Direct Git/git-annex peers over SSH or a local path.
- Optional bare Git metadata relay.
- On-demand hydration when an annexed file is opened.
- Writes committed only after all writable mount handles have closed.
- Advisory locks, durable
fsync, atomic rename-overwrite, and open-then-unlink behavior. - Persistent permissions, timestamps, and extended attributes on Linux and macOS.
- Case-preserving names and NFC-normalized Unicode paths.
- Pin, fetch, unpin, safe eviction, and LRU quota enforcement.
- Git history and non-destructive restore commits.
- Encrypted S3-compatible git-annex storage.
- Managed user services with health reporting on systemd and launchd.
- One Go codebase for Linux and macOS.
Each peer needs:
- Go 1.26 or newer to build;
- Git and git-annex;
- OpenSSH and rsync for SSH peers;
- FUSE 3 on Linux or macFUSE on macOS.
Arch Linux/CachyOS:
sudo pacman -S --needed go git git-annex openssh rsync fuse3Debian/Ubuntu:
sudo apt install golang-go git git-annex openssh-client rsync fuse3macOS:
brew install go git git-annexInstall the current macFUSE package from macfuse.io. Direct SSH access to a macOS peer also requires enabling Remote Login.
make build
./bin/dfs doctorThe binary uses the native Go FUSE protocol implementation and does not link against libfuse. FUSE or macFUSE is still required at runtime.
The recommended setup runs the foreground mount process under the operating system's user service manager. The service manager handles login startup, restart after failure, SIGTERM shutdown, and log collection; DFS continues to own FUSE unmounting and repository recovery.
Build DFS first, then install the service with the repository and mountpoint for that peer.
make build
./scripts/install-cachyos.sh \
~/.local/share/dfs/repository ~/DFS ./bin/dfsThis installs the binary as ~/.local/bin/dfs, writes and enables ~/.config/systemd/user/dfs-mount.service, waits for systemd's DFS readiness notification, and enables a watchdog. Useful commands are:
systemctl --user status dfs-mount
journalctl --user -u dfs-mount -f
systemctl --user restart dfs-mount
systemctl --user stop dfs-mount
dfs --repo ~/.local/share/dfs/repository healthUser services normally start when the user logs in. To allow this mount to start during boot before login, an administrator can enable lingering once:
sudo loginctl enable-linger "$USER"make build
./scripts/install-macos.sh \
~/.local/share/dfs/repository ~/DFS ./bin/dfsThis installs the binary under ~/Library/Application Support/DFS/bin, creates ~/Library/LaunchAgents/io.bitbeamer.dfs.mount.plist, loads it into the current GUI login session, and waits for the mount to report healthy. Inspect or control it with:
launchctl print gui/$(id -u)/io.bitbeamer.dfs.mount
launchctl kickstart -k gui/$(id -u)/io.bitbeamer.dfs.mount
dfs --repo ~/.local/share/dfs/repository health
tail -F ~/Library/Logs/DFS/mount.stderr.logRerun the platform installer after building a new DFS version. To disable and remove a service definition while retaining the installed binary and repository data:
./scripts/install-cachyos.sh --uninstall # CachyOS
./scripts/install-macos.sh --uninstall # macOSPackagers can use the templates under packaging/systemd and packaging/launchd. The installers generate equivalent definitions with absolute, safely escaped local paths.
On Linux:
dfs init ~/.local/share/dfs/repository --name linux --cache-limit 100GiB
dfs --repo ~/.local/share/dfs/repository mount ~/DFSOn macOS, clone the Linux repository over SSH:
dfs join ssh://linux.local/home/alice/.local/share/dfs/repository \
~/.local/share/dfs/repository --name mac --cache-limit 50GiB
dfs --repo ~/.local/share/dfs/repository mount ~/DFSFor transfers in both directions, also register macOS on Linux:
dfs --repo ~/.local/share/dfs/repository peer add mac \
ssh://mac.local/Users/alice/.local/share/dfs/repositoryThe mounted drive accepts normal filesystem operations:
cp report.pdf ~/DFS/Documents/
mv ~/DFS/Documents/report.pdf ~/DFS/Archive/
rm ~/DFS/Archive/old.pdfThe mount process runs automatic metadata sync after completed transactions and every 30 seconds. When running it manually, press Ctrl-C to cleanly unmount and stop it, or use dfs unmount ~/DFS from another terminal. SIGTERM from systemd or launchd uses the same clean shutdown path. If another process has its working directory inside the mount, shutdown falls back to a lazy/forced detach rather than hanging on EBUSY. A later service or manual start automatically detaches a disconnected DFS/FUSE endpoint left behind by a crashed process; it does not replace a healthy existing mount.
Writable opens use copy-on-write files under the repository's private .git/dfs/staging directory. Reads through the mount see staged content, while the locked git-annex entry is left untouched until the final writable handle successfully flushes or closes. DFS publishes a dirty staging file with a same-filesystem atomic rename and then schedules annexing and synchronization. A writable open that performs no write, truncate, or handle-level metadata mutation discards its staging copy without changing Git or triggering sync.
fsync first synchronizes the staging descriptor, then atomically publishes and directory-syncs a checkpoint without ending the open transaction. A later write can advance that checkpoint; after a crash, the last successfully synchronized destination remains valid and any newer unfinished staging payload is quarantined by startup recovery. Flush and fsync errors are returned to the application, mark the transaction failed, and prevent an unverified payload from being published on release.
DFS forwards advisory record locks to FUSE and preserves POSIX open-file behavior across unlink and rename-overwrite. Renaming an open write transaction moves its eventual publication target; unlinking it detaches the pathname so closing the descriptor cannot recreate the deleted file.
DFS preserves the mounted file's visible inode and timestamps while git-annex replaces the published regular file with its internal symlink. This prevents editors such as Vim from reporting a false external change during save.
Permissions, ownership, timestamps, and extended attributes are stored in the peer's private DFS state database so they survive git-annex relocking and remounting without mutating content-addressed annex objects. They currently remain peer-local metadata and are not replicated through Git. Names created through DFS are normalized to Unicode NFC while preserving case; case-only renames are supported on both case-sensitive and case-insensitive hosts.
Peers may commit while disconnected. After they reconnect, concurrent edits retain both contents using deterministic .variant-* names; a modification concurrent with deletion is retained as a variant, and competing rename/move destinations are both kept. DFS disables Git's heuristic rename pairing during synchronization so independent operations on different paths with identical annex content cannot be mistaken for one another and discard a valid version. Repeated synchronization reaches the same Git tree on every peer.
Mount startup holds an exclusive repository session and performs conservative recovery before exposing the filesystem. Interrupted write payloads, legacy staging files, partial annex transfers, and stale Git index locks are moved under .git/dfs/recovery/<timestamp>/; the last published destination is left untouched. Durable transaction manifests let DFS remove only proven empty placeholders from interrupted creates and recognize writes whose atomic rename completed before a crash. Pending Git index changes are committed, then Git and git-annex receive fast consistency checks. In-progress merges, rebases, cherry-picks, reverts, and bisects are copied to the recovery directory and block mounting for manual resolution rather than being destructively reset.
A session recorded by another hostname is assumed active because DFS cannot safely inspect arbitrary remote processes. After verifying that the reported mount is no longer active, pass --recover-stale-session to dfs mount. DFS preserves the displaced session record in .git/dfs/recovery/ before continuing. The option never overrides a live mount on the current host.
Mount logging uses Go's structured log/slog text format. The default error level remains quiet unless an operation fails. Use info to see mount lifecycle, filesystem changes, hydration, synchronization, pin refresh, and cache-prune activity:
dfs --repo ~/.local/share/dfs/repository mount \
--log-level info ~/DFSUse debug to additionally record Git and git-annex subprocesses and their durations. --log-file appends the same output to a mode 0600 file while continuing to write it to the terminal:
dfs --repo ~/.local/share/dfs/repository mount \
--log-level debug --log-file ~/dfs-mount.log ~/DFSFor low-level kernel/FUSE request tracing, add --fuse-debug. This is very noisy and automatically enables debug-level logging:
dfs --repo ~/.local/share/dfs/repository mount \
--fuse-debug --log-file ~/dfs-fuse.log ~/DFSAccepted log levels are debug, info, warn, and error.
Managed services use --log-format json --log-level info. systemd sends these records to the user journal; launchd writes them under ~/Library/Logs/DFS. Manual mounts default to the more readable structured text format. Either mode can explicitly select --log-format text or --log-format json.
While mounting, DFS atomically updates .git/dfs/health.json with its lifecycle state, PID, hostname, mountpoint, start time, heartbeat time, and last startup/shutdown error. It is private peer state and never appears in the mounted volume or Git history. Check it with:
dfs --repo ~/.local/share/dfs/repository health
dfs --repo ~/.local/share/dfs/repository health --jsonThe command succeeds only when the report is current, the owning process is alive on this host, and the mountpoint responds. States progress through starting, ready, and stopping/stopped; startup failures remain recorded as failed. systemd additionally waits for the ready transition and receives a watchdog heartbeat.
On restart, DFS removes a dead same-host session record and detaches Linux ENOTCONN or macOS ENXIO FUSE endpoints before mounting. A session claiming to belong to another host still requires explicit --recover-stale-session after verifying that remote mount is gone; service supervision never overrides that safety check.
dfs --repo ~/.local/share/dfs/repository fetch Documents/report.pdf
dfs --repo ~/.local/share/dfs/repository pin Photos/Vacation
dfs --repo ~/.local/share/dfs/repository unpin Photos/Vacation
dfs --repo ~/.local/share/dfs/repository evict Movies/large.mkv
dfs --repo ~/.local/share/dfs/repository cache set-limit 75GiB
dfs --repo ~/.local/share/dfs/repository cache prunefetch caches content, while pin also protects it from automatic eviction. evict delegates safety checks to git-annex and refuses to remove pinned content. The configured limit is enforced after transactions; open or pinned files and git-annex copy-safety rules can temporarily keep usage above it.
A central relay is optional. Direct peers work without it, but both peers must overlap online. A relay lets peers publish metadata at different times.
Create an empty bare repository on an SSH-accessible server:
ssh server.example.com 'git init --bare /srv/dfs.git'
dfs --repo ~/.local/share/dfs/repository relay set \
ssh://server.example.com/srv/dfs.git
dfs --repo ~/.local/share/dfs/repository sync --metadata-onlyThe relay stores Git namespace/history and annex location logs. It does not need to store file content.
git-annex reads the standard AWS credential environment variables. Add encrypted S3 storage and copy content to it with:
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
dfs --repo ~/.local/share/dfs/repository storage add-s3 archive \
--bucket my-dfs-bucket --region eu-central-1
dfs --repo ~/.local/share/dfs/repository storage copy archive DocumentsOn another peer, synchronize metadata, provide credentials, and run:
dfs --repo ~/.local/share/dfs/repository storage enable archivedfs --repo ~/.local/share/dfs/repository status
dfs --repo ~/.local/share/dfs/repository history Documents/report.pdf
dfs --repo ~/.local/share/dfs/repository restore <commit> Documents/report.pdf
dfs --repo ~/.local/share/dfs/repository conflicts
dfs --repo ~/.local/share/dfs/repository doctorRestore creates a new commit and does not rewrite shared history.
applications / Finder
│
▼
systemd / launchd supervision and health
│
▼
Go FUSE mounted view
│ open, close, rename, delete
▼
DFS transaction and quota scheduler
├── Git: namespace, history, merges
├── git-annex: hashes, locations, safe copies
├── SQLite: pins and last-access times
├── SSH/rsync: direct peer content
└── S3: optional durable content
The underlying Git working tree is an implementation detail. DFS keeps its peer-local configuration, database, mount session, staging, and recovery data under .git/dfs, so none of it is tracked by Git or exposed through the mount. A locked git-annex symlink is presented as a normal file; opening missing content runs git annex get, and opening it for writing first hydrates it into a private copy-on-write transaction.
- Peer discovery and pairing are manual; mDNS is not implemented yet.
- The conflict command lists conflicts but does not provide a full conflict-resolution UI.
- History does not itself retain old annex objects. Replicate versions to durable storage before allowing their last copy to be dropped.
- Memory mapping, sparse files, ACL replication, cross-peer POSIX-metadata replication, and large creative applications need more cross-platform stress testing.
- There is no GUI; service installation and peer administration are command-line workflows.
- Cloud storage is S3-compatible only in the CLI; additional providers can be added through git-annex special remotes later.
make test
make test-integration # mounts a temporary FUSE filesystemThe normal test suite includes real two-peer Git/git-annex flows for connected operations, disconnected edit/edit, rename/move, modify/delete, identical-content edge cases, retained content, and stable tree convergence. The integration target additionally verifies copy-on-write publication, no-op and multiple writable handles, advisory locks, fsync, flush/error propagation, atomic rename-overwrite, open rename/unlink behavior, persistent permissions/timestamps/xattrs, case-only renames, NFC normalization, stable visible metadata across annex sync, and repeated Vim saves through a real FUSE mount.
MIT