Tech Log-automated backup infrastructure set up for 3 machines
Tech Log Entry — Three-Machine Automated Backup Infrastructure
(or, OpenSSH + sshfs + rsync Across Windows and Linux)
Background and Context
This entry documents the design and implementation of a fully automated backup system spanning all three machines in my homelab LAN: one Windows 11 desktop ([desktop]) and two Linux Mint laptops ([laptop #1] and [laptop #2]). All three machines are connected via a dedicated gigabit switch on an isolated local subnet, separate from the home Wi-Fi network.
The goal was to have nightly automated backups from all three machines land on a single large external hard drive ("Coffee Canister", 14.5TB) attached to [desktop] — without any manual intervention, without data leaving the LAN (indeed, without involving the internet at all), and with appropriate security restrictions between machines.
Prior state before this session:
[desktop] had automated local backups via FreeFileSync (documented in a previous entry)
[laptop #1] and [laptop #2] had no backup solution (FFS doesn't support Linux)
No SSH infrastructure existed on any machine
Initial Goals
Enable [desktop] to receive backup data from the two Linux laptops over the LAN
Automate nightly rsync backups from both Linux machines to Coffee Canister
Implement key-based SSH authentication (no passwords stored on Linux machines)
Create a restricted service account on [desktop] with write-only access to backup folders — no delete permissions
Isolate [laptop #2]'s backup access from [laptop #1]'s backup data
Complete the backup infrastructure before beginning active Claude Code development on [laptop #2]
Hardware Used
Desktop [desktop]: Lenovo Legion T5, Windows 11 Home, i7 11th gen, 16 GB RAM, with built-in RJ45 port
Laptop [laptop #1]: Lenovo IdeaPad 3, Linux Mint (Cinnamon), i3, 8 GB RAM, USB-to-ethernet adapter
Laptop [laptop #2]: Dell Latitude Ultrabook, Linux Mint, i7, 16 GB RAM, built-in RJ45 port
External HDD Coffee Canister: 14.5TB, USB 3.0, attached to [desktop], assigned permanent drive letter
TP-Link TL-SG105 5-port gigabit unmanaged switch
CAT8 ethernet cables
Software and Tools
OpenSSH Server (Windows optional feature, built into Windows 11)
sshfs (Linux — mounts remote SSH filesystems as local folders)
rsync 3.2.7 (Linux — efficient incremental file synchronization)
PowerShell 5.1 (Windows — administration and configuration)
icacls (Windows — filesystem permission management)
LLM used for guidance and troubleshooting: Claude Sonnet 4.6 [Pro plan]
Architecture Overview
[laptop #1] (Linux) ──────────────────────────────────┐
rsync over sshfs (SSH port 22) │
[laptop #2] (Linux) ──────────────────────────────────► [desktop] (Windows)
OpenSSH Server
backupuser account
F:\backup-[laptop #1]\
F:\backup-[laptop #2]\
│
Coffee Canister (14.5TB)
Both Linux machines push their backups to [desktop] over SSH. [desktop]'s OpenSSH Server receives the data and writes it to Coffee Canister. No data leaves the local LAN.
Part 1: [desktop] — OpenSSH Server Setup (Windows 11)
Installation:
Installed OpenSSH Server via Settings → System → Optional Features (OpenSSH Client was already present; Server is a separate component requiring explicit installation)
Started sshd service and set to automatic startup via PowerShell
Verified service running: Get-Service sshd returned Status: Running
Firewall configuration:
Default firewall rule created automatically on installation; RemoteAddress was set to Any
Restricted SSH access to LAN subnet only using PowerShell
Set all three network adapters (two ethernet, one Wi-Fi) from Public to Private profile
SSH rule profile updated to Private
sshd_config modifications:
Enabled PasswordAuthentication yes (was commented out by default)
Commented out Match Group administrators block (was preventing non-administrator password authentication)
Restarted sshd service to apply changes
Restricted service account (backupuser):
Created dedicated local Windows account backupuser — no administrator rights, no interactive login needed
Set PasswordRequired, PasswordNeverExpires, AccountNeverExpires via PowerShell and net user commands
Password stored in Bitwarden on [desktop]; not stored on either Linux machine (key-based auth used instead)
Created backup destination folders on Coffee Canister: F:\backup-[laptop #1] and F:\backup-[laptop #2]
Applied write-only permissions to both folders using icacls:
Flags granted: Write (W), Add subdirectory (AD), Write data (WD), Write attributes (WA)
Flags deliberately withheld: Delete (D), Delete child (DC)
Result: Linux machines can create and update files but cannot delete existing backup data (all deletions must be done through [desktop])
Security notes:
SSH access restricted to [LAN subnet] — not reachable from Wi-Fi or internet
backupuser cannot delete files from backup destinations — protects backup integrity
Each machine backs up to its own isolated folder — [laptop #1] cannot access [laptop #2]'s backup data and vice versa
Key-based authentication means no passwords are stored on Linux machines
Part 2: SSH Key Setup (Both Linux Machines)
For each machine ([laptop #1], then [laptop #2]):
Generated ED25519 key pair with no passphrase (required for unattended automated backups):
ssh-keygen -t ed25519 -C "[machine]-backup"
Used ssh-copy-id to transfer public key to [desktop]
Key placement issue (Windows OpenSSH quirk):
ssh-copy-id on Linux to a Windows OpenSSH server does not reliably place the key in the correct location
On [laptop #1]: key was written to a file literally named ${AUTH_KEY_FILE} in the user's home directory instead of .ssh/authorized_keys
Fix: manually created .ssh directory, extracted key content from the misnamed file, placed it correctly, set strict permissions via icacls
On [laptop #2]: used cat ~/.ssh/id_ed25519.pub to display the key and Add-Content in PowerShell to append it to the existing authorized_keys file
Both keys ([laptop #1]-backup and [laptop #2]-backup) coexist in the same authorized_keys file, one per line
Permission requirements for Windows OpenSSH authorized_keys:
authorized_keys file must have restricted permissions or OpenSSH will refuse to use it
Set via icacls: backupuser Read, SYSTEM Read, Administrators Full Control, inheritance removed
Part 3: sshfs Mount Setup (Both Linux Machines)
rsync over SSH to a Windows machine fails because Windows does not have rsync installed — rsync requires the tool to be present on both ends. Solution: use sshfs to mount the remote Windows backup folder as a local Linux filesystem, then run rsync locally against the mount point.
Mount command with resilience options:
sshfs -o reconnect,ServerAliveInterval=15,ServerAliveCountMax=3 \
backupuser@[[desktop]-ip]:"F:\\backup-[machine]" \
/home/[username]/mnt/[desktop]-backup
Key options:
reconnect — automatically reconnects if SSH connection drops
ServerAliveInterval=15 — sends keepalive every 15 seconds to prevent idle disconnection
ServerAliveCountMax=3 — retries 3 times before giving up
Autostart on login:
Created shell script at ~/.config/autostart-sshfs.sh with a 10-second sleep to allow network initialization before mounting
Made executable and added to Linux Mint Startup Applications
Tested across reboot on both machines — mount re-established automatically, visible as desktop icon
Desktop icon serves as a visual confirmation that the mount is active each morning
Mount drop issue:
sshfs connection dropped after the initial 3.5-hour full backup on [laptop #1]
Subsequent rm command hung for 6+ minutes trying to reach disconnected mount
File had actually been deleted successfully before connection dropped — terminal simply never returned a prompt
Resolution: used sftp directly to verify and clean up; remounted with resilience options
Part 4: rsync Backup Scripts (Both Linux Machines)
Core rsync flags used:
-av — archive mode with verbose output
--no-perms --no-owner --no-group — suppresses Linux permission metadata that Windows NTFS cannot preserve (eliminates "operation not permitted" errors)
--checksum — compares files by content rather than timestamp, solving Linux/Windows timestamp mismatch that caused full re-transfers on every run
--ignore-errors — treats attribute-setting failures as non-fatal (metadata errors don't affect actual file data)
Exclusions applied (both machines):
mnt/ — backup mount point (must never back up inside the backup destination)
.ssh/ — SSH keys (must not be copied to remote locations)
.steam/, .steampath, .steampid — Steam runtime symlinks incompatible with Windows NTFS
.clamtk/ — ClamAV quarantine and logs
.cache/ — all cache files (browser cache, GPU shader cache, etc.) — regenerated automatically, not personal data
.config/mozilla/ — Mozilla configuration cache data isn't necessary
snap/ — snap package files (reinstallable from snap store)
.local/share/Trash/ — deleted files
Templates/, Public/ — system folders, [laptop #2]lly empty on single-user machines
[laptop #2]-specific exclusions:
Same as [laptop #1] plus explicit exclusion of system folders not needed for academic/professional use
Backup script structure: Each machine has a script (backup-to-[desktop].sh) that:
Checks whether the sshfs mount is active using mountpoint -q
If not mounted, remounts automatically before proceeding
Runs rsync with all flags and exclusions
Logs output to ~/rsync-backup.log
This ensures the backup runs correctly even if the mount dropped since the last login.
Scheduled via cron:
0 8 * * * /home/[username]/backup-to-[desktop].sh >> /home/[username]/rsync-backup.log 2>&1
Scheduled at 8:00 AM to align with actual usage patterns — both laptops are opened and logged in during morning hours, ensuring the sshfs mount is established before the cron job fires. Linux cron does not have a built-in wake-from-sleep capability, so scheduling during active hours is more reliable than a 2:00 AM run that may find the laptop suspended.
Multi-OS Challenges and Solutions
Transfer Performance
[laptop #2]'s significantly higher transfer speed reflects the performance difference between native gigabit ethernet and a USB 3.0 adapter. Both are on the same gigabit switch — the adapter is the bottleneck for [laptop #1].
Backup Verification Approach
A backup that has never been tested for restoration is not a backup — it is a hope. Verification plan:
Daily (short term): Check rsync log for completion, verify mount icon present on desktop
Weekly (medium term): Spot-check file counts and sizes on Coffee Canister; confirm logs show consistent incremental transfers
Monthly (long term): Open several files directly from the backup destination to confirm readability and integrity
After any significant event (power outage, hardware change, OS update): manual verification run
Final Backup Infrastructure State
All backups are local — no cloud dependency, no internet involvement, no data leaving the home network.
Watch Out For (Future)
sshfs mount must be established before the 8:00 AM cron job fires — open and log into both laptops before 8:00 AM; desktop mount icon confirms mount is active
If either laptop is not logged in at 8:00 AM, the backup script's mountpoint check will attempt to remount automatically — but this requires the laptop to be awake and on the network
Coffee Canister must be connected to [desktop] at backup time; if unplugged, all three backups will fail that night
After a Claude Code or VS Code update on [laptop #2], new extension paths may trigger ClamAV false positives — whitelist new paths in ClamTk if they appear
rsync log at ~/rsync-backup.log accumulates indefinitely — consider adding log rotation after several months
The authorized_keys file on [desktop] contains both [laptop #1]'s and [laptop #2]'s public keys — if either machine is decommissioned, remove its key from the file using a text editor
Lessons Learned
rsync between Linux and Windows requires a bridge — rsync is not available on Windows natively. sshfs provides a clean solution by making the Windows destination appear as a local Linux filesystem.
The --checksum flag is essential for Linux-to-Windows rsync. Without it, timestamp differences between Linux and Windows filesystems cause rsync to re-transfer the entire backup on every run, defeating the purpose of incremental backups.
Windows OpenSSH has subtle differences from Linux OpenSSH. Key placement, configuration defaults, and permission requirements all differ from what Linux administrators expect. Test every assumption.
Schedule automated tasks around actual machine usage patterns. A technically correct cron job at 2:00 AM is useless if the machine is suspended. An 8:00 AM job that runs reliably every day is worth more than a perfect 2:00 AM job that runs never.
Separate backup destinations per machine is better practice than a shared folder. It prevents one machine's backup activity from affecting another's, simplifies restoration, and makes it easier to audit what's backed up where.
The write-only permission model for backup accounts is worth the up-front setup time investment. A backup service account that cannot delete files protects its backup integrity against unintended deletions; even if something goes wrong on a source machine, previously backed-up data remains intact on the destination.
Next Steps / To-Do
Install Bitwarden Firefox extensions on [laptop #1] and [laptop #2]
Set up VPN on [laptop #1] and [laptop #2] (WireGuard-based; evaluation in progress)
Begin Claude Code introductory projects on [laptop #2]
Evaluate and uninstall CUDA toolkit on [desktop]; update FreeFileSync exclusions
Address C: drive storage situation on [desktop] (88% full)
Complete reused-password cleanup in Bitwarden
Add log rotation for rsync-backup.log on both Linux machines after several months of accumulation
Look into Trivy (Aqua Security) for a FOSS tool to identify misconfigs, secrets, and vulnerabilities.
Comments
Post a Comment