Time Machine Backup to a Windows Share — Do it More Reliably

You can find plenty of instructions on the interwebs for setting up time machine to a network share, even a Windows share.

What you can't easily find is, how to do it reliably.

I have some recommendations.

1. Use cron scripts to just-in-time mount and dismount the time machine share

This is my first and biggest point. Dismounting the backup drive after each backup removes most of the reliability problem of network backup. Before doing this, I rarely got through 3 months without some kind of “the network share got dismounted uncleanly and now it won't mount until I run Disk First Aid on it”.

2. When it comes to data security, if you don't have 3 copies you aren't being serious.

This is a lesson you can take from cloud computing. Both Microsoft and Amazon clouds treat 'at least 3 copies' as the basic level for storing data. That means you want at least 2 independent backup systems for anything on your own machine.

If you combine this thought with the standard “don't put all your eggs in one physical location” motto of backup, you realise that you need a cloud or offsite backup as well as your time machine backup. The simplest free solution for your third copy, if 5GB is enough, is to use iDrive or OneDrive.

3. Buy a copy of Alsoft Disk Warrior

This is optional, and certainly less important than the first two points but, running Disk First Aid or fsck doesn't always work. Sad but true. I typically got a “fsck can't repair it properly” incident about once a year. I had a growing stack of hard disks with a year's worth of backup each, all only mountable readonly.
DiskWarrior has so far been reliable in restoring broken volumes back to fully working state. NB as of 2021 DiskWarrior can't yet repair APFS volumes so stay with HFS+ volumes for your time machines.

Help with cron scripts and multiple backups

Cron Scripts

Here are my script and cron table for mounting a TM drive from the network, requesting a backup, and dismounting the TM drive. It uses wakeonlan to wake the server from sleep, and ping to confirm it's up before trying to mount. It uses osascript to mount the volume because that deals with saving the network password in your keychain.

#! /usr/bin/env sh
#
smbServer=NameOfYourServer.local
smbServerMacAddress='00:00:00:00:00:00' # fill in server mac address to be able to wake it from sleep
smbVolumeUrl="smb://$(whoami)@$smbServer/D"
smbMaybeMountedAtPath1="/Volumes/$smbServer/Backups"
smbMaybeMountedAtPath2="/Volumes/D/Backups"
smbMaybeMountedAtMore(){
    dirname $(find /Volumes -iname $tmDiskImageName -maxdepth 3 2>/dev/null | head -n 1) 2>/dev/null
}
tmDiskImageName='TM3.21.sparsebundle'
tmVolumeMountedAtPath='/Volumes/TM3.21'

# wakeonlan
# I got mine from https://github.com/jpoliv/wakeonlan/blob/master/wakeonlan
# otherwise try https://ddg.gg/bash%20script%20wakeonlan
#

#cron jobs get a very truncated path and can't find ping, diskutil, hdiutil, tmutil ...
PATH="$PATH:/sbin:/usr/sbin:/usr/bin:/usr/local/bin"

echo '#-------------------------------------------------------'
date

echo "$0 [email protected]"
if [[ -z "$1" && -x $(which tmux) ]]
then
    echo "tmuxing ..."
    tmux new-session $0 tmuxed
    exit
elif [[ "$1" == *unmount* ]] ; then
    tmutil status
    if [[ ! -d "$tmVolumeMountedAtPath" ]] ; then 
        echo "$tmVolumeMountedAtPath is already unmounted"
    elif [[ $(tmutil status | grep 'Running = 0') ]] ; then 
        echo "unmounting ..."
        /usr/sbin/diskutil unmount $tmVolumeMountedAtPath
    else
        echo "not unmounting $tmVolumeMountedAtPath because tm status says still running."
    fi
    exit
fi

#-------------------------------------------------------
if [[ -x $(which wakeonlan) ]] ; then   wakeonlan $smbServerMacAddress ; fi
for tried in {1..50} ; do ping -c 1 -t 5 $smbServer 2>&1 && break ; done
if (( $tried == 50 )) ; then
    echo "failed to ping $smbServer. Exiting."
    exit
fi
sleep 10
if (( $tried > 2 )) ; then 
    echo "waiting in case it was a cold-ish start..."
    sleep 20
fi
echo 'mounting ...'
osascript -e 'mount volume "'$smbVolumeUrl'"'
echo 'attaching ...'
hdiutil attach $smbMaybeMountedAtPath1/$tmDiskImageName \
    || hdiutil attach $smbMaybeMountedAtPath2/$tmDiskImageName \
    || hdiutil attach $(smbMaybeMountedAtMore)/$tmDiskImageName \
    || echo \
       "Didn't find $tmDiskImageName at $smbMaybeMountedAtPath1 or $smbMaybeMountedAtPath2 or $(smbMaybeMountedAtMore)"
[ $? -eq 0 ] && echo "attached OK"
echo 'requesting backup.'
  tmutil startbackup --auto
  echo 'Done.'

And the crontab lines:

5  10-23/4 * * * /Users/chris/Applications/tmbackupnow.sh >> /Users/chris/Applications/Logs/tmbackupnow.log 2>&1
20,35 10-23/4 * * * /Users/chris/Applications/tmbackupnow.sh -unmount >> /Users/chris/Applications/Logs/tmbackupnow.log 2>&1

This schedule does 3 or 4 backups per working day on top of the local snapshots that time machine does anyway. Possibly this is overkill if you are squeezed for disk space. It tries to mount the TM machine and kick off a backup at 5 minutes past 10am,2pm,6pm,10pm and then tries to dismount the backup disk 15 minutes later and again another 15 minutes later. Adjust the timing to the size & speed of your backup.

At least 3 copies

My third copy, on top of time machine, is syncthing “continuous file synchronization”, which is great. It's like being able to set up a load of open source CloudDrives but using your local network too.

My fourth copy is either github or bitbucket for code; and iDrive or OneDrive for documents and graphics.

My fifth copy will be scripted backups to Azure storage, which seems like the cheapest way to do cloud backups. Meanwhile I'm paying Apple or Microsoft each month for big enough cloud storage.

(Failing to) Copy a Time Machine Backup to a Network Drive with asr

The Apple support page for copying a Time Machine backup disk doesn't cover the scenario when your new backup target disk is on the network. If you try to do it by hand using cp, rsync, ditto or other, you will likely fail with inscrutable errors.

Using asr may work, but failed for me after 1 ½ days, 500GB, possibly because I had some kind of network disconnected. To rely on a network being reliable for 3 days is to ignore the 8 fallacies of distributed computing, but if your TM backup is small enough this could work.

  1. Use Disk Utility -> File -> New Image -> Blank Image … to create a new sparsebundle disk image on your network drive. The arrowed options must be set correctly (well, you don't have to use sparse bundle but it is allegedly designed specifically for efficient use across a network):

2.

2. Mount the new disk image by double-clicking it, and also attach your existing Time Machine backup drive. Then, use  -> About This Mac -> System Report… -> Hardware/Storage and look in the column BSD Name to find the device names on which your Source and Target volumes are mounted:

3. Turn off Time Machine backup. Usually by unticking “Back Up Automatically” in the Time Machine preferences, if there is no On/Off switch.

4. Then, use asr on the command line to copy the device that hosted the volume to the device hosting the new volume. Use caffeinate at the same time to stop the computer sleeping instead of copying. In my case that was:

sudo caffeinate asr restore --source /dev/disk3 --target /dev/disk4s2 --erase --puppetstrings --allowfragmentedcatalog

I got this output, and after a few seconds had to type y to confirm:

XSTA    start   512 client
XSTA setup
Validating target…done
XSTA metadata
Validating source…done
Erase contents of /dev/disk4s2 (/Volumes/LaCie2019)? [ny]:

The --puppetstrings option means what most of us might call --progress although the output is quite limited.

Expect a speed of about 4 days per terabyte. I don't know why. Watching the Network tab in Activity Monitor I can see that data is rarely going faster than 5MB/s. Even writing to a spinning disk across a 20 year old 100Mbps network should go faster than that. I tried adding --buffers 10 --buffersize 100MB, but that still only got me to about 3 days per terabyte.

Anyway …

For me it failed. Sorry I lost the error message. So I went to to Finder drag-n-drop. The first time this failed after a day; the second it succeeded after 3 days. 🤷‍♂️

Use a Windows NTFS, or Linux, or Network, or other–formatted disk for Apple Time Machine Backups

It used to be well known that you have to use an Apple formatted disk (HFS+) for Time Machine Backups and you can't use an NTFS formatted disk, or any of the popular *nix disk formats with Time Machine.

Except, you always could.

Gotchas And Caveats

It's not always a first choice option:

  • If your Windows drive is connected over USB2, it will be much slower than a Mac disk connected over Thunderbolt or USB 3 or 4. My first backup of 120GB took about 10 hours, aka all night. But further backups after that first one are much quicker.
  • On an external drive plugged in to your computer, the backup volume may not auto-mount when you plug the drive in. You must manually double-click the <YourMachineName>.sparsebundle in the Finder each time you plug the drive in. After that, Time Machine backups will run as normal.
  • Backups, especially to a network share, are best done with some extra setup to improve reliably unless your network, server, router, disks and desktop all stay reliably connected and switched on.

Backing up to a plugged-in NTFS drive

  1. Plug in your foreign-formatted disk and get a read/write driver for it.

  2. Save the the script from tmMakeImage script in, for instance, your Downloads directory.

  3. Open a terminal window.

  4. If you saved to Downloads, then something like this should make the script executable and show you command line usage:

    cd ~/Downloads
    chmod a+x tmMakeImage
    mv tmMakeImage /usr/local/bin
    tmMakeImage
  5. So if your external drive has a Volume called WinDrive, and your Mac drive is about 250GB then this command should initialise it for Time Machine backups:

    tmMakeImage 500GB /Volumes/WinDrive GO
  6. Optionally, follow up with an immediate tmutil startbackup

Opening the Time Machine preferences should now show that you have "Time Machine Backups" selected as your backup drive. Don't forget to double-click the sparsebundle each time you attach the drive, to mount your Backup Volume and allow Time Machine to do its stuff.

Network Drives

Backup to a network drive in a similar way. First mount your smb network fileshare in the Finder, then use the tmMakeImage script to put the sparsebundle image on the mounted drive.

Reliability

Reliability is a concern for backing up across a network, or to a removeable in disk. In otherwise, it's always a concern. Especially if you have a laptop which gets plugged in and out. See notes on using time machine backup to a windows share more reliably.

Background: Using Sparsebundles to create HFS+ formatted drive on a 'foreign' disk format

Apple created, it seems, sparse images and sparse bundles to solve the problem of saving backups on a network drive. Time Machine uses and recognises them. It might even auto-mount the sparsebundle disk image when you re-attach the drive in order to start running a backup.

Other Things Under the Hood of Time Machine Disks

  • You can't create a sparsebundle directly on a network or 'foreign' drive. So the tmMakeImage script first creates the sparsebundle image then copies it to its destination.
  • Sparsebundles don't use up empty space. An sparsebundle declared as 500GB but still empty will only take up a few megabytes of real disk space.
  • A sparsebundle used by Time Machine has a com.apple.TimeMachine.MachineID.plist file in the package, which contains the UUID of the physical machine it belongs to. This stops you accidentally using backups on the 'wrong' machine. (Time Machine does let you browse and use 'wrong' backups though).
  • An external disk plugged in to your Mac must have "Ignore ownership permissions" Off, whereas by default the Mac mounts external drives with "Ignore ownership permissions" set to On. This setting is not stored on the drive itself; it's stored on your machine and can be specified each time the drive is mounted (man hdiutil) or specified permanently (man vsdbutil)
  • You can set the Time Machine destination to a sparsebundle on an attached drive by first mounting it then doing tmutil setdestination /Volumes/Volumenameonceitsbeenmounted. You can mount the image either by double-clicking it in Finder, or with hdiutil attach /path/to/image.
  • Learn More:

    ```
    man tmutil
    man hdiutil
    man vsdbutil
    ```

  • You can still use your backup disks for other files alongside your Time Machine backups. Just don't touch the Backups.backupdb folder! The Finder will protect it to some extent; it doesn't let you modify things in there, but it will let you delete and add things.
  • The Script

    Most of the things in the tmMakeImage script can be found on the net going back to 2007, but you'll still find answers on the internet saying it can't be done.