Skip to content

rsync

Selective pruning of old rsync backups

In yesterday's post, I described a couple rsync oddities, and how they'd led me to this modified command for pruning old (older than four days) backups:

find /path/to/backups/ -d 1 -type d -Bmin +$((60*4*24)) -maxdepth 1 -exec rm -r {} +

After getting this working, though, I wondered if it'd be possible to keep my backups from the first day of each month, even while clearing out the other dates. After some digging in the rsync man page, and testing in Terminal, it appears it's possible, with some help from regex.

My backup folders are named with a trailing date and time stamp, like this:

back-2017-05-01_2230
back-2017-05-02_0534
back-2017-05-02_1002

To keep any backups made on the first of any month, for my folder naming schema, the modified find command would look like this:

find /path/to/backups/ -d 1 -type d -Bmin +$((60*4*24)) -maxdepth 1 -not -regex ".*-01_.*" -exec rm -r {} +

The new bits, -not -regex ".*-01_.*" basically say "find only files that do not contain anything surrounding a string that is 'hyphen 01 underscore.' And because only backups made on the first of the month will contain that pattern, they're the only ones that will be left out of the purge.

This may be of interest to maybe two people out there; I'm documenting it so I remember how it works!



How to not accidentally delete all your rsync backups

With my Time Machine-like rsync backups running well, I decided it was time to migrate over the cleanup portion of my old script—namely, the bit that removes older backups. Soon after I added this bit to my new script, though, I had a surprise: All of my backups, save the most recent, vanished.

In investigating why this happened, I stumbled across two rsync/macOS behaviors that I wasn't aware of…and if you're using rsync for backup, they may be of interest to you, too.

[continue reading…]



Cancel shell script on remote connect failure

I use a shell script to back up this site (and a variant of the same script to back up the Many Tricks site). I've been using these scripts for over a decade (wow), and though they've evolved, they're still fundamentally the same. (I did switch from cron to launchd for launching them, however.)

While the script typically runs very nicely, I recently noticed that my last backup was from a few weeks ago—uh oh. It didn't take long to figure out what had gone wrong: My ISP changed the hostname of the machine my site runs on, and my script uses ssh, scp and rsync, which connect via the hostname. Unfortunately, the failure mode is silence, because the script runs via a scheduled task. The only way I knew it failed was when I went to check the backup folder. Obviously, something more automatic than that would be desirable.

After much web searching, I couldn't find anything that seeemed like it'd do what I want: An email (and onscreen alert) if my backup failed. I found lots of Unix solutions to send mail using sendmail, but I didn't really want to enable that on my Mac. So I futzed around and built a simple checker that will mail me when it can't reach my web host.

[continue reading…]