Skip to content

bash

Revisiting a PDF page counting script

A couple of years back, I created a bash script to count PDF pages across subfolders. Here's how it looks when run on my folder of Apple manuals:

I use this script on the top-level folder where I save all my Fujitsu ScanSnap iX500 scans. Why? Partly because I'm a geek, and partly because it helps me identify folders I might not need to keep on their own—if there are only a few pages in a folder, I'll generally try to consolidate its contents into another lightly-used folder.

The script I originally wrote worked fine, and still works fine—sort of. When I originally wrote about it, I said…

I feared this would be incredibly slow, but it only took about 40 seconds to traverse a folder structure with about a gigabyte of PDFs in about 1,500 files spread across 160 subfolders, and totalling 5,306 PDF pages.

That was then, this is now: With 12,173 pages of PDFs spread across 4,475 files in 295 folders, the script takes over two minutes to run—155 seconds, to be precise. That's not anywhere near acceptable, so I set out to see if I could improve my script's performance.

In the end, I succeeded—though it was more of a "we succeeded" thing, as my friend James (who uses a very similar scan-and-file setup) and I went back-and-forth with changes over a couple days. The new script takes just over 10 seconds to count pages in the same set of files. (It's even more impressive if the files aren't so spread out—my eBooks/Manuals folder has over 12,000 pages, too, but in just 139 files in 43 folders…the script runs in just over a second.)

Where'd the speed boost come from? One simple change that seems obvious in hindsight, but I was amazed actually worked…

[continue reading…]



Semi-automatic Homebrew and video-transcode updates

As I've written about in the past, I use Don Melton's video transcoding tools to rip Blu-Ray discs. I also use Homebrew to install some of the transcode video dependencies, as well as other Unix tools.

Keeping these tools current isn't overly difficult; it only requires a few commands in Terminal:

$ brew update
$ brew upgrade
$ sudo gem update video_transcoding

My problem is that I often forget to do this, because—unlike most GUI Mac apps and the Mac App Store—there's no built-in "hey, there's an update!" system. Suddenly, two months and many revisions later, I finally remember (usually when I see a tweet about a new version of something.) So I thought I'd try to write my own simple update reminder.

I didn't really want a scheduled task, like a launchd agent—it's not like these tools need to stay current on a daily basis. (And one of them needs to run with admin privileges, which complicates things.) I just wanted something that would remind me if it'd been a while since I last checked for updates, and then install the updates if I wanted it to do so.

After mulling it over, I came up with a script that runs each time I open a Terminal window (which I do daily). The referenced script looks at the date on a check file, and asks me if I'd like to check for updates if that date is more than a week older than today's date. This is perfect for my needs: The reminder is automatic, but I can choose when to install the updates based on what I'm doing at the time. If it's been under a week since I last checked, nothing at all is different about my Terminal launch.

Read on for the script and implementation details. (Note: This is not written for a Terminal beginner, as it assumes some knowledge about how the shell works in macOS.)

[continue reading…]



View Unix man (help) pages in Preview

Today's tip goes well with yesterday's tip, which explained how to open any Finder item's folder in Terminal via Keyboard Maestro. Once in Terminal (and sometimes even when not in Terminal), I'll often want to check out the man page (help) for a given command.

You can do this directly in Terminal with man [name of command], of course, but then it opens on top of whatever you were working on, and you have to read it in Terminal. You could use another tab or window, but you'd still be reading in Terminal. There are times, too, when I'm writing about the Unix side of macOS, so I'm not even in Terminal, but still want to view a man page.

My solution to this problem is two different ways of doing the same thing: I open man pages as nicely-formatted PDFs in Preview. The method I use to get to that point depends on if I'm working in Terminal or not.

Update: I've modified the script and macro so that they properly handles two-argument man commands, such as man 3 printf.

In Terminal

Based on an old Mac OS X Hints tip, I created a very simple shell script:

The key to this little script is the -t option on the man command. From the (hehe) man man help file, here's what that does:

  -t     Use /usr/bin/groff -Tps -mandoc -c to format the manual page,
         passing the  output to stdout. The default  output format  of
         /usr/bin/groff -Tps -mandoc -c is  Postscript, refer  to  the
         manual  page  of /usr/bin/groff -Tps -mandoc -c for  ways  to 
         pick an alternate format.

In other words, the -t converts the help page into a PostScript file, which is something that Preview can easily open (which is just what the last line of the script does).

I named this script preman, because it uses Preview to open man pages. Once saved, I made it executable (chmod 755 preman), and I can then open any man page in Preview by typing, for instance, preman bash.

The output is nicely formatted, and by opening the man page in Preview, my Terminal session is uninterrupted. A quick adjustment with Moom, where I have a saved layout to position Preview and Terminal, and I can scan the man page while working in Terminal.

But what about when I'm not in Terminal? For that, I basically implement the same shell script, but with it set up to run within a Keyboard Maestro macro.

[continue reading…]



Behind OS X’s modern face lies an aging collection of Unix tools

Note: This article has been heavily modified, as I was a total knucklehead and completely forgot about GPL v3—thanks to Geordie Korper for pointing it out in the comments. Basically, the cause of the aging Unix tools in OS X is GPL v3. I've updated the following table to include the license information. In every case but one (nano), GPL v3 is the license on the non-updated apps. So that's that…I'll leave this up, though, in case anyone's curious about this stuff.

As I covered recently, the version of bash that ships with OS X is quite old, and as such, is vulnerable to the recently-revealed bash security hole.

At some point, Apple will release an OS X update containing a patched version of bash. (Update released.)

So while Apple has patched bash, this version of the shell is simply ancient. Just how old is it? bash 3.2.53(1) is roughly seven years behind the current version, 4.3.25. Seven years is like, well, forever, in Internet time!

With that bash age gap in mind, I took at look at a number of common Unix apps—in both Mavericks and Yosemite—to see which versions were in use. Then I checked the same apps in MacPorts, a tool that makes it simple to install many Unix apps.

(MacPorts doesn't necessarily have the absolute latest version of every Unix app, but they do stay relatively current. For instance, they already have a patched version of bash that fixes the above vulnerability.)

The results were interesting, to say the least—many of the core Unix utilities in OS X are years and multiple versions behind their open source, er, sources. You can thank GPL v3 for that, as noted above (and covered in more detail below). But that still leaves the big question:

Does it matter?

That is, should we care that these tools aren't keeping up with their latest and (presumably) greatest versions? Is it a problem, or merely a statement that what we have works well enough for the majority of users? (For those who don't want to bother reading, my general opinion is no, it doesn't matter.)

[continue reading…]



A [possibly dangerous] fix for the bash security exposure

In case you missed the news, there’s a big security hole in the bash shell. If you’re using an OS X machine and it’s not exposed to the internet (and even then, not running a publicly-facing service that communicates with bash), you shouldn’t have much of anything to worry about. In theory, anyway.

In practice, because I like to live dangerously, I decided I wanted to make my machine safe. Or at least try. What follows is the how-to for how I did it.

Note: Please exercise extreme caution if you're going to follow my how-to! I am replacing a couple of key system-level files with a new, untested-by-Apple version. It's quite possible this may break my system in ways I haven't yet discovered. Also, given that there's probably not any exposure for a typical user not running web-facing apps that interface with bash, this really isn't worth doing for most users.

With that disclaimer out of the way, read on for my how-to…

[continue reading…]