The Robservatory

Robservations on everything…

 

VMware Fusion and the vmmon broken pipe error

As part of my struggles to fix my constantly-crashing suggestd process (and the Spotlight failures that it seems to cause), I reinstalled macOS Mojave a couple of times using the Internet Recovery partition. These were in-place reinstalls, so I didn't start completely clean, but the OS is reinstalled from scratch—so much so that I even had to run the 10.14.6 updater and a couple of security updates again.

In the end, not only did this not fix the suggestd problem, but it also broke my all-important VMware Fusion virtual machines. When I tried to launch any of them, I was greeted with an error message:

Error: Could not open /dev/vmmon: Broken pipe

This is apparently so common that VMware has a knowledge base article on the issue. The error is due to VMware Fusion not loading its required kernel extensions, but nobody seems to be sure of the cause of the problem. However, that article is supposed to fix it…unfortunately for me, it did not. I never saw an entry in the Security & Privacy System Preferences panel to allow VMWare's extensions. As a result, they didn't load.

Here's the tl;dr for the fix that did work: Rebooting in Recovery mode (hold down Command-R at startup) and deleting the KernelExtensionManagement folder located in /private/var/db. There was a lot more to it than that, though, which I cover for my own possible future needs in the rest of this post, in case the linked sources ever vanish.


When the fixes in the knowledge base article didn't work for me, I asked a friend who works at VMware if he had any additional suggestions. He eventually came back with a pointer to this forum thread, in particular, the post by Alex_Romeo, which I'll replicate here in case it ever goes away:

  1. In Terminal, get the team identifier for VMware Fusion with this command:

    codesign -dv --verbose=4 -i TeamIdentifier /Applications/VMware\ Fusion.app 2>&1 | grep TeamIdentifier

    Note that I modified this command slightly from the forum post, so it only returns the value you need, TeamIdentifier.

  2. Write down the value for TeamIdentifier. For Fusion 11.5.7, it should be EG7KH642X6. You need it somewhere not on your Mac, because you won't have access to it when you need it.
  3. Reboot into Recovery mode by holding Command-R at reboot, then launch Terminal from the menu.
  4. Run this command in the Terminal window, changing the string at the end to match what you wrote down in the prior step: /usr/sbin/spctl kext-consent add EG7KH642X6
  5. Reboot.

I did all of this, to no avail. I then realized I hadn't followed one of the steps in the knowledge base article, which was a reinstall. I thought I might be stuck here, as being on Mojave, I needed the installer for version 11 of Fusion, not the current version 12.

Thankfully, VMware makes the version 11 installer available on their customer portal. If you have an account, you should be able to download it via this URL.

Before I reinstalled, I did a complete manual uninstallation, removing all traces of VMware Fusion (and its three kexts) from my system. I then rebooted and installed from scratch. I was thinking that, for certain, this should solve the problem. Nope, same issue—the kexts wouldn't load.

By this time, through all the fixes and changes I had tried, I had probably rebooted close to 50 times. To say I was aggravated would be an understatement. So I gave up for the evening, and tackled the problem again this morning.

Today I went back to the VMware forums and started digging through all the posts that matched /dev/vmmon broken pipe. There are many, but I got quite lucky—the first search result had the answer, though it's buried on the very last page of the discussion. On that page, user kevinp2 posted this gem:

I ran into this problem with Mojave 10.14.6 and VMWare Fusion 11.5.7
I tried many of the solutions in this thread but none of them worked other than turning SIP off (csrutil disable) I wasn't comfortable with leaving SIP off.
I finally found the solution in this Github post: kOSKextReturnNotLoadable on Mojave 10.14.6
The solution that I executed was to delete the folder /private/var/db/KernelExtensionManagement and reboot. Upon the reboot, and starting VMWare, the folder was recreated with the correct settings and file permissions.

The linked post on Github has the actual explanation for what's going wrong, courtesy of StoneJT:

Based on the temporary directory error, can you check the permissions and flags for the following:

  • /private/var/db/KernelExtensionManagement should be 0755 with the "restricted" flag set, and the com.apple.rootless attribute set to KernelExtensionManagement (you can check the value of the attribute with xattr -l)
  • /private/var/db/KernelExtensionManagement/Staging should be 0755 with the "restricted" flag set.

I recently had an issue where VMWare Fusion broke after upgrading to 10.14.6, and after some digging around through the console log, I found that it was trying to use these directories while staging the kexts.

When compared to another Mac running 10.14.6, I found that the "restricted" flag was not set against KernelExtensionManagement, which appears to have affected how SIP treats the directory.

If that's the case for you, you should be able to repair it using chflags (e.g chflags restricted /Volumes/Macintosh\ HD/private/var/db/KernelExtensionManagement) from the terminal in recovery mode.

I didn't try the chflags solution, I simply booted to Recovery and used Terminal to wipe the directory. After I rebooted to normal mode, the directory wasn't created. However, when I launched Fusion, it was created, and I was able to open and use all of my saved virtual machines.

Note that because I had tried the other solutions first, I have no idea if they're actually required or not. If I run into this problem again, though, I'm going to try deleting the folder from Recovery before any of the other steps. Hopefully that's all that's required. (Changing the flags is just as time consuming as deleting the folder, as it too must be done from Recovery. I figure by deleting the folder, I'll ensure the permissions are set properly.)

A big thank you to both StoneJT and kevinp2, as without their posts, I never ever would have figured this one out on my own. This was one of the more aggravating and frustrating issues I've ever run into on my Mac; I have no doubt I've rebooted it more in the last 18 hours than I have since it was new.

Leave a Reply

Your email address will not be published. Required fields are marked *

The Robservatory © 2021 • Privacy Policy Built from the Frontier theme