Migrate Homebrew from Intel to M4 MacBook Pro
A big part of the time used to move from my 2017 Intel MacBook Pro to my new 2024 M4 MacBook Pro was spent on migrating my Homebrew installation.
There are three aspects to the migration.
- One is the change in CPU architecture (which means to replace my Intel-only binaries with arm64 binaries).
- The other one is the change of the Homebrew root folder from /usr/local to /opt/homebrew.
- And the last one is the change in OS version (from macOS 13 to macOS 15).
The impact of the change in CPU architecture was mostly taken care of by enabling the Rosetta 2 emulator to run Intel-binaries on the M4.
This immediately fixed the problem of my Terminal windows auto-closing on startup (as they now no longer failed to run the zsh Intel-binary).
But it did not solve all problems, particularly annoying was that vim could not be started and always failed with the following error:
dyld[31382]: Library not loaded: /System/Library/Perl/5.30/darwin-thread-multi-2level/CORE/libperl.dylib Referenced from: <77AC96DE-0453-3E1C-B442-EFE624110BAE> /usr/local/Cellar/vim/9.1.0750_1/bin/vim Reason: tried: '/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE/libperl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE/libperl.dylib' (no such file), '/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE/libperl.dylib' (no such file, not in dyld cache), '/usr/local/lib/libperl.dylib' (no such file), '/usr/lib/libperl.dylib' (no such file, not in dyld cache)
My workaround for this was to use the default vim that comes bundled with macOS, by typing /usr/bin/vim.
Although it complained about some unsupported parts in my .vimrc, it at least started up and was usable enough to edit files.
The error itself seems due to some Perl system libraries that vim was compiled with and which now no longer exist in the new macOS version.
This is a typical problem which can be solved by re-installing Homebrew.
Re-installing Homebrew
The steps below follow this FAQ entry and are extended with what was needed to make it work for my situation.
Your mileage might vary!
- In the Terminal.app settings, set the login shell to Default. This to avoid Rosetta's Intel emulated zsh from the previous installation interfering with things.
- Verify that you're running in native ARM mode with the arch command. It should return arm64.
- Create a dump of the previous Homebrew installation by explicitly running brew through Rosetta's Intel emulation:
arch -x86_64 /usr/local/bin/brew bundle dump --global
- Review the resulting ~/.Brewfile and adjust as desired (I didn't make any adjustments).
- Ensure that native arm binaries are preferred over Intel binaries from the previous installation:
export PATH=/bin:/usr/bin:/sbin:/usr/sbin:$PATH
- Install homebrew into /opt/homebrew with:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
- Adjust your shell config file to prefer /opt/homebrew/bin over /usr/local/bin in the PATH.
As I manually set PATH in my .zshrc, I needed to adjust it. Following these commands recommended by the brew installer was not enough!==> Next steps: - Run these commands in your terminal to add Homebrew to your PATH: echo >> /Users/aj/.zprofile echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> /Users/aj/.zprofile eval "$(/opt/homebrew/bin/brew shellenv)" - Open a fresh terminal window and verify with which brew, it must return /opt/homebrew/bin/brew.
- Now have brew install all the applications from the dump in the ~/.Brewfile:
/opt/homebrew/bin/brew bundle install --global
- This process will print some warnings and errors, thus make sure to keep the output.
I will go through all the errors and warnings I encountered and list how I fixed them below (for the ones I could fix, unfortunately not all have been fixed yet).
It usually ends with a summary of the number of failures:Homebrew Bundle failed! 30 Brewfile dependencies failed to install.Homebrew Bundle failed! 30 Brewfile dependencies failed to install. - Once the process is completed and you see the above summary, open the Terminal.app settings and set the Login shell to /opt/homebrew/bin/zsh
- Open a fresh terminal window and verify that you are now running in native arm64 mode with arch.
- Now we are ready to address the errors, and then at the end we will remove the old Homebrew installation under /usr/local.
Outdated Xcode and missing Xcode command-line utilities
Due to the change in macOS version, the Xcode installation was outdated and also no usable command-line utilities were available.
This usually manifests in errors like these:
Installing ssh-audit ==> Fetching ssh-audit ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-core/723e7ef695e15c790aab727c1da11a2a4610ad53/Formula/s/ssh-audit.rb ==> Cloning https://github.com/jtesta/ssh-audit.git Cloning into '/Users/aj/Library/Caches/Homebrew/ssh-audit--git'... ==> Checking out branch master Already on 'master' Your branch is up to date with 'origin/master'. Error: Your Xcode (15.1) at /Applications/Xcode.app is too outdated. Please update to Xcode 16.0 (or delete it). Xcode can be updated from the App Store. Error: Your Command Line Tools are too outdated. Update them from Software Update in System Settings. If that doesn't show you any updates, run: sudo rm -rf /Library/Developer/CommandLineTools sudo xcode-select --install Alternatively, manually download them from: https://developer.apple.com/download/all/. You should download the Command Line Tools for Xcode 16.0. Installing ssh-audit has failed!
Installing tdsmith/ham/direwolf ==> Fetching dependencies for tdsmith/ham/direwolf: portaudio ==> Fetching portaudio ==> Downloading https://ghcr.io/v2/homebrew/core/portaudio/manifests/19.7.0-1 ==> Downloading https://ghcr.io/v2/homebrew/core/portaudio/blobs/sha256:8ad9f1c15a4bc9c05a9dd184b53b8f5f5d13a2458a70535bfb01e54ce4f8b4bd ==> Fetching tdsmith/ham/direwolf ==> Downloading https://github.com/wb2osz/direwolf/archive/1.4-dev-E.tar.gz ==> Downloading from https://codeload.github.com/wb2osz/direwolf/tar.gz/refs/tags/1.4-dev-E ==> Installing direwolf from tdsmith/ham Error: Your Xcode (15.1) at /Applications/Xcode.app is too outdated. Please update to Xcode 16.0 (or delete it). Xcode can be updated from the App Store. Error: Your Command Line Tools are too outdated. Update them from Software Update in System Settings. If that doesn't show you any updates, run: sudo rm -rf /Library/Developer/CommandLineTools sudo xcode-select --install Alternatively, manually download them from: https://developer.apple.com/download/all/. You should download the Command Line Tools for Xcode 16.0. Installing tdsmith/ham/direwolf has failed!
This can be fixed in two steps.
First update Xcode via the App Store.
Then run the mentioned commands to install/update the command-line utilities:
sudo rm -rf /Library/Developer/CommandLineTools sudo xcode-select --install
With the latest Xcode and command-line utilities in place, trigger a new installation attempt for the affected applications:
brew install ssh-audit brew install tdsmith/ham/direwolf
Missing Rosetta 2 Intel emulation
Some of the Casks distributed by Homebrew are currently only available as Intel-binaries.
This usually manifests in warnings like these:
Installing gqrx ==> Caveats gqrx is built for Intel macOS and so requires Rosetta 2 to be installed. You can install Rosetta 2 with: softwareupdate --install-rosetta --agree-to-license Note that it is very difficult to remove Rosetta 2 once it is installed. ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/g/gqrx.rb ==> Downloading https://github.com/gqrx-sdr/gqrx/releases/download/v2.17.5/Gqrx-2.17.5.dmg ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask gqrx Error: It seems there is already an App at '/Applications/Gqrx.app'. ==> Purging files for version 2.17.5 of Cask gqrx Installing gqrx has failed!
Installing jitsi ==> Caveats jitsi is built for Intel macOS and so requires Rosetta 2 to be installed. You can install Rosetta 2 with: softwareupdate --install-rosetta --agree-to-license Note that it is very difficult to remove Rosetta 2 once it is installed. ==> Downloading https://github.com/jitsi/jitsi/releases/download/Jitsi-2.10/jitsi-2.10.5550.dmg ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask jitsi Error: It seems there is already an App at '/Applications/Jitsi.app'. ==> Purging files for version 2.10.5550 of Cask jitsi Installing jitsi has failed!
Installing keepassx Password: ==> Caveats keepassx is built for Intel macOS and so requires Rosetta 2 to be installed. You can install Rosetta 2 with: softwareupdate --install-rosetta --agree-to-license Note that it is very difficult to remove Rosetta 2 once it is installed. ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/k/keepassx.rb ==> Downloading https://www.keepassx.org/releases/2.0.3/KeePassX-2.0.3.dmg ==> Installing Cask keepassx ==> Changing ownership of paths required by keepassx with sudo; the password may be necessary. Error: It seems there is already an App at '/Applications/KeePassX.app'. ==> Purging files for version 2.0.3 of Cask keepassx Installing keepassx has failed!
Installing sequel-pro Warning: sequel-pro has been deprecated because it is discontinued upstream! It will be disabled on 2024-12-17. ==> Caveats sequel-pro has been deprecated in favor of Sequel Ace. brew install --cask sequel-ace sequel-pro is built for Intel macOS and so requires Rosetta 2 to be installed. You can install Rosetta 2 with: softwareupdate --install-rosetta --agree-to-license Note that it is very difficult to remove Rosetta 2 once it is installed. ==> Downloading https://github.com/sequelpro/sequelpro/releases/download/release-1.1.2/sequel-pro-1.1.2.dmg ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask sequel-pro Error: It seems there is already an App at '/Applications/Sequel Pro.app'. ==> Purging files for version 1.1.2 of Cask sequel-pro Installing sequel-pro has failed!
This can be fixed by running the mentioned command:
softwareupdate --install-rosetta --agree-to-license
With this in place, Intel-binaries will not be a problem anymore.
But as you can see there was actually a second failure in all these errors, which we are going to address next.
Old applications linger around in /Applications and block updates/re-installation
Homebrew nicely installed all the opensource applications into the new /opt/homebrew, but for the Casks which are distributed as binary application bundles and live in /Applications we end up with conflicts (as the new Homebrew installation does not know about the previously installed application bundles there).
This typically manifests in errors like these (also seen in the ones from the Rosetta section):
Installing 0-ad ==> Downloading https://releases.wildfiregames.com/0ad-0.0.26-alpha-osx-aarch64.dmg ==> Installing Cask 0-ad Error: It seems there is already an App at '/Applications/0 A.D..app'. ==> Purging files for version 0.0.26-alpha of Cask 0-ad Installing 0-ad has failed!
Installing brave-browser ==> Downloading https://updates-cdn.bravesoftware.com/sparkle/Brave-Browser/stable-arm64/173.91/Brave-Browser-arm64.dmg ==> Installing Cask brave-browser Error: It seems there is already an App at '/Applications/Brave Browser.app'. ==> Purging files for version 1.73.91.0 of Cask brave-browser Installing brave-browser has failed!
Installing burp-suite ==> Downloading https://portswigger-cdn.net/burp/releases/download?product=community&version=2024.10.3&type=MacOsArm64 ==> Installing Cask burp-suite Error: It seems there is already an App at '/Applications/Burp Suite Community Edition.app'. ==> Purging files for version 2024.10.3 of Cask burp-suite Installing burp-suite has failed!
Installing docker ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/d/docker.rb ==> Downloading https://desktop.docker.com/mac/main/arm64/175267/Docker.dmg ==> Installing Cask docker Error: It seems there is already an App at '/Applications/Docker.app'. ==> Purging files for version 4.36.0,175267 of Cask docker Installing docker has failed!
Installing gitup ==> Downloading https://github.com/git-up/GitUp/releases/download/v1.4.2/GitUp.zip ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask gitup Error: It seems there is already an App at '/Applications/GitUp.app'. ==> Purging files for version 1.4.2 of Cask gitup Installing gitup has failed!
Installing keeweb Password: ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/k/keeweb.rb ==> Downloading https://github.com/keeweb/keeweb/releases/download/v1.18.7/KeeWeb-1.18.7.mac.arm64.dmg ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask keeweb ==> Changing ownership of paths required by keeweb with sudo; the password may be necessary. chown: /Applications/KeeWeb.app/Contents/CodeResources: Operation not permitted [...] chown: /Applications/KeeWeb.app: Operation not permitted Error: It seems there is already an App at '/Applications/KeeWeb.app'. ==> Purging files for version 1.18.7 of Cask keeweb Installing keeweb has failed!
Installing netnewswire ==> Downloading https://github.com/Ranchero-Software/NetNewsWire/releases/download/mac-6.1.4/NetNewsWire6.1.4.zip ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask netnewswire Error: It seems there is already an App at '/Applications/NetNewsWire.app'. ==> Purging files for version 6.1.4 of Cask netnewswire Installing netnewswire has failed!
Installing obs ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/o/obs.rb ==> Downloading https://cdn-fastly.obsproject.com/downloads/OBS-Studio-30.2.3-macOS-Apple.dmg ==> Installing Cask obs Error: It seems there is already an App at '/Applications/OBS.app'. ==> Purging files for version 30.2.3 of Cask obs Installing obs has failed!
Installing transmission ==> Downloading https://github.com/transmission/transmission/releases/download/4.0.6/Transmission-4.0.6.dmg ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask transmission Error: It seems there is already an App at '/Applications/Transmission.app'. ==> Purging files for version 4.0.6 of Cask transmission Installing transmission has failed!
Installing tunnelblick Password: ==> Caveats For security reasons, tunnelblick must be installed to /Applications, and will request to be moved at launch. ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/t/tunnelblick.rb ==> Downloading https://github.com/Tunnelblick/Tunnelblick/releases/download/v4.0.1/Tunnelblick_4.0.1_build_5971.dmg ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask tunnelblick ==> Changing ownership of paths required by tunnelblick with sudo; the password may be necessary. chown: /Applications/Tunnelblick.app/Contents/CodeResources: Operation not permitted [...] chown: /Applications/Tunnelblick.app: Operation not permitted Error: It seems there is already an App at '/Applications/Tunnelblick.app'. ==> Purging files for version 4.0.1,5971 of Cask tunnelblick Installing tunnelblick has failed!
Installing vlc ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/v/vlc.rb ==> Downloading https://get.videolan.org/vlc/3.0.21/macosx/vlc-3.0.21-arm64.dmg ==> Downloading from http://mirror.easyname.ch/videolan/vlc/3.0.21/macosx/vlc-3.0.21-arm64.dmg ==> Installing Cask vlc Error: It seems there is already an App at '/Applications/VLC.app'. ==> Purging files for version 3.0.21 of Cask vlc Installing vlc has failed!
Installing wireshark Password: ==> Downloading https://2.na.dl.wireshark.org/osx/all-versions/Wireshark%204.4.2%20Arm%2064.dmg ==> Installing Cask wireshark ==> Running installer for wireshark with sudo; the password may be necessary. installer: Package name is ChmodBPF installer: Installing at base path / installer: The install was successful. ==> Running installer for wireshark with sudo; the password may be necessary. Error: It seems there is already an App at '/Applications/Wireshark.app'. installer: Package name is Add Wireshark to the system PATH installer: Installing at base path / installer: The install was successful. ==> Purging files for version 4.4.2 of Cask wireshark Installing wireshark has failed!
Now that we have the full list, here is how I fixed these errors:
- For each of them, move the existing application bundle from /Applications to Trash.
- Then trigger the installation again with:
brew install --cask 0-ad brew install --cask brave-browser ...
- The --cask part is important, as sometimes a headless source-only formula exists with the same name which does not include the GUI parts.
By using --cask we make sure that the upstream binary application bundle is used.
With this, most of the fixing is concluded.
Thus let's go to the warnings and errors that I consciously ignore.
Warnings and errors to ignore
Some of the warnings and errors I consciously ignore.
A lot of them come from no longer maintained/available software which Homebrew no longer provides, or which have become obsolete as another software does the same job now.
Also in some cases it is just that I no longer use the software and thus do not need to spend the time to fix their installation problems.
Of course there are some errors in software which I still want to use and which I have not managed to fix yet, we will get to them later.
Here now the warnings and errors which I choose to ignore:
Tapping homebrew/cask Error: Tapping homebrew/cask is no longer typically necessary. Add --force if you are sure you need it for contributing to Homebrew. Tapping homebrew/cask has failed!
Tapping homebrew/core Error: Tapping homebrew/core is no longer typically necessary. Add --force if you are sure you need it for contributing to Homebrew. Tapping homebrew/core has failed!
Skipping gdb (no bottle for Apple Silicon)
Installing hping Error: hping has been disabled because it is not maintained upstream! It was disabled on 2024-02-15. Installing hping has failed!
Skipping hyperkit (no bottle for Apple Silicon)
Installing xhyve Warning: 'xhyve' formula is unreadable: No available formula with the name "xhyve". Warning: No available formula with the name "xhyve". Error: No formulae found for xhyve. ==> Searching for similarly named formulae... Installing xhyve has failed!
Installing jsmin Warning: 'jsmin' formula is unreadable: No available formula with the name "jsmin". Did you mean jsmn, jasmin or jsign? Warning: No available formula with the name "jsmin". Did you mean jsmn, jasmin or jsign? ==> Searching for similarly named formulae... ==> Formulae jsmn jasmin jsign To install jsmn, run: brew install jsmn Installing jsmin has failed!
Installing sslyze Error: sslyze has been disabled because it uses deprecated `openssl@1.1`! It was disabled on 2024-04-05. Installing sslyze has failed!
Installing virtualbox-extension-pack Warning: Cask 'virtualbox-extension-pack' is unavailable: No Cask with this name exists. ==> Searching for similarly named casks... ==> Casks virtualbox@beta To install virtualbox@beta, run: brew install --cask virtualbox@beta Installing virtualbox-extension-pack has failed!
Installing docker-compose-completion Warning: 'docker-compose-completion' formula is unreadable: No available formula with the name "docker-compose-completion". Did you mean docker-completion? Warning: No available formula with the name "docker-compose-completion". Did you mean docker-completion? ==> Searching for similarly named formulae... ==> Formulae docker-completion To install docker-completion, run: brew install docker-completion Installing docker-compose-completion has failed!
Installing visitors Error: visitors has been disabled because it has a removed upstream repository! It was disabled on 2024-02-21. Installing visitors has failed!
Installing osxfuse Password: ==> Caveats `osxfuse` has been succeeded by `macfuse` as of version 4.0.0. To update to a newer version, do: brew uninstall osxfuse brew install macfuse osxfuse requires a kernel extension to work. If the installation fails, retry after you enable it in: System Settings → Privacy & Security For more information, refer to vendor documentation or this Apple Technical Note: https://developer.apple.com/library/content/technotes/tn2459/_index.html You must reboot for the installation of osxfuse to take effect. ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/o/osxfuse.rb ==> Downloading https://github.com/osxfuse/osxfuse/releases/download/osxfuse-3.11.2/osxfuse-3.11.2.dmg ==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-[...] ==> Installing Cask osxfuse ==> Running installer for osxfuse with sudo; the password may be necessary. installer: Error - Das FUSE for macOS Installationspacket ist nicht kompatibel mit dieser macOS Version. Error: Failure while executing; `/usr/bin/sudo -u root -E LOGNAME=aj USER=aj USERNAME=aj -- /usr/sbin/installer -pkg /opt/homebrew/Caskroom/osxfuse/3.11.2/Extras/FUSE\ for\ macOS\ 3.11.2.pkg -target /` exited with 1. Here's the output: installer: Error - Das FUSE for macOS Installationspacket ist nicht kompatibel mit dieser macOS Version. ==> Purging files for version 3.11.2 of Cask osxfuse Installing osxfuse has failed!
Installing tuntap Warning: Cask 'tuntap' is unavailable: No Cask with this name exists. ==> Searching for similarly named casks... ==> Casks tunetag To install tunetag, run: brew install --cask tunetag Installing tuntap has failed!
Installing mailtrackerblocker Warning: mailtrackerblocker has been deprecated because it is now exclusively distributed on the Mac App Store! It will be disabled on 2025-04-22. ==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/1ef784f287cce93f2bb54bea66ae7ac0953f623b/Casks/m/mailtrackerblocker.rb Error: This cask does not run on macOS versions newer than Ventura. Installing mailtrackerblocker has failed!
This concludes the warnings and errors I ignored.
And there is a little exception.
For mailtrackerblocker I did not completely ignore the error, but rather installed the paid MailTrackerBlocker for Mail application from the App Store.
Cleaning up the old Homebrew installation
We still have all the no longer needed Intel-binaries and old Homebrew framework in /usr/local.
Time to clean this up.
Homebrew nicely provides a script to help (partially) with this:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/uninstall.sh)" -- --path=/usr/local
This will remove most of the old Homebrew installation but will likely fail to remove everything.
Especially in my situation with years of cruft accumulated, this was the case.
I ended up with the following output from the cleanup script:
==> /usr/bin/sudo rmdir /usr/local rmdir: /usr/local: Operation not permitted Warning: Failed during: /usr/bin/sudo rmdir /usr/local Warning: Homebrew partially uninstalled (but there were steps that failed)! To finish uninstalling rerun this script with `sudo`. The following possible Homebrew files were not deleted: /usr/local/.com.apple.installer.keep /usr/local/Cellar/ /usr/local/Frameworks/ /usr/local/Homebrew/ /usr/local/MacGPG2/ /usr/local/bin/ /usr/local/etc/ /usr/local/include/ /usr/local/lib/ /usr/local/man/ /usr/local/opt/ /usr/local/remotedesktop/ /usr/local/sbin/ /usr/local/share/ /usr/local/tests/ /usr/local/var/ You may wish to remove them yourself.
I manually went through all these folders and removed what I could identify as obsolete and no longer needed leftovers.
Important though for example the MacGPG2 folder seems to come from the GPGTools installation, thus be aware that there are other things than Homebrew which might have installed some software under /usr/local and not everyhting must necessarily be removed.
Unsolved problems
As mentioned there are some things which I haven't figured out how to fix yet.
I'm missing a good solution for encfs (which is no longer provided by Homebrew due to license changes of the underlying MacFuse project).
Compiling from source (at least through Homebrew) is not supported either (as it is marked as Linux-only and complilation is not supported anymore on macOS).
Update: found a solution for how to install EncFS on macOS Sequoia.
Another missing thing (which is more a problem of not having time to deal with it yet), is the Perl ecosystem including CPAN package management. Here probably some research to find the canonical way for installing Perl in macOS 15 is all I need. 🐪
