I upgraded the blog to the newest Jekyll 4.4.0 which was released yesterday.
Unfortunately this first resulted in the following segfault while running jekyll build 🙈

/usr/gem/gems/sass-embedded-1.83.4/ext/sass/embedded_sass_pb.rb:11: [BUG] Segmentation fault at 0x0000000000004410
ruby 3.1.1p18 (2022-02-18 revision 53f5fc4236) [x86_64-linux-musl]

Turns out that this is a known problem of the google-protobuf gem (which is used by jekyll-sass-converter which is part of the default Jekyll).
Luckily there is a workaround.
Adding the following to my Gemfile fixed it 🎉

gem 'google-protobuf', force_ruby_platform: true if RUBY_PLATFORM.include?('linux-musl')