While most modern OSes share many ideas and concepts, the underlying philosophy behind the kernels tends to vary. Linux uses a more monolithic kernel with some user space additions, MacOS uses the Mach kernel which is a microkernel, while Windows strikes a balance with a hybrid kernel.
The difference between these types of kernels affects the gradient between reliability and latency among other things. Let’s go over the difference between monolithic kernels and microkernels, then address the mix known as hybrid kernels, as well as how this impacts modern OS design and implementation.
Monolithic Kernels
The oldest and most tested type of kernel is a monolithic kernel. This means that the kernel process in the OS is (for all intents and purposes) a single process which runs in kernel mode. One process handles memory, scheduling, drivers, system calls, etc.
This one process can be fast, but tends to suffer from slowdowns in certain operations since it is a single process ultimately driving operations. Monolithic kernels also tend to be extremely fault sensitive. If you were around for the old days of Windows 98 and similar, you remember the Bluescreens of Death (BSoD) which occurred when you plugged in the wrong device or did the wrong operation on the wrong driver.
Modern systems using a monolithic kernel tend to require more rigorous testing in order to make sure the kernel is fault tolerant enough. The philosophy behind these systems is that you measure twice and cut once for kernel code. Sane operation should not make you reach conditions where the system is going to break. This works well in practice for something like the mainstream Linux kernel, but if done wrong or you pull in the wrong driver, can lead to an extremely unstable system.
Expanding Monolithic Kernels
A lot of monolithic architectures work in extensible modules. Linux has kernel modules which can be loaded and unloaded as necessary. These modules still exist in kernel mode and can still bring down the whole system if something goes wrong.
Microkernels
On the other hand, there are microkernels which have the basic scheduler, memory management, etc. in a core, kernel mode process, but the rest of the kernel operates at various levels in user space. A fault in a device driver doesn’t necessarily bring down the whole system anymore.
Another benefit of this system is that each piece is more modular and more fault resilient. You can just disable or remove a process if it is too impactful on the system or crashes. The downside is that this does increase the latency of communication between the kernel and non-kernel processes which can impact system usability.
MacOS uses the Mach kernel which is a microkernel. My tuned Linux system can get about 10% better audio latency over my MacOS machine while recording despite the Linux machine’s hardware being substantially less powerful, but it comes at the cost of flexibility. With modern hardware, this improvement isn’t really all that noticeable except for very specific operations.
Hybrid Kernels
Microkernels offer some great features over traditional, monolithic kernels, but come with their own caveats. The compromise is to build a hybrid kernel. The NT kernel for Windows is an example of a true hybrid kernel.
Hybrid kernels bake more than a microkernel would into a monolithic, core kernel process, but use microkernel style services for extensibility. Certain fault sensitive parts of the kernel live in user space. Reading the wrong file off of a network share no longer has a chance to take down the entire kernel in a BSoD or kernel panic.
Striking a Balance
Hybrid kernels try to balance the features of both kernel types to create something more usable for the average use case. Microkernels tend to have higher fault tolerance at the expense of higher latency. Microkernels also tend to have certain conditions which can lead to instability or slowness more than traditional monolithic kernels (race conditions). What does the kernel do when something like the queueing system gets flooded by two classes of operations which are conflicting?
A properly balanced hybrid kernel deals with this by making core functions built into a monolithic piece of the kernel in kernel mode while offloading others. Ideally, if you have a queue being saturated, part of it will die off without bringing down the entire OS. You can’t do this with most microkernels without wasting a good number of cycles assessing each task. Monolithic kernels run the risk of crashing from the OS oversaturating the same queue and not being able to keep up (though on most mature systems, this is only going to happen in extremely rare circumstances). Which approach do you bet on if you don’t know the system intimately?
Modern Problems Require Modern Solutions
Even though the Linux kernel is a monolithic kernel, it has implemented some of the ideas of microkernels and hybrid kernels. Linus has traditionally been a bit vocal about his views on kernels, but they work out for the current kernel architecture. When Linux started, it wasn’t built with the excess of resources we have at present. A monolithic kernel was computationally cheaper at the expense of certain tasks being a little harder.
Many kernel tasks in Linux have been offloaded out of the kernel in some way while others stay. FUSE allows many file systems to be implemented in user space which means a crash is an application issue rather than a kernel issue. That weird NTFS versioning problem is now a segfault rather than a kernel panic.
MacOS took Mac OS 9 and completely redid it with a new microkernel (one which wasn’t arguably a glorified Motorola 68000 emulator with extra steps) and a new user space based on BSD. The old system had its merits, but just couldn’t keep up. Even Microsoft moved kernel types with the jump from Windows 98 and Windows ME to Windows 2000 which marked the start of the modern NT era.
What Works Best?
Microkernels are much more fault resilient than than monolithic kernels if everything else is equal. A good microkernel will resist a driver crash which will bring down a Windows or Linux box. On the other hand, the microkernel is also slower and less efficient with I/O and other operations. With modern systems, this impact usually isn’t that noticeable.
My Linux box records audio with 10% less latency than my Mac for the same buffer rate because there is less cross talk between the kernel and the audio driver. When everything is optimized, I don’t see any change in the overall stability. If your kernel is constantly crashing, you probably have worse issues to deal with than fault tolerance. The MacOS system is plenty fine for audio work, and the amount of effort required to get my Linux machine where it is won’t be worth it for most casual users.
Hybrid kernels try to strike a balance between the two approaches. I only use Windows for work, but I’ve watched drivers crash over and over while the rest of the OS brings itself back up without crashing. The balance means less latency in certain operations a Mac might struggle with, but with a little more stability than a Linux box running experimental drivers.
Implications of the Types
Monolithic kernels inherently require better testing and a better approach to avoid halts. Microkernels require better testing for latency and hits to throughput than the other types. Hybrid kernels can lean either way for better and for worse. Each type requires a balancing act, but some parts of the act are easier than the others.
Linux requires code to be better at the expense of developers having to be much more cautious about the type of drivers they commit. Experimental drivers can bring down the whole operating system easily. Macs are more fault resilient, but have conditions where they may as well have crashed from spinlocks between clashing drivers and applications trampling the same kernel operations. Windows machines mix the approaches and end up with certain operations having the benefits of both a monolithic and microkernel approach, while others have the faults of both.
No one type is the right answer to every problem, but monolithic gets you speed in most situations, microkernels get you get reliability, and a hybrid kernel gets a mix. The other side of this is that monolithic kernels can be less stable, microkernels tend to be slower, and hybrid kernels can be a bit more variable on whether the monolithic or microkernel side benefits or harms the individual process.
Monolithic kernels encourage a more conservative approach to drivers and coding for the kernel, while microkernels allow experimentation with minor faults being inconvenient rather than a kernel panic. Hybrid kernels depend on which portion is outsourced from the kernel and which stays as a core component. Each kernel type has its own set of advantages and disadvantages.
Featured image by sunnysun0804 from Pixabay