Part 6: Comprehensive Research of Linux Operating System

Linux is one of the most widely used operating systems in the world. Being an open-source OS, it is a prime choice not only for personal computer, but for many IoT devices. In this research series, we examined Linux in detail, discussed everything from its architecture to the drivers and exploits. Today, we conclude the comprehensive research of Linux with an overview of its main elements.

Linux Architecture

Linux operating system is composed of the combination of Bootloader, Kernel, Init System, Daemons, Graphical Server, Desktop Environment, Applications.

Linux architecture also includes other components. Although some high-level components change in each Linux distribution, certain components remain the same. These are the parts that make up the basic structure of the Linux operating system. Let’s take the Kernel as an example.

Kernel forms the basic structure and control mechanism in Linux, as in every operating system. It is written using just C and runs with the highest privilege level. There is a Protection Ring structure on the CPU. With this structure, privilege differences emerge. In other words, every operation performed on the computer runs in a section on the Ring, and each section has certain privileges. The highest of these privilege levels is Ring 0. Some instructions and registers can only be controlled by processes running at Ring 0. hlt, cr3, MSR_LSTAR are examples of these.

Protection ring consists of 4 parts: Ring 0, Ring 1, Ring 2, and Ring 3. As we said before, Ring 0 is the part with the highest privilege and codes working with this privilege have unlimited authority. As you go up in the rings, the privilege level decreases and, in general, user mode applications run with the Ring 3.

In addition to these, there is a ring called Ring -1 in modern operating systems. This is called hypervisor mode and is the privilege class used for virtualization.

There are different types of kernels in modern operating systems – Monolithic, Micro, and Hybrid. Linux operating system has a monolithic kernel. In Monolithic kernel, all drivers and operating system components run on Ring 0. In the micro kernel, none of these components work in Ring 0. Hybrid kernel can be characterized as a mixture of these two types. There are also libraries in the Linux operating system for user mode applications to communicate with the kernel and to use certain features offered by the kernel. These are called system libraries and contain certain functions for user mode applications. This component is a common unit in all Linux distributions.

Linux Privilege Modes

There are two different privilege modes in Linux operating system – User Mode and Kernel Mode. These modes are distinguished from each other by privilege level. Kernel Mode has unlimited privileges while User Mode has limited privileges to the operating system. In the operating system, many processes, except critical processes, run in User Mode. It is also possible to switch between these modes, which are designed to avoid security problems. Syscall Instruction is used for these transitions. A process running in User Mode can perform operations that require high privilege with the help of syscall. With the syscall instruction, the operations are performed by Kernel for User Mode applications, with the handler previously prepared by the Kernel. Another difference between User Mode and Kernel Mode is their location in virtual memory. The virtual memory of Kernel Mode processes takes place at higher addresses than User Mode processes. The basic logic here is the same as the stack mechanism.

Device Drivers

Operating systems are complex structures formed by the combination of multiple parts. Device drivers are one of these parts in the Linux OS. Every device on the computer has a driver in the operating system, and this driver acts as a bridge between the device and the kernel. Drivers are differentiated from each other according to the devices they represent, which are also divided into three classes. These classes are Char Devices, Block Devices, and Network Devices.

Device drivers in the Linux Operating system have certain general features. Since Linux has a monolithic kernel, the device drivers in the operating system work in Ring 0, which is the Kernel Mode, and have unlimited privilege on the computer. They can be loaded and unloaded in the operating system at runtime, so they are quite dynamic. They can be configured and changed according to the user’s request. Another detail that should be mentioned here is just like Kernel, Device Drivers are mainly pieces of code written in C. These structures, which are bridges between the kernel and the devices, perform communication using methods such as Polling and Interrupts.

Some device drivers have Direct Memory Access(DMA) capability. As we mentioned above, devices divided into three different classes also differ in terms of their working structure and mechanics. The way Char Block and Network devices process data, the communication channels with the kernel and the methods they use in communication are different. But the common point for all of them is that they have unlimited privilege over the operating system. In other words, any vulnerability in device drivers can be enough to take over the entire computer.

Attack Surfaces

One of the important rules in exploitation is to find a suitable attack surface. The greatest need in a remote exploitation is to have a connection to an external network. In this regard, exploitation methods can be performed on the target device with different attack surfaces. When we look at the Linux operating system, it is possible to divide the attack surfaces into three main classes. These are Network, Userspace, and Devices.

There are multiple attack methods that can originate from different locations. With these attack methods, the target device can be accessed and a possible attack scenario results in success. We examined some examples of Userspace, which is one of these attack surfaces, under the title of Vulnerability Analysis. We discussed two different LPE vulnerabilities named DirtyCow and Sudo – Baron Samedit that appeared in the Linux OS. Among these vulnerabilities, DirtyCow was a Linux kernel related vulnerability, while Sudoedit was a vulnerability that emerged with the sudo SUID binary found in many Linux distributions.

In general, LPE vulnerabilities are not strong in a stand-alone attack scenario. In other words, it is necessary to combine them with other vulnerabilities and turn them into a chain. If a remote code execution is performed with an application other than critical applications in the operating system, it will have low privileges. This is where LPEs come into play, helping to raise this low privilege to the root privilege level. Thus, a full chain attack scenario is successfully completed. In the Linux operating system, the Kernel and almost all critical components are written in C. In addition, the SUID binaries used in many Linux distribution are also built in C. The biggest source of LPEs emerging today is memory corruption-based vulnerabilities. Of course, although mitigations have been produced to eliminate such vulnerabilities, it should be noted that the strongest mitigation is a human.

Mitigations

So far, we have talked about vulnerabilities and attack scenarios. There are certain mitigations to prevent vulnerabilities in the Linux operating system. The main purpose here is to make the exploitation of a memory-based vulnerability in any component or application difficult, or even impossible. Mitigations are basically divided into two different branches. These are User Mode Mitigations and Kernel Mode Mitigations.

In the previous part of the research, we discussed User Mode Mitigations under the titles of PIE, NX, Canary, and RELRO. The Kernel Mode Mitigations we went over were ASLR, KASLR, SELinux and SMAP/SMEP. In general, mitigations such as PIE, ASLR, KASLR are based on Randomization logic. As soon as any application is running, the area it allocates in virtual memory is randomly assigned. Although the offsets do not change, this randomization makes it difficult for attackers to write any exploit chains and run commands. Although only their names have changed, today’s modern operating systems all have such randomization-based defense mechanisms. It is the most popular and effective solution on the security side.

On the other hand, mitigations such as NX and Canary are defense mechanisms created to prevent stack-based buffer overflow vulnerabilities that we encountered in the past. However, since the exploitation methods we encounter in the modern period mainly use heap-based overflows, the precautions taken for stack-based overflow are weakening against modern attacks day by day. Otherwise, SELinux is a defense mechanism used especially on the android side, which functions as a filter between User Mode and Kernel Mode.

All these mitigations may have different names in different operating systems. But the basic logic is the same and does not change. Another important detail is that every action taken by the operating system has the potential to protect against exploitation. However, as we see in the next section, as long as the necessary conditions are met, every mitigation can be bypassed and the exploitation can result in success.

Syscall

As we mentioned before, User Mode applications may need high privilege in runtime. This could even be reading or writing a file, for example. When applications start running, they perform their operations on a virtual memory assigned to them by the operating system. When they want to perform an operation outside of this memory, they should get help from the Kernel. For such needs, there is an instruction called syscall. The operating system creates syscall handlers for the User Mode applications to use.

With these handlers, User Mode executes the syscall instruction by placing the necessary information in registers when the application wants to perform any operation that requires high privilege. With instruction, the User Mode application enters the syscall handler and the CPU switches to Ring 0. During this transition, necessary controls are made on the handler and the operation is performed. In operating systems, each syscall has an integer value. Although syscalls show structural and mechanical differences according to the architecture and the OS, the logical operation is the same in every operating system and architecture.

Linux is a Kernel with a monolithic architecture. The general features of monolithic kernels are that the collective code structure called Kernel is a whole. In other words, Kernel codes, Drivers, Components all form the whole that we call Kernel working with Ring 0 privilege. As we can understand, the drivers on Linux are also part of the kernel and have high privileges. In the Linux operating system, the drivers are written using C/C++. There are certain common features found in each driver. For example, the init_module() and cleanup_module() functions are used to specify a driver’s entry and exit points. In other words, these functions are triggered when the drivers are installed and deleted in the kernel.

Apart from this, there are certain libraries that should be in every driver, thanks to these libraries, driver codes can access and use certain capabilities in the kernel. They have special logging mechanisms and these log records have certain flags that indicate their importance level. Building Modules or Drivers is different from User Mode C codes and they are built with Makefiles.

It is worth mentioning that in the Linux operating system, Modules and Device Drivers are almost identical except that they are responsible for a physical device. That is, if the component running in the kernel undertakes the task of communicating with a physical device, it is called a Device Driver, while when it is used only as a pure piece of code in the Kernel, it is called a Module.

Mitigations

Almost all modern operating systems have certain defense mechanisms against attacks. We examined these mechanisms under two headings in the Mitigations section. These were the mitigations that required and did not require leaks. These defense mechanisms aim to minimize or even completely prevent the effects of attacks using different methods. But as we mentioned before, any mitigation can be circumvented as long as the necessary conditions are met. We also touched on this issue in the last section, Exploit Per Mitigation. We examined the exploit codes written for binaries created for educational purposes. There is a bypass method for all the mitigations we mentioned, but it also changes according to this scenario. All these issues, such as the source of the vulnerability discovered in Binary, and what this vulnerability offers the attacker, constitute the key points for a successful attack scenario.

Each exploit is distinguished from each other by its target binary and the methods it uses. At this point, we can talk about a puzzle mechanism. Each binary puzzle has a unique piece, and there is also another unique piece named exploit that can manipulate this piece. So, we can think of it as a key-lock model. Among the mitigations we mentioned, those that require leaks are usually randomization-based ones. For this reason, a leak data is required, but the issue of where and how to get this leak data is completely different.

It is clear that a leak data is required for randomization-based mitigations, but how this data is obtained depends on the attacker’s scenario. On the other hand, in mitigations that do not require leaks, when we need to somehow bypass the mitigation, a specific and useful attack method must be developed. And at this point, the probability of success of the attack increases in direct proportion to the attacker’s skill and ability.

Released in 1991, Linux operating system is now one of the most prominent open-source software. It is used not only for computers but mobile devices, servers, mainframes, and embedded devices. Hundreds of available distributions, thousands of applications, and options for configuring and compiling makes Linux a powerful platform, capabilities of which are worth exploring.

View the previous Linux OS research.